HEPiX bit-preservation WG Dmitry Ozerov/DESY Germán Cancio/CERN HEPiX Fall 2013, Ann Arbor.
JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA...
-
Upload
sharon-nichols -
Category
Documents
-
view
215 -
download
0
Transcript of JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA...
![Page 1: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/1.jpg)
JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS presentation IN2P3 Computer Centerhttp://www.in2p3.fr/CC
Campus de la DOUA27, Boulevard du 11 Novembre 1918
69622 VILLEURBANNEFRANCE
![Page 2: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/2.jpg)
JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS : JBQS :
Graphical InterfaceGraphical Interface
for BQSfor BQS
![Page 3: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/3.jpg)
Slide No 3JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
BQS Summary
• Software component for batch management
• Allows– The submission of jobs;
– The scheduling and execution of jobs on workers;
– The monitoring of a job.
• Developed and used at the IN2P3 Computer Centre (France)In October 2000, BQS drives
– 900 workpoints on;
– 250 workers and;
– 10 classes.
![Page 4: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/4.jpg)
Slide No 4JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• Worker : A physical machine where jobs can run.
• Class : – Description of resources such as
Amount of CPU, memory, scratch, spool size, etc.
– Other characteristics such asprivileged access, AFS token information.
– Example :
Workers, Classes, Workpoints
225000 300000 256 512 1000 1000 5 50
R class exampleCPU (IU) Virt. Mem (MB) Scratch (MB) Spool (MB)
![Page 5: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/5.jpg)
Slide No 5JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• Workpoint : – Used to get a better control over job production;
– Defined as a set of ordered classes allocated to a worker;
– Runs only one job at a time, but
– Multiple workpoints on a worker.
– Example : workpoints definition for ccwasn01 (two processors SUN worker) [email protected] : T,S
[email protected] : G,C,A
[email protected] : I,R,J,P
[email protected] : S,T,I,R
Up to 4 jobs can run at the same time for the ccwasn01 worker.
– About 900 workpoints at the IN2P3 Computer Center.
Workers, Classes, Workpoints
![Page 6: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/6.jpg)
Slide No 6JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
JBQS design goals• A graphical interface for
– production manager;
– BQS administrator;
– on duty staff.
• Avoid knowledge of BQS Unix command syntax.
• Allows – Graphical query;
– Results display with• Editing abilities (finding, sorting, selecting);
• Interactions facilities on single or multiple selections.
![Page 7: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/7.jpg)
Slide No 7JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Structure of JBQS
• Batch management Production manager Available
• BQS administration BQS staff Available
• Batch status, On duty staff Under development
Resources optimisation
![Page 8: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/8.jpg)
Slide No 8JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Batch management : Description
• Batch management allows query & control over– jobs;– workers;– workpoints;– dispatching authorisation.
• Queries are made through various criteria from menus.
• Results interaction with– Popup menu allows actions on single or multiples selected cell.
– Editing abilities allows finding, extracting, sorting.
• Predefined options such as job type or frequently used command options can be set.
![Page 9: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/9.jpg)
Slide No 9JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Batch management : Diagram
Workers
Jobs Workpoints
Query by:•user•group•platform•worker•class•workpoint
Query by:•platform•name
Query by:•platform•worker•name
Switchingability
![Page 10: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/10.jpg)
Slide No 10JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• First example :– CPU limit modification on a running job.
• Second example– Draining a set of workers.
Screenshots examples
![Page 11: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/11.jpg)
Slide No 11JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Job query by account
Account choice
Job menu selection
![Page 12: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/12.jpg)
Slide No 12JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Job results
Job results panel
Text-area panel
![Page 13: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/13.jpg)
Slide No 13JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Action on a job
Popup menufor the selected cell
Job action choice panel
![Page 14: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/14.jpg)
Slide No 14JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Multiple workers drain action
Worker menu selection
Platform choice
![Page 15: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/15.jpg)
Slide No 15JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Worker results
worker results panel
Action status
![Page 16: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/16.jpg)
Slide No 16JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Multiple workers drain action
3) Worker action panel
1) Selection of multiple workers
2) Popup menu for selected cells
![Page 17: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/17.jpg)
Slide No 17JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
BQS administration description
• Allows modification on all BQS objects i.e. :Farm, platform, machine, workpoint, class and account.
• Objects can be created or deleted
• Parameters can be modified and then– applied to the object itself;
– propagated to other selected object.
![Page 18: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/18.jpg)
Slide No 18JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
• First example :– Default CPU modification on one class (G).
• Second example– Swap space modification on multiple workers.
Screenshots examples
![Page 19: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/19.jpg)
Slide No 19JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Default CPU modification on class G
New default AND max CPU values
(previous 30,000)
Apply new values(default and max CPU)
to the class G
![Page 20: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/20.jpg)
Slide No 20JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
New swap space for multiple workers
2) New Swap Space value(previous 4,000)
1) Selection of multiple workers
3) Button for (only) Swap Space modification
![Page 21: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/21.jpg)
Slide No 21JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Implementation features
• A Java standalone application with BQS calls made through an C API(uses of JNI *).
• Available on Solaris 2.7 and Linux RH 6.1.On any platform as soon as RMI * functionality is implemented.
• Multithreaded application for BQS calls,in order to get a better response time.
• Dynamically dialog boxes generation (thanks to Java).
• Multiple windows with menus.
• Text area panel with file Open/Save et Cut/Copy/Paste facilities.
* JNI : Java Native Interface. * RMI : Remote Method Invocation.
![Page 22: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/22.jpg)
Slide No 22JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Next implementation
• Editing functions for results– Finding,
– Sorting,
– Selecting
• RMI * functionality
* RMI : Remote Method Invocation.
![Page 23: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/23.jpg)
Slide No 23JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
Future development
• BQS status functionality– Worker status;
– Job running status (dead lock, infinity loops).
• Batch production flow (in order to optimise)– Estimated ending time of running and queued jobs;
– Workpoints and classes occupation rate.
• JDBC * implementation in JBQS
(After a data-base migration for BQS software)
* JDBC : Java Data Base Connectivity
![Page 24: JBQS - Bernard CHAMBON - HEPIX, Nov. 2000 JBQS presentation IN2P3 Computer Center Campus de la DOUA 27, Boulevard du 11 Novembre.](https://reader035.fdocuments.net/reader035/viewer/2022062518/56649f355503460f94c53b65/html5/thumbnails/24.jpg)
Slide No 24JBQS - Bernard CHAMBON - HEPIX, Nov. 2000
More information
• About BQS– http://www.in2p3.fr/ CC/interne/pub/bqs/index.html
– email : [email protected] (Yves FOUILHE)
[email protected] (Bernard CHAMBON)
• About CC-IN2P3– http://www.in2p3.fr/CC/
• About Java– http://www.javasoft.com