Http://egee.cesnet.cz/ VO AUGER.ORG Preparation and First Applications Enabling Grids for E-sciencE...
1
http://egee.cesnet.cz/ VO AUGER.ORG Preparation and First Applications Enabling Grids for E-sciencE VO AUGER.ORG - Preparation and First Applications J. Chudoba 1,2 , L. Fiala 1,2 , J. Kmuníček 2,3 , J. Kosina 1,2 , T. Kouba 1,2 , D. Kouřil 2,3 , M. Lokajíček L. Matyska 2,3 , M. Ruda 2,3 , J. Schovancová 1 , Z. Sebastianová 2,3 , J. Sitera 2,4 , J. Švec 1,2 1) Institute of Physics of the Academy of Sciences of the Czech Republic 2) CESNET 3) Institute of Computer Science, Masaryk University, Brno 4) Centre for Information Technology, University of West Bohemia, Pilsen Poster presented at EGEE'06 International Conference, Septeber 25 - 29, 2006 in Geneva, Switzerland. Who are the Auger Collaborators? Pierre AUGER Collaboration consists of more than 300 scientists from 17 participating countries (highlighted in green on the map above): Argentina, Australia, Bolivia, Brazil, the Czech Republic, France, Germany, Italy, Mexico, Netherlands, Poland, Portugal, Slovenia, Spain, the United Kingdom, the United States, and Vietnam. Given the size and the geographical distribution of the collaboration, usage of the EGEE grid midleware may help in the organization of the access to the data and to add more computing resources for Monte Carlo simulations. In 2006 collaboration charged the Czech group with a creation and a setup of the VO. What is the Auger Observatory? Pierre Auger Cosmic Ray Observatory (http://www.auger.org) is studying the universe's highest energy particles (E>10 18 eV), which shower down on Earth in the form of cosmic rays. The Auger Observatory is a "hybrid detector", emplying two independent methods. The first detecting method uses 1600 water tanks that cover enormous section of the Pampa Amarilla (yellow prairie) in western Argentina (see the Pampa map below) and serve as a particle detector measuring Cherenkov light. The second detecting method tracks the development of air showers by observing ultraviolet light emitted high in the Earth's atmosphere. To this task will employ the GRID environment with VO AUGER.ORG. The Pampa map below shows, which parts are covered by functional tanks - roughly two thirds of area. Each tank separated from each of its neighbors by 1.5 kilometers, is completely dark inside - except when particles from a cosmic ray air shower pass through it. How to join? Registration form is available on the URL: https://auger-register.farm.particle.cz/auger/ Only members of the Pierre Auger collaboration can register. How to contribute with resources? Use your standard configuration tools and add auger.org to the list of supported virtual organizations. Here are pointers to ldap servers: groupldap://meta-ldap.cesnet.cz/ou=AUGER,o=AUGER,dc=eu- egee,dc=org .auger authldap://meta-ldap.cesnet.cz/ou=People,o=AUGER,dc=eu- egee,dc=org Current version of yaim does not yet support a dot in the name of the VO. Here is a description of our temporary patches: • add auger to yaim.conf as a new VO with these parameters: •VO_AUGER_SW_DIR=$VO_SW_DIR/auger •VO_AUGER_DEFAULT_SE=$SE_HOST •VO_AUGER_USERS=ldap://meta-ldap.cesnet.cz/ ou=AUGER,o=AUGER,dc=eu-egee,dc=org •VO_AUGER_QUEUES="auger“ •VO_AUGER_STORAGE_DIR=$CE_CLOSE_SE1_ACCESS_POINT/auger •add auger to the list of VO's in VOS="..." and list of queues in QUEUES="...“ Values above may differ on different sites. You may use a generic queue instead of a dedicated one. •add generic accounts augerXXX to the config_users •run configure_node yaim-all.def YOUR_ELEMENT, where YOUR_ELEMENT may be your type of CE, SE, UI, WN, ... •postconfiguration changes: •UI: rename the /opt/edg/etc/auger to /opt/edg/etc/auger.org and substitute 'auger' to 'auger.org' in /opt/edg/etc/auger.org/edg_wl_ui.conf •RB, WMS: no changes •CE: sed -i 's/AUGER_/AUGER_ORG_/g' /etc/profile.d/lcgenv*sh sed -i 's/\([^-^/]auger\)/\1.org/g' /opt/lcg/var/gip/ldif/* /opt/lcg/var/gip/* (we don't want the queue names and storage paths to be changed ) service globus-mds restart && service bdii restart •classicSE: sed -i 's/\([^:^/]auger\)/\1.org/g' /opt/lcg/var/gip/ldif/* /opt/lcg/var/gip/* (the path attribute in GlueSARoot should stay unchanged) service globus-mds restart && service gridice-mds restart •WN: sed -i 's/AUGER_/AUGER_ORG_/g' /etc/profile.d/lcgenv*sh These changes must be repeated after every run of configure_node. EGEE grid middleware and Air Showers Program CORSIKA (http://www-ik.fzk.de/corsika/) is often used for air showers simulations. We used a set of bash-scripts to submit CORSIKA jobs to the GRID environment. Every CORSIKA simulation needs a set of initial values data (IVD). We can put a set of IVD into one configuration file and the scripts take care of the rest - they submit several jobs to the GRID environment, each job corresponds to one CORSIKA simulation with different IVD set. We simply run script split_corsika.sh with input parameters <WorkingDirectory> <corsika.jdl> <Count Of Jobs> <Max Time For Job Checking in sec> <Interval Between Checks in sec> <Configuration File for CORSIKA> <[email protected]> <VO name>, e.g. ./split_corsika.sh `pwd` corsika.jdl 1 86400 1800 config_corsika [email protected] auger.org The example configuration file is shown below. # config_corsika EVENTS=10 ERANGE=1.0E+5 5.0E+5:5.1E+5 THETAP=0.0 0.1 0.3:0.35 Every record on ERANGE (M records in total) or THETAP (N records in total) line in the configuration file means new CORSIKA job. Simulation with the sharp value of energy or angle has a single value of energy or angle. Simulation with an interval of energy or angle has the lower bound and upper bound separated by a colon. Records are separated by a space. The number of simulations started from one configuration file is M x N. When all the jobs from corresponding configuration file are Done or Aborted, user is informed by an email. The output data files are placed onto a predefined Storage Element. User can download and further process the downloaded data at any time after the run of simulation stops. We realise that output data management is not user friendly for larger productions and we consider to test other systems like CHARON (http://egee.cesnet.cz/en/voce/Charon.html) to more automate job submissions and data management.
-
Upload
bertha-dennis -
Category
Documents
-
view
223 -
download
0
Transcript of Http://egee.cesnet.cz/ VO AUGER.ORG Preparation and First Applications Enabling Grids for E-sciencE...
http://egee.cesnet.cz/
VO AUGER.ORG Preparation and First Applications
Enabling Grids for E-sciencE
VO AUGER.ORG - Preparation and First ApplicationsJ. Chudoba1,2, L. Fiala1,2, J. Kmuníček2,3, J. Kosina1,2, T. Kouba1,2, D. Kouřil2,3, M. Lokajíček1,2,
L. Matyska2,3, M. Ruda2,3, J. Schovancová1, Z. Sebastianová2,3, J. Sitera2,4, J. Švec1,2
1) Institute of Physics of the Academy of Sciences of the Czech Republic2) CESNET3) Institute of Computer Science, Masaryk University, Brno4) Centre for Information Technology, University of West Bohemia, Pilsen
Poster presented at EGEE'06 International Conference, Septeber 25 - 29, 2006 in Geneva, Switzerland.
Who are the Auger Collaborators?Pierre AUGER Collaboration consists of more than 300 scientists from 17 participating countries (highlighted in green on the map above): Argentina, Australia, Bolivia, Brazil, the Czech Republic, France, Germany, Italy, Mexico, Netherlands, Poland, Portugal, Slovenia, Spain, the United Kingdom, the United States, and Vietnam.
Given the size and the geographical distribution of the collaboration, usage of the EGEE grid midleware may help in the organization of the access to the data and to add more computing resources for Monte Carlo simulations. In 2006 collaboration charged the Czech group with a creation and a setup of the VO.
What is the Auger Observatory?
Pierre Auger Cosmic Ray Observatory (http://www.auger.org) is studying the universe's highest energy particles (E>1018 eV), which shower down on Earth in the form of cosmic rays. The Auger Observatory is a "hybrid detector", emplying two independent methods. The first detecting method uses 1600 water tanks that cover enormous section of the Pampa Amarilla (yellow prairie) in western Argentina (see the Pampa map below) and serve as a particle detector measuring Cherenkov light. The second detecting method tracks the development of air showers by observing ultraviolet light emitted high in the Earth's atmosphere. To this task will employ the GRID environment with VO AUGER.ORG.
The Pampa map below shows, which parts are covered by functional tanks - roughly two thirds of area. Each tank separated from each of its neighbors by 1.5 kilometers, is completely dark inside - except when particles from a cosmic ray air shower pass through it.
How to join?Registration form is available on the URL:
https://auger-register.farm.particle.cz/auger/Only members of the Pierre Auger collaboration can register.
How to contribute with resources?Use your standard configuration tools and add auger.org to the list of supported virtual organizations.Here are pointers to ldap servers:
groupldap://meta-ldap.cesnet.cz/ou=AUGER,o=AUGER,dc=eu-egee,dc=org .augerauthldap://meta-ldap.cesnet.cz/ou=People,o=AUGER,dc=eu-egee,dc=org
Current version of yaim does not yet support a dot in the name of the VO. Here is a description of our temporary patches:
• add auger to yaim.conf as a new VO with these parameters:•VO_AUGER_SW_DIR=$VO_SW_DIR/auger•VO_AUGER_DEFAULT_SE=$SE_HOST•VO_AUGER_USERS=ldap://meta-ldap.cesnet.cz/ou=AUGER,o=AUGER,dc=eu-egee,dc=org•VO_AUGER_QUEUES="auger“•VO_AUGER_STORAGE_DIR=$CE_CLOSE_SE1_ACCESS_POINT/auger•add auger to the list of VO's in VOS="..." and list of queues in QUEUES="...“
Values above may differ on different sites. You may use a generic queue instead of a dedicated one.•add generic accounts augerXXX to the config_users•run configure_node yaim-all.def YOUR_ELEMENT, where YOUR_ELEMENT may be your type of CE, SE, UI, WN, ... •postconfiguration changes:
•UI: rename the /opt/edg/etc/auger to /opt/edg/etc/auger.org and substitute 'auger' to 'auger.org' in /opt/edg/etc/auger.org/edg_wl_ui.conf•RB, WMS: no changes •CE: sed -i 's/AUGER_/AUGER_ORG_/g' /etc/profile.d/lcgenv*sh sed -i 's/\([^-^/]auger\)/\1.org/g' /opt/lcg/var/gip/ldif/* /opt/lcg/var/gip/* (we don't want the queue names and storage paths to be changed)service globus-mds restart && service bdii restart•classicSE: sed -i 's/\([^:^/]auger\)/\1.org/g' /opt/lcg/var/gip/ldif/* /opt/lcg/var/gip/* (the path attribute in GlueSARoot should stay unchanged)service globus-mds restart && service gridice-mds restart •WN: sed -i 's/AUGER_/AUGER_ORG_/g' /etc/profile.d/lcgenv*sh
These changes must be repeated after every run of configure_node.
Pictures and description of the experiment were provided by the Auger Collaboration.
EGEE grid middleware and Air ShowersProgram CORSIKA (http://www-ik.fzk.de/corsika/) is often used for air showers simulations. We used a set of bash-scripts to submit CORSIKA jobs to the GRID environment. Every CORSIKA simulation needs a set of initial values data (IVD). We can put a set of IVD into one configuration file and the scripts take care of the rest - they submit several jobs to the GRID environment, each job corresponds to one CORSIKA simulation with different IVD set. We simply run script split_corsika.sh with input parameters <WorkingDirectory> <corsika.jdl> <Count Of Jobs> <Max Time For Job Checking in sec> <Interval Between Checks in sec> <Configuration File for CORSIKA> <[email protected]> <VO name>, e.g. ./split_corsika.sh `pwd` corsika.jdl 1 86400 1800 config_corsika [email protected] auger.org The example configuration file is shown below.
# config_corsikaEVENTS=10ERANGE=1.0E+5 5.0E+5:5.1E+5THETAP=0.0 0.1 0.3:0.35
Every record on ERANGE (M records in total) or THETAP (N records in total) line in the configuration file means new CORSIKA job. Simulation with the sharp value of energy or angle has a single value of energy or angle. Simulation with an interval of energy or angle has the lower bound and upper bound separated by a colon. Records are separated by a space. The number of simulations started from one configuration file is M x N.
When all the jobs from corresponding configuration file are Done or Aborted, user is informed by an email. The output data files are placed onto a predefined Storage Element. User can download and further process the downloaded data at any time after the run of simulation stops.
We realise that output data management is not user friendly for larger productions and we consider to test other systems like CHARON (http://egee.cesnet.cz/en/voce/Charon.html) to more automate job submissions and data management.
