GACRC User Quick Guide...All Sapelo and zcluster users, as well as all GACRC PIs, can access the...
Transcript of GACRC User Quick Guide...All Sapelo and zcluster users, as well as all GACRC PIs, can access the...
GACRCUserQuickGuide:
MigratingfromZclustertoSapelo
8/4/17 1
TheGACRCStaffVersion1.0
DiscussionPoints
I. RequestSapeloUserAccount
II. Systems
III. TransferFiles
IV. ConfigureSoftwareEnvironment
V. JobSubmissionWorkflow
VI. JobSubmissionScript
VII. Appendices
8/4/17 2
Sapeloclusteruseraccount:[email protected]
Note:AvalidofficialUGAMyIDisaMUSTtocreateauseraccount!
38/4/17
RequestSapeloUserAccount
PIRequestUserAccount
PIVerification
NewUserTraining
UserAccount
Provisioning
WelcomeLetter
1. TheUGAPIusestheGACRConlineformhttp://help.gacrc.uga.edu/userAcct.php to
requestauseraccountforagroupmember.
2. Oncewereceivedtherequest,wewillverifyitwiththePI.
3. AfterverificationbythePI,thenewuserwillberequiredtoattendatrainingsession.
4. Aftertheuserattendedtraining,wewillprovisionaSapeloaccountfortheuser.
5. Awelcomeletterissenttotheuseronceuseraccountisready.
Systems– OverviewDiagram
TransferNodexfer.gacrc.uga.edu
LoginNodezcluster.rcc.uga.edu
LoginNodesapelo1.gacrc.uga.edu
InteractiveNode
InteractiveNode/home/abclab/userID /escratch4/userID
/home/MyID /lustre1/MyID
/project/abclab
ssh
ssh
ssh/scp
qlogin
qlogin
Zcluster
Sapelo
AccessingLine
8/4/17 4
Systems– Overview
LogontoZclusterLogin Node: ‘ssh [email protected]’(usingzclusterpassword)
OpenZclusterInteractiveNode: type‘qlogin’commandfromloginnode
ZclusterHomeDirectory: /home/abclab/userID
ZclusterGlobalScratch: /escratch4/userID (4TBquota)
Create WorkingDirectoryonZclusterGlobalScratch: type‘make_escratch’commandfromloginnode
Log onto SapeloLoginNode: ‘[email protected]’(usingUGAMyIDpassword)
OpenSapeloInteractiveNode: type‘qlogin’ commandfromloginnode
SapeloHomeDirectory: /home/MyID(100GBquota)
SapeloGlobalScratch: /lustre1/MyID
Create WorkingDirectoryonSapeloGlobalScratch: use‘mkdir’commandin/lustre1/MyIDfrominteractivenode
LogontoGACRCTransferNode: ‘ssh [email protected]’ (usingUGAMyIDpassword)
GroupProjectSpace: /project/abclab(“abc”areinitialsofthePI ofthegroup)
8/4/17 5
Systems– StorageEnvironmenthttps://wiki.gacrc.uga.edu/wiki/Disk_Storage
Name Role Purpose Zcluster Sapelo
Home sshlandingspot staticdata, e.g.,scripts,software /home/abclab/userID /home/MyID
(100GBquota)
GlobalScratch
Jobworkingspace
Current Jobdatabeingused byrunningjobs
/escratch4/userID(4TBquota)
/lustre1/MyID(noquota–limitedbycapacity)
Project Activedatastorage
Non-currentactivedataforfutureanalysis
/project/abclab(1TB initialquota)
8/4/17 6
Systems– ComputingResourcesOverviewhttps://wiki.gacrc.uga.edu/wiki/Systems
8/4/17 7
Zcluster
150x8-core16GBRAMnodeswithIntelXeon processors(total1200cores)45x12-core48GBRAMnodeswithIntelXeon processors(total540cores)
4x8-core192GBRAMnodeswithIntelXeon processors(total32cores)10x12-core256GBRAMnodeswithIntelXeon processors(total120cores)2x32-core512GBRAMnodeswithIntelXeon processors(total64cores)
6x32-core64GBRAMnodeswithAMDOpteronprocessors(total192cores)
4x12-core96GBRAMnodeswithIntelXeonprocessors, 8NVIDIAK20XmGPUcardseach
Sapelo
112x48-core128GBRAMnodeswithAMDOpteronprocessors(total5376cores)4x48-core256GBRAMnodeswithAMDOpteronprocessors(total192cores)
6x48-core512GBRAMnodeswithAMDOpteronprocessors(total288cores)1x48-core1TBRAMnodewithAMDOpteronprocessors (total48cores)3x 28-core1TBRAMnodewithIntelXeon processors(total84cores)
2x16-core128GBRAMnodeswithIntelXeonprocessors, 8NVIDIAK40mGPUcardseach
85+buy-innodeswithvariableconfigurations
TransferofFileshttps://wiki.gacrc.uga.edu/wiki/Transferring_Files
8/4/17 8
AllSapeloandzclusterusers,aswellasallGACRCPIs,canaccesstheGACRCtransfernodes(xfer.gacrc.uga.edu)andusingUGAMyID andMyIDpasswordtoauthenticate.
Notethatauser'shomedirectoryontransfernodeisthesameastheuser'sSapelohomedirectory,whichisnotthesameastheuser'sZclusterhomedirectory.
Transfernodescanaccessthefollowingfilesystemsusingtheirfullpaths:
SapeloHomeDirectory: cd/home/MyID
ZclusterHomeDirectory: cd/panfs/pstor.storage/home/abclab/userID
ZclusterGlobalScratch: cd/escratch4/userID
SapeloGlobalScratch: cd/lustre1/MyID
GroupProject Folder cd/project/abclab
Detailedinstructionscanbefoundatthelinkabove
local
FileTransferringTreehttps://wiki.gacrc.uga.edu/wiki/Transferring_Files
8/4/17 9
scp/ssh
TransferNode
/home/MyID
/lustre1/MyID
/project/abclab
/escratch4/userID
/panfs/pstor.storage/home/abclab/userID
ü scp:Remotefiletransferringbetweenyourlocalcomputerandyourfoldersonfilesystems
ü ssh:Loggingontotransfernode.Onceloggedon,youcanusecp ormv forfiletransferbetweenfilesystems.
ConfigureSoftwareEnvironmenthttps://wiki.gacrc.uga.edu/wiki/Software
8/4/17 10
SapeloUser issuesmodule commandstoconfigurevariouspathsfor softwarepackages
(inajobsubmissionscriptoronthecommandlineofaninteractivesession):
1. moduleavail: showallmodulesavailableoncluster2. moduleloadmoduleName/version:loadtheneededmodule“moduleName”withaversion3. modulelist:listallmodulesyoualreadyloaded
Forexample:
$moduleloadpython/2.7.8$moduleloadR/3.2.3$modulelistCurrentlyLoadedModules:1) moab/8.1.1 2)StdEnv 3)python/2.7.8 4) R/3.2.3
ZclusterUser “manually”exportsvariouspathsforsoftwarepackages(inajobsubmissionscriptoronthecommandlineofaninteractivesession),forexample:
export PATH=……:$PATH
JobSubmissionWorkflow(general)
8/4/17 11
Zcluster Sapelo
1. Logontozclusterloginnode:
2. Createaworkingdir onglobalscratch:
make_escratch
3. Changetoworkingdironglobalscratch:
cd/escratch4/userID/userID_mmm_dd
(mmm, dd:dateabbreviations)
4. Create azclusterjobsubmissionscript:
nano./sub.sh
5. Submitjob:
qsub-qrcc-30d./sub.sh
1. LogontoSapelologinnode:
2. GotoSapelo Interactivenode:
qlogin
3. ChangetoSapeloglobalscratch:
cd/lustre1/MyID
4. Createaworkingdironglobalscratch:
mkdir./workDir
5. Changetoworkingdironglobalscratch:
cd./workDir
6. CreateaSapelojobsubmissionscript:
nano./sub.sh
7. Submitjob:
qsub./sub.sh
JobSubmissionScript(NCBIBlastasexample)
8/4/17 12
sub.shonZcluster sub.shonSapelo
#!/bin/bash
cd`pwd`
exportPATH=/usr/local/ncbiblast/latest/bin:$PATH
blastall-pblastn-d/db/ncbiblast/latest/nt \-a4 -iQUERY\-oout.QUERY
#PBS-S/bin/bash#PBS-qbatch#PBS-NtestBlast#PBS-lnodes=1:ppn=4:AMD#PBS-lmem=20gb#PBS-lwalltime=480:00:00
cd$PBS_O_WORKDIR
moduleloadncbiblast+/2.2.26
blastall-pblastn-d/db/ncbiblast/nrte/latest/nt \-a4 -iQUERY\-oout.QUERY>outputfile2>&1
qsub-qrcc-30d-pethread4./sub.sh qsub./sub.sh
138/4/17
Appendix1:
148/4/17
Tasks Zcluster Sapelo
Commonfile/diroperations
ls,cd, pwd,file,cat, less,more,cpmv,mkdir,rm,rmdir,dos2unix,mac2unix,etc.
Transfer remotefiles scp, sftporSSHSecureClientFileTransfer,FileZilla,WinSCP
Openinteractivesession qlogin
Createworkingdirectoryonglobalscratch make_escratch mkdirsubdirectoryName
Edit scriptfiles nanosub.sh or vim sub.sh
Submitabatchjob qsub–qqueueName–l…–pe …sub.sh qsubsub.sh
checkjobstatus qstat
Checkdetailedjobinfo qsj jobID,qstat–jjobID,qacct–jjobID qstat–fjobID,showjobsjobID
Cancelajobfromqueue qdeljobID, qdel–uuserID qdeljobID
Appendix2:CommonCommandSummaryhttps://wiki.gacrc.uga.edu/wiki/Command_List
158/4/17
Appendix3:CurrentprocessforaUGAPItorequest(re)activationorcreationofaMyIDforanexternalcollaborator
Tokeepauser'sMyIDactiveafter he/sheleavesUGA,ortocreateaMyIDforanexternalresearchcollaborator:
1. ThePIaccessestheEITShelpdeskportalathttps://uga.teamdynamix.com/TDClient/Home/
2. Click on"MyIDAccountRequest”ontherightsideofthepagein“PopularServices”section.
3. UseMyID credentialstologontheservicepage.4. IntheMyIDRequestFrom,enterhis/herinformationandselect"GACRC
RemoteCollaborator"fortheAccountTypefield.5. AfterverificationbytheGACRCDirector,EITSwillgeneratecollaborator’sMyID6. IfcollaboratorisvisitingUGAcampus,MyID canbeobtainedthroughOVPR’s
VisitingResearcher&Scholarprogram– https://research.uga.edu/visiting/
• GACRCWebsite:http://gacrc.uga.edu
• GACRCWiki:https://wiki.gacrc.uga.edu/wiki/Main_Page
• Systems: https://wiki.gacrc.uga.edu/wiki/Systems
• HowtoTransferFiles:https://wiki.gacrc.uga.edu/wiki/Transferring_Files
• CurrentSoftwareList: https://wiki.gacrc.uga.edu/wiki/Software
• HowtoSubmitaJob:https://wiki.gacrc.uga.edu/wiki/Running_Jobs_on_Sapelo
• HowtoGetHelp: http://help.gacrc.uga.edu/
• GACRCTrainingOpportunities:https://wiki.gacrc.uga.edu/wiki/Training
• HowtoregisteranewGACRCNewLab:http://help.gacrc.uga.edu/labAcct.php
• HowtoregisteranewGACRCUser:http://help.gacrc.uga.edu/userAcct.php
8/4/17 16
Appendix4:UsefulLinks