Introduction to remote command line LinuxResearchComputingTeamUniversityofBirmingham

Linux/UNIX/BSD/OSX/what?

v Alldifferentv UNIXistheoldest,mostlynowcommercialonlyinlargeenvironments(e.g.bankingmainframe)

v Practicallyfromauserperspective,thesamev Focusoncommandlinetools

Distributions of Linux

v Groupsofsoftwareatspecificversionsv Lotsofthem:

– RedHat– Fedora– Suse– Debian– Ubuntu– Mint…..

Remote graphical applications

v UoB onlyprovideBlueBEARv Nographicaldesktop(Gnome/KDE/Unity)v Remoteconnectionviassh (PuTTy)v Xserverrunningforgraphicalapplications

– Exceed(UoB desktops)– Xming (free)– Xquartz (OSX)

Practical

v http://www.birmingham.ac.uk/bear– Thenfollowtheselinksinorder:

v Servicesv LinuxCompute(BlueBEAR)vAccess&FileTransfervWindowssoftwaretoaccessBlueBEARv “thehelppageforinstallingandconfiguringPuTTYandExceed”(lastparagraphofthe“WindowssoftwaretoaccessBlueBEAR”section)

orv http://tinyurl.com/bear0915

Graphical text editingv Fromthecommandpromptrun‘gedit’togetagraphicaltexteditorrunningontheremotesystem

v IntheXserveryouhavetwocopybuffers– Copyandpaste,asnormal– ‘Middleclick’buffer

Exceed … (useful tip)

v Selecttexttoplaceitintothecopybufferv Middleclicktopastetext

A word about directory layoutv UnlikeWindows(andDOSsystems)thereareno“drives”(i.e.no

‘C:\’,‘D:\’,‘U:\’etc.)v Everythingexistsunderadirectory,‘/’,calledtherootdirectory.v Usefuldirectoriestoknow:

– ‘/home’– eachuserusuallyhasadirectoryherewiththeirownfilesinit,knownastheir‘homedirectory’.v OnBlueBEAR users’homedirectoriesarelocatedat‘/gpfs/bb’,not‘/home’,fortechnicalreasons,but‘/home’stillexists.

– ‘/tmp’– temporaryfiles.Anyusercancreatefilesinthisdirectory.Oftendeletedwhenthecomputerreboots.v OnBlueBEAR thereisalso‘/scratch’oneachcomputenodewhichisforthesamepurposebutmuchlarger.

What is a path?

v Aconcretestriptothebottomofthegardenv Wherethesystemlookstofindprograms

echo $PATH

– Command not found – theprogramisn’tinyourpath

Welcome to the command line

v FilesandprogramsareCaSe-SeNsItIvEv Youtypecommandsintoaprogramcalledthe“shell”v Commandsarecase-sensitivev The‘tab’keyismagic,pressingit:

– once:completethecurrentcommandorfilenameifthereis1match

– twice:listallthematchingoptionsifthereismorethan1match

Command line syntax

v Variesfromprogramtoprogramv Typicallyoftheform

tar -cpf foo.tar ~/to_archive/

<command> -<character> <argument> <file>

tar -cpf foo.tar ~/to_archive/

Help? How do I use this command?v Eachcommandshould haveamanualpageforit.– Toviewittypeman command,e.g.toviewthemanualforthemancommandtype:man man

– Pressqtoquit,usethearrowkeystoscroll

Looking at your filesv Basicfilecommands:

ls – listfiles

cd – changedirectory

pwd – printcurrent(working)directory

Looking at your files (continued)v Basicfilecommands:

less filename – view the contents of your (text) files (press q to quit), space to page down, up/down arrows

Looking at your files (continued)v Basicfilecommands:

tail -f filename – watchtheendoffileasitgrows

Organising your filesv Basicdirectorymanagementcommands:

mkdir newdirectory – createdirectory

rmdir directory – removedirectory(onlyworksifdirectoryisempty)

Organising your filesv Basicfilemanagementcommands:

touch file –createanemptyfile

rm file – removeafile(thereisnorecyclebin!)

cp file newfile – copyafile(createsaduplicate)

mv file newfile – moveafile(renamesthefile)

dot files and directories

v Userconfig settingsstoredinthemv Pleasedon’tdelete!v Usefulonestoedit(tosetthingseverytimeyoulogin):– .bash_profile– .bashrc– BUTBECAREFUL!

Linksv Therearetwotypesoflinksonunix systems:

– symboliclinks– knownas“symlinks”,likea‘shortcut’tothefile

– hardlinks– anewreferencetothesamefile(youareveryunlikelytoeverusethese)

v Creatingasymlink:– Usetheln linkcommandwiththe-s,symlink,option.E.g.tocreatealinkto/gpfs/bb/username/myfile called/tmp/link_to_myfiletype:ln –s /gpfs/bb/username/myfile/tmp/link_to_myfile

Users and groups

v UsernamemapstoanumericUID– Commandid returnsdetails

v Memberofleastonegroupv GroupnamesmaptoanumericGID

– Commandgroups listsyourgroupsgroups username

User permissions

v Bydefaultusershavenospecialpermissionv rootisequivalentofadministratorv Commandsu - username wouldchangetothatuserifyouknowthepassword

v Modernsystemsusesudo todocommandswithelevatedpermission(orasanotheruser)

v UoB centralsystemsdon’tallowsudo orsu tonormalusers

File Permissions

v Files(anddirectories)canhavedifferentpermissionsetsforgroupsandusers

Files Directory

r– readthefile r– listcontentsofdirectory

w– writetothefile w– createnewfiles/folders

x– execute(run) x– traverse(e.g.cdtodirectory)

File Permissions

v Trydoingls –ld ~/

v Thisshowspermissionsonhomedirectoryv Firstcharacterindicatesfile(-),directory(d)orlink(l),next3charactersisuser,next3group,final3isallotherusers

v +signindicatesextendedACLsapplied

File permissions

mkdir testdirtouchtestdir/testfile

chmod isusedtochangepermissionschmod g+rwx testdir(thencheckwithls -ld testdir)chmod o+x testdir/testfile

File permissions

v Aswellasusingrwx etc,bitmaskscanalsobeused,thefollowingareequivalent:chmod 755 directorychmod u+rwx,go+rx directory

Shell variablesv Lotssetatlogin

envv Cansetown

export FOO=barecho $FOO– Valueof$FOOisnowavailabletoother“things”inshell

v ~/ - shortcuttohomedirectoryv ~someuser/ - shortcuttoanotheruserhome

directory

Some useful toolsv grep stringfilename

– Lookforstringoccurrencesinthefilegrep ldap /etc/nsswitch.conf

– Whatdoesgrep –c ldap /etc/nsswitch.conf

do?

Some useful toolsv diff file1 file2

– Showsthedifferencesbetweentwofilesdiff /etc/nsswitch.conf

~/nsswitch.conf

Editing files

v CaneditfromWindowsanduseshare– Windowsfilesusedifferentlineendings

dos2unix ~/filename

v LotsofeditorsunderLinux– nano isabasic,easytouseone– Others– joe,vim,emacs

Editing filesv Createandeditanewfileinhomedirectory:

nano ~/testfile.txt

– Typeinsometext– SavethefilewithCTRL+o– ExitwithCTRL+x– Checkthecontentsofthefile(Remembercatorless?)– Editthefileagain,tryCTRL+k,CTRL+u,CTRL+w toseewhattheydo

Pipes and redirection

v |(pipe)canbeusedtosendoutputofcommandintoanotherls –al /gpfs/bb | grep $USER

v >redirectstheoutputofacommand– E.g.toafile– STDOUT– STDERR– 2>&1

Pipes and redirection

v Trythese:ls –al /gpfs/bb | grep $USERls –al ~/ > ~/ls.outputCdgffgffg > ~/error.outputCdgffgffg > ~/error.output 2>&1

Scripts

v Allowseriesofcommandstoberepeatedv Canpassargumentsin,usevariablesetc.v Needtobe“executable”torunfromcommandline

v Firstlineshowsthe“interpreter”(orshell)touse,e.g.#!/bin/bash#!/usr/bin/perl

Scriptsv Usethetexteditortocreateanewscriptfileto:

– Createadirectory(mkdir)– Changetoadirectory(cd)– Listthedirectorycontentswithalldetails(ls)– Createanewemptyfile(touch)– Listthedirectorycontentswithalldetails(ls)

v Remembertosettheshellinthefirstline#!/bin/bash

v Makeitexecutable(chmod)v Runthescript

./SCRIPTNAME– (Youcouldalsotryredirectingtheoutputtoafile)

Scripts

#!/bin/bash

mkdir ~/tempdircd ~/tempdirls -altouch somefilels -al

Archiving and compressing

v Atar file(traditionallytapearchive)isawayofgroupingasetoffilestogetherintoasinglefile

v gzip orbzip2 areoftenusedtocompressfilesv Soforexamplea.tar.gz filewouldbesimilartoazipfile

Archiving and compressingv Createanewtarfile,beingverboseofthefiles,preserving

filepermissions:tar -cvpf ~/newfile.tar ~/testdir

v Compressthefile:gzip -9 ~/newfile.tar

v Listcontentsofcompressedfile:tar -tzf ~/newfile.tar.gz

v Extractcontentstoanewdirectory:mkdir ~/newdircd ~/newdirtar -xvzpf ~/newfile.tar.gz

Process management

v Everythingyouexecutecausesatleast1processtobegenerated

v Programswhich“fork”maycreatemultiplesub-processes

v Thecommandtop showstheprocessesrunningusingmostCPU(qtoexit)

Process managementv Processesmaybeinseveraldifferentstates:

– Runninginforeground– Suspended(stopped,nofurtherprocessinghappens)

– Backgrounded (runningbutnolongerattachedtoterminal)

– Zombie(Inabadway,typicallynolongerdoinganythinguseful)

Process management

v CommandlineprocessescanbestoppedusingCTRL+z

v Runningbg willthenbackgroundtheprocessv Jobslistsprocessesfromthecurrentterminalandtheirstate

v fg returnsasuspendedorbackgroundprocesstothecontrollingshell

Process Priority

v EachprocesshasaprioritywhichdetermineshowmuchCPUtimewillbeallocatedwhentheCPUisbusywithmanyapplications

v renice canbeusedtochangethepriorityofaprocess(userscanonlyreduce)

v Onsharedsystems,itis“sociable”toreniceheavycomputeprocesseswhichyouareleavingrunning

Looking for processes

v Thecommandps willlistyourrunningprocessesandtheirprocessID

v ps -ef | grep vi– Wouldlookforprocessescalledvi

Signals

v Signalsareusedtotellprocessestodosomething,typically:– Totellaprocesstorereadaconfig (HUP)– Totellaprocesstoterminate(TERM)– Toforcefullykillaprocess(KILL)– HUPandTERMrelyonsignalhandlerincode

kill -SIGNAL processID

Any questions?