Scientific Computing Division File Transfers from/to SCD Supercomputers Siddhartha S Ghosh...
-
Upload
neal-jacobs -
Category
Documents
-
view
214 -
download
0
Transcript of Scientific Computing Division File Transfers from/to SCD Supercomputers Siddhartha S Ghosh...
Scientific Computing Division
File Transfers from/to
SCD Supercomputers
Siddhartha S Ghosh
Consulting Services Group
SCD/NCAR
Scientific Computing Division
Current Security PolicyExposed Network
UCARExposed
Hosts
gatekeeper
ServersOffices
Servers
MSS
Protected Network
RAS
VPNssh, scp, sftp
ftp, telnet, ssh, rcp/rlogin
ssh
Semi Exposed Network token + ssh
Supers
Data Analysis
Super Net
MSS
roy
token + ssh
sshscpsftp
Scientific Computing Division
Options
1. Invoke scp/sftp from Supers to remote Computers
2. Invoke ftp-proxy from Supers to remote Computers
3. Invoke scp from remote Computers to roy.ucar.edu
4. Invoke scp from Supers to salo-s.ucar.edu and a following scp from remote Computers to salo and vise-versa.
5. Stage into MSS and download using ftp/TLS to peewink.ucar.edu
Scientific Computing Division
Invoke scp/sftp from Supers to remote Computers• Most convenient, may be automated through scripts by
Installation of ssh-keys.• Secure
Ref: http://www.scd.ucar.edu/docs/ibm/scp.html
Caveats• Remote site must allow incoming ssh• Little computational overhead for encryption/decryption
Scientific Computing Division
Invoke ftp-proxy from Supers to remote Computers
• Issue ftp roy-s from Supers• At the proxy prompt give [email protected]• Follow usual ftp syntax• Convenient for anonymous remote ftp access
• Caveats - Remote site should allow incoming active mode ftp connection - Password for remote account is transmitted in clear - Few reported cases of failures particularly when remote ftp server is behind a firewall
Scientific Computing Division
Invoke scp from remote Computers to roy.ucar.edu
• Scp files from remote computers [email protected]:supercomputer-name• Periodically roy transfers those files tousername@supercomputer:/ptmp/username/file• Some ssh key and other setup is required, Ref:https://www.scd.ucar.edu/docs/access/internal/inbound.html
• Caveats: - Only works for inbound transfers
Scientific Computing Division
Invoke scp from Supers to salo-s.ucar.edu following scp from remote Computers to salo and vice versa.
• Works when remote site too is hardened
• Transfer to salo (3rd machine)
[email protected]:ptmp
• Pick it up from salo within 2 hrs.
(Refer to the last section of previous reference)
• Requires account in salo, please email [email protected].
Scientific Computing Division
Stage into MSS and download using ftp/TLS to peewink.ucar.edu
• Convenient, if it needs to be archived in MSS anyway• May be used for transferring files in both ways.
Ref: http://www.scd.ucar.edu/docs/mss/ftp.html
Caveats
- Installation of ftp/TLS client is required, (usually
straightforward in Linux)
- MSS holdings and transfers incur charges.
Scientific Computing Division
Installation of ssh-keys in roy• This is described in the web page referred to earlier.• Create key in your remote computer: ssh-keygen –t rsa press <return> at passphrase prompts• Grab the .pub key using mouse• type “key roy” when prompted• type “key supercomputer-name” e.g. “key bluesky”
and type passwd for bluesky when prompted.• Once completed you can do: automated file-transfers (method-1) You only need to give One Time Crypto passwd
while logging into the supercomputers.