Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy...
-
Upload
wyatt-toon -
Category
Documents
-
view
216 -
download
2
Transcript of Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy...
![Page 1: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/1.jpg)
Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,
Antonio Rosa
2012 IEEE High Performance Extreme Computing Conference
10 - 12 September 2012
Scalable Cryptographic Authentication for High
Performance Computing
This work is sponsored by the Department of the Air Force under Air Force contract FA8721-05-C-0002. Opinions, interpretations, conclusions and recommendations are those of the author and are not necessarily endorsed by the United States Government.
![Page 2: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/2.jpg)
HPEC 2012 - 2AJP 9/12/2012
• What is the LLGrid
• The Problem: External services authentication
• The Solution: Cryptographic authentication
• Results
Outline
![Page 3: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/3.jpg)
HPEC 2012 - 3AJP 9/12/2012
• LLGrid is a ~500 user ~2000 processor system
• World’s only desktop interactive supercomputer– Dramatically easier to use than any other supercomputer– Highest fraction of staff using (20%) supercomputing of any
organization on the planet
• Foundation of Supercomputing in Massachusetts
LLGrid System Architecture
LAN Switch
Network Storage
Resource Manager
ConfigurationServer
Compute NodesService Nodes Cluster Switch
To Lincoln LAN
Users
LLAN
![Page 4: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/4.jpg)
HPEC 2012 - 4AJP 9/12/2012
All jobs run on LLGrid
LLGrid Usage
1 10 100 1000
Tota
l Jo
b d
ura
tio
n (
seco
nd
s)
1
10
0
100
00
1M
Classic Supercomputing
Inte
ract
ive
Sup
erco
mpu
ting
Processors used by Job
TX-2500 (952 Cores)TX-X (220 Cores)TX-3d (540 Cores)
• Desktop Computing– CPU-time <20 minutes
• Classic Supercomputing– Wall-clock time >3 hours
• Interactive Supercomputing – Between desktop and classic
supercomputing– Shortens the “time to
insight”– Ten development turns/day
instead of one turn/week
• Desktop Computing– CPU-time <20 minutes
• Classic Supercomputing– Wall-clock time >3 hours
• Interactive Supercomputing – Between desktop and classic
supercomputing– Shortens the “time to
insight”– Ten development turns/day
instead of one turn/week
Des
ktop
Com
putin
g
![Page 5: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/5.jpg)
HPEC 2012 - 5AJP 9/12/2012
• What is the LLGrid
• The Problem: External services authentication
• The Solution: Cryptographic authentication
• Results
Outline
![Page 6: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/6.jpg)
HPEC 2012 - 6AJP 9/12/2012
• As the line between a shared supercomputer and a “really powerful personal computer” blurs, users expect to have access to network resources (storage, svn, cvs, etc).
Challenges withInteractive Supercomputing
Challenge: Users expect seamless access to other network resources from the HPC.
![Page 7: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/7.jpg)
HPEC 2012 - 7AJP 9/12/2012
• However these commands raise security concerns.– They store passwords as plain-text on the HPC central storage.– Password synchronization has made this password very sensitive.
Challenges withInteractive Supercomputing
Challenge: Ensure seamless access without putting the user’s “one common password” at risk.
“S3cr3t”
![Page 8: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/8.jpg)
HPEC 2012 - 8AJP 9/12/2012
• What is the LLGrid
• The Problem: External services authentication
• The Solution: Cryptographic authentication
• Results
Outline
![Page 9: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/9.jpg)
HPEC 2012 - 9AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
![Page 10: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/10.jpg)
HPEC 2012 - 10AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
![Page 11: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/11.jpg)
HPEC 2012 - 11AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
Authentication RequestA
![Page 12: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/12.jpg)
HPEC 2012 - 12AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
Authentication RequestA
A
![Page 13: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/13.jpg)
HPEC 2012 - 13AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
Authentication Request
Signed Authentication Responseand copy of PKI certificate
A
A
![Page 14: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/14.jpg)
HPEC 2012 - 14AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
Authentication Request
Signed Authentication Responseand copy of PKI certificate
A
A
A
![Page 15: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/15.jpg)
HPEC 2012 - 15AJP 9/12/2012
• Cryptographic authentication of clients using X509 PKI certificates has long been part of the SSL and TLS standards.
• The root of trust will certify that a specific keypair belongs to a specific user or process.
Cryptographic Authentication
User Server
Connection Request
Authentication Request
Signed Authentication Responseand copy of PKI certificate
A
A
A
Access Granted: Welcome Andy!
![Page 16: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/16.jpg)
HPEC 2012 - 16AJP 9/12/2012
• Cryptographic authentication depends on both the security of the user’s private key and access to it.– Storing the private key on central storage is little different than
storing a user’s password.
Challenges withCryptographic Authentication
Challenge: Where to store the private key?
![Page 17: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/17.jpg)
HPEC 2012 - 17AJP 9/12/2012
• Cryptographic authentication depends on both the security of the user’s private key and access to it.– Storing the private key on central storage is little different than
storing a user’s password.
Challenges withCryptographic Authentication
No guarantee the key won’t be lost, copied or left unprotected.
![Page 18: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/18.jpg)
HPEC 2012 - 18AJP 9/12/2012
• One traditional solution is to store the key on the client system and forward authentication requests back to the user’s system.– Could be on the client system or in a smart card.
Challenges withCryptographic Authentication
![Page 19: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/19.jpg)
HPEC 2012 - 19AJP 9/12/2012
Challenges withCryptographic Authentication
Forwarding requests back doesn’t work forsemi-interactive computing or background jobs.
Poof!
• One traditional solution is to store the key on the client system and forward authentication requests back to the user’s system.– However this fails if the user disconnects from the HPC.
![Page 20: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/20.jpg)
HPEC 2012 - 20AJP 9/12/2012
Challenges withCryptographic Authentication
Poof!
• Connecting smart cards to the HPC is not practical.– Some network-attached key storage devices exist, but their practical
benefit in this scenario is questionable.
![Page 21: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/21.jpg)
HPEC 2012 - 21AJP 9/12/2012
Challenges withCryptographic Authentication
Poof!
• We implemented a virtual smart card to run on each node.– Allows for keys to be used on any node, connected or disconnected.– Allows for different keys on each node.
![Page 22: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/22.jpg)
HPEC 2012 - 22AJP 9/12/2012
• Uses the smart card communication API: PKCS#11.
• Authenticates users and allows authorized users to perform cryptographic operations.
• Protects private keys from being copied, even by authorized users of the key.
• High throughput capability & low latency.– Physical smart cards have a latency approximately 800-900ms.
Virtual Smart Card Defined
![Page 23: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/23.jpg)
HPEC 2012 - 23AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.
The keyd Daemon: A Virtual Smartcard
Keyd
![Page 24: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/24.jpg)
HPEC 2012 - 24AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.– Has access to all the keys.
The keyd Daemon: A Virtual Smartcard
Keyd
![Page 25: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/25.jpg)
HPEC 2012 - 25AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.– Has access to all the keys.
• We then created a library that conformed to the PKCS#11 standard and could talk to this daemon.– Loaded by applications running as a HPC user.
The keyd Daemon: A Virtual Smartcard
Keyd
PKCS#11
![Page 26: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/26.jpg)
HPEC 2012 - 26AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.– Has access to all the keys.
• We then created a library that conformed to the PKCS#11 standard and could talk to this daemon.– Loaded by applications running as a HPC user.– Connects through a unix socket.– User credentials passed through the socket
Secure, provided you trust your linux kernel.
The keyd Daemon: A Virtual Smartcard
Keyd
PKCS#11
![Page 27: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/27.jpg)
HPEC 2012 - 27AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.– Has access to all the keys.
• We then created a library that conformed to the PKCS#11 standard and could talk to this daemon.– Loaded by applications running as a HPC user.– Connects through a unix socket.– User credentials passed through the socket
Secure, provided you trust your linux kernel.
• The SVN client can then load the PKCS#11 library and use the keys to authenticate to the SVN server.
The keyd Daemon: A Virtual Smartcard
Keyd
PKCS#11
![Page 28: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/28.jpg)
HPEC 2012 - 28AJP 9/12/2012
• We created the keyd daemon to be the brains of our virtual smartcard.– Runs as it’s own user account.– Has access to all the keys.
• We then created a library that conformed to the PKCS#11 standard and could talk to this daemon.– Loaded by applications running as a HPC user.– Connects through a unix socket.– User credentials passed through the socket
Secure, provided you trust your linux kernel.
• The SVN client can then load the PKCS#11 library and use the keys to authenticate to the SVN server.– Other applications can be enabled in the
future.
The keyd Daemon: A Virtual Smartcard
Keyd
PKCS#11
![Page 29: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/29.jpg)
HPEC 2012 - 29AJP 9/12/2012
• The SVN server was configured to accept the LLGrid’s root of trust.
• The SVN client on the LLGrid was configured to load the keyd daemon PKCS#11 library.– One configuration entry: ssl-pkcs11-provider=libkeyd_pkcs11
Configuring SVN for TLS Client Auth
SVN User SVN Server
Connection Request
Authentication Request
Signed Authentication Responseand copy of PKI certificate
A
A
A
Keyd Daemon
![Page 30: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/30.jpg)
HPEC 2012 - 30AJP 9/12/2012
• What is the LLGrid
• The Problem: External services authentication
• The Solution: Cryptographic authentication
• Results
Outline
![Page 31: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/31.jpg)
HPEC 2012 - 31AJP 9/12/2012
• Keypair generation and X509 PKI certificate creation is performed during user account creation.– LLGrid Adminstrators act as the root of trust.
• We developed scripts that execute parallel key generation across nodes in the cluster.
X509 PKI Certificate Enrollment
1 10 100 10000
50
100
150
200
250
300
350
400
450
500
Serial
Parallel
Nodes
Tim
e (s
econ
ds)
Keypair & Certificate Generation
– Each certificate asserts both the user identity and the node identity to meet the guidelines to be used for either server or client TLS authentication.
![Page 32: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/32.jpg)
HPEC 2012 - 32AJP 9/12/2012
• Created a general purpose key storage and certificate management solution for HPC.– Keys are not managed by the end-user, ensuring a low risk of
compromise requiring revocation.
• Demonstrated that it can be used to enable single sign-on integration to systems outside of the HPC.– Mitigated security concerns over passwords being stored on the
LLGrid central storage.– Avoided the issue of periodic password changes impacting batch
processing.
Results
![Page 33: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/33.jpg)
HPEC 2012 - 33AJP 9/12/2012
• Future work will look to use these PKI certificates to secure inter-node web services communication.– Certificates are valid for both TLS client or server authentication.
Future Work
![Page 34: Andrew Prout, William Arcand, David Bestor, Chansup Byun, Bill Bergeron, Matthew Hubbell, Jeremy Kepner, Peter Michaleas, Julie Mullen, Albert Reuther,](https://reader036.fdocuments.net/reader036/viewer/2022062515/56649cb15503460f949762d3/html5/thumbnails/34.jpg)
HPEC 2012 - 34AJP 9/12/2012
Questions?