The SSL Problem & Deploying SHA2 Certificates

Gabriella DavisThe Turtle Partnership

Mark MyersLDC Via

T1-1086Monday 1st February 2016

Who Are We?• Adminofallthingsandespeciallyquitecomplicatedthingswherethefunis

• Workingwithsecurity,healthchecks,singlesignon,designanddeploymentofDomino,ST,Connectionsandthingsthattheytalkto

• Stubbornandrelentlessproblemsolver

• LivesinLondonabouthalfofthetime• gabriella@turtlepartnership.com• twitter:gabturtle

Who Are We?• Member of the London Developer Co-op

▪ Co-writer of LDCViahttp://www.ldcvia.com

▪ IBM Connections, Domino, Mobile and Web development

▪ Hire me!

• Developer from a support background

• 14+ years on Domino, 17+ years in IT

• Speaker at 6x Lotuspheres/Connects, 6 x UKLUGs, 1 x ILUG, 3 x BLUG/Engage

Why This Session?• Encrypting and securing information is not just a thing you add to

complete a project, it’s a necessity• Information can be intercepted en route between client and

server or between servers• This can include credentials which can they expose further

information• The speed at which hackers are working around encryption

standards is growing, major attacks and vulnerabilities appear every week

• You need to understand where the vulnerabilities are, how to watch for them and how to protect against them

Encryption and Certificates• What is Encryption?

▪ It is a process of making data unrecognizable • Unless you have “key” to unlocking the data.• Without the key, it should be imposable or more commonly

unfeasible to read the data in a reasonable timeframe. • What are Certificates?

▪ Digital Certificates are a way of trying to prove that the security “key” they contain actually belong to the person they were issued to.

▪ This is done via a trusted third party that both parties in communication can rely on.

Let’s Talk Acronyms

SSL• Stands for Secure Socket Layer • A cryptographic protocol (A set of agreed rules for coding and

decoding messages so as to keep those messages secure)• Each version was replaced by another version due to security

flaws and now is completely deprecated ( June 2015 by RFC 7568)

• Its death knell was the block cipher attack used by Poodle (see later slide) in 2014

• Replaced by...

TLS• “Transport Layer Security”• Like SSL it is cryptographic protocol (A set of agreed rules for

coding and decoding messages so as to keep those messages secure)

• The successor to SSL (TLS 1.0 is actually SSL 3.1 but was renamed to mark the change to an open standard rather than Netscape's protocol)

• Currently has 3 versions 1.0, 1.1, 1.2 (1.3 in Draft)• Like SSL it is a constantly changing protocol

Man in the Middle Attack• An attack where someone intercepts communication between

two systems and acts as a proxy between the parties without either of them knowing

S/MIME• This is another protocol, this time for allowing email in the MIME

format (basically all SMTP mail) to be both signed or/and encrypted

▪ Signed: To ensure to your email recipients that you actually sent the email

▪ Encrypted: To protect the content from being read by other entities than the intended recipients.

• Just about the only intersystem security standard that all vendors can agree on.

HTTPS• This is the transfer of data using the Hypertext Transfer Protocol

over a link secured be either SSL or TLS• Provides:

▪ Bidirectional encryption of your data in transit▪ A reasonable guarantee that you are talking to who you

think you are.▪ Defends against "Man in the Middle" and third party

snooping attacks.

SHA2• An upgrade to the popular hashing algorithm used by the

majority of SSL certificates• Its predecessor SHA1 was found to be more insecure that was

previously thought (not broken just not as secure)• Microsoft Google and Mozilla all announced deprecation plans

for SHA1• When people talk about "SHA256" they are talking about one of

the 6 hash functions with digests that make up the SHA2 family

AES• “Advanced Encryption Standard”• Based on 3 members of the Rijndael cipher family (developed

by Joan Daemen and Vincent Rijmen) each with a block size of 128 bits, but three different key lengths: 128, 192 and 256 bits.

• Used worldwide and a federal government standard• It supersedes DES (Data Encryption Standard) which you can

now brute force attack

What Went Wrong… and keeps going wrong

POODLE • "Padding Oracle On Downgraded Legacy Encryption“• An exploit that allowed attackers to trick a session to use SSL

rather than TLS then during that session use a design flaw in SSL 3.0 to Snoop on the session

• What it did▪ It allowed attackers to perform a man in the middle attack

• How it was stopped▪ We all turned off SSL V3.0 on the servers (and were then

grumbled at by people who used old browsers)

ShellShock or Bashdoor• An Bug from the original version of Bash* allowed you to launch

child instances of Bash but supply your own variables• What it did

▪ It allowed an attacker to execute bash commands on the target server

• How it was stopped▪ Patched all servers running Bash

*A program that all lot of Unix-based systems use to execute command lines and command scripts

Heartbleed• A "buffer over-read" vulnerability in the TLS heartbeat extension of

OpenSSL caused by a missing input validation check (but really by not enough peer review)

• What it did?▪ Allowed an attacker to read up to 64 kilobytes of the servers

active memory for each attack, memory that was very likely to contain secure information.

• How it was stopped?▪ We updated all clients/servers to a patched version of OpenSSL▪ Reissued all certificates where there was any chance they could

have been compromised.

Freak - “Factoring RSA Export Keys”• A vulnerability cased by the growth of cheap commuting power

▪ A "512-bit export-grade key“ now be broken with a bit of math's called the "Number Field Sieve algorithm"* and about ~ $150 of cloud computing

• What it did?▪ Allowed the attacker to perform a man in the middle attack

• How it was stopped?▪ We disabled "TLS export cipher suites" either by updating

browsers, disabling the feature in servers or updating libraries that used them (such as OPENSSL to versions that did not)

*https://en.wikipedia.org/wiki/General_number_field_sieve

What’s Next? • Who knows!! the important thing we have all learnt is that just

because something has been around for a while is no guarantee of it being bomb proof.

• No system is perfect▪ Open source software is a great concept, but relies on

ACTIVE members picking at each others code and performing peer review.

▪ Closed Source relies on the competency of hidden processes and testers not succumbing to pressure to “Ship now”.

• Just remember there is always someone cleverer than you.

Constructing A Certificate

Certificate Structures• Certificate authorities• Private keys• Trusted roots• Generating a certificate

• You’ll need a keyfile• You’ll need a request with all the details of your certificate• You’ll need the trusted roots and intermediate certificates or

your CA• You’ll need the final certificate itself

File Extensions For Certificates• More Acronyms

▪ PEM▪ DER▪ CRT▪ CER▪ KEY

▪ ….CSR

OpenSSL• An open source library of SSL and TLS cryptography• Available for most platforms• Developed and managed by https://www.openssl.org

• repository for downloads on https://github.com/openssl/openssl

• Create certificates• Convert certificates• Extract certificates

HERE BE TIGERS

Installing OpenSSL - For the brave• https://www.openssl.org/source/

• ftp://ftp.openssl.org/source/ previous version • ftp://ftp.openssl.org/source/old older versions

• Download the compressed file and extract• Read the ReadME for instructions e.g run

• INSTALL Linux, Unix, etc.• INSTALL.W32 Windows (32bit)• INSTALL.W64 Windows (64bit)

• https://wiki.openssl.org/index.php/Compilation_and_Installation

Installing OpenSSL Under Windows• I found the easiest solution (as an Admin) is to install the pre

built Windows executable from Shining Light - there are other’s out there

• https://slproweb.com/products/Win32OpenSSL.html• Download the most recent “lite” version

• Currently 1.0.2e (Win32OpenSSL_Light-1_0_2e)

Installing OpenSSL For Linux• For Linux many distros come with a pre compiled version of

OpenSSL• yum install openssl• each OS may have its own method for configuration

Let’s Create Some Certificates

Domino – Creating A SHA2 Certificate• Domino no longer uses the Secure Server Certificate database

to generate keyfiles or merge certificates• We use a combination of OpenSSL and an IBM utility for

Domino called kyrtool• download kyrtool from IBM Fixcentral

http://ibm.co/1SAYX5E• copy it to your Notes or Domino program directory

• The program files must be 9.0.1 FP3 or higher

Domino – Creating A SHA2 Certificate• We need to decide the size of the key pair we want to create

• the larger the key pair the harder it is to decrypt• not all software systems support the largest key pairs

• If using Windows set the environment variable for OpenSSL first • Set OpenSSL_Conf=c:\openssl\bin\openssl.cfg

• verify openssl.cfg actually exists in that directory• To create a 4096 key pair

• c:\openssl\bin\openssl genrsa -out mynewserver.key 4096

Create a Certificate Signing Request • When buying a new certificate this sends to your CA• openssl req -new -sha256 -key mynewserver.key -out

mynewserver.csr• note that we are requesting a SHA2 certificate• the CSR will be verified by the CA when you submit it so you

can check that it’s right• if not you can recreate it by running the command again

MyNewServer.CSR• -----BEGIN CERTIFICATE REQUEST-----• MIIEvjCCAqYCAQAweTELMAkGA1UEBhMCR0IxDzANBgNVBAgMBkxvbmRvbjEjMCEG• A1UECgwaVGhlIFR1cnRsZSBQYXJ0bmVyc2hpcCBMdGQxCzAJBgNVBAsMAklUMScw• JQYDVQQDDB50cmF2ZWxlci50dXJ0bGVwYXJ0bmVyc2hpcC5jb20wggIiMA0GCSqG• SIb3DQEBAQUAA4ICDwAwggIKAoICAQDG5S3l7CtwiZQDHPXPxZMt3tQa8styCuZ+• CyipKAyqAKvaurqGfb232kYjLdR9hDh/TAswAeG40+DuQN4LKW4efWB91tQTKyZp• R9Kt5y6hVgKLjWbkZUqJcBRq60w7E1x+ufAqADLlhQAH0Q5fVe8aLhkYc5qIz4u/• JIm1Y+RgO3M/80v4xl85s6R/wEUSOdynKjrpBOsgWXUWu6pkCmxQOTD0lZfII5Lj• GztF9m7It8KcUojV4IdlsBNGlmOwdRgRwV1oqR0C3wdK9325xEbZcQgBnLBYprcN• GxZTwQpkIkv9tHVs7jhmrJsIYCRv7uDgIVpd3VXcTpGJXdBNgAxy7zW2q/EBlFMe• nPoavA8yyEID4tRHAQwCsDd4aoM/y3ZJRdU9ZyJE6fbcja2lDoB1r0dQWzA17UTC• o4qFgdLqJ94IKlEhnkYF7Dotj3lt0tBpNLRdL3MQwMdpGpetYYhLATQRNaXaOz9n• IsSFI/kIb5KKmFJX39vX7LjeAi9uRe4TbUBWBIWl+kmIT8n4xjUbjIeLrFWYUD4E• Aft6qEmXyScIRufqorbWMz88juuC9Svkcm3zjGcLFjGSuxXOhrrMA6LpCqQJXHI1• 5NCjZMdh/1xD1K39JhcYvSdfcpEtOe3CIXMpmkmJK0kANWrUOgeajoz7xC1vsUcE• H4btBohD7B6fiqdozsOsvN1s• -----END CERTIFICATE REQUEST-----

Now Comes The Domino Bit• We have to create a keyring file in a format Domino will be able to

read• For that we use the kyrtool we downloaded from FixCentral• From your Notes program directory

• kyrtool create -k c:\notes\data\mynewserver.kyr -p <passwordyouwanttouse>

• this will create two files• mynewserver.kyr• mynewserver.sth (this is the stashed password that unlocks

the keyring)

In the past we used a Domino database to create this keyring file pair but no more

Nearly There…• We have our keyring file• We have sent our request for a certificate, generated off our

new key pair to our CA• When the CA sends the certificate back we can merge the new

certificate into our keyring file▪ we need to merge ALL the certificates, root, intermediate

and server into a single “key” file• c:\openssl\bin\type mynewserver.key server.crt intermediate.crt

root.crt >mynewserver.txt

Last Step• We now add our new txt file with all the certificates in it into our

new Domino keyring• c:\ibm\notes\kyrtool import all -k c:\notes\data\mynewserver.kyr

-i c:\openssl\bin\mynewserver.txt

• That’s it. We now have a shiny keyring pair to use with our Domino server

Installing A SHA2 Certificate Under Domino• Install Using Internet Site Documents

• The first keyring file in the Internet Site docs view that matches the server configuration “wins”.

• Avoid too many wildcard or duplicate Internet Site Documents

• What can you use it for▪ HTTPS (Traveler, Websites)▪ S/MIME (encrypted mail)▪ TLS (secure outbound and inbound mail delivery)

More Domino SSL• Remove weak ciphers from the site documents• Add Disable_SSLV3=1 to the notes.ini on the server• Domino support TLS 1.2 now

▪ SSL_DISABLE_TLS_10▪ https://www-10.lotus.com/ldd/dominowiki.nsf/dx/TLS_1.2

Working With WebSphere Certificates• WebSphere installs with its own keystores for each cell and

node you add• The keystores are created and owned by IBM and have the

hostname of the server you’re installing onto by default• The cell keystores are found in

• /profiles/Dmgr01/config/cells/{cellname}/trust.p12• /profiles/Dmgr01/config/cells/{cellname}/key.p12

Accessing The SSL Configuration• Login to the WebSphere ISC• Security - SSL Certificate and Key Management

the default passwords for the stores,set by IBM during install are webAS

Adding A New Certificate To WebSphere• Go to the CellDefaultTrustStore if the certificate existing on

another server already you can “Retrieve from port”• Add your root and intermediate certificates here

Personal Certificate Request• The simplest way to generate a WAS certificate

• create a CSR in WAS• “receive” it into WAS when sent from the CA

• you can’t “receive” a certificate you didn’t request

WebSphere and 4096 Key Length Certificates• A 4096 certificate can generate an error when attempting to add

to WebSphere• “RSA premaster secret”

• You need to add the unrestricted policy files to WebSphere for the 4096 certificates to be imported

The Unrestricted Policy Files• ibm.co/1JZGs3z

Exporting A Certificate From WebSphere• Export a WAS certificate so that it can be imported onto other

systems• Such as a keyfile database generated by ikeyman and used

by IBM HTTP Server

Working With Ikeyman• There are different versions of ikeyman that create keyfile

databases recognised by different products• Look in the program directory for your installed product to find

the right one• For IBM HTTP Server the file is in /IBM/HTTPServer/bin• On Linux you’ll need to configure X11 forwarding if you don’t

have a graphical interface

Working With IKeyMan - Signer Certificates• Import the WebSphere certificate we extracted earlier• Add root and intermediate certificates

Working With IKeyman - Personal Certificates

Editing httpd.conf to add SSL configuration• Example content

• LoadModule ibm_ssl_module modules/mod_ibm_ssl.so • Listen 0.0.0.0:443 • <VirtualHost *:443> • SSLEnable • SSLProtocolDisable SSLv2 • </VirtualHost> • KeyFile /opt/IBM/HTTPServer/Plugins/config/webserver1/plugin-cfg.kdb • SSLDisable

• Restart IHS - use netstat to see if 443 is active and listening• Check IHS logs for SSL errors

• If WebSphere doesn’t have a copy of the IHS certificate and IHS doesn’t have a copy of the WebSphere certificate or they don’t share a trusted root, they won’t be able to communicate

SSL and Development• Despite the initial pain see if you can get a proper production

SSL certificate to use on your development environment.• If you can not (for cost reasons) ensure you create a self cert

that is EXACTLY the same type as your production environment• Identify ALL your third party libraries to your Admins as well as

any changes in versions in a proper release document. particularly if you are overriding an existing library on the server

Testing SSL On Your Site• https://www.ssllabs.com/ssltest/

• You can’t stay ahead with the hackers but you must be vigilant and keep up

• Have a plan for monitoring • Have a plan for lock down at the first appearance of exposure• Have a plan to fix the vulnerability• Have a plan to identify what information may be compromised• Have a plan to make that information of as little value as

possible

Resources• Working with OpenSSL

https://www.feistyduck.com/books/openssl-cookbook/• Creating SHA2 For Domino

http://turtleblog.info/2015/06/22/creating-sha-2-4096-ssl-certificates-for-domino/

• Unrestricted policy files for WebSphere http://www-01.ibm.com/support/docview.wss?uid=swg21663373

Thank you

Acknowledgements and DisclaimersAvailability. References in this presentation to IBM products, programs, or services do not imply that they will be available in all countries in which IBM operates.

The workshops, sessions and materials have been prepared by IBM or the session speakers and reflect their own views. They are provided for informational purposes only, and are neither intended to, nor shall have the effect of being, legal or other guidance or advice to any participant. While efforts were made to verify the completeness and accuracy of the information contained in this presentation, it is provided AS-IS without warranty of any kind, express or implied. IBM shall not be responsible for any damages arising out of the use of, or otherwise related to, this presentation or any other materials. Nothing contained in this presentation is intended to, nor shall have the effect of, creating any warranties or representations from IBM or its suppliers or licensors, or altering the terms and conditions of the applicable license agreement governing the use of IBM software.

All customer examples described are presented as illustrations of how those customers have used IBM products and the results they may have achieved. Actual environmental costs and performance characteristics may vary by customer. Nothing contained in these materials is intended to, nor shall have the effect of, stating or implying that any activities undertaken by you will result in any specific sales, revenue growth or other results.

Acknowledgements and Disclaimers cont.© Copyright IBM Corporation 2015. All rights reserved.

• U.S. Government Users Restricted Rights - Use, duplication or disclosure restricted by GSA ADP Schedule Contract with IBM Corp.

• IBM, the IBM logo, ibm.com, IBM WebSphere, IBM Domino are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both. If these and other IBM trademarked terms are marked on their first occurrence in this information with a trademark symbol (® or ™), these symbols indicate U.S. registered or common law trademarks owned by IBM at the time this information was published. Such trademarks may also be registered or common law trademarks in other countries. A current list of IBM trademarks is available on the Web at “Copyright and trademark information” at www.ibm.com/legal/copytrade.shtml

Other company, product, or service names may be trademarks or service marks of others.