Shield talk elasticsearch meetup Zurich 27.05.2015
63
Shield security for Elasticsearch
Transcript of Shield talk elasticsearch meetup Zurich 27.05.2015
Shieldsecurity for Elasticsearch
Agenda• Introduction
• Security requirements
• Alternative approaches
• Shield features
• Demo
• Q&A
Security requirements
• Prevent unauthorised access
• Preserve data integrity
• Provide an audit trail
Alternative approaches
• Network separation, Firewall etc.
• Reverse proxy
• Apache HTTPD
• nginx
es cluster
node
node
node
client
reverse proxy
Shield introduction
• provides security features for Elasticsearch
• installs as a plugin on an Elasticsearch node
• License required
• integrates security at a low-level
Shield installation
• bin/plugin -i elasticsearch/license/latest
• bin/plugin -i elasticsearch/shield/latest
Shield features• IP filtering
• Authentication
• Authorization
• Node authentication & encryption
• Auditing
/doc_store/doc/7384
es cluster
node
node
{ "_id": "837826", "_source": { "type": “delivery_note”, "content": … } }
/doc_store/doc/7384
es cluster
node
nodenode
user access
node access
IP filtering• Application-level filtering of IP addresses
• Prevent clients from accessing a cluster
• Prevent nodes from joining a cluster
• configuration in elasticsearch.yml
• Dynamic update of rules possible
shield.transport.filter.allow: “10.0.0.100”shield.transport.filter.deny: “10.0.0.0/24”
one allow and deny statement
allow takes precedence
shield.transport.filter.allow: [“10.0.0.1”, “10.0.0.2”]shield.transport.filter.deny: _all
statements can contain an array
“_all” denies everything not declared before
shield.transport.filter.allow: company.comshield.transport.filter.deny: *.google.com
statements can contain domain names
shield.transport.filter.allow: localhostshield.transport.filter.deny: '*.google.com'shield.http.filter.allow: 172.16.0.0/16shield.http.filter.deny: _all
transport and http client separation
/doc_store/doc/7384
es cluster
node
node
Authentication IP filtering
Authentication• Users must prove their identity to access a resource
• Authentication provider implemented as realms
• Realms live in an realm chain
• Multiple providers
• esusers
• LDAP
• Active Directory
shield.authc: realms: es_users: type: esusers order: 0 ldap1: type: ldap order: 1 enabled: false url: "ldap://path/to/ldap"
elasticsearch.yml
shield.authc: realms: es_users: type: esusers order: 0 ldap1: type: ldap order: 1 enabled: false url: "ldap://path/to/ldap"
elasticsearch.yml
realm definition
shield.authc: realms: es_users: type: esusers order: 0 ldap1: type: ldap order: 1 enabled: false url: "ldap://path/to/ldap"
elasticsearch.yml
each realm has a specific type
shield.authc: realms: es_users: type: esusers order: 0 ldap1: type: ldap order: 1 enabled: false url: "ldap://path/to/ldap"
elasticsearch.yml
each realm has a execution order
esusers
• File based internal realm
• Easy to use
• command line administration tools
esusers
• bin/shield/esusers list
• bin/shield/esusers useradd <user> -r <role>
• bin/shield/esusers userdel <user>
Demo
LDAP & Active Directory realms
• authentication against external ldap or ad server
• supports SSL/TLS using ldap
• Elasticsearch roles can be mapped to LDAP/AD groups
• cached users in memory
monitoring: - "cn=admins,dc=example,dc=com"user: - "cn=users,dc=example,dc=com" - "cn=admins,dc=example,dc=com" - "cn=John Doe,cn=contractors,dc=example,dc=com"
config/shield/rolemapping.yml
monitoring: - "cn=admins,dc=example,dc=com"user: - "cn=users,dc=example,dc=com" - "cn=admins,dc=example,dc=com" - "cn=John Doe,cn=contractors,dc=example,dc=com"
config/shield/rolemapping.yml
Elasticsearch role name
monitoring: - "cn=admins,dc=example,dc=com"user: - "cn=users,dc=example,dc=com" - "cn=admins,dc=example,dc=com" - "cn=John Doe,cn=contractors,dc=example,dc=com"
config/shield/rolemapping.yml
values are ldap groups and users
/doc_store/doc/7384
es cluster
node
node
Authorization
Authentication IP filtering
Authorization• Determines if the user has the right to execute a
action over a resource
• Role based access control (RBAC)
• Every role defines a set of permissions over clusters and indices
• set of predefined permissions
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
Elasticsearch role name
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
cluster related privileges
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
indices and aliases privileges
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
access to all indices with any action
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
access to suggest and search permission
on all indices
administrator: cluster: all indices: '*': all
user: indices: '*': search
logs_user: indices: 'logs-*': read
config/shield/roles.yml
access to read every index which starts with “logs-“
Complete privileges reference:
http://www.elastic.co/guide/en/shield/current/reference.html#privileges-list
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
admin
/documents/doc/_search
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
admin
/documents/doc/_search
admin: indices: ‘documents’: all
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
logistics
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
logistics
"filter": {"terms": {"type": ["pod", "delivery_note"]}}
{ content: … type: offer}
{ content: … type: pod}
{ content: … type: delivery_note}
documents
/documents_logistics/_search
logistics: indices: ‘documents_logistics’: all
"filter": {"terms": {"type": ["pod", "delivery_note"]}}
logistics
Demo
es cluster
node node
es cluster
node node
es cluster
node node
Node authentication and encryption
• traffic encryption using SSL/TLS
• node authentication using certificates
• certificates can be self signed or signed by CA (recommended)
• traffic separation
• JCE extensions for stronger encryption
shield.ssl.keystore.path: /usr/local/es/config/node01.jksshield.ssl.keystore.password: mypassword
elasticsearch.yml
full path to the keystore file
shield.ssl.keystore.path: /usr/local/es/config/node01.jksshield.ssl.keystore.password: mypassword
elasticsearch.yml
the password to decrypt the keystore
shield.transport.ssl: trueshield.http.ssl: true
elasticsearch.yml
enable SSL on the HTTP layer
enable SSL on the network layer
Auditing
• Logging of security events
• Currently outputs to file
• Logging level
shield.audit.enabled: true
elasticsearch.yml
enable audit logging
Auditing examples
• access
• failed authentications
• anonymous requests
• unauthorised access
Q&A
Credits• mimacom: www.mimacom.com
• Elasticsearch: https://www.elastic.co/
• Images: • https://icons8.com/ • http://dilbert.com/ • http://c279160.r60.cf1.rackcdn.com/assets/blog_post/meta_og_image/115/hiring.png • http://funny-pictures-blog.com/wp-content/uploads/funny-pictures/Security-Gate-Fail.jpg • http://2.bp.blogspot.com/-o-0b_5ReFyQ/VOwNd6ktk4I/AAAAAAAAMkE/B8HL_zQAjGY/
s1600/security-chain-fence-funny-demotivational-posters-1296250082.jpg • http://s2.quickmeme.com/img/5c/
5c9e05a465c753417dbde949ee285fd9e56f0739ad0254de838a7d8c61c1a318.jpg • http://media.giphy.com/media/YdaqReCo5D51m/giphy.gif
