Privileged Access Manager POC Guidelines

Hitachi ID Privileged Access Manager

Product Evaluation Guidelines

© 2011 Hitachi ID Systems, Inc. All rights reserved.

http://hitachi-id.com/

http://Hitachi.com/

Contents

1 Introduction 1

2 Focusing on Technical Challenges 1

3 Replication, Latency and Bandwidth 2

4 Automatic Discovery and Classification 6

5 Firewalls 9

6 Strong Authentication 11

7 Access Controls 12

8 Approval Workflow 14

9 Building the Lab Environment 15

APPENDICES 17

A Why Replication Across Cities is Mandatory 18

i

Hitachi ID Privileged Access Manager Product Evaluation Guidelines

1 Introduction

This document outlines best practices for evaluating privileged access management solutions. It offersguidance regarding how to construct a lab environment, what features to test and how to test them. Prod-uct evaluators can use this document to guide a robust testing process that highlights architectural andtechnological differences between competing products.

This document is intended for a technical audience. In particular, it should be helpful to anyone planninga lab evaluation of a privileged access management system. It describes the systems that must be madeavailable in the lab and the key tests which should be performed on candidate products.

2 Focusing on Technical Challenges

This document begins with the assumption that every product being considered can meet the most ba-sic requirements – randomize passwords for privileged accounts, store passwords in an encrypted vault,disclose passwords to users after they have been authenticated and subjected to access control policies.Rather than focusing on these basics, this document will help prospective buyers focus on more advancedelements – fault tolerance, scalability, workflow processes and more.

By focusing on the more technically difficult components of the system, prospective customers will be ableto see clear differences between products.

© 2011 Hitachi ID Systems, Inc.. All rights reserved. 1


3 Replication, Latency and Bandwidth

Once a PAM system is deployed, it will change passwords on privileged accounts to random values on ascheduled basis. This is good for security – attackers have a shorter time interval to guess passwords anddeparted staff automatically lose access.

Randomizing passwords goes hand in hand with storing those password values in a secure vault, so thatauthorized users will still be able to sign into these accounts. With passwords changing frequently, IT userswill become dependent on the vault’s availability to do their work.

While this is a secure arrangement, it introduces the potential for disaster: if the password vault is discon-nected, damaged or destroyed, system administrators will be unable to do their jobs. There is a potentialhere to escalate a disaster from a purely local problem – with the system where the password vault is stored– to a global one, where IT change management is interrupted everywhere.

To avoid this problem, a PAM system must:

1. Replicate the vault between different servers, so that loss of a single server does not cause data loss.

2. Place replicated servers in different data centers, as far apart (physically, geographically) as possible,so that a regional disaster does not cause data loss at other locations.

For a sobering historical review that explains why geo-diverse replication is important, refer to the notes inAppendix A on Page 18.

Replication is conceptually simple, but can be complicated in practice:

1. Wide area networks, between data centers in different cities and perhaps on opposite sides of theworld, have the following characteristics:

(a) High network latency – up to 50ms across a continent and 350ms around the planet.

(b) Low bandwidth – typically no more than 10Mbps on a private WAN across a continent and 1Mbpson a private WAN across an ocean.

(c) More frequent connection outages than LANs (e.g., a few times per year).

(d) Higher packet loss than LANs (e.g., 1 out of 1000 packets may be dropped).

2. Configuring the replication features native to database software can be very complex, requiring databaseadministration expertise, a deep knowledge of how each application uses the (to be replicated)database and many days of configuration and testing work.

3. WAN connections are normally carried on infrastructure owned by third parties, so they usually cannotbe trusted to be secure against packet sniffing or injection.

When evaluating a PAM system, organizations should always:

1. Deploy at least two replicated nodes, each with its own vault. This simulates an arrangement whereloss of a single application server or database instance does not cause loss of current passwordvalues.



2. Simulate a WAN between the two nodes – with high latency, low bandwidth and a small amount ofpacket loss.

An evaluation using a single node is simply unrealistic and may be dangerous, since the selected productmay demo well in the lab but not be able to support a reliable, replicated architecture in practice.

An evaluation using replication over a high speed, low latency network is also unsafe, since replication thatworks in a lab may not work well in a multi-site real-world deployment.

To evaluate products in this way, at least two PAM servers plus a WAN emulator are required. Fortunately,this is easy to setup using virtual machines and open source software, as illustrated in Figure 1.

WAN Emulator

Latency: 100msBandwidth: 1MbpsPacket loss: 0.1%

Privileged AccessManagement SystemNode 2


Application Level Replication

Password Vault:Node 1


Figure 1: Multiple, replicated nodes and a wide area network simulator

A more robust evaluation would use three replicated nodes, where each node replicates through its ownWAN emulator instance to the other two.

The WAN emulator can be downloaded from this project:http://wanem.sourceforge.netincluding using this (free) virtual appliance:http://wanem.sourceforge.net/download-vma.html

For products where replication is provided by the database server, rather than by the PAM system, theconfiguration should be as shown in Figure 2.

To summarize:


http://wanem.sourceforge.net

http://wanem.sourceforge.net/download-vma.html


WAN Emulator

Latency: 100msBandwidth: 1MbpsPacket loss: 0.1%



Database Server 1

Database Server 2


DB Native Replication


Figure 2: Using a WAN emulator with native database replication

1. Evaluate with at least two nodes.

2. Replicate data between the nodes.

3. Pass replication traffic through a WAN emulator.

Testing with a single node or without a WAN emulator represents a significant business risk once the systemmoves to production.

Once a replicated environment is configured, test with load and failure conditions:

1. Have each password management system change many passwords in a short time interval. Verifythat new password values are available on other nodes after a short time (ideally, just a few seconds).

2. Turn off one of the password management nodes. Verify that other nodes still have a full data setand that they respond to the failure in a suitable fashion – for example, by stopping the scheduledpassword change process so that new passwords are not stored in just one vault.

3. Re-enable the disabled node and verify that queued up updates from other nodes are automaticallywritten to its database.



4. Turn off and delete one node and walk through the process to rebuild its replacement. Gauge difficultyand estimate the amount and impact of data loss.



4 Automatic Discovery and Classification

It is not practical to manually configure thousands of managed systems where a PAM system will changepasswords and control administrative access. In practice, automatic discovery of systems and accounts,combined with automatic application of appropriate policies to each discovered system, is essential.

It is not practical to deploy thousands of servers or workstations in a lab environment. On the other hand,it is essential to test the system’s capability to discover, auto-manage and auto-unmanage systems andaccounts on thousands of systems. The solution to this problem is to simulate thousands of systems,without actually provisioning them physically or even virtually.

A simulator is a complex piece of software. Organizations considering deployment of a PAM system cannotseriously be expected to develop one, purely to perform a lab evaluation. Moreover, there are no enterprise-scale infrastructure simulators available for download (at least none that Hitachi ID Systems is aware of).The challenge here is to acquire and use an infrastructure simulator in the PAM evaluation despite it beingtoo costly to develop one and there being none available for purchase or as open source downloads.

PAM software vendors must have such a simulator, or they would not be able to test the scalability andflexibility of the auto-discovery infrastructure in their solution, especially as they prepare new versions forrelease.

This leads to just three possibilities:

1. The vendor does not have an infrastructure simulator. Such vendors should be disqualified immedi-ately, since they cannot test their own ability to deliver scalable, flexible solutions.

2. The vendor has a proprietary infrastructure simulator but is unable or unwilling to share it with prospec-tive customers. This is suspicious – either the simulator is needlessly complex, which means that itsuse by the vendor may be limited, or the simulator is not very capable, which means that the vendor’ssoftware is not well tested.

3. The vendor has a proprietary infrastructure simulator and is willing to share it with customers – typicallyunder terms of a non-disclosure agreement (NDA). This is the best outcome as it shows that thevendor has robust internal quality assurance processes and is willing to show them to customers.

Organizations are encouraged to only consider PAM systems from vendors who can provide a robust in-frastructure simulator for use in their prospective customers’ lab environments.

Using an infrastructure simulator, Hitachi ID Systems recommends testing that has the PAM system:

1. Automatically discover at least 1000 managed systems, drawn from computer objects in Active Direc-tory. Increase this number based on the ultimate scale of the deployment and the capacity of PAMservers available in the lab.

2. Automatically probe these systems, to extract their local users, groups, services, machine identifiersand so on.

3. Simulate latency and connectivity problems when probing individual systems – i.e., some are slow torespond (simulating network distance or busy systems) and others are simply unreachable.

4. Apply rules to decide which systems to manage at all. How flexible are these rules?



5. Apply rules to decide what security policies to apply to each system.

6. Apply rules to decide which accounts on managed systems are considered “privileged” and thereforeshould be managed.

7. Apply rules to decide what security policies to apply to each account.

8. Apply rules to decide what systems and accounts to stop managing – for example because systemshave been decommissioned and are no longer responsive, or accounts have been deleted.

Since Windows servers are often both the most numerous and the easiest to discover, in the context of aproof of concept (POC) it makes sense to focus the auto-discovery testing on them. With this in mind, it ishelpful to construct an Active Directory domain with many computer objects and link this to the simulator,so that they can each simulated computer has a "state" (login IDs, groups, group memberships, servicesand service accounts, password values, etc.). Each computer can then be "probed" by the PAM system, tofind its state. This is illustrated in Figure 3.

As can be seen in Figure 3, a simulator for auto-discovery can be thought of as a test harness for the PAMsystem. It should include components to:

1. Create a landscape of computers and load this data into Active Directory computer objects.

2. Track the state of each simulated computer.

3. Simulate daily changes – new computers are provisioned, old ones modified, moved and deactivated.

4. Simulate connectivity problems – slow connections and failures to connect.

5. Allow the PAM system to "discover" both administrator and service accounts. For the latter, it shouldbe able to not only change passwords on the service accounts, but also notify suitable operatingsystem components of new password values.

A robust auto-discovery system is crucial. It should be able to execute in a reasonable amount of time (sayno more than a few hours, running daily) and be both flexible and reasonably easy to manage. Anythingelse will lead to a system with very high total cost of ownership – which might ultimately be abandoned.

Always keep in mind that just because a PAM system can manage access to accounts on a few hundredsystems, this does not imply that the same system will be an appropriate solution in a larger organization,with thousands of managed systems and where many of these managed systems are added or retired,daily.



Generate Computer State

Generate Computer List

LDIFDE

Active Directory Connector

Auto discoveryprocess

Scheduled password randomization

process

Script file

Randomly change some computer state

Simulated connector (model Windows

servers)

Simulated connector set service account PW

List ofcomputers(all days)

List ofAD domain

service accts

1 file per PC:Accts, Grps,

Services, etc.LDIF: Create computers,

service accts

Active Directory(real, not simulated)

PasswordVault

LDIF: changecomputers

Add/mod/delete

List accts, computers Verify pw: chg svc acct pw

(day 1,2,3,...) (day 0)

Initial load

Represent the stateof each “computer.”

Read previous state of computer

Write: new state

Change admin,service accountpassword

Change passwords

Store new serviceaccount password

Randomly “fail to connect”to some systems

Privileged AccessManagement System

Figure 3: Components required to simulate managed system auto-discovery



5 Firewalls

In larger organizations, the corporate network may be segmented into different parts, where segments areisolated using firewalls. If a PAM system is deployed on one side of a firewall but systems with privilegedaccounts exist on the other side, then the firewall is likely to prevent connections to target systems, asshown in Figure 4.

System withPrivileged Accounts


Password Vault

Figure 4: The need to traverse firewalls to securing access to privileged accounts

A simple approach is to modify firewall configuration to allow connections from the PAM system, as illus-trated in Figure 5. This is a reasonable approach if there are very few firewall rules to change, but if thefirewall protects a large network segment with hundreds or thousands of systems, this approach will notscale.

To address problems of scale and complexity (too many target systems on the other side of the firewall);security (protocols used to connect to target systems are not intrinsically encrypted and safe) and perfor-mance (protocols are sensitive to bandwidth or latency and the distance between the PAM system andtarget systems is too great), a proxy server is needed, as illustrated in Figure 6.

A proxy server can be fast, secure, manageable and efficient. The drawback is that another system mustbe deployed – typically a VM – in each isolated network segment.

If a proxy will be needed in production, the PAM system POC should include one and connect to somesystems through it. Test the installation footprint (should be small), configuration complexity and bandwidthconsumption of the proxy. Verify that communication between the main PAM system and proxy is encrypted.





Password Vault

Figure 5: Enabling access by opening ports through the firewall



PAM Proxy Server

Password Vault

Figure 6: Enabling access by deploying a proxy server



6 Strong Authentication

A PAM system controls access to the most sensitive accounts across an entire organization. It is there-fore reasonable to apply stronger authentication policies to it as compared to other applications, includingindividual systems access to which it controls.

In practical deployments, strong authentication may mean use of smart cards or one time password tokens.

In a lab environment, setting up a PKI and CA for smart cards or a token server may be cost-prohibitive. Aless costly alternative suitable to demonstrating a flexible authentication model is therefore appropriate.

A simple approach to multi-factor authentication is to enroll mobile phone numbers from users and sendthem a random PIN - via SMS - as one of the login steps. Another inexpensive authentication factor issimply asking users to answer personal security questions.

In a POC environment, it is reasonable to use one or both of these – both if the ability to route differentusers to different login processes is needed:

1. Authentication process A: user types his own Active Directory password, which the PAM server vali-dates against an AD DC.

2. Authentication process B: user is sent a random PIN via SMS to his mobile phone. He must enter thisfirst, before being prompted to enter his AD password, as above.

3. Authentication process C: user completes AD/password login as in A above and then is asked toanswer 2 or 3 security questions.

The ability to use more than simply AD ID/password authentication and to dynamically select appropriatelogin processes for different users should be tested and the above guidance shows how this can be donewithout special hardware or software.



7 Access Controls

Once the PAM system has randomized passwords to accounts with elevated privileges, users will not beable to sign into those accounts except by using the PAM system. The PAM system must then enforceaccess controls: who can connect to what?

Access controls are defined using access control lists, which connect users to rights.

Assigning ACLs directly to users or directly to managed systems is not a good solution, since there may behundreds of users and thousands of systems under management. The number of ACLs connecting usersto systems would simply be unmanageably large.

In order to make access control manageable, it should be possible to group users, group accounts andsystems and link the two types of groups with rights. This means that the PAM system must support:

1. Defining sets of users that should be granted some rights. It is important to leverage existing informa-tion about users when doing this:

(a) Define a set of users based on their membership in one or more Active Directory or LDAP groups.

(b) Define a set of users based on values of their attributes in a trusted system, such as AD or LDAP.

(c) Define a set of users based on the OU that houses their personal (unprivileged) account.

2. Defining sets of managed systems that users should get rights to. It is important to leverage dataabout systems when doing this, such as the following

(a) Managed system’s DNS or NetBIOS name.

(b) Where in a directory (OU) the machine’s computer object was found. Active Directory or LDAPgroups.

(c) IP, in particular via subnet matching.

(d) Operating system type and patchlevel.

3. Identifying managed accounts through characteristics of the account within its system:

(a) Account is a member of a specified group (group name, SID).

(b) Account is used to run a service.

(c) Account name matches a pattern.

Access controls can then be defined in a pattern such as the following:

User group Rights System Account

• Member of AD groupWindows Admins ondomain AD-NA.

• Account in OU IT ondomain AD-NA.

Launch RDPconnections

• IP in subnet 1.2.3.4/24.• OS type "Windows"

• Member of group withSID S-1-5-32-544(Administrators)

• Member of LDAP groupLinux Admins indirectory LDAP1.

Launch SSHconnections

• IP in subnet 5.6.7.8/24.• OS type "Linux|Unix"

• UID 0 (root)



A lab evaluation should verify that:

1. Users can be collected into sets, which can be used to assign rights, based on their group membershipand attributes on multiple directories.

2. Systems can be collected into sets, based on their name, address, type and other attributes.

3. Accounts can be collected into the same sets as systems, based on their name, group memberships(group ID or name) and whether they are used to run services (Windows-only).

4. Access rights can be assigned using the pattern described above – sets of users linked via rights tosets of systems and accounts.

Anything less would create an excessive administrative problem.



8 Approval Workflow

In some cases, someone who does not satisfy the criteria described in Section 7 on Page 12 above will needaccess to a privileged account. This can be thought of as a one-time or non-pre-authorized scenario. Insuch cases, an authorization process is needed to control who can request access and who must approveit before the user is allowed access to a privileged account.

Authorization means that multiple human beings are involved:

1. Exactly one requester – the person asking for access rights.

2. At least one recipient – the person who will get access rights.

3. Zero or more authorizers – people who will review the request and either approve or reject it.

The PAM must support:

1. Limiting who can act as a requester at all.

2. Limiting who can be a recipient:

• For some requesters, only self-service (requester and recipient are the same person).• For others, only some recipients, based on their relationship (e.g., requester must be a manager

of the recipient, or in the same location, etc.)

3. Limiting what access can be requested:

• For example, the system being requested must be in the same location as the requester, or of aspecific type for a given recipient.

4. Requiring and validating form inputs, such as start/end date or an incident number.

5. Automatically selecting authorizers:

• Zero authorizers means that the request is automatically approved due to some business rule.• One or more authorizers means that these people must review and approve the request before

it proceeds.• Support an unlimited number of authorizers (not just one).

6. Intelligently cope with non-responsive authorizers:

• Check an authorizer’s e-mail out-of-office status, and select an alternate if the authorizer is knownto be unavailable.

• Send reminder e-mails to non-responsive authorizers.• Be able to invite multiple authorizers at once, and accept approval from just a subset (e.g., any

two of these five...). Invitations sent all at once result in faster approvals as does the ability toapprove a request with just a subset of the invited people.

• Assign alternate authorizers in the event that a request remains open after multiple remindershave been sent, as the original authorizers are likely unavailable.

The product evaluation should attempt to exercise as many of these options as possible – especially dy-namic authorizer routing, multiple approvers and N of M approvers.



9 Building the Lab Environment

Pulling together the requirements from the preceding sections, a reasonable lab environment in which toevaluate a PAM system should include the following:

1. Two or three servers on which to install the PAM software. A reasonable configuration for each ofthese in a lab is:

(a) Virtual hardware: 8GB RAM, single core CPU, 250GB disk.

(b) Operating system: Windows 2008 32-bit (in most organizations it is reasonable to avoid 2008R2as it requires a 64-bit virtual host).

(c) Database server (installed on the same server as the application in the POC): SQL Server 2008Standard.

2. A virtual machine to host a WAN emulator.

(a) This is essential to test real-world, multi-site data replication.

(b) WANem is a 32-bit Linux-based VM.

3. A virtual Active Directory domain controller (AD DC).

(a) Should have at least 1000 users, several hundred groups, extensive group membership and anorganizational hierarchy. This data will be used to test authentication, authorization policies andapproval workflows.

(b) Hitachi ID Systems can provide sample contents for an arbitrarily-sized AD environment in theform of LDIF files.

4. A target system simulator, so that auto-discovery, auto-classification and management of administratorand service account passwords on thousands of systems can be evaluated.

(a) This has to be provided by each PAM vendor, since one or more simulated connectors are usedto accomplish this and since connector architectures are unique to each product.

(b) Hitachi ID Systems provides a simulator to every customer who wishes to use it.

5. A load balancer, capable of making web sessions “sticky” to a single server. A simple and free one,again for Linux VMs, is Pen: http://siag.nu/pen/ – definitely suitable for a POC, though productionsystems will likely demand more.

6. At least one real target system of each major type. Consider deploying the following (VMs are appro-priate in all cases):

(a) A Windows Server 2003 target system.

(b) A Windows Server 2008 target system.

(c) A Linux server target system (RedHat, SuSe, Debian, Ubuntu, etc.).

(d) A SQL Server where access to SA accounts will be controlled.

(e) An Oracle Database where access to DBA accounts will be controlled.

7. Multiple workstation operating systems, both to test the PAM system’s ability to secure access toadministrator and service accounts on laptops and as user interfaces to the system:

(a) Windows XP.


http://siag.nu/pen/


(b) Windows 7.

(c) Others as required.

In practice, all of the above can be provisioned as virtual machines on two or three large virtual hosts –quad-core CPUs, 24GB RAM, 1TB RAID disk with ESXi, Xen or similar. It can all be provisioned on a cloudIaaS vendor such as Amazon EC2 as well.

The lab environment should be able to connect to the public Internet. This will support:

1. Remote assistance by the PAM vendor.

2. Sending e-mails to mobile phones through public e-mail-to-SMS gateways, for two factor authentica-tion (AD password plus SMS random PIN).

The lab environment may include an e-mail service, to test workflow use of e-mails. A simple mail servicesuch as Postfix (http://postfix.org) is appropriate as it is much easier to setup than Exchange. Alternately,the PAM system can simply drop e-mail messages on its filesystem.

The lab environment may include an incident management system (BMC Remedy, HP ServiceDesk, etc.).This is used to test its ability to create, update and close incidents and also to verify that ticket numbersentered by IT staff are valid, open, etc.

The lab environment may include a security incident and event management system (SIEM), or just aLinux server with a networked SYSLOG service, to test log aggregation from the PAM system to a centralmonitoring solution.


http://postfix.org


APPENDICES



A Why Replication Across Cities is Mandatory

If replication across multiple vaults, each in a different city, seems extreme, consider a recent history ofdisasters and what would have happened to a global organization that had a data center containing theirsole password vault in an affected area. The human toll of each event is major, but in the following we focuson the physical effects:

1. 1995: Kobe earthquake – destroys thousands of buildings in Kobe, Japan.

2. 2003: Northeast blackout – power lost in much of Eastern and Midwest US and Canada for up to 24hours in some areas. Telecommunications and other infrastructure disrupted.

3. 2004: Indonesian tsunami – destroys towns and villages on Indonesian, Thai, Indian and other coast-lines.

4. 2005: Hurricane Katrina – knocked out much of Louisiana and Mississippi.

5. 2008: Major earthquake in Szechuan – knocked out major infrastructure in Chengdu (a city of tens ofmillions).

6. 2010: Haiti earthquake – destroyed much of Port-au-prince and the country’s infrastructure.

7. 2011: Tohoku earthquake and subsequent meltdown at the Fukushima power plant – physically de-stroyed and made uninhabitable a large area in Northeast Japan.

8. 2011: New Zealand earthquake damages much of Christchurch.

The point here is that major natural and man-made disasters happen somewhere in the world with somefrequency and cannot be safely ignored as “infrequent” or “only happens somewhere else.” A robust systemmust be built with the assumption that a single data center may be destroyed or at least disconnected frompower and/or telecommunications for at least a few hours and perhaps permanently.

www.Hitachi-ID.com

500, 1401 - 1 Street SE, Calgary AB Canada T2G 2J3 Tel: 1.403.233.0740 Fax: 1.403.233.0725 E-Mail: [email protected]

File: /home/dawnm/hipam-poc-guidelines.texDate: 2011-09-23

http://Hitachi-ID.com/

Privileged Access Manager POC Guidelines

Technology

Transcript of Privileged Access Manager POC Guidelines