Easier Platform Administration

using SAS 9.4 Grid Option Sets

SAS New South Wales User Group - Nov 2015

Andrew HowellANJ Solutions Pty Ltd

AgendaBackground• What is a grid & why use it

How it works• Platform Computing (LSF)

• Running SAS clients on a grid

Grid configuration• Prior to SAS v9.4

• SAS v9.4 - Grid Option Sets

• Examples

Recommendations

Wrap up• References, questions, etc.

Why use Grid Computing?

Maximise existing environment

• Workload management

• Parallel workload management

• Scalability

• High-availability environmentTraditional

client-serverGridded

client-server

ExamplesTexas Tech University

• Investigate stock market triggers• 10,000 samples, 10,000 simulations• 100,000,000 sample data sets• Single machine: 25 hours• SAS/Grid (100 Windows nodes): 40 mins• 2.7% of original runtime

National Institute for Environmental Health Statistics

• Gene sample profiling• 124,000 gene samples• Single machine: 448 hours• SAS/Grid (32 Linux nodes): 14.5 hours• 3.2 % of original runtime

How does it work

IBM Platform Computing

• Platform LSF

• Platform Process Manager

• Platform Grid Management Service

• Enterprise Grid Orchestrator

SAS Grid Components

• Grid Manager Plug-in for SAS Mgt Console, Env Manager

• Platform RTM for SAS

SAS Grid components

IBM Platform Computing

IBM Platform ComputingGrid definitions

Host: Each computer

Cluster: All computers

Job: Unit of work

Queue: Waiting list

Slot: Number of processes per host(By default, number of processor cores)

Queue settings

PRIORITY - jobs with higher priorities always take preference.

QJOB_LIMIT - the maximum job slots queue can use.

UJOB_LIMIT - the maximum job slots per user in a queue.

CPULIMIT - a time limit applied to jobs.

DISPATCH_WINDOW - the time window when jobs can be dispatched to a queue.

FAIRSHARE - Adjusts priority based on workload to prevent resource hogging.

IBM Platform Computinglsb.queues file• one of MANY LSF configuration files

Platform RTM for SASBegin QueueQUEUE_NAME = normalPRIORITY = 30DESCRIPTION = default queueEnd Queue

Begin QueueQUEUE_NAME = priorityPRIORITY = 50DESCRIPTION = high priority usersEnd Queue

Begin QueueQUEUE_NAME = nightPRIORITY = 40DISPATCH_WINDOW = (18:00-07:30)RUN_WINDOW = (18:00-08:00)DESCRIPTION = night time batch jobsEnd Queue

LSF Queue Configuration

Running Enterprise Guide on SAS Grid

Running Data Integration Studio on SAS Grid

Grid configuration prior to SAS v9.4

Apply default queue for a (gridded) Application Context

Can override:•Data Step Batch

Server•SASGSUB

script

Grid configuration prior to SAS v9.4

Typical SAS Grid scenario:

•Multiple business units / departments

•Each BU with own application context withBU-specific configuration:- cores, batch windows, folder

locations, etc.

•Each with own logical grid server, own queue

But with same priority !

•All have similar usage, but must be managed

separately

Grid configuration in SAS v9.4

Same as previous versions.. .. but with more !!

SAS Grid Option Sets

• New to SAS v9.4

• Apply default SAS options& grid settings

• Can apply alternate SASoptions & grid

settings basedon users, user

groups and/orSAS applications

SAS Grid Option Sets

Example:

• Give SAS PlatformAdministrators higherpriority when runningEnterprise Guide

SAS Grid Option Sets

Other possibilities:

• User specific options: Reporting teams, data miners, etc.

• Client specific options: AMO, EG, EM, DI Studio, etc

Recommendations

Support multiple business groups centrally

Leave SASApp for SAS Solutions• Credit Risk, etc

Have a dedicated Application Context (SASUsr?) for “regular” SAS usage• Enterprise Guide

• DI Studio

• Enterprise Miner, etc.

When NOT to run on a grid• Small programs

• Lots of workspace server stored processes?

Wrap upReferences

SAS Support Documentation• Grid Computing in SAS 9.4, Third Edition

• SAS/Connect Users Guide, Third Edition

SAS Training Course• SAS Grid Manager Administration (PCAD94)

SAS White Paper• “Grid Computing and SAS”, Merry Rabb & Cheryl Doninger

• http://support.sas.com/rnd/scalability/papers/101948_1204.pdf

SAS Global Forum Paper• “Queues for Newbies - How to speak LSF in a SAS World”, Andrew Howell

Questions?

Many thanks for your interest in SAS/Grid

Andrew HowellANJ Solutions Pty Ltd

• Email: ahowell@anjsolutions.com.au• LinkedIn: https://au.linkedin.com/in/howellandrew• Twitter: @AndrewAtANJ