Automated Configuration of Firmware

APOLLO GROUP

Automated Configuration of Firmware

Michael ArnoldPrincipal Systems Engineer27 September 2012

2APOLLO GROUP

Who

Tools

Example Workflow

Example Code

Links

Questions

Agenda

© 2012 Apollo Group

3APOLLO GROUP

APOLLO GROUP

Who


4APOLLO GROUP

Who is Apollo?


Apollo Group is a leading provider of higher education programs for working adults.

5APOLLO GROUP

Systems Administrator

Automation geek

13 years in IT

Hadoop Cluster Admin

I deal with:

–Server hardware specification/configuration

–Server firmware

–Server operating system

–Hadoop Cluster performance and availability

Who is Michael Arnold?


6APOLLO GROUP

It has nothing to do with Puppet.

–Unless someone writes a module to program firmware from the running OS.

• Which could work for RAID controllers.

How does firmware tie in with Puppet?


7APOLLO GROUP

APOLLO GROUP

Tools


8APOLLO GROUP

HP SmartStart Scripting Toolkit

Dell OpenManage Deployment Toolkit

IBM ServerGuide Scripting Toolkit

Fujitsu ServerView Scripting Toolkit

Hardware Toolkits


9APOLLO GROUP

Configuration of

–BIOS

–BMC (DRAC/iLO/RSA/etc.)

–Hardware RAID

–FC HBA

What do the tools provide?


10APOLLO GROUP

Manually configure reference hardware

Capture the firmware configuration

Clone the configuration to multiple machines

How does the toolkit work?


11APOLLO GROUP

Toolkit Components


Kernel & Initrd

Firmware Tools

Toolkit

12APOLLO GROUP

Toolkit Requirements


Toolkit

DHCP

TFTP

NFS

13APOLLO GROUP

APOLLO GROUP

Example Workflow


14APOLLO GROUP

Hardware is racked and cabled

Switch ports are up

Server is manually powered on

Firmware defaults are to boot from network if no other boot device is found.

–No other boot devices are found

Assumptions


15APOLLO GROUP

All server hardware is from the same vendor (HP)

Constraints


16APOLLO GROUP

Firmware Update

Hardware Discovery

Firmware Configuration

BuildsystemRegistration

The Process


17APOLLO GROUP

Screencast


18APOLLO GROUP

Unknown system netboots

Workflow


19APOLLO GROUP


Boot into HP SmartStart Scripting Toolkit

Workflow


20APOLLO GROUP



Gather hardware information (CPU/RAM/disk/NIC) via hpdiscovery

Workflow


21APOLLO GROUP




If this is HP hardware, then

–Update firmware

–Configure BIOS

–Configure iLO

–Configure RAID

Workflow


22APOLLO GROUP





–Update firmware

–Configure BIOS

–Configure iLO

–Configure RAID

cobbler-register to the hardware profile

Workflow


23APOLLO GROUP





–Update firmware

–Configure BIOS

–Configure iLO

–Configure RAID

cobbler-register to the hardware profile

System reboots

Workflow


24APOLLO GROUP

Modify Cobbler system profile, giving it an identity (hostname, IP, OS metadata)

Workflow


25APOLLO GROUP


Known system netboots

Workflow


26APOLLO GROUP



Boot into OS installer

Workflow


27APOLLO GROUP




Install OS

Workflow


28APOLLO GROUP




Install OS

Install Puppet

Workflow


29APOLLO GROUP




Install OS

Install Puppet

…

Workflow


30APOLLO GROUP




Install OS

Install Puppet

…

Profit!

Workflow


31APOLLO GROUP

APOLLO GROUP

Example Code


32APOLLO GROUP

LABEL dtk-install_hadoop

kernel /images/dtk-3.5.1/SA.1

append initrd=/images/dtk-3.5.1/SA.2

ramdisk_size=156482 Stage3_type=cdrom quiet DEBUG=0

share_location=192.168.1.1:/srv/nfs/ro/dtk-3.5.1

share_type=nfs selinux=0 share_opts=ro,nolock

share_script=install_hadoop.sh cprofile=Hadoop

ipappend 2

Dell TFTP Config


33APOLLO GROUP

LABEL sstk-install_hadoop

kernel /images/sstk-8.50/vmlinuz

append initrd=/images/sstk-8.50/initrd.img

root=/dev/ram0 rw ramdisk_size=184248 ide=nodma

ide=noraid pnpbios=off network=1

sstk_mount=192.168.1.1:/srv/nfs/ro/sstk-8.50

sstk_mount_type=nfs sstk_mount_options=ro,nolock

sstk_script=install_hadoop.sh cprofile=Hadoop

ipappend 2

HP TFTP Config


34APOLLO GROUP

cd /opt/dell/toolkit/template/scripts

/opt/dell/toolkit/template/scripts/syscap.sh /tmp/syscap

/opt/dell/toolkit/template/scripts/raidcap.sh /tmp/raidcap

/opt/dell/toolkit/template/scripts/raccap.sh /tmp/raccap

# MegaCLI does not come with the DTK so "install" it.

cp -a /opt/dell/toolkit/systems/MegaCLI/opt/MegaRAID \

/opt/MegaRAID

/opt/MegaRAID/MegaCli/MegaCli -CfgSave -a0 -f /tmp/mega.0

Dell Config Capture


35APOLLO GROUP

cd /TOOLKIT

./conrep -s -f /tmp/conrep.dat # BIOS

./hpacuscripting -c /tmp/cpqacuxe.dat # RAID

./hplpcfg /s /tmp/hplpcfg.dat # Emulex HBA

./hpqlarep /S /tmp/hpqlarep.dat # Qlogic HBA

if ./ifhw /tmp/hw_discovery.dat allboards.xml \

"PCI:Integrated Lights-Out" 2> /dev/null ; then

./hponcfg -w /tmp/hponcfg.dat # iLO 2/3

else

./lo100cfg -o /tmp/lo100cfg.dat # LO100i

fi

HP Config Capture


36APOLLO GROUP

*** Detecting system type ***

System Information:

Name : ProLiant DL360 G5

Serial : USE123D456

Arch : x86_64

LNAME : ProLiantDL360G5

SNAME : DL360 G5

Asset Tag : 01234567

Platform Detection


37APOLLO GROUP

HP SSSTK has a function library that sets many variables.

–$PRODUCT_NAME =

• "ProLiant SL170z G6"

• "ProLiant DL360 G6"

–$SERIAL_ID = hardware serial number

–$PRODUCT_LNAME =

• "ProLiantSL170zG6"

• "ProLiantDL360G6"

–$PRODUCT_SNAME =

• "SL170z G6"

• "DL360 G6"

–$TOOLKIT_MNTPNT = /mnt/main

–$TOOLKIT_SERVER_IP = NFS server IP

–$HWDISC_FILE = /TOOLKIT/hpdiscovery.xml

Some Available Variables


38APOLLO GROUP

Set Variables Based on Hardware Type

export "`./hwquery ${HWDISC_FILE} allboards.xml SERVERNAME=SystemName`"

case "${SERVERNAME}" in

"ProLiant DL160 G6"* )

CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl160_conrep.dat

LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/dl160_lo100cfg.dat

;;

"ProLiant SL170z G6"* )

CONREPDAT="${TOOLKIT_MNTPNT}/data_files/sl170z_conrep.dat -x \

${TOOLKIT_MNTPNT}/conrep_xml/conrep_SL170zg6_20100401.xml"

LO100CFGDAT=${TOOLKIT_MNTPNT}/data_files/sl170z_lo100cfg.dat

;;

#...

Inside install_hadoop.sh


39APOLLO GROUP

#...

"ProLiant DL360 G5"* )

CONREPDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_conrep.dat

if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array E200i Controller" \

2> /dev/null ; then

HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saE200i_cpqacuxe.dat

fi

if ./ifhw ${HWDISC_FILE} allboards.xml "PCI:Smart Array P400i Controller" \

2> /dev/null ; then

HPACUDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_saP400i_cpqacuxe.dat

fi

HPONCFGDAT=${TOOLKIT_MNTPNT}/data_files/dl360g5_hponcfg.dat

;;

esac



40APOLLO GROUP

if [ -n "$CONREPDAT" ] ; then

echo "** Applying BIOS Configuration"

./conrep -l -f ${CONREPDAT}

fi

if [ -n "$HPACUDAT" ] ; then

echo "** Clearing RAID Configuration"

./hpacucli/hpacuscripting -i ${HPACUDAT}_clear; sleep 5

echo "** Applying RAID Configuration"

./hpacucli/hpacuscripting -i ${HPACUDAT}

fi

if [ -n "$LO100CFGDAT" ] ; then

echo "** Applying BMC Configuration"; ./lo100cfg -i ${LO100CFGDAT}

elif [ -n "$HPONCFGDAT" ] ; then

echo "** Applying iLO Configuration"; ./hponcfg -f ${HPONCFGDAT}

fi



41APOLLO GROUP

echo "** Registering with cobbler"

# Koan does not come with SSSTK, so "install" it.

${TOOLKIT_MNTPNT}/koan/install

# cprofile is set via kernel boot parameter and points at part of a

# Cobbler profile name.

# SERIAL_ID and PRODUCT_LNAME are set via SSSTK.

# PRODUCT_LNAME = ProLiantSL170zG6, ProLiantDL360G6, etc.

# SERIAL_ID = hardware serial number

echo "** Registering to profile ${cprofile}-${PRODUCT_LNAME}"

cobbler-register --server=${TOOLKIT_SERVER_IP} --fqdn=${SERIAL_ID} \

--profile=${cprofile}-${PRODUCT_LNAME} --batch

Register with Buildsystem


42APOLLO GROUP

$ cat dl360g5_saE200i_cpqacuxe.dat_clear

; Version: 8.50.6.0

; http://h30499.www3.hp.com/t5/ProLiant-Servers-ML-DL-SL/CPQACUXE-reset-utility-within-WinPE/td-p/3939475

Action= Reconfigure

Method= Custom

Controller= SLOT 0

ClearConfigurationWithDataLoss= Yes

Clear the HP RAID Controller


43APOLLO GROUP

$ cat dl360g5_saE200i_cpqacuxe.dat

; Version: 8.50.6.0

Action= Configure

Method= Custom

; Controller Specifications

; Controller HP Smart Array E200i

; Firmware Version 1.82

Controller= SLOT 0

ReadCache= 100

WriteCache= 0

RebuildPriority= Medium

ExpandPriority= Medium

SurfaceScanDelay= 15

DriveWriteCache= Disabled

; Array Specifications

Array= A

; Array Drive Type is SAS

; 1I:1:1 (72 GB), 1I:1:2 (72 GB)

Drive= 1I:1:1, 1I:1:2

OnlineSpare= No

; Logical Drive Specifications

LogicalDrive= 1

RAID= 1

Size= 69973

Sectors= 32

StripeSize= 128

ArrayAccelerator= Enabled

Configure the HP RAID Controller


44APOLLO GROUP

You may have to test for any failures when applying configurations.

When reusing HP hardware, adding a user to the BMC that already exists will cause a failure to apply all of the configuration.

Newer firmware can have different config options that will cause an older capture to fail.

Error Handling


45APOLLO GROUP

Firmware Update

Hardware Discovery

Firmware Configuration

BuildsystemRegistration

Summary


46APOLLO GROUP 46APOLLO GROUP

HP SmartStart Scripting Toolkit

http://h18013.www1.hp.com/products/servers/management/toolkit/index.html

Dell OpenManage Deployment Toolkit

http://en.community.dell.com/techcenter/systems-management/w/wiki/1772.dell-openmanage-deployment-toolkit.aspx

IBM ServerGuide Scripting Toolkit

http://www-03.ibm.com/systems/be/management/sgstk/

Fujitsu ServerView Scripting Toolkit

http://www.fujitsu.com/fts/products/computing/servers/primergy/management/deploy/

Cobbler - Linux Installation Server

http://cobbler.github.com/

LSI MegaRAID MegaCLI

http://www.lsi.com/downloads/Public/MegaRAID%20Common%20Files/8.02.24_MegaCLI.zip

Links


47APOLLO GROUP 47APOLLO GROUP

https://intelligentsysadmin.wordpress.com/

https://github.com/razorsedge

http://forge.puppetlabs.com/razorsedge

Contact


48APOLLO GROUP

APOLLO GROUP

Questions?


Automated Configuration of Firmware

Technology

Transcript of Automated Configuration of Firmware