Driving Test Lab Power Savings with Automation

WH

ITE PAPER

10/2013

systems

Test labs represent a large ongoing investment and an opportunity for signi�cant savings. One major area of signi�cant waste is energy usage. Test labs are prodigious consumers of power. It is possible to save up to millions of dollars in energy costs by implementing test lab infrastructure management software to automate the operation of test lab hardware. Test lab management software can bring electrifying reductions in power consumption. More importantly, an investment in test lab automation also brings signi�cant side-bene�ts in terms of capital equipment-savings and increased productivity.

A key way to save millions of dollars is to implement test lab infrastructure management software

Introduction

Automation is Key to Power SavingsTest labs are a prime target to “go green”, because they consume massive amounts of electricity for the operation and cooling of power-hungry equipment, including:

If reducing the amount of time that test lab equipment spends powered up is the key to reducing power consumption, then it is essential to understand the aspects of the typical test lab’s operational processes that cause test devices to be powered up longer than is truly necessary. One look at the typical test lab, and one can see that the lack of information technology-driven automation in the test lab environment is the root cause of much ine�ciency in the lab, including power consumption.

No one in the front o�ces of companies that own large test labs would consider it e�cient to coordinate meetings and resources without a shared calendaring system like Microsoft Exchange, nor would they consider post-it notes to be a state-of-the art form of messaging. It is laughable to restrict communication in a large o�ce building setting to telephones connected manually by an operator using cables and a physical switchboard. Yet that, in a sense, is the state of business process automation in many test labs. Very costly DUTs and test generators are often connected together with an unruly mess of cables and manual patch panels. Testing teams coordinate usage of equipment through “hands o�” post-it notes placed on equipment that is in use, and through emails. Visibility and calendaring of resources is typically non-existent as well - it’s all handled in a very manual fashion.

Devices under test (DUTs) such as switches, routers, computing and storage

Test generation-switches

Any equipment that provides connectivity between all the actual testing devices

”

”

Driving Test Lab Power Savings with Automation

ROI

AutomationI nve s t m e n t

Energy & cost

savings

systems

CloudShell

How Test Lab Automation Drives Power SavingsTest lab automation software provides the means to realize signi�cant power savings by changing the way test labs are operated. Lab automation software does this by:

Without Automation

Poor device sharing

No utilization visibility

Time consuming con�guration

Equipment powered up while idle

Ine�cient testing Process: 80% setup time vs. 20% testing time

Driving Test Lab Power Savings with Automation

The lack of automation dramatically increases the amount of time it takes to connect test equipment together and to con�gure test devices so that they’re ready for testing. Sorting through a spaghetti mess of cables and physically connecting the various test components into a testing topology can take hours or even days. A test-engineer may take further hours or days to load the right OS or �rmware versions and con�gure the devices in a large test bed topology. Due to the downside risk of making mistakes during setup and con�guration, equipment is often kept locked down and powered up during that entire period.

And it gets even worse. Delivery of the actual software that must be tested is often delayed. Since the time investment just to prepare for the test is so high, engineers often will simply keep test topologies locked up to avoid the risk of losing access to a critical test component or having their con�gurations changed. As a result, the ratio of time spent in setup and con�guration vs. actual testing can be as lopsided as 80% to 20%. From a power consumption and cost containment point of view, this way of doing business presents a huge opportunity for improvement.

Connecting to, and controlling of, all the equipment in the lab, including all the intelligent Power Distribution Units (PDU’s) in the lab and how the PDUs are connected to each other. This capability enables much better control over power consumption because the lab automation software can easily power on and o� required resources, and even manage power as regularly scheduled provisioning procedures in the lab, or as a resource to manage which devices should be powered on at any given time.

Enabling testers to build and schedule the precise resources they need in a test topology, including the design and activation of their connectivity, provisioning of proper OS and �rmware versions onto devices, and con�gurations of logical parameters and features. This advanced scheduling minimizes setup time and wasted power.

Tracking scheduling and activation indicators of device utilization and test completion providing vital business intelligence reports to help managers and architects continue to re�ne best practices and improve e�ciency.

Automating the responses to power outage conditions so as to set devices to baseline con�gurations and gracefully power them down.

1

2

3

4

do not touch!

systems

Key Resources for Test Lab Automation

A Phased Approach Brings Fast ROI

Test lab software is best combined with some key resources to ensure successful deployment:

Software-based automation bene�ts from a structured, documented, and easy to operate physical connectivity environment. The state-of-the-art practices in test lab automation include deployment of Layer 1 switching to virtually eliminate manual cable patching. Of course, Layer 1 switching should be combined with sound, TIA standards-compliant data center layout and structured cabling, so that the entire physical environment can �ex to changing requirements over time.

The most successful lab automation deployments involve dedicating personnel resources with data architecture and programming skills to build and maintain the object library of inventory resources, test topologies, provisioning and shared testing objects and work�ows. The broader user community can then leverage this library to build and reserve topologies, easily perform provisioning, and progress into test automation as the library is built up. Dedicating resources to maintaining the object library as an infrastructure service is strongly recommended, since if the utility and ease of use of the object library is not maintained at high levels, users will abandon the automation system, wasting the investment.

Test lab automation can be rolled out in phases, so that a positive energy savings return on investment (ROI) can be achieved early in the process, while laying the foundation for additional increases of levels of automation, e�ciency and overall savings.

Generally speaking, achieving “hands-o�” visibility and reservation of lab resources using Layer 1 switching and automation software is the �rst major goal. This level of automation drives the most immediate drop in power consumption because devices can simply be powered down when they are not in use.

The second automation phase is to free testers from the time-consuming tyranny of low-level device provisioning tasks. This involves turning manual provisioning processes into easy to invoke, menu-driven tasks from the automation GUI. The best practice in this stage is to ensure the sustainability of the system, by avoiding a reliance on �xed scripts. While it may be relatively easy to create a �rst set of provisioning scripts for some usage scenarios, the time-consuming nature of script maintenance will too often cause the provisioning capabilities of the system to become out dated.

Driving Test Lab Power Savings with Automation

Test lab Automation

Phase 1 “hands-o�” visibility and reservation of lab resources

Phase 2 free testers from repeated provisioning tasks.

Phase 3 create automated maintenance routines

Phase 4 full test processauto-mation

Highly Automated Physical Layer Connectivity

Resourcing the Automation Infrastructure Service

systems

1

2

3

4

The negative experience of using scripts that don’t work will end up alienating users and deepening their reliance on manual processes. Automation of provisioning tasks typically start with the basic provisioning steps needed to get DUTs to a particular state, such as uploading OS images or applying patches. More advanced provisioning tasks involve common con�guration steps to ready the logical layer of a test topology, such as con�guring VLANs, routing adjacencies or tunnels, on physical, or virtual, switches. These automated provisioning objects help test-engineers more easily accomplish the routine tasks that often dominate their workdays, and allows them to focus more on higher order thinking to achieve maximal test coverage. From a power savings perspective, the time savings associated with automated provisioning tasks means that overall powered-up time of equipment is shortened.

A third phase of test lab automation that is short of full test automation is to create automated maintenance routines. Examples include auto-discovery, which helps keep inventory up to date, and auto base-lining, which returns devices back to their default provisioning states at a pre-determined time. These types of routines require development of a comprehensive set of device-control/interface-automation objects for all necessary devices in the test infrastructure, so that they can be leveraged across multiple maintenance automation processes. From a power savings perspective, automated maintenance ensures that tests are run more accurately, which prevents the need to re-run tests due to con�guration errors.

The fourth phase of test lab automation is full test process-automation. This more advanced form of test lab automation means the creation of a library of reusable testing task-objects. Best practices are to invest the time and programming personnel to develop and maintain the automation object library, and then have non-programmers use GUI tools to create work�ow. This is the only way for test automation to achieve a high penetration of the overall testing process. Organizations that invest in this methodology �nd that they can dramatically increase the speed and coverage of testing. From an energy savings standpoint, test automation can cut down test cycle times dramatically - sometimes by up to 90% - which means that test equipment consumes that much less electricity.

Driving Test Lab Power Savings with Automation

TestShell - The Industry’s Choice for Test Lab AutomationA best-of-breed commercial solution deployed by industry-leading organizations worldwide, TestShell o�ers the fastest path to successful test lab automation, allowing managers and engineers to:

Manage test lab inventory including physical DUT and testing equipment, L1 switches, and virtual resources such as virtual machines and virtual switches in a real-time, searchable database of resource objects. The objects are tagged with searchable attributes, thus eliminating manual searches for

systems

Test automation can cut down test cycle times dramatically - sometimes by

up to 90%

TestShell o�ers the fastest path to successful test lab automation

Time toMarket

PowerSavings

Calendaringof Resources

ControlDevices & PDUs

CentralizedLab Inventory

ON

Green Lab AutomationSoftware

equipment in racks, and allowing engineers to interface with the data center infrastructure e�ciently, via software. TestShell’s inventory and resource management allows for object hierarchies which can represent relatively simple nested resources such as chassis, blades and ports, or complex, pre-integrated resources stacks such as converged infrastructure and “data center in-a-box” solutions.

Populate device inventory objects with power consumption attributes so that optimal power consumption can be considered in the selection process.

Create test topologies via a software GUI that allows dragging and dropping of resource objects onto a canvas, visually ascertain availability, design and sanity-check connectivity. The entire topology can then be saved as a higher level object in the resource library, so that it can be reused later, or by other engineers.

Schedule resources and entire test topologies through a shared calendaring system. Resource con�icts can be easily resolved, since it is easy to �nd out who is using resources at any time. Power consumption can be managed as an attribute against known power resource limitations to prevent the lab from overwhelming the available electrical supply.

Manage connectivity remotely by generating patching or cabling requests to lab administrators, or if Layer 1 switches are in use, to automatically connect test topologies, enabling true ‘hands-o�’ test topology manage-ment.

Make device provisioning error-free by building a library of easily updated automation objects for common provisioning tasks that can be launched from a right click menu in the graphical test topology view. Device provisioning can include uploading OS images, resetting device con�gurations to baselines, or creating routing adjacencies between virtual switches.

Create auto-discovery and auto-base lining processes that leverage TestShell’s extensive array of control interfaces, GUI automation and script-ing capabilities to streamline the management of inventory and device states.

Automated maintenance routines can be created to handle power outage situations. In case of a power outage, TestShell can con�gure devices back to a baseline state and gracefully power them down while Uninterruptible Power Supply (UPS) power is available. This ensures that recovery is as smooth as possible after power is restored.

Roll out full test automation by the creation of an automation object library. This includes the ability to integrate legacy automation scripts as testing objects as well as the creation of new test automation objects through screen, GUI and other capture processes.

Generate real-time reports and dashboards on data center device utilization, topology reservations vs. activations, and even comprehensive test results.

Driving Test Lab Power Savings with Automationsystems

The “Fringe” Bene�ts of Automating Test LabsThe really good news about automating test labs is that not only are there powerful savings to be had in terms of energy costs, but there are also major “fringe” bene�ts:

AutomationBene�ts:

Utilization increase from 15% to 50%

Setup to Testing time ratio down from 80/20, to 50/50

Shorter testing cycle – higher revenues

Capital savings - Software-automated test labs increase the utilization of costly devices from a typical low baseline of 15% to anywhere from 30% to 50%. This utilization increase directly impacts the following year’s capital spending, since greater e�ciency when using existing assets reduces the need to increase capacity. Given the high costs of typical test lab equipment, savings from avoiding purchases along with lower annual write-downs can easily add up to millions of dollars.

Higher productivity - Automation by its nature saves time. Test labs that implement software-based automation of test lab infrastructure alone can drive down setup to testing time ratios from 80/20 to 50/50. Further dramatic gains in e�ciency can be achieved through automated provisioning and full test automation.

Faster time to market or deployment and greater customer responsiveness - Whatever the purpose of the lab, software-based automation makes testing cycles shorter. The result is:

Better coverage and higher service or product quality for bringing products and services to market

Faster �eld error-replication and �x-veri�cation for technical services organizations

Increased sales through higher delivery of lab-based demonstrations and proof of concept tests for customers

Speedier completion of testing in pre-production labs to speed-up deployment of services to IT end-users

Higher revenues by speeding-up turn-around times for testing and validate manufactured products intended for shipment to customers

TestShell automation accompanied by best practices methodologies o�er organizations an opportunity to dramatically save on energy costs, decrease capital spending, and enjoy increased business velocity.

Conclusion

For more information about QualiSystems, visit our website at www.qualisystems.com

Driving Test Lab Power Savings with Automation

WP

-PS

-1

15%50%

systems