Business need

The NCSA Private Sector Program

sought to build a readily accessible,

high-performance computing (HPC)

system to help manufacturing and

engineering companies increase their

competitiveness.

Solution

The organization manages several

supercomputing resources, including

the iForge HPC cluster based on

Dell and Intel technologies, to help

solve industry’s most demanding

engineering and science problems.

Benefits• Helps industry customers solve

finite element analysis and

computational fluid dynamics

problems with its HPC cluster

• Large memory nodes perform

53 percent faster on demanding

simulations

• Provides benchmarking so users

can predict performance and

scalability benefits of larger

software licenses

• Maintains high reliability and

availability

• Performs live upgrades without

disrupting partner projects

Solutions at a glance• High-Performance Computing

• Server

Satisfying industry’s appetite for performance and insightThe NCSA Private Sector Program creates a high-performance computing cluster to help corporations overcome critical challenges

Customer profile

Company National Center for

Supercomputing

Applications (NCSA)

Industry Manufacturing

Country United States

Employees 200

Website ncsa.illinois.edu

“iForge is the most reliable HPC system NCSA has ever had, and a big part of that is the technologies we’re getting from Dell and Intel. That gives us the confidence to upgrade faster than we would otherwise.” Evan Burness, Program Manager for the Private Sector Program National Center for Supercomputing Applications

2

“If you think about all the simulations that a large company’s fleet of engineers might run in a year, the time savings that iForge delivers really adds up.” Merle Giles, Director of Private Sector Programs and Economic Impact, National Center for Supercomputing Applications

“NCSA began on a platform of science and industry,” says Merle Giles, the director of private sector programs and economic impact at NCSA. “That was our legacy, and we pushed it even further to create the Private Sector Program.”

Through its Private Sector Program (PSP), NCSA has provided supercomputing, consulting, research, prototyping and development, and production services to more than one-third of the Fortune 50, in manufacturing, oil and gas, finance, retail/wholesale, bio/medical, life sciences, technology and other sectors. “We’re not the typical university supercomputer center, and PSP isn’t a typical group,” Giles says. “Our focus is on helping companies leverage high-performance computing in ways that make them more competitive.”

Over time, PSP customers have become increasingly interested in using HPC to solve more of their most critical business problems. Very few companies, however, possess a full complement of HPC systems and expertise. “The return on investment from HPC is something every company struggles with in one way or another,” says Evan Burness, program manager for the PSP. “The key is to demonstrate the ROI of HPC across all phases of a product design cycle.” But not many supercomputing centers are organized and funded well enough to do so. “If you look at the history of government-funded open science supercomputing, it’s very much focused on basic research,” says Burness. “That can limit the kind of work that industry can do with a place like NCSA.”

Eventually, the Private Sector Program decided it needed to have its own supercomputer. “We needed an HPC system built purely around the interests, challenges, requirements and desires of industry,” Burness says. “That was going to be the only way we could appropriately serve this community.”

As it began the design process, the PSP knew that reliability would be a key factor. “The world’s biggest supercomputers can do things few other systems can do, but high uptime is usually not part of the mission,” says Giles. “And if corporations don’t have the trust in that system because it’s not reliable, they tend to avoid it.” The organization also needed to address the aforementioned ROI challenges. “Commercial engineering software is extremely expensive, and it’s difficult to quantify in advance what the return on investment will be for altering or increasing your license,” states Burness. “We realized that there was nobody out there acting like an independent third party to help facilitate that information-gathering and decision-making process

When the National Center for Supercomputing Applications

(NCSA) was created at the University of Illinois 27 years ago, it

had a unique proposition—its computing, data and networking

resources were designed for industry as well as academia.

Over the years, NCSA’s efforts to serve industry have grown

and matured.

Products & Services

Hardware

Dell PowerEdge C8000 Series Chassis

Dell PowerEdge C8220 Compute Nodes featuring 10-core Intel® “Ivy Bridge” Xeon E5 2680 v2 processors

3

for both industry users and independent software vendors.”

NCSA Private Sector Program creates powerful HPC cluster for industryThe “A” in NCSA stands for “Applications,” which are exactly what the PSP focused on in making its final design decisions. “We designed every aspect of the system around the needs of widely used engineering applications, and we had a keen sense of what architectural aspects would cause performance on those applications to go up or down,” says Burness.

Using that expertise, in 2011 NCSA created “iForge,” an HPC cluster built to address the needs of power users from some of the nation’s leading engineering companies. The solution consisted of three distinct hardware platforms, each configured for a different computational need.

But whereas many HPC systems maintain the same configuration until all technologies are simply too outdated, the PSP wanted to embark on a highly aggressive upgrade strategy starting in 2012. “Newer technologies come out every year that can accelerate these applications, so to help our industry partners stay on the cutting edge, iForge has to incorporate those advancements as or before they come to market,” says Burness. Only a year after iForge’s debut, the Private Sector Program swapped out all of its blade servers for newer models. When completed, iForge featured 128 PowerEdge M620 2-socket servers featuring 8-core Intel® “Sandy Bridge” Xeon E5 2670 processors and 128 GB of RAM.

And early in 2014, the PSP performed a full system upgrade yet again. Today, iForge features 144 Dell PowerEdge C8220 2-socket compute nodes featuring 10-core Intel® “Ivy Bridge” Xeon E5 2680 v2 processors, all housed inside 18 Dell PowerEdge C8000 Series chassis. During this most recent upgrade, NCSA also diversified its memory-per-

server offerings. “Computational fluid dynamics (CFD) simulations typically need less memory than those of a finite element analysis (FEA) domain,” explains Burness. “So we reduced the amount of memory in our nodes for CFD, and doubled our investment in the memory in our nodes targeting FEA applications.”

Meeting customers’ needs for performance and efficiencyThe newest upgrade also gave the PSP the ability to better meet customers’ needs. “With our most recent upgrade to the Dell and Intel configuration, we actually saw a 25 percent improvement in network throughput. We get faster memory and more memory in a node, so we can handle industry’s most demanding finite element analysis and computational fluid dynamics applications,” says Burness.

With this latest upgrade, iForge now offers a complementary presence to NCSA’s “Blue Waters” supercomputer, one of the world’s largest and most powerful HPC environments. “iForge is designed for optimal performance on moderate- and large-scale workloads from industry, whereas Blue Waters really targets the most extreme-scaling science and engineering applications on the planet,” says Giles. “iForge allows us to say we’re serving the full spectrum of industry’s HPC needs.”

Indeed, iForge is impressive in its performance and efficiency. Running Dassault Systèmes’ Abaqus, one of the most widely used and powerful FEA applications, iForge’s large memory nodes perform 53 percent faster on demanding simulations compared with the system’s previous Sandy Bridge nodes. CFD applications have also seen a healthy performance boost of 22 percent compared with previous benchmark highs. “These are big jumps not just in performance, but in the efficient use of software licenses,” says Burness. “If you think about all the simulations that a large company’s fleet of engineers might run in a year, the

“We ask ourselves, ‘will this help companies perform their work faster and better?’ We have to enable that, because industry has an insatiable appetite for performance and insight.” Evan Burness, Program Manager for the Private Sector Program, National Center for Supercomputing Applications

4

time savings that iForge delivers really adds up,” adds Giles.

The latest incarnation of iForge has also begun to field a larger number of requests from other industrial sectors. These include genomic sequencing applications for bioinformatics, reservoir simulation for oil and gas, and a range of other industrial applications. In the process, the PSP is learning more and more about how insights from one industrial sector can benefit others. “Bioinformatics workflows are incredibly complex,” notes Giles. “Learning about how companies manage that challenge has made us better at tackling the workflows of other sectors.”

iForge has also caught the attention of commercial software vendors. “We offer a collaboration model whereby software companies contribute licenses, our partners contribute real-world engineering problems, and we perform the work to show how well those problems will perform with the applications as they use larger and larger fractions of a supercomputer,” says Giles.

And while the PSP offers a variety of services to industry, the organization is not a “cloud” in the conventional sense. “Public clouds are great for the markets they’re designed to serve,” says Burness. “But from a performance and access model standpoint, public clouds are inherently not a good fit for the toughest HPC challenges from industry.” says Burness. “For example, high-fidelity, complex multi-physics simulations of a jet engine have much higher demands than serving web pages or file servers. The modeling and simulation challenges that companies bring to NCSA require an entirely different kind of computer.”

iForge helps corporations solve complex problems fasterAccording to Giles, every design decision and upgrade to iForge has come down to one essential question. “We ask ourselves, ‘will this help companies perform their work faster and

better?’ We have to enable that, because industry has an insatiable appetite for performance and insight.” That appetite isn’t without purpose, either. “Most large companies doing HPC will tell you they are still very far away from being able to simulate conditions that capture the full physics, resolution and time scale they desire,” he further explains. “When you’re capturing everything a machine is doing as it moves from one nanosecond to the next, simulating just a few seconds of time is computationally very, very difficult. Now imagine you want to simulate the entire multi-year life of the product!”

Business model enables flexible testing and a better understanding of ROI With iForge, the NCSA PSP has an HPC cluster that corporations can use to test and deploy their mission-critical applications on new and various hardware platforms. By doing so, these corporations can better understand the ROI of upgrading or replacing their own HPC systems, without having to set aside these production-focused resources for experimental use. “We can use iForge to show our users what performance and scalability gains would look like under a variety of conditions, so they can make more informed decisions in terms of how they use and invest in their own HPC systems,” says Giles.

Meeting industry’s demand for reliability, handling live upgrades without disruptions Customers taking advantage of iForge get access to a highly reliable computing environment to support their work. “Users from industry need to focus on their applications and the insight they get from them. Our challenge at NCSA is to design, operate and support iForge in such a way that the users can have that clarity of focus,” says Burness. “Sometimes, that process is extremely difficult.”

Burness points to the fact that the PSP does not take system downtime to upgrade iForge, despite the enormous

“With our most recent upgrade to the Dell and Intel configuration, we actually saw a 25 percent improvement in network throughput. We get faster memory and more memory in a node, so we can handle industry’s most demanding finite element analysis and computational fluid dynamics applications.” Evan Burness, Program Manager for the Private Sector Program, National Center for Supercomputing Applications

5

Dell, the Dell logo and PowerEdge are trademarks of Dell Inc. AMD and Opteron are registered trademarks or trademarks of Advanced Micro Devices, Inc. Intel and Xeon are registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. Dell disclaims any proprietary interest in the marks and names of others. Availability and terms of Dell Software, Solutions and Services vary by region. This case study is for informational purposes only. Dell makes no warranties – express or implied—in this case study. Reference Number: 10013433 © July 2014 Dell Inc. All Rights Reserved

View all Dell case studies at Dell.com/CustomerStories

complexity of these systems. This practice is virtually unheard of in the world of supercomputing, where is it far more common to take an HPC system offline for hours, days or even weeks to perform such work. For example, when it came time for the upgrade to the newest iForge configuration, the PSP had to balance migrating systems with allowing customers to continue their HPC work. “We started testing Ivy Bridge many months before it was released through companies like Dell, first remotely on a system hosted by Intel and later in the form of demo servers Intel sent us,” says Burness. “This allowed us to identify the nuances of the new architecture, which in turn helped us ask the right questions to Dell in terms of how to proceed with a live upgrade.”

The Private Sector Program has grown accustomed to annual upgrades, and it offers credit to the technologies that it is working with. “iForge is the most reliable HPC system NCSA has ever had, and a big part of that is the technologies we’re getting from Dell and Intel. That gives us the confidence to upgrade faster than we would otherwise,” says Burness.

Giles notes that while high reliability can be an afterthought in the world of HPC, it’s essential in the world of business. “With iForge, we built a system that’s high performing and has greater than 99 percent uptime, and we package that with high-end expertise to help our user communities do what they do better.”

“We can use the iForge cluster to show our users what performance and scalability gains would look like, so they can have a clear idea of what they’d be getting if they upgraded their licenses.” Merle Giles, Director of Private Sector Programs and Economic Impact, National Center for Supercomputing Applications

For more information go to www.DellHPCSolutions.com