Solutions at a glanceBusiness needs

Business results

Supercomputing Wales | Academic and Industrial Research

United Kingdom

Supercomputing Wales needed a high-performance computing resource to empower the work of researchers across the nation of Wales.

• Two supercomputer hubs with: - Dell EMC PowerEdge R640, R740 and C6420 servers with Intel® Xeon® processors - Dell EMC Networking S3048 - Dell EMC PowerVault MD3420 storage

• Atos® BullSequana S Datalake appliances • Atos Extreme Factory for supercomputing simulation

and services

• Fuelling research breakthroughs

• Creating highly skilled research jobs

• Helping Wales capture more research funding

Drawing on technologies and expertise from Atos and Dell EMC, Supercomputing Wales delivers leading-edge compute facilities to researchers across Wales.

WORLD-CLASS RESEARCH

13,080 coresdelivering 1 petaflop

The HPC facilities comprise two hubs with

>2 petabytesof total storage

Connected to

A national mission Mangrove characterisation and monitoring. New solutions for electrical steel coatings in electrical transformers. Pushing the boundaries of boating speed and stability. Bioinformatic insights to pH. This is just a small sample of the diverse scientific investigations powered by the high-performance computing (HPC) clusters of Supercomputing Wales.

Supercomputing Wales is a strategic programme of investment in the university sector in Wales that is changing the way supercomputing facilities are used to support research activities. The programme is led by Cardiff University, in a consortium with Aberystwyth, Bangor and Swansea universities.

Supercomputing Wales provides researchers across Wales with access to powerful computing facilities for science and innovation projects. The programme’s facilities are used by research groups at Cardiff, Swansea, Bangor and Aberystwyth along with companies and other partners working on collaborative projects.

At a higher level, Supercomputing Wales is helping the nation capture more research funding, increase scientific partnerships, create highly-skilled research jobs and support collaborations among research partners. Funded in part by the European Regional Development Fund through the Welsh Government, the programme supports Wales in its efforts to compete globally for research and innovation that requires state-of-the-art computing facilities to simulate and solve complex scientific problems.

With its high-powered HPC resources, Supercomputing Wales is facilitating a step change in supercomputing activity across strategically important sectors of the Welsh economy — including nano-scale materials and advanced engineering; energy and the environment; and life sciences and health.

A Centre of ExcellenceAt the heart of the Supercomputing Wales initiative is a Supercomputing Centre of Excellence spearheaded by two global leaders in digital transformation, Atos and Dell EMC.

The joint Atos Dell EMC Supercomputing Centre of Excellence provides Welsh researchers with a full suite of leading-edge HPC equipment, software and services. The centre provides two supercomputer systems, two Atos BullSequana S Datalake appliances, and on-demand cloud-based supercomputing simulation and services powered by Extreme Factory, HPC on-demand from Atos.

The Centre also offers community benefit provisions from Atos and its partners, including in-depth technology workshops for staff and guest lecturers by Atos scientific community experts.

In addition, Supercomputing Wales employs Research Software Engineers (RSEs), some embedded in specific domain research groups and some working across multiple domains. In tandem, they work to develop algorithms and customised software that harnesses the power of the supercomputers to perform multiple computational tasks simultaneously at high speeds.

Two supercomputer hubsThe supercomputer hubs under Supercomputing Wales are based at Cardiff and Swansea universities, with research teams across the consortium universities accessing the hubs. These hubs contain more than 13,000 cores, tens of terabytes of memory and hundreds of terabytes of high-performance storage, all interconnected by low-latency / high-bandwidth networking.

Table 1 shows the specifications for the supercomputing hubs, named Hawk (based at Cardiff University) and Sunbird (located at Swansea University).

Cardiff Hub – Hawk

Swansea Hub – Sunbird

Key specifications

• 201 nodes• 8,040 cores• 46 TB of total

memory

• 126 nodes• 5,040 cores• 48.4 TB of total

memory

Compute nodes

• CPU: 2x Intel® Xeon® Gold 6148 CPU @ 2.40GHz with 20 cores each

• 26 nodes with 384 GB to support memory-intensive applications

• CPU: 2x Intel® Xeon® Gold 6148 CPU @ 2.40GHz with 20 cores each

• RAM: 384 GB per node - 4 nodes have dual NVIDIA V100 GPUs

Storage

• 692 TB (usable) scratch space on a high-performance parallel Lustre file system

• 420 TB of home directory space

• 808 TB (usable) scratch space on a high-performance parallel Lustre file system

• 231 TB of home directory space on a high-performance parallel Lustre file system

Interconnect• Mellanox EDR 100Gb/s

InfiniBand compute and storage network

• Mellanox EDR 100Gb/s InfiniBand compute and storage network

“ A typical simulation uses 450 compute cores running for 72 hours to get about .5 seconds of physically simulated time. It would take years on a desktop.”Dr. Edward Bennett Research Software Engineer based at Swansea

Fuelling ground-breaking researchSupercomputing Wales supports research across a wide range of academic disciplines, from traditional science, technology, engineering and mathematics fields to environmental, e-science, health, medical and social sciences.

Here are a few examples of the ground-breaking research initiatives that are capitalising on the computational resources of Supercomputing Wales.

Mangrove characterisation and monitoringAt Aberystwyth University, Dr. Pete Bunting and colleagues are creating baseline maps of mangrove forests, which are important players in maintaining the health of coastal ecosystems in the tropics and subtropics. Mangroves grow in a saline environment in the boundary between the ocean and the land.

“There’s a whole bunch of ecosystem services that mangroves provide,” Bunting says. “To try to replace that with sea defences would obviously be billions of dollars, so they have an intrinsic monetary value in terms of the services they provide.”

Mangroves face threats on multiple fronts. Many mangrove forests have been cleared to make way for human-driven uses, such as agricultural ponds and urban expansion. They are also threatened by natural or indirect anthropogenic events or processes, including sea-level fluctuations. To create a baseline for historical and future comparisons, the research team leveraged satellite datasets from the Japanese Space Exploration Agency (JAXA) and the U.S. National Aeronautics and Space Administration (NASA). To put the data to work, the team wrote a special software program and trained an algorithm to identify mangroves in satellite images.

The team plans to makes its baseline dataset, showing the state of mangroves in 2010, publicly available via the United Nations Ocean Data Viewer. The accessibility of this data will help research teams and governments

around the world characterise mangroves and detect changes in their extent from established baselines — which is one of the keys to protecting an invaluable natural resource.

Steel coatingsAt Cardiff University, Dr. Sachin Nanavati and doctoral student Lee Edwards are using the resources of Supercomputing Wales in their search for a new generation of coatings for the types of steel used in electrical transformers. For this work, the researchers are teamed with an industrial partner in Wales, a country with deep roots in the steel industry.

If you’ve seen the movie Erin Brockovich, you already know a little bit of the backstory. The movie highlighted the dangers of chromium, a carcinogenic chemical element, in drinking water. Chromium, which is highly resistant to corrosion, is widely used in coatings for electrical steel.

“Once we understand more about the properties and role of chromium in aluminium phosphates, maybe in the future we could replace it with an environmentally friendly, non-toxic material that reduces power losses,” Nanavati says. “We use supercomputers to simulate the properties of these materials.”

“We’re not going to be using chromium in the future,” Edwards adds. “We need to reduce power losses in the transformer and do it in a way that’s not toxic. These are two problems, and we need to solve both of them.”

Solving these problems, and understanding the underlying theory behind the solutions, requires more than experimentation, the researchers point out. It also requires computationally complex simulations that demand the power and scale of a supercomputer — like those operated by Supercomputing Wales.

Share this storyLearn more about

supercomputing.walesView customer stories at

DellEMC.com/hpc

Speed boatsWith choppy water, high winds and other environmental factors, often companies cannot test designs without destroying boats. So, companies like Norson Design work with a team of researchers at Swansea University who can test the boundaries between water and air.

The company provides CAD models, geometry and boat design. The research team then virtually puts the boat in wind-tunnels and applies different forces to the boat using customised computational fluid dynamics to determine the best possible mix of performance and stability.

“A typical simulation uses 450 compute cores running for 72 hours to get about .5 seconds of physically simulated time. It would take years on a desktop,” says Dr. Edward Bennett, a Research Software Engineer based at Swansea.

“With supercomputing resources, it’s exciting to see advances in the marine industry. We are working on radical boat designs based on hydrodynamics and aerodynamics, carefully optimising the hull to help create the world’s fastest boat, and we’re very excited to see how it’s going to turn out.”

Bioinformatic insights to pHOnce the largest copper mine in Europe, Parys Mountain is currently the largest provider of zinc and copper to the Irish Sea. At Bangor University, a team of researchers is studying microorganisms including bacteria that thrive in the acid mine drainage.

One of the aims of this research is the study of extreme environments, i.e., those where the environmental conditions (temperature, pH, etc) can be tolerated only by a small range of living beings, mostly microorganisms from Bacteria and Archaea domains.

The identification, classification and study of this microbial community may help shed some light on ways bacteria and acid tolerance play a role in areas such as dental plaque, intestinal colonisation and food-borne pathogens.

The developed computational pipeline included the use of bioinformatics tools like ClustalW and in-house scripts coded in R and Perl to perform the multiple alignment, phylogenetic tree and bootstraping of the 16S rRNA sequences. To accelerate the computational process and calculations, all of these operations have been performed on the Supercomputing Wales systems.

Key takeawaysWith the HPC systems and expertise of Supercomputing Wales and the Atos Dell EMC Supercomputing Centre of Excellence, researchers across Wales have access to the resources they need to simulate and solve complex scientific problems — and keep the nation of Wales at the forefront of theoretical science and technical innovation.

As Professor Roger Whitaker, Academic Director for Supercomputing Wales, notes, “Supercomputing Wales, with its supercomputing hub facilities and new group of Research Software Engineers, is a unique national technological asset that is crucial to underpin our scientific excellence in Wales.”1

1 Atos news release, “Atos wins deal to establish supercomputing Centre of Excellence in Wales,” June 21 2018.

Copyright © 2018 Dell Inc. or its subsidiaries. All Rights Reserved. Dell, EMC, and other trademarks are trademarks of Dell Inc. or its subsidiaries. Intel, Xeon and the Intel logo are trademarks of Intel Corporation in the U.S. and/or other countries. Other trademarks may be trademarks of their respective owners. This case study is for informational purposes only. The contents and positions of staff mentioned in this case study were accurate at the point of the interview conducted in October 2018. Dell and EMC make no warranties — express or implied — in this case study.