LUMI again among the fastest and greenest supercomputers in the world

EuroHPC Joint Undertaking’s LUMI supercomputer reached the third spot on the new Top500 list of world’s fastest supercomputers released at the SC22 conference in Dallas, USA, on 14 November 2022. LUMI reached a measured High-Performance Linpack (HPL) performance of 309 petaflops on this biannual list. This makes LUMI the fastest supercomputer in Europe.

LUMI is a unique endeavour thanks to its scale, sustainability and pan-European nature, hosted by a consortium of ten countries.

– To be able to reach the very top spots of the Top500 ranking is a remarkable achievement and proves the power of European cooperation. By joining forces, the LUMI consortium, in cooperation with EuroHPC JU, has managed to acquire the kind of state-of-the-art device to support its RDI efforts none of us in the ten consortium countries could have achieved on our own, says Kimmo Koski, Managing Director of CSC – IT Center for Science, Finland, on behalf of the LUMI consortium.

– We are delighted about LUMI’s position in the ranking, and we are even more excited about seeing the societal impact that this extremely powerful platform will have, by making it possible to tackle next-level computational challenges in all fields of research, involving larger datasets than most European researchers have ever been able to use before. LUMI will advance digitalization across the whole society, Koski continues.

– I am very pleased to see LUMI on the Top500 podium. With this remarkable machine, we can really see the results of European collaboration at work. LUMI’s exceptional technologies will bolster European competitiveness and innovation, and further support Europe’s digital sovereignty, says Anders Dam Jensen, Executive Director of the EuroHPC Joint Undertaking.

The LUMI system is supplied by Hewlett Packard Enterprise (HPE), based on an HPE Cray EX supercomputer.

– We are proud to deliver LUMI, Europe’s fastest supercomputer, in partnership with the LUMI consortium, CSC, EuroHPC JU, and AMD, says Justin Hotard, executive vice president and general manager, HPC, AI & Labs, HPE.

– LUMI is Europe’s first supercomputer that integrates exascale technology with advanced developer tools, software, accelerators, and interconnect. This will allow users to develop richer models and simulations, and integrate AI-at-scale, to deliver new insights. LUMI will advance critical research in areas such as climate modeling, medicine, and new materials to strengthen European competitiveness and sovereignty, Hotard continues.

LUMI’s second pilot phase for nearly 30 research projects started in November 2022. The academic fields of the pilot projects are diverse ranging from astrophysics to climate and cancer research, but also disciplines that have recently started to increase their use of computational methods are represented – such as natural language processing. After the second pilot phase the system is generally available for users across Europe.

– As LUMI’s HPL figure keeps ramping up and we also received podium places on other complementing rankings like HPCG and HPL-AI, we have not just yet seen its full potential. However, we have already seen glimpses of its potential and impressive capability in real-world scientific workloads during the system pilot use. The system is now ready to serve European scientific discovery and innovation, says Pekka Manninen, Director of LUMI, CSC – IT Center for Science, Finland.

Super sustainable supercomputing

LUMI supercomputer reached the seventh spot on the new Green500 list which biannually ranks supercomputers from the Top500 list in terms of energy efficiency. The GFlops/Watts ratio for LUMI is 51.38, making it one of the greenest supercomputers in the world.

– I am not only pleased with LUMI’s achievements for its computing power, but also its result on the Green500 list. It is ever more important to consider energy efficiency and sustainability in supercomputing, and I am very proud that LUMI can be an example of eco-conscious computing allied with top-of-the-range technology and world-leading supercomputing power, adds Jensen.

CSC’s data center in Kajaani, Finland, where LUMI is placed, is one of the world’s most eco-efficient data centers. LUMI uses 100% carbon-free hydroelectric energy for its operations, and its waste heat can be reused to heat up the surrounding houses in city of Kajaani. Thanks to the cool climate of the region very little energy is needed to keep the supercomputer cool. Thus, LUMI’s energy-efficiency is in a league of its own.

– The LUMI consortium is extremely proud to run a supercomputer with such a small carbon footprint. We are also keen to see what LUMI can achieve on the handprint side, i.e., how it can contribute to RDI efforts looking for solutions to climate challenges. One promising track are the digital twins of the Earth that will use LUMI’s remarkable computing power to model, monitor and simulate the impact that different natural phenomena and human activities will have on our planet, Koski complements.

List complementing the HPL measurement

On the new HPCG list (High-Performance Conjugate Gradient) LUMI was ranked number three. The HPCG benchmark provides an alternative metric for assessing supercomputer performance and is meant to complement the HPL measurement.

In addition, LUMI was listed in the HPL-AI benchmark as number two with its 2.168 exaflops performance. The HPL-AI benchmark probes the emerging convergence of high-performance computing (HPC) and artificial intelligence (AI) workloads, which was one of the design goals of LUMI.

LUMI’s architecture

The full system architecture of LUMI is the following:
• The LUMI system is supplied by Hewlett Packard Enterprise (HPE), based on an HPE Cray EX supercomputer.
• The GPU partition consists of 2560 nodes, each node with one 64 core AMD Trento CPU and four AMD MI250X GPUs.
• Each GPU node features four 200 Gbit/s network interconnect cards, i.e. has 800 Gbit/s injection bandwidth.
• The committed Linpack performance of LUMI-G in its final configuration is 375 Pflop/s.
• The MI250X GPU comes with a total of 128 GB of HBM2e memory offering over 3.2 TB/s of memory bandwidth.
• A single MI250X card is capable of delivering 42.2 TFLOP/s of performance in the HPL benchmarks. More in-depth performance results for the card can be found on AMD’s website.
• In addition to the GPUs in LUMI there is another partition (LUMI-C) using CPU only nodes, featuring 64-core 3rd-generation AMD EPYC™ CPUs, and between 256 GB and 1024 GB of memory. There are 1,536 dual-socket CPU nodes in total.
• LUMI also has a partition with large memory nodes, with a total of 32 TB of memory in the partition.
• For visualization workloads LUMI has 64 Nvidia A40 GPUs.
• LUMI’s storage system, based on the Cray Clusterstor E1000 storage system from HPE, consists of three components. First, there is a 8 petabyte all flash Lustre system for short term fast access. Next, there is a longer term more traditional 80 petabyte Lustre system based on mechanical hard drives.
• For easy data sharing and project lifetime storage LUMI has 30 petabytes of Ceph based storage.
• All the different compute and storage partitions are connected to the very fast HPE Slingshot interconnect of 200 Gbit/s.
• LUMI takes nearly 300m2 of space, which is about the size of two tennis courts. The weight of the system is nearly 150 000 kilograms (150 metric tons).

Image from the Top500 award ceremony at SC22 conference. Image: CSC

More information:

Media gallery: www.lumi-supercomputer.eu/media

Interview requests: media@lumi-supercomputer.eu

Image on top: Fade Creative