“Weather” or not to use GPUs – improving weather forecasts with AI-dedicated HPC

Through a successful collaboration with EuroCC Denmark, researchers at Danish Meteorological Institute (DMI) gained access to high-performance GPUs, enabling them to accelerate the training of their weather models. This article was originally published on the DeiC website.

Image: Irene Kruse (left), Eleni Biola and Simon Christiansen from the Danish Meteorological Institute. Copyright: DeiC

Across Europe, Industry and public entities face the challenge of delivering solutions to society’s most critical problems. At the Danish Meteorological Institute, DMI, providing accurate and timely weather forecasts is essential not only for daily decisions but also for safeguarding communities during extreme weather events and supporting the green transition by optimizing the forecasts of green energy production. Through the support of EuroCC Denmark, DMI was able to access cutting-edge HPC resources and expertise, paving the way for faster, AI-driven weather models and demonstrating the transformative potential of HPC for the public sector.

A shift in weather models – From CPU to GPUs

Traditional weather forecasting has long relied on solving complex differential equations that describe atmospheric dynamics, such as the flow of water and air. These simulations traditionally run on high-performance computing (HPC) systems packed with CPUs (central processing units), which handle the immense computational workload.

In recent years, however, machine learning (ML) has transformed the landscape of weather prediction, offering faster and increasingly accurate forecasts. Unlike traditional numerical models, ML-based models can process massive datasets and generate forecasts in a few minutes rather than hours.

At DMI, the wish to train AI-driven models like LDCast[1] (which is trained on radar data to predict rain) and SHADECast[2] (which is trained on satellite images to predict solar radiation, critical to the energy sector) exemplifies this shift toward cutting-edge technology. These models are based on advanced tools developed by other institutions, but DMI is now adapting and retraining them using Danish data to meet local needs.

The efficiency of these AI models depends heavily on GPUs (Graphics Processing Units), which are specifically designed for the parallel processing required by ML algorithms. GPUs can perform these tasks significantly faster than CPUs, making them essential for scaling AI in weather forecasting.

– We aim to run these AI models alongside our traditional ones for now. Time will tell what the future of weather forecasting looks like, but these models have the potential to be competitive with numerical methods, offering much faster and hopefully equally accurate predictions, says Simon Christiansen, DMI.

Access: 5000 GPU hours

At an AI training workshop for LUMI, hosted by EuroCC Denmark, organized together with LUMI User Support, EuroCC Finland and CSC DMI researchers Irene Livia Kruse, Research Scientist, Machine Learning Scientist Eleni Briola and Machine learning scientist Simon Kamuk Christiansen first met EuroCC Denmark team. As the researchers shared their challenges with accessing GPUs and scaling their models, the EuroCC team recognized the potential impact of a collaboration that could secure DMI the resources and know-how to take their weather models to the next level.

Through its partnership with DeiC as the hosting institution, the EuroCC Denmark is helping DMI upgrade their weather models by providing access to LUMI, one of the most powerful supercomputers in Europe, equipped with GPUs.

– The DeiC support team found us access to GPU hours on LUMI, giving us 5000 GPU hours through the DeiC sandbox, enough to get started on LUMI, and start training a first algorithm, and testing how to scale up on multiple GPUs, says Irene Livia Kruse, DMI.

While CPUs can handle complex tasks, GPUs can more efficiently process thousands of small calculations simultaneously.

HPC Hackathon and Competence Building

To learn how to use LUMI effectively and to make optimal use of the computing power of GPUs, EuroCC DK advised the researchers in consultations and in meetings and ultimately supported them by preparing the organisers of a LUMI hackathon to address the researchers’ challenges. At the hackathon, the DMI team improved their workflows and scaled their training from 1 to 8 GPUs, making the process faster and more efficient. For example, by processing more data at once — doubling the batch size — they reduced the training time per epoch from 72 seconds to 50 seconds, saving both time and energy.

– The LUMI hackathon made a real difference for us. Initially, we struggled to set up the container for our work due to challenges with PyTorch Lightning. With the guidance we received, we became able to use PyTorch Lightning on LUMI effectively, including profiling and scaling across multiple GPUs, says Eleni Briola, DMI.

Impact and future plans

With the right tools and knowledge, DMI is now training AI-based weather models that can deliver faster predictions – for both short-term and long-range forecasting. In the long run, this means real-time weather data, allowing for better preparedness when it comes to both everyday weather and extreme weather events.

– Before connecting with EuroCC2, our access to high-performance GPU resources was limited, which posed significant challenges in advancing our weather models. The support we received—from gaining access to LUMI’s powerful GPUs to the technical guidance at the hackathon — has been instrumental in enabling and optimizing our AI-driven workflows. Without this connection, none of what we’ve achieved so far would have been possible, tells Rune Carbuhn Andersen, Head of Weather Modelling at DMI.

DMI plans to expand their model training by using more GPU hours and collecting data from other countries to improve the generalizability of their models. EuroCC2 and DeiC continue to provide support as DMI works to harness HPC and AI to improve their weather predictions.

1: Leinonen, J., Hamann, U., Nerini, D., Germann, U., & Franch, G. (2023). Latent diffusion models for generative precipitation nowcasting with accurate uncertainty quantification. arXiv preprint arXiv:2304.12891

2: Carpentieri, A., Folini, D., Leinonen, J., & Meyer, A. (2025). Extending intraday solar forecast horizons with deep generative models. Applied Energy, 377(Part A), 124186. https://doi.org/10.1016/j.apenergy.2024.124186

Author: Anne Rahbek-Damm, DeiC