Papers
arxiv:2409.13598

Prithvi WxC: Foundation Model for Weather and Climate

Published on Sep 20
ยท Submitted by akhaliq on Sep 23
#3 Paper of the day
Authors:
,
,
,
,
,
,

Abstract

Triggered by the realization that AI emulators can rival the performance of traditional numerical weather prediction models running on HPC systems, there is now an increasing number of large AI models that address use cases such as forecasting, downscaling, or nowcasting. While the parallel developments in the AI literature focus on foundation models -- models that can be effectively tuned to address multiple, different use cases -- the developments on the weather and climate side largely focus on single-use cases with particular emphasis on mid-range forecasting. We close this gap by introducing Prithvi WxC, a 2.3 billion parameter foundation model developed using 160 variables from the Modern-Era Retrospective Analysis for Research and Applications, Version 2 (MERRA-2). Prithvi WxC employs an encoder-decoder-based architecture, incorporating concepts from various recent transformer models to effectively capture both regional and global dependencies in the input data. The model has been designed to accommodate large token counts to model weather phenomena in different topologies at fine resolutions. Furthermore, it is trained with a mixed objective that combines the paradigms of masked reconstruction with forecasting. We test the model on a set of challenging downstream tasks namely: Autoregressive rollout forecasting, Downscaling, Gravity wave flux parameterization, and Extreme events estimation. The pretrained model with 2.3 billion parameters, along with the associated fine-tuning workflows, has been publicly released as an open-source contribution via Hugging Face.

Community

Paper submitter

This is an automated message from the Librarian Bot. I found the following papers similar to this paper.

The following papers were recommended by the Semantic Scholar API

Please give a thumbs up to this comment if you found it helpful!

If you want recommendations for any Paper on Hugging Face checkout this Space

You can directly ask Librarian Bot for paper recommendations by tagging it in a comment: @librarian-bot recommend

๐ŸŒŽ ๐“๐ก๐ž ๐Ÿ๐ข๐ซ๐ฌ๐ญ ๐ž๐ฏ๐ž๐ซ ๐…๐จ๐ฎ๐ง๐๐š๐ญ๐ข๐จ๐ง ๐ฐ๐ž๐š๐ญ๐ก๐ž๐ซ ๐ฆ๐จ๐๐ž๐ฅ: ๐๐ซ๐ข๐ญ๐ก๐ฏ๐ข ๐–๐ฑ๐‚ ๐ž๐ง๐š๐›๐ฅ๐ž๐ฌ ๐ฅ๐ข๐Ÿ๐ž-๐ฌ๐š๐ฏ๐ข๐ง๐  ๐ฐ๐ž๐š๐ญ๐ก๐ž๐ซ ๐Ÿ๐จ๐ซ๐ž๐œ๐š๐ฌ๐ญ๐ฌ

Hurricane Katrina killed hundreds of people as it made landfall on New Orleans in 2005 - many of these deaths could have been avoided if alerts had been given one day earlier. Accurate weather forecasts are really life-saving.

๐Ÿ”ฅ Now, NASA and IBM just dropped a game-changing new model: the first ever foundation model for weather! This means, it's the first time we have a generalist model not restricted to one task, but able to predict 160 weather variables!

Prithvi WxC (Prithvi, โ€œเคชเฅƒเคฅเฅเคตเฅ€โ€, is the Sanskrit name for Earth) - is a 2.3 billion parameter model, with an architecture close to previous vision transformers like Hiera.
But it comes with some important tweaks: under the hood, Prithvi WxC uses a clever transformer-based architecture with 25 encoder and 5 decoder blocks. It alternates between "local" and "global" attention to capture both regional and global weather patterns.

And boy, does it deliver.

๐—ž๐—ฒ๐˜† ๐—ถ๐—ป๐˜€๐—ถ๐—ด๐—ต๐˜๐˜€:
๐Ÿ”ฎ ๐—ก๐—ฎ๐—ถ๐—น๐˜€ ๐˜€๐—ต๐—ผ๐—ฟ๐˜-๐˜๐—ฒ๐—ฟ๐—บ ๐—ณ๐—ผ๐—ฟ๐—ฒ๐—ฐ๐—ฎ๐˜€๐˜๐˜€ - Prithvi WxC crushed it on 6-12 hour predictions, even outperforming some traditional numerical weather models
๐ŸŒ€ ๐—ง๐—ฟ๐—ฎ๐—ฐ๐—ธ๐˜€ ๐—ต๐˜‚๐—ฟ๐—ฟ๐—ถ๐—ฐ๐—ฎ๐—ป๐—ฒ๐˜€ ๐—น๐—ถ๐—ธ๐—ฒ ๐—ฎ ๐—ฐ๐—ต๐—ฎ๐—บ๐—ฝ - For Hurricane Ida, it predicted the landfall location within 5 km (vs 20+ km errors from other AI models), which is a huge progress!
๐Ÿ” ๐Ÿฒ๐˜… ๐—ฑ๐—ผ๐˜„๐—ป๐˜€๐—ฐ๐—ฎ๐—น๐—ถ๐—ป๐—ด ๐—ฝ๐—ผ๐˜„๐—ฒ๐—ฟ - Can zoom in on weather data to 6x higher resolution with 4x lower error than basic methods
๐ŸŒŠ ๐— ๐—ผ๐—ฑ๐—ฒ๐—น๐˜€ ๐—ฒ๐—น๐˜‚๐˜€๐—ถ๐˜ƒ๐—ฒ ๐—ด๐—ฟ๐—ฎ๐˜ƒ๐—ถ๐˜๐˜† ๐˜„๐—ฎ๐˜ƒ๐—ฒ๐˜€ - Accurately simulates these crucial but hard-to-capture atmospheric oscillations
๐Ÿ˜Ž The coolest part? ๐—ฃ๐—ฟ๐—ถ๐˜๐—ต๐˜ƒ๐—ถ ๐—ช๐˜…๐—– ๐—ถ๐˜€๐—ป'๐˜ ๐—ฎ ๐—ผ๐—ป๐—ฒ-๐˜๐—ฟ๐—ถ๐—ฐ๐—ธ ๐—ฝ๐—ผ๐—ป๐˜†. Its flexible design lets researchers fine-tune it for all kinds of specialized tasks. They've already adapted it for things like detailed regional climate projections, modeling tiny atmospheric gravity waves, and hurricane tracking.

This opens up tons of possibilities for improving climate models, severe weather prediction, and more. As climate change intensifies, tools like Prithvi WxC will become more and more crucial to avoid disasters!
image.png

Paper author

Thanks, @m-ric . ๐—ฃ๐—ฟ๐—ถ๐˜๐—ต๐˜ƒ๐—ถ ๐—ช๐˜…๐—– ๐—ถ๐˜€๐—ป'๐˜ ๐—ฎ ๐—ผ๐—ป๐—ฒ-๐˜๐—ฟ๐—ถ๐—ฐ๐—ธ ๐—ฝ๐—ผ๐—ป๐˜†: one of the key characteristics we wanted to have is structure simplicity. This helps other groups to extend and adapt the model.

Sign up or log in to comment

Models citing this paper 3

Datasets citing this paper 0

No dataset linking this paper

Cite arxiv.org/abs/2409.13598 in a dataset README.md to link it from this page.

Spaces citing this paper 0

No Space linking this paper

Cite arxiv.org/abs/2409.13598 in a Space README.md to link it from this page.

Collections including this paper 9