New high-resolution exascale Earth-modeling system announced for energy

Supercomputing advances will improve analysis of Earth system variability and energy sector needs

April 23, 2018

The high-resolution E3SM earth system model simulates the strongest storms with surface winds exceeding 150 mph—hurricanes that leave cold wakes that are 2 to 4 degrees Celsius cooler than their surroundings. This simulation from E3SM represents how sea surface temperature changes evolve as a hurricane (seen here approaching the U.S. East Coast) moves across the Atlantic and how the resultant cold wake affects subsequent intensification of the next hurricane.

A new Earth-modeling system unveiled today will have weather-scale resolution and use advanced computers to simulate aspects of Earth’s variability and anticipate decadal changes that will critically impact the U.S. energy sector in coming years.

This multilaboratory effort will be a huge advance in our already important capabilities for Earth-systems modeling and energy-related analysis,” said John Sarrao, Principal Associate Director for Science, Technology and Engineering at Los Alamos National Laboratory. “Our laboratory, along with our sister institutions, has made significant contributions to Earth-systems modeling over the previous decades, but this latest contribution takes our work to an entirely new level.”

After four years of development, the Energy Exascale Earth System Model (E3SM) will be released to the broader scientific community this month. The E3SM project is supported by the Department of Energy's Office of Science in the Biological and Environmental Research Office. The E3SM release will include model code and documentation, as well as output from an initial set of benchmark simulations.

The Earth, with its myriad interactions of atmosphere, oceans, land and ice components, presents an extraordinarily complex system for investigation. Earth-system simulation involves solving approximations of physical, chemical and biological governing equations on spatial grids at resolutions that are as fine in scale as computing resources will allow.

“E3SM is the first end-to-end multi-scale Earth system model, meaning that we can focus model resolution and computer resources toward specific locations to help answer specific questions that are important to the DOE,” said researcher Steve Price of Los Alamos. “For example, Los Alamos is using E3SM with focused resolution around Antarctica to improve the modeling of how ocean waters melt ice shelves — the critical process that controls the likelihood of abrupt sea-level rise.”

The E3SM project will reliably simulate aspects of earth system variability and project decadal changes that will critically impact the U.S. energy sector in the near future. These critical factors include a) regional air/water temperatures, which can strain energy grids; b) water availability, which affects power plant operations; c) extreme water-cycle events (e.g. floods and droughts), which impact infrastructure and bio-energy; and d) sea-level rise and coastal flooding which threaten coastal infrastructure.

“There is no shortage of important problems that we can tackle with this new modeling capability,” said Todd Ringler, also of Los Alamos. “Take the Arctic, for example. It is changing rapidly — and this presents new opportunities and new security risks. This new modeling capability — particularly the new approaches that we developed for the ocean and sea-ice systems — will be critical for predicting the how, when and why of the changing Arctic.”

The goal of the project is to develop an earth system model (ESM) that has not been possible because of limitations in current computing technologies. Meeting this goal will require advances on three frontiers:

  1. better resolving earth system processes through a strategic combination of developing new processes in the model, increased model resolution and enhanced computational performance;
  2. representing more realistically the two-way interactions between human activities and natural processes, especially where these interactions affect U.S. energy needs; and
  3. ensemble modeling to quantify uncertainty of model simulations and projections.

“The quality and quantity of observations really makes us constrain the models,” said David Bader, Lawrence Livermore National Laboratory scientist and lead of the E3SM project. “With the new system, we’ll be able to more realistically simulate the present, which gives us more confidence to simulate the future.”

Simulating atmospheric and oceanic fluid dynamics with fine spatial resolution is especially challenging for ESMs. The E3SM project is positioned on the forefront of this research challenge, acting on behalf of an international ESM effort. Increasing the number of earth-system days simulated per day of computing time is a prerequisite for achieving the E3SM project goal. It also is important for E3SM to effectively use the diverse computer architectures that the DOE Advanced Scientific Computing Research (ASCR) Office procures to be prepared for the uncertain future of next-generation machines.

A long-term aim of the E3SM project is to use exascale machines to be procured over the next five years. The development of the E3SM is proceeding in tandem with the Exascale Computing Initiative (ECI). (An exascale refers to a computing system capable of carrying out a billion billion (109 x 109 = 1018) calculations per second. This represents a thousand-fold increase in performance over that of the most advanced computers from a decade ago).

“We are particularly interested in accurately assessing the risk of abrupt sea-level rise, say more than 3 feet, sometime during this century,” said Ringler. “To accomplish this, Los Alamos constructed entirely new computer models of the ocean, land-ice and sea-ice systems -- this is a huge accomplishment by the Los Alamos modeling team.”

“This model adds a much more complete representation between interactions of the energy system and the earth system,” Bader said. “The increase in computing power allows us to add more detail to processes and interactions that results in more accurate and useful simulations than previous models.”

To address the diverse critical factors impacting the U.S. energy sector, the E3SM project is dedicated to answering three overarching scientific questions that drive its numerical experimentation initiatives:

  • Water Cycle: How does the hydrological cycle interact with the rest of the human-Earth system on local to global scales to determine water availability and water cycle extremes?
  • Biogeochemistry: How do biogeochemical cycles interact with other Earth system components to influence the energy sector?
  • Cryosphere Systems: How do rapid changes in cryosphere (continental and ocean ice) systems evolve with the Earth system and contribute to sea-level rise and increased coastal vulnerability?

In the E3SM, all model components (atmosphere, ocean, land, ice) are able to employ variable resolution to focus computing power on fine-scale processes in regions of particular interest. This is implemented using advanced mesh-designs that smoothly taper the grid-scale from the coarser outer region to the more refined region.

The E3SM project includes more than 100 scientists and software engineers at multiple DOE Laboratories as well as several universities; the DOE laboratories include Argonne, Brookhaven, Lawrence Livermore, Lawrence Berkeley, Los Alamos, Oak Ridge, Pacific Northwest and Sandia national laboratories. In recognition of unifying the DOE earth system modeling community to perform high-resolution coupled simulations, the E3SM executive committee was awarded the Secretary of Energy's Achievement Award in 2015.

In addition, the E3SM project also benefits from DOE programmatic collaborations including the Exascale Computing Project (ECP) and programs in Scientific Discovery through Advanced Computing (SciDAC), Climate Model Development and Validation (CMDV), Atmospheric Radiation Measurement (ARM), Program for Climate Model Diagnosis and Intercomparison (PCMDI), International Land Model Benchmarking Project (iLAMB), Community Earth System Model (CESM) and Next Generation Ecosystem Experiments (NGEE) for the Arctic and the Tropics.

For more information, see the E3SM website.

About Los Alamos National Laboratory

Los Alamos National Laboratory, a multidisciplinary research institution engaged in strategic science on behalf of national security, is managed by Triad, a public service oriented, national security science organization equally owned by its three founding members: Battelle Memorial Institute (Battelle), the Texas A&M University System (TAMUS), and the Regents of the University of California (UC) for the Department of Energy’s National Nuclear Security Administration.

Los Alamos enhances national security by ensuring the safety and reliability of the U.S. nuclear stockpile, developing technologies to reduce threats from weapons of mass destruction, and solving problems related to energy, environment, infrastructure, health, and global security concerns.