When performing numerical weather predictions, it is important that the simulation itself be accurate, but it is also key for real-world data, based on observations, to be accurately entered into the model. Typically, weather simulations work by having the computer conduct a number of simulations based on the current state, and then entering observational data into the simulation to nudge it in a way that puts it closer to the actual state. The problem of incorporating data in the simulation — data assimilation — has become increasingly complex with the large number of types of available data, such as satellite observations and measurements taken from ground stations. Typically, supercomputers today spend an approximately equal amount of time running the simulations and incorporating the real-world data.

Now, with research that could lead to more accurate forecasts, researchers from the RIKEN Advanced Institute for Computational Science in Japan have run an enormous global weather simulation. They ran 10,240 simulations of a model of the global atmosphere divided into 112-km sectors, and then used data assimilation and statistical methods to come up with a model closely fitting the real data for a historical time period, between November 1 and November 8, 2011. The simulations were run on Japan’s flagship 10-petaflop K computer using NICAM, a simulation intended to accurately model the atmosphere.

One of the key findings is that faraway observations, several thousand kilometers in distance, may have an impact on the eventual state of the weather forecast. Data from the Great Lakes region in the United States, for instance, can have an impact on the eventual state in Europe. This finding suggests the need for further research on advanced methods that can make better use of faraway observations, as this could potentially lead to an improvement of weather forecasts.

According to Takemasa Miyoshi, who led the research team, “Forecasting is becoming better thanks to more powerful computers and better observational data from satellites and radars. We attempted to use a large number of samples using a relatively coarse simulation, and found that it performed quite well, fitting the actual data from the time period we chose. We are planning to use the power of the K computer’s successor, as it develops, to create tools that could be used for better weather forecasting.”

The following two research projects funded by the Japan Science and Technology Agency (JST) CREST programs contributed to this achievement:

• “Innovating ‘Big Data Assimilation’ technology for revolutionizing very-short-range severe weather prediction” (led by Dr. Takemasa Miyoshi of RIKEN), a project in the research area of Advanced Application Technologies to Boost Big Data Utilization for Multiple-Field Scientific Discovery and Social Problem Solving (Research Supervisor: Prof. Yuzuru Tanaka of Hokkaido University), and
• “EBD: Extreme Big Data: Convergence of Big Data and HPC for Yottabyte Processing” (led by Prof. Satoshi Matsuoka of the Tokyo Institute of Technology with Dr. Takemasa Miyoshi of RIKEN acting as co-PI), which is a project in the Advanced Core Technologies for Big Data Integration area (Research Supervisor: Prof. Masaru Kitsuregawa of the National Institute of Informatics).

In addition, the project was funded by:

• “Ensemble-based Data Assimilation of TRMM/GPM Precipitation Measurements” funded by the 7th RA of JAXA PMM project (led by Dr. Takemasa Miyoshi), and
• “Advancement of meteorological and global environmental predictions utilizing observational ‘Big Data’” of the social and scientific priority issues (Theme 4) to be tackled by using post K computer of the FLAGSHIP2020 Project” (led by Dr. Keiko Takahashi of the Japan Agency for Marine-Earth Science and Technology (JAMSTEC) with Dr. Takemasa Miyoshi of RIKEN acting as the sub-theme organizer)

The research is featured in the November 2015 issue of Computer, IEEE Computer Society’s flagship publication.

Citation: Takemasa Miyoshi, Keiichi Kondo, and Koji Terasaki, “Big Ensemble Data Assimilation in Numerical Weather Prediction,” Computer, vol. 48, no. 11, 2015, pp. 15–21; DOI: 10.1109/MC.2015.332.