Climate: When data fail us, nonlinear/non-Gaussian estimation

State estimation techniques are used in weather and climate prediction, hydrogeology, seismology, as a way to blend model output and real data in order to improve on predictions from the exclusive use of the model or the data alone.

Techniques that are based upon least-squares ideas, such as the family of Kalman Filter/Smoothers, or Variational Data Assimilation, are optimal in linear/Gaussian problems. However, they often fail in problems in which nonlinearities are important and/or when Gaussianity in the statistics cannot be assumed. Even linearization may fail, and so do ensemble techniques that make nonlinear predictions but rely on linear analyses. These comprise the practical state of the art, at least in weather forecasting and in hydrogeology. I will describe these as well as how failures arise in these methods.

We have created a number of nonlinear/non-Gaussian data assimilation techniques. Our present efforts are to make them computationally practical as well as to use of these to do problems that are otherwise intractable using conventional means.

One such application is in Lagrangian data assimilation: here we tackle the problem of blending data that has been sampled along paths, which when blended in traditional ways on Eulerian grids will lead to loss of critical features even though the estimates may be variance-minimizing.