1 2 3 4 5 arXiv:2104.12506v1 [physics.ao-ph] 26 Apr 2021

Topical Review

Bridging observation, theory and numericalsimulation of the ocean using Machine Learning

Maike Sonnewald1,2,3‡, Redouane Lguensat4,5, Daniel C.Jones6, Peter D. Dueben7, Julien Brajard5,8, V. Balaji1,2,4

E-mail: [email protected] University, Program in Atmospheric and Oceanic Sciences,Princeton, NJ 08540, USA2NOAA/OAR Geophysical Fluid Dynamics Laboratory, Ocean and CryosphereDivision, Princeton, NJ 08540, USA3University of Washington, School of Oceanography, Seattle, WA, USA4 Laboratoire des Sciences du Climat et de l’Environnement (LSCE-IPSL), CEASaclay, Gif Sur Yvette, France5 LOCEAN-IPSL, Sorbonne Universite, Paris, France6British Antarctic Survey, NERC, UKRI, Cambridge, UK7European Centre for Medium Range Weather Forecasts, Reading, UK8 Nansen Center (NERSC), Bergen, Norway

April 2021

Abstract.Progress within physical oceanography has been concurrent with the

increasing sophistication of tools available for its study. The incorporation ofmachine learning (ML) techniques offers exciting possibilities for advancing thecapacity and speed of established methods and also for making substantial andserendipitous discoveries. Beyond vast amounts of complex data ubiquitous inmany modern scientific fields, the study of the ocean poses a combination ofunique challenges that ML can help address. The observational data available islargely spatially sparse, limited to the surface, and with few time series spanningmore than a handful of decades. Important timescales span seconds to millennia,with strong scale interactions and numerical modelling efforts complicated bydetails such as coastlines. This review covers the current scientific insightoffered by applying ML and points to where there is imminent potential. Wecover the main three branches of the field: observations, theory, and numericalmodelling. Highlighting both challenges and opportunities, we discuss both thehistorical context and salient ML tools. We focus on the use of ML in situsampling and satellite observations, and the extent to which ML applicationscan advance theoretical oceanographic exploration, as well as aid numericalsimulations. Applications that are also covered include model error and biascorrection and current and potential use within data assimilation. While notwithout risk, there is great interest in the potential benefits of oceanographic MLapplications; this review caters to this interest within the research community.

Keywords: Ocean Science, physical oceanography, machine learning, observations,theory, modelling, supervised machine learning, unsupervised machine learning.Submitted to: Environ. Res. Lett.

‡ Present address: Princeton University, Program in Atmospheric and Oceanic Sciences, 300 ForrestalRd., Princeton, NJ 08540

arX

iv:2

104.

1250

6v1

[ph

ysic

s.ao

-ph]

26

Apr

202

1

Bridging observation, theory and numerical simulation of the ocean using ML 2

1. Introduction

1.1. Oceanography: observations, theory, andnumerical simulation

The physics of the oceans have been of crucialimportance, curiosity and interest since prehistorictimes, and today remain an essential element in ourunderstanding of weather and climate, and as a keydriver of biogeochemistry and overall marine resources.The eras of progress within oceanography have gonehand in hand with the tools available for its study.Here, the current progress and potential future roleof machine learning (ML) techniques is reviewed andbriefly put into historical context. ML adoption isnot without risk, but is here put forward as havingthe potential to again accelerate scientific insight,performing tasks better and faster, along with allowingavenues of serendipitous discovery.

Perhaps the principal interest in oceanographywas originally that of navigation, for exploration,commercial and military purposes. Knowledge of theocean as a dynamical entity with predictable features–the regularity of its currents and tides – must have beenknown for millennia: Knowledge of oceanography likelyhelped the successful colonization of Oceania [165], andsimilarly Viking and Inuit navigation [110], the oldestknown dock was constructed in Lothal with knowledgeof the tides dating back to 2500–1500 CE[48], and AbuMa’shar of Baghdad in the 8th century CE correctlyattributed the existence of tides to the Moon’s pull.

The ocean measurement era, determining temper-ature and salinity at depth from ships, starts in the late18th century CE. While the tools for a theory of theocean circulations started to become available in theearly 19th century CE with the Navier-Stokes equation,observations remained at the core of oceanographic dis-covery. The first modern oceanographic textbook waspublished in 1855 by M. Mauri, whose work in oceanog-raphy and politics served the slave trade across the At-lantic, around the same time CO2s’ role in climate wasrecognized [90, 223]. The first major global observa-tional synthesis of the ocean can be traced to the Chal-lenger expeditions of 1873-75 CE [64], which gave afirst look at the global distribution of temperature andsalinity including at depth, revealing the 3-dimensionalstructure of the ocean.

Quantifying the time mean ocean circulation re-mains challenging, as ocean circulation features stronglocal and instantaneous fluctuations. Improvements inmeasurement techniques allowed the Swedish oceanog-rapher Ekman to elucidated the nature of the wind-driven boundary layer [81]. Ekman used observationstaken while he was intentionally frozen into the Arc-tic ice on the Fram expedition with the Norwegianoceanographer and explorer Nansen. The “dynamic

method” was introduced by Swedish oceanographerSandstrom and the Norwegian oceanographer Helland-Hansen [198], allowing the indirect computation ofocean currents from density estimates under the as-sumption of a largely laminar flow. This theory wasdeveloped further by Norwegian meteorologist Bjerk-nes into the concept of geostrophy, from the Greek geofor earth and strophe for turning. This theory was putto the test in the extensive Meteor expedition in theAtlantic from 1925-27 CE; they uncovered a view of thehorizontal and vertical ocean structure and circulationthat is strikingly similar to our present view of the At-lantic meridional overturning circulation [163, 191].

While the origins of Geophysical Fluid Dynamics(GFD) can be traced back to Laplace or Archimedes,the era of modern GFD can be seen to stemfrom linearizing the Navier-Stokes equations, whichenabled progress in understanding meteorology andatmospheric circulation. For the ocean, pioneeringdynamisists include Sverdrup, Stommel, and Munk,whose theoretical work still has relevance today [209,167]. As compared to the atmosphere, the oceancirculation exhibits variability over a much largerrange of timescales, as noted by [168], likely spanningthousands of years rather than the few decades ofdetailed ocean observations available at the time.Yetthere are phenomena at intermediate timescales(that is, months to years) which seemed to involveboth atmosphere and ocean, e.g [171], and indeedSverdrup suggests the importance of the coupledatmosphere-ocean system in [211]. In the 1940s muchprogress within GFD was also driven by the secondworld war (WWII). The introduction of accuratenavigation through radar introduced with WWIIworked a revolution for observational oceanographytogether with bathythermographs intensively used forsubmarine detection. Beyond in situ observations, thelaunch of Sputnik, the first artificial satellite, in 1957heralded the era of ocean observations from satellites.Seasat, launched on the 27th of June 1978, was thefirst satellite dedicated to ocean observation.

Oceanography remains a subject that must beunderstood with an appreciation of available tools,both observational and theoretical, but also numerical.While numerical GFD can be traced back to theearly 1900s [2, 29, 190], it became practical withthe advent of numerical computing in the late 1940s,complementing that of the elegant deduction andmore heuristic methods that one could call “patternrecognition” that had prevailed before [10]. The firstocean general circulation model with specified globalgeometry were developed by Bryan and Cox [44, 43]using finite-difference methods. This work pavedthe way for a what now is a major component ofcontemporary oceanography. The first coupled ocean-


atmosphere model of [153] eventually led to their usefor studies of the coupled Earth system, includingits changing climate. The low-power integratedcircuit that gave rise to computers in the 1970s alsorevolutionized observational oceanography, enablinginstruments to reliably record autonomously. Thishas enabled instruments such as moored currentmeters and profilers, drifters, and floats through tohydrographic and velocity profiling devises that gaverise to microstructure measurements. Of note is thefleet of free-drifting Argo floats, beginning in 2002,which give an extraordinary global dataset of profiles[193]. Data assimilation (DA) is the important branchof modern oceanography combining what is oftensparse observational data with either numerical orstatistical ocean models to produce observationally-constrained estimates with no gaps. Such estimatesare referred to as an ’ocean state’, which is especiallyimportant for understanding locations and times withno available observations.

Together the innovations within observations,theory, and numerical models have produced distinctlydifferent pictures of the ocean as a dynamicalsystem, revealing it as an intrinsically turbulent andtopographically influenced circulation [239, 94]. Keylarge scale features of the circulation depend onvery small scale phenomena, which for a typicalmodel resolution remain parameterized rather thanexplicitly calculated. For instance, fully accountingfor the subtropical wind-driven gyre circulation andassociated western boundary currents relies on anunderstanding of the vertical transport of vorticityinput by the wind and output at the sea floor,which is intimately linked to mesoscale (ca. 100km)flow interactions with topography [123, 79]. It hasbecome apparent that localized small-scale turbulence(0-100km) can also impact the larger-scale, time-meanoverturning and lateral circulation by affecting how theupper ocean interacts with the atmosphere [219, 89,115]. The prominent role of the small scales on thelarge scale circulation has important implications forunderstanding the ocean in a climate context, and itsrepresentation still hinges on the further developmentof our fundamental understanding, observationalcapacity, and advances in numerical approaches.

The development of both modern oceanographyand ML techniques have happened concurrently, asillustrated in Fig. 1. This review summarizes thecurrent state of the art in ML applications for physicaloceanography and points towards exciting futureavenues. We wish to highlight certain areas wherethe emerging techniques emanating from the domainof ML demonstrate potential to be transformative.ML methods are also being used in closely-relatedfields such as atmospheric science. However, within

oceanography one is faced with a unique set ofchallenges rooted in the lack of long-term and spatiallydense data coverage. While in recent years the surfaceof the ocean is becoming well observed, there is still aconsiderable problem due to sparse data, particularlyin the deep ocean. Temporally, the ocean operates ontimescales from seconds to millennia, and very few longterm time series exist. There is also considerable scale-interaction.

There remains a healthy skepticism towards someML applications, and calls for “trustworthy” ML arealso coming forth from both the European Union andthe United States government (Assessment List forTrustworthy Artificial Intelligence [ALTAI], and man-date E.O. 13960 of Dec 3, 2020). Within the physi-cal sciences and beyond, trust can be fostered throughtransparency. For ML, this means moving beyond the“black box” approach for certain applications. Mov-ing away from this black box approach and adoptinga more transparent approach involves gaining insightinto the learned mechanisms that gave rise to ML pre-dictive skill. This is facilitated by either building a pri-ori interpretable ML applications or by retrospectivelyexplaining the source of predictive skill, coined inter-pretable and explainable artificial intelligence (IAI andXAI, respectively [195, 124]). With such insights fromtransparent ML, a synthesis between theoretical andobservational branches of oceanography could be pos-sible. Traditionally, theoretical models tend towardsoversimplification, while data can be overwhelminglycomplicated. For advancement in the fundamental un-derstanding of ocean physics, ML is ideally placed toidentify salient features in the data that are compre-hensible to the human brain. With this approach, MLcould significantly facilitate a generalization beyondthe limits of data, letting data reveal possible struc-tural errors in theory. With such insight, a hierarchyof conceptual models of ocean structure and circulationcould be developed, signifying an important advance inour understanding of the ocean.

In this review, we introduce ML concepts(Section 1.2), and some of its current roles in theatmospheric and Earth System Sciences (Section 1.3),highlighting particular areas of note for oceanapplications. The review follows the structure outlineillustrated in Fig. 2, with the ample overlap notedthrough cross referencing the text. We reviewocean observations (Section 2), sparsely observed formuch history, but now yielding increasingly clearinsight into the ocean and its 3D structure. InSection 3 we examine a potential synergy betweenML and theory, with the intent to distill expressionsof theoretical understanding by dataset analysis fromboth numerical and observational efforts. We thenprogress from theory to models, and the encoding


of theory and observations in numerical models(Section 4). We highlight some issues involved withML-based prediction efforts (Section 5), and end witha discussion of challenges and opportunities for MLin the ocean sciences (Section 6). These challengesand opportunities include the need for transparent ML,ways to support decision makers and a general outlook.Appendix A1 has a list of acronyms.

1.2. Concepts in ML

Throughout this article, we will mention some conceptsfrom the ML literature. We find it then natural to startthis paper with a brief introduction to some of the mainideas that shaped the field of ML.

ML, a sub-domain of Artificial Intelligence(AI), is the science of providing mathematicalalgorithms and computational tools to machines,allowing them to perform selected tasks by “learning”from data. This field has undergone a seriesof impressive breakthroughs over the last yearsthanks to the increasing availability of data andthe recent developments in computational and datastorage capabilities. Several classes of algorithms areassociated with the different applications of ML. Theycan be categorized into three main classes: supervisedlearning, unsupervised learning, and reinforcementlearning (RL). In this review, we focus on the first twoclasses which are the most commonly used to date inthe ocean sciences.

1.2.1. Supervised learning Supervised learning refersto the task of inferring a relationship between a setof inputs and their corresponding outputs. In orderto establish this relationship, a “labeled” dataset isused to constrain the learning process and assessthe performance of the ML algorithm. Given adataset of N pairs of input-output training examples{(x(i), y(i))}i∈1..N and a loss function L that representsthe discrepancy between the ML model prediction andthe actual outputs, the parameters θ of the ML modelf are found by solving the following optimizationproblem:

θ∗ =arg minθ

1

n

n∑i=1

L(f(x(i);θ

), y(i)

). (1)

If the loss function is differentiable, then gradientdescent based algorithms can be used to solveequation 1. These methods rely on an iterative tuningof the models’ parameters in the direction of thenegative gradient of the loss function. At each iterationk, the parameters are updated as follows:

θk+1 = θk − µ∇L (θk) , (2)

where µ is the rate associated with the descent and iscalled the learning rate and ∇ the gradient operator.

Two important applications of supervised learningare regression and classification. Popular statisticaltechniques such as Least Squares or Ridge Regression,which have been around for a long time, are specialcases of a popular supervised learning technique calledLinear Regression (in a sense, we may consider alarge number of oceanographers to be early MLpractitioners.) For regression problems, we aim toinfer continuous outputs and usually use the meansquared error (MSE) or the mean absolute error(MAE) to assess the performance of the regression. Incontrast, for supervised classification problems we sortthe inputs to a number of classes or categories thathave been pre-defined. In practice, we often transformthe categories into probability values of belonging tosome class and use distribution-based distances suchas the cross-entropy to evaluate the performance of theclassification algorithm.

Numerous types of supervised ML algorithms havebeen used in the context of ocean research, as detailedin the following sections. Notable methods include:

• Linear univariate (or multivariate) regression(LR), where the output is a linear combinationof some explanatory input variables. LR is one ofthe first ML algorithms to be studied extensivelyand used for its ease of optimization and its simplestatistical properties [166].

• k-Nearest Neighbors (KNN), where we consider aninput vector, find its k closest points with regardto a specified metric, then classify it by a pluralityvote of these k points. For regression, we usuallytake the average of the values of the k neighbors.KNN is also known as “analog methods” in thenumerical weather prediction community [150].

• Support Vector Machines (SVM) [57], where theclassification is done by finding a linear separatinghyperplane with the maximal margin between twoclasses (the term “margin” here denotes the spacebetween the hyperplane and the nearest pointsin either class.) In case of data which cannotbe separated linearly, the use of the kernel trickprojects the data into a higher dimension wherethe linear separation can be done. Support VectorRegression (SVR) are an adaption of SVMs forregression problems.

• Random Forests (RF) that are a composition ofa multitude of Decision Trees (DT). DTs areconstructed as a tree-like composition of simpledecision rules [27].

• Gaussian Process Regression (GPR) [237], alsocalled kriging, is a general form of to the optimalinterpolation algorithm, which has been used inthe oceanographic community for a number ofyears


Figure 1. Timeline sketch of oceanography (blue) and ML (orange). The timelines of oceanography and ML are movingtowards each other, and interactions between the fields where ML tool as are incorporated into oceanography has the potential toaccelerate discovery in the future. Distinct ‘events’ marked in grey. Each field has gone through stages (black), with progress thatcan be attributed to the available tools. With the advent of computing, the fields were moving closer together in the sense that MLmethods generally are more directly applicable. Modern ML is seeing an very fast increase in innovation, with much potential foradoption by oceanographers. See table A1 for acronyms.

• Neural Networks (NN), a powerful class of uni-versal approximators that are based on composi-tions of interconnected nodes applying geometrictransformations (called affine transformations) toinputs and a nonlinearity function called an “ac-tivation function” [61]

The recent ML revolution, i.e. the so-called DeepLearning (DL) era that began in the early 2010s,sparked off thanks to the scientific and engineeringbreakthroughs in training neural networks (NN),combined with the proliferation of data sources and theincreasing computational power and storage capacities.The simplest example of this advancement is theefficient use of the algorithm of backpropagation(known in the geocience community as the adjointmethod) combined with stochastic gradient descentfor the training of multi-layer NNs, i.e. NNswith multiple layers, where each layer takes theresult of the previous layer as an input, appliesthe mathematical transformations and then yields

an input for the next layer [24]. DL researchis a field receiving intense focus and fast progressthrough its use both commercially and scientifically,resulting in new types of ”architectures” of NNs, eachadapted to particular classes of data (text, images,time series, etc.). Multilayer perceptrons (MLP)for tabular data, Convolutional NNs (ConvNet) forimagery and Recurrent NNs (RNN) for temporal dataare ubiquitous in the recent ML literature [200, 142].

1.2.2. Unsupervised learning Unsupervised learningis another major class of ML. In these applications,the datasets are typically unlabelled. The goal isthen to discover patterns in the data that can beused to solve particular problems. One way tosay this is that unsupervised classification algorithmsidentify sub-populations in data distributions, allowingusers to identify structures and potential relationshipsamong a set of inputs (which are sometimes called“features” in ML language). Unsupervised learning


PredictionsModelsTheoryObservationsDecisionSupport

- Observation operators- Gap filling- Error detection and

bias correction- Synthesis of

observations- In situ feature

detection

- Learn equations and boundary conditions

- Unsupervised learning to understand dynamics and causality

- Learn process interactions

- Learn sub-grid-scale representation of models

- Learn low-order models

- In situ updates of boundary conditions

- Speed-up simulations via emulation and preconditioning

- Compare models against observations

- Uncertainty quantification

- Data assimilation- Error correction- Down-scaling- Understand climate

response- Improve signal-to-noise- In situ alarm systems

- Alarm systems- Climate mitigation- Route planning- Oil spilling- Flooding

Figure 2. Machine learning within the components of oceanography. A diagram capturing the general flow of knowledge,highlighting the components covered in this review. Separating the categories (arrows) is artificial, with ubiquitous feed-backsbetween most components, but serves as an illustration.

is somewhat closer to what humans expect from anintelligent algorithm, as it aims to identify latentrepresentations in the structure of the data whilefiltering out unstructured noise. At the NeurIPS 2016conference, Yann LeCun, a DL pioneer researcher,highlighted the importance of unsupervised learningusing his cake analogy : ”If machine learning is acake, then unsupervised learning is the actual cake,supervised learning is the icing, and RL is the cherryon the top.”

Unsupervised learning is achieving considerablesuccess in both clustering and dimensionality reductionapplications. Some of the unsupervised techniques thatare mentioned throughout this review are:

• k-means, a popular and simple space-partitioningclustering algorithm that finds classes in adataset by minimizing within-cluster variances[208]. Gaussian Mixture Models (GMMs) can beseen as a generalization of the k-means algorithmthat assumes the data can be represented by amixture (i.e. linear combination) of a number ofmulti-dimensional Gaussian distributions [162].

• Kohonen maps [also called Self Organizing Maps(SOM)] is a NN based clustering algorithm thatleverages topology of the data; nearby locations ina learned map are placed in the same class [136].K-means can be seen as a special case of SOM

with no information about the neighborhood ofclusters.

• t-SNE and UMAP are two other clusteringalgorithms which are often used for not onlyfinding clusters but also because of their attractivedata visualization properties [225, 161]. Thesemethods are useful for representing the structureof a high-dimensional dataset in a small numberof dimensions that can be plotted. They use ameasure of the distance between points.

• Principal Component Analysis (PCA) [176], thesimplest and most popular dimensionality reduc-tion algorithm. Another term for PCA is Empir-ical Orthogonal Function analysis (EOF), whichhas been used by physical oceanographers formany years.

• Autoencoders (AE) are NN-based dimensionalityreduction algorithms, consisting of a bottleneck-like architecture that learns to reconstruct theinput by minimzing the error between the outputand the input which are the same data (i.e. thesame data is fed into both the input and outputof the autoencoder). A central layer with a lowerdimension than the original inputs’ dimensionis called a “code” and represents a compressedrepresentation of the input [138].

• Generative modeling : a powerful paradigm that


learns the latent features and distributions ofa dataset and then proceeds to generate newsamples that are plausible enough to belong to theinitial dataset. Variational Auto-encoders (VAEs)and Generative Adversarial Networks (GANS) aretwo popular techniques of generative modelingthat benefited much from the DL revolution [133,103].

Between supervised and unsupervised learning liessemi-supervised learning. It is a special case whereone has access to both labeled and unlabeled data. Aclassical example is when labeling is expensive, leadingto a small percentage of labeled data and a highpercentage of unlabeled data.

Reinforcement learning is the third paradigm ofML; it is based on the idea of creating algorithmswhere an agent explores an environment with the aimof reaching some goal. The agent learns through a trialan error mechanism, where it performs an action andreceives a response (reward or punishment), the agentlearns by maximizing the expected sum of rewards[215]. The DL revolution did also affect this fieldand led to the creation of a new field called deepreinforcement learning (Deep RL) [210]. A popularexample of Deep RL that got huge media attention isthe algorithm AlphaGo developed by DeepMind whichbeat human champions in the game of Go [204].

The goal of this review paper is not to delveinto the definitions of ML techniques but only tobriefly introduce them to the reader and recommendreferences for further investigation. The textbook byChristopher Bishop [28] covers essentials of the fieldsof pattern recognition and ML. William Hsieh’s book[121] is probably one of earliest attempts at writinga comprehensive review of ML methods targeted atearth scientists. Another notable review of statisticalmethods for physical oceanography is the paper byWikle et al. [235]. We also refer the interested readerto the book of Goodfellow et al. [24] to learn moreabout the theoretical foundations of DL and some ofits applications in science and engineering.

1.3. ML in atmospheric and the wider Earth systemsciences

Precursors to modern ML methods, such as regressionand principal component analysis, have of course beenused in many fields of Earth system science for decades.The use of PCA for example, was popularized inmeteorology by [149], as a method of dimensionalityreduction of large geospatial datasets, and Lorenz alsospeculates here on the possibility of purely statisticalmethods of long-term weather prediction based ona representation of data using PCA. Methods ofdiscovering correlations and links, including possible

causal links, between dataset features using formalmethods have also been much in use in Earth systemscience. e.g [17]. Walker [231], tasked with discoveringthe cause for the interannual fluctuation of the Indianmonsoon, (whose failure meant widespread droughtin India, and in colonial times, famine as well, [63]),Walker put to work an army of Indian clerks tocarry out a vast computation by hand to discoverpossible correlations across all available data. Thisled to the discovery of the Southern Oscillation, theseesaw in the West-East temperature gradient in thePacific, which we know now by its modern name, ElNino Southern Oscillation (ENSO). Beyond observedcorrelations, theories of ENSO and its emergence fromcoupled atmosphere-ocean dynamics appeared decadeslater [244]. Walker speaks of statistical methods ofdiscovering “weather connections in distant parts ofthe earth”, or teleconnections. The ENSO-monsoonteleconnection remains a key element in diagnosis andprediction of the Indian monsoon [214], [213]. Theseand other data-driven methods of the pre-ML eraare surveyed in [41]. ML-based predictive methodstargeted at ENSO are also being established [111].Here, the learning is not directly from observations butfrom models and reanalysis data, and outperform somedynamical models in forecasting ENSO.

There is an interplay between data-driven meth-ods and physics-driven methods that both strive tocreate insight into many complex systems, where theocean and the wider Earth system science are exam-ples. As an example of physics-driven methods [10],Bjerknes and other pioneers discussed in Section 1.1formulated accurate theories of the general circulation,that were put into practice for forecasting with theadvent of digital computing. Advances in numericalmethods led to the first practical physics-based atmo-spheric forecast [182]. Until that time, forecasting of-ten used data-driven methods “that were neither algo-rithmic nor based on the laws of physics” [172]. MLoffers avenues to a synthesis of data-driven and physics-driven methods. In recent years, as outlined below inSection 4.2, new processors and architectures withincomputing have allowed much progress within forecast-ing and numerical modelling overall. ML methods arepoised to allow Earth system science modellers to in-crease the efficient use of modern hardware even fur-ther. It should be noted however that “classical” meth-ods of forecasting such as analogues also have becomemore computationally feasible, and demonstrate equiv-alent skill, e.g [68]. The search for analogues has be-come more computationally tractable as well, althoughthere may be limits here as well [70].

Advances in numerical modeling brought inadditional understanding of elements in Earth systemscience which are difficult to derive, or represent from


first principles. Examples include cloud microphysicsor interactions with the land surface and biosphere.For capturing cloud processes within models, theactual processes governing clouds takes place at scalestoo fine to model and will remain out of reach ofcomputing for the foreseeable future [201]. A practicalsolution to this is finding a representation of theaggregate behavior of clouds at the resolution of amodel grid cell. This has proved quite difficult andprogress over many decades has been halting [35]. Theuse of ML in deriving representations of clouds isnow an entire field of its own. Early results includethe results of [98], using NNs to emulate a “super-parameterized” model. In the super-parameterizedmodel, there is a clear (albeit artificial) separationof scales between the “cloud scale” and the largescale flow. When this scale separation assumptionis relaxed, some of the stability problems associatedwith ML re-emerge [40]. There are also a fundamentalissue of whether learned relationships respect basicphysical constraints, such as conservation laws [147].Recent advances ([241], [25]) focus on formulating theproblem in a basis where invariances are automaticallymaintained. But this still remains a challenge in caseswhere the physics is not fully understood.

There are at least two major efforts for thesystematic use of ML methods to constrain thecloud model representations in GCMs. First, thecalibrate-emulate-sample (CES [54, 75]) approachuses a forward model for a broad calibration ofparameters also referred to as “tuning”[119]. Thisis followed by an emulator (as the forward modelis computationally expensive) for further calibrationand uncertainty quantification. It is important toretain the uncertainty quantification aspect in theML context as it is likely that the data in a chaoticsystem only imperfectly constrain the loss function.Second, emulators can be used to eliminate implausibleparameters from a calibration process, demonstratedby the HighTune project [59, 120]. This processcan also identify “structural error”, indicating thatthe model formulation itself is incorrect, when noparameter choices can yield a plausible solution. Modelerrors are discussed in Section 5.1. In an ocean context,the methods discussed here can be a challenge due tothe necessary forwards model component. Note also,that ML algorithms such as GPR are ubiquitous inemulating problems thanks to their built-in uncertaintyquantification. GPR methods are also popular becausetheir application involves a low number of trainingsamples, and function as inexpensive substitutes fora forward model.

Model resolution that is inadequate for manypractical purposes has led to the development of data-driven methods of “downscaling”. For example climate

change adaptation decision-making at the local levelbased on climate simulations too coarse to featureenough detail. Most often, a coarse-resolution modeloutput is mapped onto a high-resolution referencetruth, for example given by observations [226, 4].Empirical-statistical downscaling (ESD, [23]) is anexample of such methods. While ESD emphasized thedownscaling aspect, all of these downscaling methodsinclude a substantial element of bias correction.This is highlighted in the name of some of thepopular methods such as Bias Correction and SpatialDownscaling [238] and Bias Corrected ConstructedAnalogue [157]. These are trend-preserving statisticaldownscaling algorithms, that combine bias correctionwith the analogue method of Lorenz (1969)[151]. MLmethods are rapidly coming to dominate the fieldas discussed in Section 5.1, with examples rangingfrom precipitation (e.g [227]) to unresolved rivertransport [101]. Downscaling methods continue tomake the assumption that transfer functions learnedfrom present-day climate continue to hold in the future.This stationarity assumption is a potential weaknessof data-driven methods ([177, 69]), that requires asynthesis of data-driven and physics-based methods aswell.

2. Ocean observations

Observations continue to be key to oceanographicprogress, with ML increasingly being recognised as atool that can enable and enhance what can be learnedfrom observational data, performing conventional tasksbetter/faster, as well as bring together different formsof observations, facilitating comparison with modelresults. ML offers many exciting opportunities for usewith observations, some of which are covered in thissection and in section 5 as supporting predictions anddecision support.

The onset of the satellite observation era broughtwith it the availability of a large volume of effectivelyglobal data, challenging the research community touse and analyze this unprecedented data stream.Applications of ML intended to develop more accuratesatellite-driven products go back to the 90’s [218].These early developments were driven by the dataavailability, distributed in normative format by thespace agencies, and also by the fact that modelsdescribing the data were either empirical (e.g. marinebiogeochemistry [199]) or too computationally costlyand complex (e.g. radiative transfer [132]). Morerecently, ML algorithms have been used to fuse severalsatellite products [107] and also satellite and in-situdata [170, 49, 156, 131]. For the processing ofsatellite data, ML has proven to be a valuable toolfor extracting geophysical information from remotely


sensed data (e.g. [76]), whereas a risk of using onlyconventional tools is to exploit only a more limitedsubset of the mass of data available. These applicationsare based mostly on instantaneous or very short-termrelationships and do not address the problem of howthese products can be used to improve our ability tounderstand and forecast the oceanic system. Furtheruse for current reconstruction using ML [155], heatfluxes [99], the 3-dimensional circulation[206], andocean heat content[125] are also being explored.

There is also an increasingly rich body of literaturemining ocean in-situ observations. These leveragea range of data, including Argo data, to study arange of ocean phenomena. Examples include assessingNorth Atlantic mixed layers [158], describing spatialvariability in the Southern Ocean [128], detectingEl Nino events [118], assessing how North Atlanticcirculation shifts impacting heat content [66], andfinding mixing hot spots [194].

Modern in-situ classification efforts are oftenproperty-driven, carrying on long traditions withinphysical oceanography. For example, characteristicgroups or “clusters” of salinity, temperature, densityor potential vorticity have typically been used to de-lineate important water masses and to assess their spa-tial extent, movement, and mixing [117, 112]. However,conventional identification/classification techniques as-sume that these properties stay fixed over time. Thetechniques largely do not take interannual and longertimescale variability into account. The prescribedranges used to define water masses are often somewhatad-hoc and specific (e.g. mode waters are often tied tovery restrictive density ranges) and do not generalizewell between basins or across longer timescales [8]. Al-though conventional identification/classification tech-niques will continue to be useful well into the future,unsupervised ML offers a robust, alternative approachfor objectively identifying structures in oceanographicobservations [128, 194, 180, 31].

To analyze data, dimensionality and noise reduc-tion methods have a long history within oceanogra-phy. PCA is one such method, which has had a pro-found influence on oceanography since Lorenz first in-troduced it to the geosciences in 1956 [149]. Despitethe method’s shortcomings related to strong statisticalassumptions and misleading applications, it remains apopular approach [164]. PCA can be seen as a su-per sparse rendition of k-means clustering [67] withthe assumption of an underlying normal distributionin its commonly used form. Overall, different forms ofML can offer excellent advantages over more commonlyused techniques. For example, many clustering algo-rithms can be used to reduce dimensionality accordingto how many significant clusters are identifiable in thedata. In fact, unsupervised ML can sidestep statis-

Figure 3. Cartoon of the role of data withinoceanography. While eliminating prior assumptions withindata analysis is not possible, or even desirable, ML applicationscan enhance the ability to perform pure data exploration. The’top down’ approach (left) refers to a more traditional approachwhere the exploration of the data is firmly grounded in priorknowledge and assumptions. Using ML, how data is used inoceanographic research and beyond can be changed by takinga ’bottom up’ data-exploration centered approach, allowing thepossibility for serendipitous discovery.

tical assumptions entirely, for example by employingdensity-based methods such as DBSCAN [205]. Ad-vances within ML are making it increasingly possibleand convenient to to take advantage of methods suchas t-SNE [205] and UMAP, where the original topol-ogy of the data can be conserved in a low-dimensionalrendition.

Interpolation of missing data in oceanic fields isanother application where ML techniques have beenused, yielding products used in operational contexts.For example, Kriging is a popular technique thatwas successfully applied to altimetry [141], as itcan account for observation from multiple satelliteswith different spatio-temporal sampling. EOF-basedtechniques are also attracting increasing attention withthe proliferation of data. For example, the DINEOFalgorithm [6] leverages the availability of historicaldatasets, to fill in spatial gaps within new observations.This is done via projection onto the space spannedby dominant EOFs of the historical data. The useof advanced supervised learning, such as DL, for thisproblem in an oceanographical contexts is still in itsinfancy. Attempts exist in the literature, such asthe use of NN for regression to reconstruct pCO2

data [65], or deriving a DL equivalent of DINEOF forinterpolating SST [18].

3. Exchanges between observations and theory

Progress within observations, modeling, and theory gohand in hand, and ML offers a novel method for bridg-ing the gaps between the branches of oceanography.When describing the ocean, theoretical descriptions of


circulation tend to be oversimplified, but interpretingbasic physics from numerical simulations or observa-tions alone is prohibitively difficult. Progress in the-oretical work has often come from the discovery orinference of regions where terms in an equation maybe negligible, allowing theoretical developments to befocused with the hope of observational verification. In-deed, progress in identifying negligible terms in fluiddynamics could be said to underpin GFD as a whole[224]. For example, Sverdrup’s theory [212] of oceanregions where the wind stress curl is balanced by theCoriolis term inspired a search for a predicted ‘level ofno motion’ within the ocean interior.

The conceptual and numerical models thatunderlie modern oceanography would be less valuableif not backed by observational evidence, and similarly,findings in data from both observations and numericalmodels can reshape theoretical models [94]. MLalgorithms are becoming heavily used to determinepatterns and structures in the increasing volumes ofobservational and modelled data [158, 128, 129, 194,217, 207, 45, 118, 180, 31, 66]. For example, MLis poised to help the research community reframethe concept of ocean fronts in ways that are tailoredto specific domains instead of ways that are tiedto somewhat ad-hoc and overgeneralized propertydefinitions [51]. Broadly speaking, this area ofwork largely utilizes unsupervised ML and is thuswell-positioned to discover underlying structures andpatterns in data that can help identify negligible termsor improve a conceptual model that was previouslyempirical. In this sense, ML methods are well-placedto help guide and reshape established theoreticaltreatments, for example by highlighting overlookedfeatures. A historical analogy can be drawn tod’Alembert’s paradox from 1752 (or the hydrodynamicparadox), where the drag force is zero on a bodymoving with constant velocity relative to the fluid.Observations demonstrated that there should be adrag force, but the paradox remained unsolved untilPrandtl’s 1904 discovery of a thin boundary layersthat remained as a result of viscous forces. Prandtl’sdiscovery offered credibility to fluid mechanics as ascience, and ML is ideally posed to make similardiscoveries possible through its ability to objectivelyanalyze the increasingly large and complicated dataavailable.

With an exploration of a dataset that movesbeyond preconceived notions comes the potentialfor making entirely new discoveries. It can beenargued that much of the progress within physicaloceanography has been rooted in generalizations ofideas put forward over 30 years ago[94, 169, 127]. Thisfoundation can be tested using data to gain insightin a “top-down” manner (Fig. 3). ML presents

a possible opportunity for serendipitous discoveryoutside of this framework, effectively using data asthe foundation and achieving insight purely throughits objective analysis in a “bottom up” fashion. Thiscan also be achieved using conventional methods butis significantly facilitated by ML, as modern data in itsoften complicated, high dimensional, and voluminousform complicates objective analysis. ML, through itsability to let structures within data emerge, allowsthe structures to be systematically analyzed. Suchstructures can emerge as regions of coherent covariance(e.g. using clustering algorithms from unsupervisedML), even in the presence of highly non-linear andintricate covariance [205]. Such structures can thenbe investigated in their own right and may potentiallyform the basis of new theories. Such exploration isfacilitated by using an ML approach in combinationwith interpretable/explainable (IAI/XAI) methods asappropriate. Unsupervised ML lends itself morereadily to IAI and to many works discussed above.Objective analysis that can be understood as IAIcan also be applied to explore theoretical branches ofoceanography, revealing novel structures [45, 207, 217].Examples where ML and theoretical exploration havebeen used in synergy by allowing interpretability,explainability, or both within oceanography include[206, 243], and the concepts are discussed further insections 6.2 and 6.

As an increasingly operational endeavour, physicaloceanography faces pressures apart from fundamentalunderstanding due to the increasing complexityassociated with enhanced resolution or the complicatednature of data from both observations and numericalmodels. For advancement in the fundamentalunderstanding of ocean physics, ML is ideally placedto break this data down to let salient features emergethat are comprehensible to the human brain.

3.0.1. ML and hierarchical statistical modelingThe concept of a model hierarchy is described byHeld (2005) as a way to fill the “gap” betweenmodelling and understanding[116] within the Earthsystem. A hierarchy consists of a set of modelsspanning a range of complexities. One can potentiallygain insights by examining how the system changeswhen moving between levels of the hierarchy, i.e.when various sources of complexity are added orsubtracted, such as new physical processes, smaller-scale features, or degrees of freedom in a statisticaldescription. The hierarchical approach can helpsharpen hypotheses about the oceanographic systemand inspire new insights. While perhaps conceptuallysimple, the practical application of a model hierarchyis non-trivial, usually requiring expert judgementand creativity. ML may provide some guidance


here, for example by drawing attention to latentstructures in the data. In this review, we distinguishbetween statistical and numerical ML models used forthis purpose. The models discussed in Sections 2and 3 constitute largely statistical models, suchas ones constructed using a k-means application,GANs, or otherwise. This section discusses theconcept of hierarchical models in a statistical sense,and Section 4.1.1 explores the concept of numericalhierarchical models. A hierarchical statistical modelcan be described as a series of model descriptions of thesame system from very low complexity (e.g. a simplelinear regression) to arbitrarily high. In theory, anystatistical model constructed with any data from theocean could constitute a part of this hierarchy, but herewe will restrict our discussion to models constructedfrom the same or very similar data. This conceptof exploring a hierarchy of models, either statisticalor otherwise, using data could also be expressed assearching for the “underlying manifold” or “attractor”within the data[148].

In oceanographic ML applications, there aretunable parameters that are often only weaklyconstrained. A particular example is the totalnumber of classes K in unsupervised classificationproblems [158, 128, 129, 207, 205]. Although one canestimate the optimal value K∗ for the statistical model,for example by using metrics that reward increasedlikelihood and penalize overfitting [e.g. the Bayesianinformation criteria (BIC) or the Akaike informationcriterion (AIC)], in practice it is rare to find a clearvalue of K∗ in oceanographic applications. Often,tests like BIC or AIC return either a range of possibleK∗ values, or they only indicate a lower bound forK. This is perhaps because oceanographic data ishighly correlated across many different spatial andtemporal scales, making the task of separating the datainto clear sub-populations a challenging one. Thatbeing said, the parameter K can also be interpretedas the complexity of the statistical model. A modelwith a smaller value of K will potentially be easierto interpret because it only captures the dominantsub-populations in the data distribution. In contrast,a model with a larger value of K will likely beharder to interpret because it captures more subtlefeatures in the data distribution. For example, whenapplied to Southern Ocean temperature profile data, asimple two-class profile classification model will tendto separate the profiles into those north and southof the Antarctic Circumpolar Current, which is awell-understood approximate boundary between polarand subtropical waters. By contrast, more complexmodels capture more structure but are harder tointerpret using our current conceptual understandingof ocean structure and dynamics [128]. In this way, a

collection of statistical models with different values ofK constitutes a model hierarchy, in which one buildsunderstanding by observing how the representationof the system changes when sources of complexityare added or subtracted [116]. Note that for theexample of k-means, while a range of K values maybe reasonable, this does not largely refer to merelyadjusting the value of K and re-interpreting the result.This is because, for example, if one moves from K=2to K=3 using k-means, there is no a priori reasonto assume they would both give physically meaningfulresults. What is meant instead is similar to the type ofhierarchical clustering that is able to identify differentsub-groups and organize them into larger overarchinggroups according to how similar they are to oneanother. This is a distinct approach within ML thatrelies on the ability to measure a “distance” betweendata points. This rationale reinforces the view thatML can be used to build our conceptual understandingof physical systems, and does not need to be usedsimply as a “black box”. It is worth noting that theaxiom that is being relied on here is that there existsan underlying system that the ML application canapproximate using the available data. With incompleteand messy data, the tools available to assess the fit ofa statistical model only provide an estimate of howwrong it is certain to be. To create a statisticallyrigorous hierarchy, not only does the overall co-variancestructure/topology need to be approximated, but alsothe finer structures that would be found within theseoverarching structures. If this identification process issuccessful, then the structures can be grouped withaccuracy as defined by statistical significance. Thiscan pose a formidable challenge that ML in isolationcannot address; it requires guidance from domainexperts. For example, within ocean ecology, [205]derived a hierarchical model by grouping identifiedclusters according to ecological similarity. In physicaloceanography. [194] grouped some identified classestogether into zones using established oceanographicknowledge, in a step from a more complex statisticalmodel to a more simplified one that is easier tointerpret. When performing such groupings, one hasto pay attention to a balance of statistical rigour anddomain knowledge. Discovering rigorous and usefulhierarchical models should hypothetically be possible,as demonstrated by the self-similarity found in manynatural systems including fluids, but limited andimperfect data greatly complicate the search, meaningthat checking for statistical rigour is important.

As a possible future direction, assessing modelsusing IAI and XAI and known physical relationshipswill likely make finding hierarchical models that aremeaningful much easier. Unsupervised ML is moreintuitively interpretable than supervised ML and may


prove more useful for identifying such hierarchicalmodels. Moving away from using ML in a “black box”sense, with IAI and XAI or otherwise, may yield asynthesis of observations and theory, allowing the fieldto go beyond the limitations of either; theory may allowone to generalize beyond the limits of data, and datamay reveal possible structural errors in theory.

4. From theory to numerical models

The observation of patterns in data is the precursor toa theory, which can lead to predictive models, providedtheory can be converted to practical computation. Inthis section, we discuss how ML could change theway theory is represented within ocean modelling. Torepresent the ocean using numerical models is to helpfill in missing information between observations. Inaddition, models act as virtual laboratories in whichwe can work to understand physical relationships (forexample how the separation of boundary currents suchas the Gulf stream dependents on local topographyor boundary conditions.) The focus of this discussionwill be on models that represent the three-dimensionalocean circulation, but most of these ideas can alsobe used in the context of modelling sea-ice, tides,waves, or biogeochemistry. We also discuss a recurringissue within ocean modeling: the presence of coastlinesthat complicate the application of methods that areconvolutional or spectral.

4.1. Timescales and space scales

When building numerical models, the ocean is largelytreated like a typical fluid that follows the Navier-Stokes equations, and the challenges faced therein aresimilar to those presented by general computationalfluid dynamics. The filamentation of the flow results inscale interactions that make it necessary to representall spatial scales within the model, while the modelresolution needs to be truncated due to the finitenature of computational power. The dynamics atdifferent scales can either be represented via theexplicit, resolved representation within the modelor via the parametrization of sub-grid-scales as aturbulent closure.

Much research has gone into the formulationof parametrizations to represent the sub-grid-scales.Such representations range from classical closures forturbulent fluids, using formulations such as Gent-McWilliams [97] that take the dynamics of sub-gridocean eddies into account, to empirical closure schemesthat are determined by comparing simulations at atarget resolution to simulations at higher resolution[56, 196]. Lately, ML has also been used to learnthe sub-grid-scale, either via the direct learning ofthe terms using NNs [32] or via the learning of the

underlying equations [242]. Similar and promising DAapplications are also emerging, discussed in section 5.2.

Next to the representation of the sub-grid-scale,numerical ocean models are also prone to errorsdue to the necessary discretization of the differentialequations on a numerical grid. A number of methodsare used to discretize the equations [77], includingfinite difference, finite volume, and finite elementmethods. In comparison to the atmosphere, spectraldiscretization methods cannot easily be applied to theocean due to the presence of coastlines, as creatinga representation using global basis functions is notstraightforward.

In the presence of perfect data and adequatecomputational power with which to train a DLapplication, it would be theoretically possible to learnthe dynamics of the ocean with no knowledge ofthe equations of motion. This is because DL canlearn the update of the physical fields based on time-series of observations or model data. This has beendone successfully for certain atmospheric applications[72, 234, 186] and for an idealized ocean model [93].However, DL representation of the ocean is moredifficult than for the atmosphere, as there is muchless reliable three-dimensional training data availablefor the ocean spatially and because the relevant time-scales of the ocean are much longer. This is becausethe ocean has important low-frequency variability,resulting in a need for longer training data sets.Furthermore, coast lines form lateral boundaries thatmay be difficult to incorporate into NN solutions, thattypically require a certain stencil of local informationto update the physical fields at a given gridpoint, suchas convolutional networks.

ML tools could also serve as a method to representthe ocean with fewer degrees of freedom than a fullconventional numerical model. Such use cases for MLinclude being used (1) as part of a coupled Earthsystem model that is either used for short-term weatherforecasts, or (2) in long climate simulations. Forexample, if a model is only trying to represent thesurface fields that are most important for the couplingto the atmosphere, the model could focus on theuse of the leading principal components (if these canbe derived in the presence of coastlines), and learnthe interactions between the different componentsusing data from a time-series extracted from longmodel (or observational) trajectories. However, afirst approach towards building low-order ML modelsusing a barotropic model showed that results fromDL may not necessarily improve on more classicalapproaches that combine regression techniques andstochastic forcing [5].


4.1.1. Concepts of ML and hierarchical numerical mod-eling This section discusses hierarchical modelling ina numerical sense, complementing Section 3.0.1 thatdiscusses hierarchical modelling in a statistical sense.Within oceanography, observations and theory aremore meaningful when viewed together. Observationalscientists (see Section 2) make choices of what to sam-ple based on some prior conceptions of relevance, andof course theory is ungrounded without data. In epis-temology, this is often summarized in Duhem’s formu-lation, “theory is data-laden, and data is theory-laden”[74]. In talking about climate and weather model-ing, Edwards made the corollary, “models are data-laden, and data is model-laden” [80]. For example, theconcept of a reanalysis dataset comes from a model.The sequence from observations to theory to modelsand predictions shows this interplay. This is a key se-quence where we expect ML to display its strengths,e.g. where IAI and XAI methods may yield a syn-thesis of observations and theory, allowing one to gobeyond the limitations of either: theory allowing oneto generalize beyond the limits of data, and data re-vealing possible structural errors in theory as detailedin Section 3. Ideally, we would like to go beyond theseand use ML to discover the underlying equations (e.g[42]), and deliver a model hierarchy that can then beimplemented numerically ([116],[10]). While simple inprinciple, in practice this concept is less straightfor-ward to implement. An example of a form of equationdiscovery can be seen in Zanna and Bolton’s [242] re-duction of resolved turbulent dynamics into a represen-tation suitable for use in coarse-grained models. Thecoarse-grained models represent a different level of athe hierarchy, if tiers are set by horizontal resolution.This ML model was arrived at giving an RVM differentequation terms. Within the context of attempting todiscover underlying equations, using IAI/XAI can bevery powerful, where for the Zanna and Bolton exam-ple one can intuitively interpret the results as they areexpressed in mathematical terms. Using XAI, it wouldalso be possible to infer what gave the ML applicationits predictive skill, which could eliminate e.g. contami-nation from numerical issues that are model resolutionspecific. Methods constituting equation discovery arean exciting, and potentially powerful, way ML couldimpact numerical modelling, particularly if IAI/XAIcan be applied to ensure the ML application predictiveskill is grounded in physics.

4.2. Computational challenges

Since the first ocean general circulation model [44, 43],available computational power has grown exponen-tially, following Moore’s law. The realization that theocean is fundamentally turbulent and topographicallyinfluenced [239, 94] resulted in numerical model devel-

opment focused on increasing model complexity andrefining the model discretization. Numerical modelperformance is often measured in simulated years perday (SYPD). Computational challenges largely man-ifest as a balance between preserving the significantlegacy present in current ocean modeling codes andharnessing the significant advances within the field ofhigh performance computing, which is often tailored toML. ML is a trillion dollar industry which is based onhigh-performance computing power [53]. It is thereforedriving developments in modern supercomputing.

The growth of processing speed in supercomput-ers is no longer exponential, but improvements in thecomputational efficiency of ocean models are still pos-sible through customisation of the computing hard-ware. ML may likely have a place within a revisionof ocean models to improve their computational effi-ciency. Even within Earth system models as a whole, a“digital revolution” has been called for [19], where har-nessing efficiency in modern hardware is central. Com-puters can increasingly be customised as hardware isbecoming more heterogeneous, meaning that differentcomponents for data movement and processing can becombined [20]. Examples of such heterogeneous hard-ware include the so-called Graphical Processing Units(GPU), Tensor Processing Units, Field-ProgrammableGate Arrays, and Application Specific Integrates Cir-cuits, which largely are highly compatible with ML. Totake advantage of this heterogeneous hardware, makingcurrent ocean models “portable”, a significant effortwould be necessary [20]. Current ocean models use theFortran programming language and are parallelised torun on many processors via interfaces such as MPI andOpenMP. This parallelisation approach is not compat-ible with hardware accelerators such as GPUs. Com-patibility could be achieved via re-writing or enhance-ment by programming interfaces such as OpenACC orCuda. Some model groups are investigating a moveto newer computing languages, such as Julia (such asthe Oceananigans model as part of the CliMA project[185]). Languages like Julia can hide technical detailsin high-level descriptions of the model code makingit more portable. So-called domain-specific languagescan be used to facilitate portability [109]. Here, themain algorithmic motives are formulated into libraryfunctions that can be ported to different systems withno need to change the model code that is used by themodel developer.

ML is expected to play a role in issues associatedwith the purely computational approach to oceanmodelling, beyond devising portability to differenthardware accelerators such as GPUs. Hardwareaccelerators are best suited to problems of highoperational intensity (floating-point operations permemory operation). The discretized differential


equations governing fluid flow typically result in sparseoperations resulting from near-neighbour dependencies(“stencils”). Stencil codes remain notorious fortheir low operational intensity [14] resulting in poorcomputational performance, and despite substantialefforts in recent years there has been little progress[173, 11]. This problem is accentuated in oceans, whoselong timescales often require OO(1000 SYPD) for thebasic dynamics to emerge. The role of ML in emulatingturbulent ocean dynamics is likely to be critical inachieving the level of performance required. This isbecause resolving key phenomena such as mesoscaleeddies remain computationally out of reach, and thecurrent parameterizations such as from Gent and McWillians (1990) [97] discussed in Section 4.1 continueto exhibit deficiencies in simulating meridional eddytransport [92].

ML, and in particular DL, could play a significantrole in improve computational efficiency of oceanmodels due to its ability to work with low numericalprecision. Many operations are memory bandwidthbound, and as DL is based on dense linear algebrait is capable of working with very low numericalprecision, such as IEEE half precision with 16 bitsper variable [139]. The trend towards ML hardwarethat is optimised for dense linear algebra and lownumerical precision may have an impact on futureocean modelling. The use of low numerical precisionhas been discussed for weather and climate models[73]. The NEMO model [108] was run in singleprecision with 32 bits per variable instead of the defaultof double precision with 64 bits per variable [220],and half precision with 16 bits per variable is beingexplored for weather and climate models [134] andhardware that is customised for ML has been testedto speed-up expensive components of conventionalmodels [113]. However, in particular for the long-term simulations needed in the ocean, care needs tobe taken to make sure that rounding errors do notimpact on conservation laws. Certain specific aspectsof ocean dynamics require a large dynamic range.For instance, sea level rise, which is a secular changemeasured in cm/century, must be simulated againsta backdrop where surface waves have an amplitudemeasured in O(m) and a phase speed of O(100 m/s),at least 8 orders of magnitude larger over a typicalocean timestep. It is worth noting, that for subsequentanalysis, that using lower numerical precision wouldalso impact the ability of doing analysis on budgets, asclosing these would be complicated.

ML is being explored as a method to emulatecomputationally costly components of ocean models.This was done successfully in a number of studies[188, 40] for physical parametrisation schemes ofatmospheric models. For ocean modelling, NN

emulators could for example speed-up biogeochemicalcomponents [175], which often form a large cost-fraction for ocean models in climate predictions, or sea-ice models, which are often a computational bottleneckas they are difficult to parallelise. ML could also beuseful for improving advection schemes and learninglocal corrections and limiters of fluxes between grid-cells [135]. Furthermore, it may also be possible toimprove efficiency of ocean models with semi-implicittimestepping schemes. Here, ML could be used toprecondition solver for the large linear problem thatneeds to be solved in every timestep by estimating theresults [3].

The exponential growth of computing power hasbeen accompanied by an exponential growth in datavolume. This growth represents a big challenge foroperational weather and climate predictions [12]. Asdata movement is very expensive and a bottleneck inperformance, ocean models need to be “data-centric”and the workflow of the model should be designed in away that is reducing data movement to a minimum.For example, data is conventionally simply writtento discs or tapes after a model simulation, to beretrieved by users afterwards for analysis. A data-centric workflow would process data on-the-fly beforeit is stored. ML, and in particular unsupervised ML,would be essential in enabling domain scientists toextract the relevant information in such a data-centricworkflow. However, such a workflow would also resultsin additional requirements in terms of the training ofstaff and the software and hardware infrastructure ofweather and climate centres [71].

Also of note is the increased difficulty in extractingscientifically interesting information from the vastamounts of data produced by numerical models thatis stored. The complexity and sheer size of many ofthese data hinder data dissemination and analysis andseverely hamper efforts to analyze the data and addressresearch goals. This emerging class of problems canbe illustrated by the Coupled Model IntercomparisonProject (CMIP) ensemble now in its’ sixth phase,which is expected to generate an estimated 40,000TB of climate model data, a 20-fold increase in datavolume from the previous phase [86, 84]. Manyvariables needed for analysis are effectively unavailabledue the difficulty in saving or sharing the data. MLhas the capacity to efficiently analyze large datasetsas shown in section 2 and 3, but it has also beenused to infer, for example, information about sub-surface currents [50, 155], eddy heat fluxes [99] andfull 3-dimensional dynamics in CMIP6 [206]. ML inmany forms has the potential to be highly valuablefor researchers interested in the analysis of data thatis increasingly large, potentially sparse, and partiallyunavailable for logistical reasons [85].


4.3. Enforcing physical priors in ML methods

When physical constraints are enforced within MLtechniques, this is equivalent to incorporating physicalunderstanding into the applications. Using statisticallanguage, we can describe this process as “enforcingphysical priors”. ML techniques backed by massivedatasets have achieved groundbreaking results invision, speech, and natural language processing, butthey have yet to reach the physical oceanographycommunity or largely the physical sciences in general.The ocean is governed by complex phenomena thathave been studied by oceanographers for centuries,and taking advantage of this scientific heritageis one way of helping ML techniques reduce thesearch space of solutions, i.e. by guiding themusing physical theories. This research direction isincreasingly attracting attention as it helps constrainML algorithms to be physically plausible and alsofacilitates the interpretation of the results by domainexperts. There is a broad spectrum of techniquesto supplement ML with physical constraints [236], ofwhich only the most directly relevant are discussedhere.

The simplest way to enforce physical priors isthrough the loss function used to train the ML model.Concretely, this is done by adding an error term relatedto the physical constraint that needs to be respected,such as a conservation law. For example, if theoutput field F in a regression problem need to bedivergence-free, the term ‖∇F‖ is added to the totalloss function to ensure that the divergence of F is closeto zero. This approach has its mathematical roots inthe theory of Lagrange multipliers. It can also be seenas a way of doing regularization, meaning that findingsolutions that generalize well to unseen data is morelikely. However, adding physical priors as terms in lossfunctions comes with a price, which is the problem ofweighting different loss terms to impose which onesare most important. This problem of weighting canbe solved using cross-validation techniques, but thesecould be prohibitive when the number of constraints ishigh.

A second strategy that has gained much attentionin the recent years is enforcing the constraints directlyin the mapping function used for learning. Thisstrategy is best suited to NNs given their flexibility andthe rich design choices that enable them to be tailoredto specific data. The NN architecture is designedwith the physical priors in mind. For example, if wealready know that the quantity we want to find is amultiplication of two quantities, then we can encodethis inside the neural net by creating two sub-networkswhose outputs are multiplied in the last layer [88, 37].

While enforcing physical priors has been a veryactive area of research in the atmospheric community

(see section 1.3), few papers investigating the potentialof combining ML and physics can be found in theocean science literature. In the following we citesome of these examples. Authors in [33] reconstructsubgrid eddy momentum forcing using ConvNets andfound that enforcing a constraint on global momentumconservation can best be done by either postprocessingthe ConvNet’s output or hardcoding a last layer inthe ConvNet that removes the spatial mean of thedata. [243] proposes to use an equation discoveryalgorithm, namely Relevance Vector Machines (RVM),for ocean eddy parameterizations. Few attempts havebeen made to forecast ocean variables using a mix ofphysical models and DL tools, notably in [26] whereauthors model an advection diffusion equation in aDL architecture used to forecast SST, while [82] tacklethe same problem by combining an autoencoder withideas from Lyapunov analysis, and [145], where aNN is embedded inside a one-layer quasi-geostrophicnumerical model to reduce its bias towards a 3D oceanmodel.

Enforcing physical priors by solving differentialequations with ML techniques is an active researchdirection that features the development of interestingtools for the ocean community, which are still under-exploited. Physically Informed Neural Networks(PINNs) [184] is a notable example of a techniquethat leverages the power of NNs to solve differentialequations such as the incompressible Navier-Stokesequation [126] without a need for mesh generation,which could accelerate model development. Otherrecent techniques for learning ordinary differentialequations using either NNs [52] or a combinationof NNs and physical-based components [183] are apromising line of research at the interface of NNs anddifferential equations, which to the best of the authors’knowledge has not yet been applied to ocean modeling.

5. From models to predictions

A basic goal and a test of the understanding of aphysical process is the ability to predict its behaviour.Predictions of the weather for several days are a majorgeoscientific success. Such forecasts have improvedwith the increasing availability of computationalpower and observational networks, as well as betteralgorithms and process understanding [21]. However,predictions of the Earth system on longer timescalesare still a major challenge. This is problematic, aspredictions often form the basis of decision making.Understanding model error, and combining modelswith observations, is also at the core of supportingdecision makers discussed in Section 6.


5.1. Model bias and model error

Bias and error in models are addressed through a sys-tematic process of improvements in our understanding,but the needs of decision support can be immediate.Constraining simulations using observations is the pro-cess of data assimilation, covered below in Section 5.2.But where errors are recalcitrant, oceanographers andapplied scientists in general use methods of “artificial”error reduction, driven by comparisons against data.A key early example related to the ocean’s role in cli-mate is the use of “q-flux adjustments” or simply fluxadjustments [154], where there was a persistent errorin evaporative flux from the ocean surface. The ad-justment to ameliorate this was a correction to restoreenergy balance to the coupled system by artificiallyadding a compensation term. This adjustment methodfell into disfavor owing to its blatant “fudge factor” na-ture [203], although recent studies indicate that “fluxadjusted” models continue to exhibit greater predictiveskill [228].

When assessing a prediction from a model,the accuracy of the output can be assessed bycomparing to a ’truth’ benchmark. Such a benchmarkcan for example be from observations or a targetmodel representation of the system. Observations,although mostly not complete, constitute a best guess.This process can also identify “structural error”,also mentioned in Section 1.3, indicating that themodel formulation itself is incorrect. Compared toobservations, model outputs can show differences thatcannot only be attributed to differences in initialconditions, but instead reflect errors within the modelitself discussed in Section 5.2 below. Some of theseerrors can be explained by unresolved scales in thediscretized version of numerical models, but modelerrors can also originate from incomplete physicalknowledge. For example, within a sub-gridscaleparameterisation the exact physics that need to berepresented may be unclear, as discussed in section4. Incomplete physical knowledge also impactsuncertainties in the parameters used, for examplein the coupling terms between model components.Within model error as a whole, there may be asystematic component, which is referred to as modelbias.

For post-processing of model output, statisticalmethods, related to ML, have been used to correctbiases (for example [202, 140] or flux adjustments).Bias correction methods are used frequently inoperational weather predictions with DL playing anever-increasing role [187, 13, 104]. However, usingdownscaling as described in Section 1.3, ML can alsobe used to relate model output with local information,such as the local topography at very high resolution orobservations that are available, to improve predictions

when model simulations have already finished. Calledup-scaling within the ML community, some of themapping procedures used for downscaling, such asGANs, even allow for uncertainty quantification [143].Within climate models, the LRP XAI method havesuccessfully been used to identify key model biasesfor certain prediction tasks [15], with potential forapplication to the ocean. However, the LRP methodapplication is still in its infancy.

5.2. Ocean data assimilation

5.2.1. Data assimilation methods: A brief historyand main assumptions Data assimilation (DA) is theprocess of constraining a theoretical representation of asystem, usually using a numerical or statistical model,using a collection of observations. The results of thisprocess typically include optimized estimates of (1) thetime-evolving state of the system (sometimes calledthe “trajectory”), (2) initial conditions, (3) boundaryconditions, and (4) other intrinsic model parameters(e.g. mixing coefficients). The optimization processtypically consists of correcting the values of theinitial conditions, boundary conditions, and modelparameters in order to minimize a selected model-datamisfit metric. To use the language of the theory ofdifferential equations, one may think of DA as a setof methods for rigorously identifying which solution,among the family of solutions to a system of differentialequations, best satisfies the given constraints.

Although there is a long history of DA innumerical weather prediction stretching across much ofthe 20th century, oceanographic DA only began in thelate 1980s. The first experiments were regional [192],followed a few years later by the ambitious WorldOcean Circulation Experiment (WOCE, [240]), anda community was subsequently assembled underthe Global Ocean Data Assimilation Experiment(GODAE, [22]). These first DA approaches used inweather and ocean prediction were directly derivedusing optimal interpolation [100] and were based onstrong assumptions, namely that the evolution model islinear and perfect and that the data error distributionis unbiased and well-represented by a Gaussian. Intime, DA algorithms evolved to relax some of theseassumptions, extending the scope of DA applicationsto the ocean.

The developments within DA have led to two mainsets of techniques. These are ensemble approaches,of which the ensemble Kalman filter (EnKF) is astandard example, and variational approaches suchas four dimensional variational assimilation (4DVar).Both classes of methods conceptually represent theabstract trajectory of the target system as a probabilitydistribution across possible trajectories. EnKFconstructs an ensemble of forecast states such that


the ensemble mean and the sample covariance areexpected to be the best estimates. A core assumptionis that the ensemble probability distribution can bewell-represented by a Gaussian function [83]. The4DVar method uses a linear model to calculate whichperturbations to the initial conditions, boundaryconditions, and parameters tend to increase theagreement between the time-evolving state of themodel and the observational constraints [58].

Each of the DA classes of methods are usedin their various flavours for both global or regionalstudies [159, 91, 229, 174, 144, 197]. DA is usedroutinely both in operational forecast and reanalysismode. DA is used in the framework of several nationaland international projects. In no particular order,examples include the ECCO§ project, ECMWF‖ orthe NOAA NCEP¶ Global Ocean Data AssimilationSystem (GODAS) in the USA.

In idealized comparisons between the two classesof methods, EnKF produces more accurate estimatesfor shorter assimilation windows, whereas 4DVar pro-duces more accurate estimates when data constraintsare sparse. For ocean applications, data is often sparse,making 4DVar attractive [130]. In practice, differ-ent DA approaches derived from optimal interpolation,3DVar, the EnKF, or 4DVar are used [60]. The typedepends on the application (e.g. short-term forecastor climate application), the available computing re-sources, the type of observations that are assimilated,and also likely the historical expertise in each group.

5.2.2. Model errors and ML within data assimilationHistorically, DA techniques mainly focus on theestimation of the state of the system, but theestimation of model error in the DA process isincreasingly important [47]. Several approaches thatare used to handle model error apply DA frameworksthat can be considered ML approaches [216, 55]. Theestimation of model errors is particularly importantif DA is being used to calculate forecasts over longtimescales, i.e. from sub-seasonal to decadal scales.This is of particular importance for ocean forecasts,where timescales are longer than in the atmosphere;DA has been shown to be effective in this context [232].

5.2.3. Data assimilation and ML Several studies havehighlighted the connection between DA and ML [1,30, 36, 95]. The connection is more direct with4DVar, in which a function that quantifies model-datadisagreement (i.e. a “cost function”) is minimizedusing a gradient descent algorithm, wherein an adjoint

§ Estimating the Circulation and Climate of the Ocean‖ The European Centre for Medium-Range Weather Forecasts¶ National Oceanic and Atmospheric Administration, NationalCenters for Environmental Prediction

model calculates the gradient. In this perspective,4DVar is approximately equivalent to the process oftraining of a neural network for regression. This isbecause the adjoint model can be seen as equivalent tothe gradient backpropagation process [122, 137].

There are several ways ML can be used incombination with a DA framework. First, a data-driven model can be used to emulate a numericalmodel, partially or totally to provide the forecast.The objective is then to correct the model error,or to decrease the computational cost [146]. Notethat emulation could become instrumental, since DAmethods increasingly rely on ensemble runs, which arecostly [46]. As DA allows one to bring the model andobservations close enough together to represent thesame physical situation, DA can in principle be used toextend the learning of parametrization to the learningof improved models directly from observations [38, 34],described further in the section 4. It is still unclearwhether observations are too sparse for this approachto be successful within ocean modeling, in particular asthe time period where dense observations are availableis relatively short, as compared to the long timescalesthat are important for ocean dynamics. Anotherbenefit of using an ML emulators arises because mostML tools, such as NNs, are easy to differentiate.Given the structure of NN (interconnected simpleoperators), and the libraries used to implement them,the computation of the gradient of the NN model isstraightforward and efficient. This means that thecomputation can be used to efficiently develop tangentlinear and adjoint model code, which is required forDA methods such as 4DVar [114]. This is noteworthy,because traditionally the development of tangent linearand adjoint models has required major efforts fromthe research community, either by manually coding anadjoint or by the semi-automatic process of algorithmicdifferentiation (e.g. [102]).

Second, ML can be instrumental in stronglycoupled DA. Strongly coupled DA consists of correctinga coupled system (e.g. ocean-atmosphere) in aunified way. This allows, for example, atmosphericobservations to constrain the ocean state and viceversa, which is not the case in uncoupled DA, whereonly ocean observations are used to constrain theocean system. Strongly coupled DA is expected tobe efficient but challenging due to the high variety oftemporal and spatial scales [181]. In this sense, MLcan be used to relax some strong assumptions of theDA algorithm (e.g. the assumption that the errorsfollow a Gaussian distribution), or to isolate relevantscales in observational and model states: a ML processcan learn to compute the DA correction in optimalspace. Some examples of this approach have beendeveloped [7, 152, 105, 87], but so far none of them


have been applied to realistic ocean DA setups.Finally, ML can help deal with the mass of

available observations. In section 2, we discussedhow ML can help derive new type of productsfrom observations. These new products are goodcandidates for inclusion in a DA system. ML canalso be used to provide more accurate and/if or fasterobservation operators, for example to emulate satelliteobservations [34].

6. Discussion: towards a new synthesis ofobservations, theory, modeling, and predictionin ocean sciences using ML

6.1. The need for transparent ML

To increase confidence in the use of ML, steppingout of the “black box” is advisable. Towardsthis, having ML methods be transparent is veryimportant. A transparent ML application is onewhere source of skill is known, or put differentlywhy the ML came to its conclusion. Possibly thelargest hurdles for ML adoption are a lack of trustand the difficulty of generalization. These two arelinked, and if generalization is not reached trustis certainly not merited for ML applications withinoceanography. Generalization refers to a model’sability to properly adapt to new, previously unseendata. Within oceanography and beyond, the idealgeneralization would come from the ML applicationlearning the underlying physics. With a lack of gooddata coverage, the possibility underspecification [62]and shortcut learning [96] are important to keep inmind, where a model can seemingly perform wellfor example in the current climate but will fail ina future scenario as something physical was notlearned. These issues are ubiquitous and not uniqueto oceanography or to Earth science, with a call for‘physics informed’ ML[189]. Accordingly, the field ofML already has, and is, developing methods to addressthese issues such as “few shot learning” and “transferlearning”. If it is possible to reliably quantify andaccount for the uncertainties associated with an MLapplication during training, this could also increaseconfidence in the model, but assessing the reliabilitymay face challenges similar to underspecification.Recent works show promising progress toward thisby learning a probability distribution of outcomesthat can be stochastically sampled[75],[106]. Foruncertainty quantification, the uncertainty could bedetermined during training with ML, likely increasingthe reliability of the results. Other methods such asregularisation, invariances, dimensionality reductionsare also a powerful tool to increase the generalizationskill. For climate applications, a key issue whentraining ML applications is that the system they

are being trained on is largely non-stationary. Thiscomplicates the problem of generalization even further,but ML methods have demonstrated that having goodgeneralization skills in a non-stationary context ispossible [179]. Increasingly, the ML community issuggesting a focus on using IAI [195], driven amongother things by the consistent racial and gender biasrevealed in DL applications. With the ability tointerpret the ML model itself, and intuitively discernif it is meaningful, the danger of introducing suchbias is likely reduced dramatically. Similarly, XAImethods for example for NN, that retrospectivelyexplain the source of ML predictive skill, can alsohelp inspire confidence [160, 221, 78]. XAI methodssuch as layerwise relevance propagation (LRP [178, 9])have been gaining traction within the atmosphere[16,15, 39, 221], and ocean, but making their applicationexplicitly appropriate to oceanography, and indeedthe physical sciences in general, may require targetedmethod development.

6.2. Decision support

There is a need for accurate and actionable informationabout the ocean for a wide range of decision making.As noted above in Section 5.1, the need for actionablepredictions and decision support can short-circuitthe scientific process of error elimination. This isbecause the information may have “customers”/userswith an immediate need: for example decisions onshorelines ranging from building seawalls, issuinghousing permits, to setting insurance premiums. MLmay play a role in bridging the gap between whatmodel-based predictions are able to provide, and whatusers wish to know. The role of data-driven methodscould be particularly important for in filling in thegaps where theory and models underspecify the system,potentially leaving considerable uncertainty as noted inSection 5.2.

The reliable quantification of uncertainties is of-ten essential to support decision making. However,uncertainty quantification is often difficult for conven-tional approaches used in ocean science. This is be-cause model errors cannot be described by physicalequations or physical reasoning in most applications,and errors are often noisy and non-linear. On the otherhand, model error can often be diagnosed against a ref-erence truth, such as observations or target model sim-ulations. Therefore, ML can be useful for the quantifi-cation of uncertainties. In particular as datasets fromdifferent data sources, on different reference grids andfor different variables can be fused and compared us-ing ML techniques. For example, ML can be used topost-process ensemble simulations [202], and BayesianML techniques can also be used to learn the uncer-tainty quantification together with the ML tool. In


addition, targeted loss function design could help tar-get, and lessen areas of uncertainty important for spe-cific decisions. In an ML context, the models can becalibrated or tuned toward a particular loss function([119]), and for specific decision support those metricscould be specifically those required for the particularapplication. An open area of research remains in re-lating the results that may be obtained from differentcalibrations (loss functions) of the same system trainedon the same data.

ML can help to map model data and observationsto predict or detect events for which we cannot providea useful physical representation of the interactions.This could, for example, be a mapping fromobservational data of a time series in a specific localor observations from a buoy, to large-scale modeldata with the goal of making customised predictionsof surface waves and local wind. Such data couldfor example be used for a sailing competition. Suchtools based on ML could become essential for decisionsupport, for example when used to predict sea levels[230]. ML based mapping tools could also be usefulto inform where more observational data is needed,for example when deciding where to sample on acruise or where to send autonomous platforms. Todate, satellite images are largely used, but addedguidance from ML techniques could be very valuable,particularly if sub-surface observations are the target[155, 49, 206]. ML may eventually be used tosupport observational campaigns in near-real time byinteractively connecting networks of non-autonomousand autonomous observing platforms (e.g. gliders)to decision planning systems. These systems cantake environmental conditions, target observations,and task scheduling into account. The vision is tohave a “cyberinfrastructure” that can maximize thespatiotemporal coverage of the observations without aspecific need for human intervention. The potential useof such observational planning and adjustment systemsis being explored by international initiatives such asthe Southern Ocean Observing System (SOOS, https://www.soos.aq/). Similarly, for planing legislation,having knowledge of what is within a nations marinearea and how this may connect to the surroundingocean can be very valuable. Here ML has been usedto provide actionable information [205], as the oceandoes not adhere to borders drawn by humans.

Next to DL methods, the calibration of parame-ters is very important as many parameters within at-mosphere and ocean models cannot be validated withintheir physical uncertainty range and need to be tuned[222, 54]. Given this physical uncertainty, using MLand DL in particular will likely be very valuable asnoted in Section 4. If successful, such breakthroughscould help inform a wide range of decisions including

those based on climate models such as CMIP, or ina more general sense. This is particularly the casefor longer timescale integrations from seasonal and on-ward, due to the longer timescale active within theocean.

An important component of supporting decisionmakers is communication. The ability to communicateeffectively between the people that are makingdecisions and oceanographers can pose a problem.Oceanographers would need to be aware of what isuseful information, and how to provide this. Decisionmakers largely may not have intimate knowledge ofwhat available tools are capable of addressing, butmainly knowledge of the problem at hand. Whileseeming trivial, improving this line of communicationis an important component of increasing the utility ofoceanographic work.

6.3. Challenges and opportunities

In this review, we have highlighted some of the manychallenges within observational, computational, andtheoretical oceanography where ML offers an excitingopportunity to improve the speed and efficiency ofconventional work and also to explore completely newavenues. As a merger of two distinct fields, there isample opportunity to incorporate powerful, establishedML methods that are largely new to oceanography as afield. While not without risk, the potential benefits ofML methods is creating increasing interest in the field.This review has presented some of the challenges andopportunities involved when leveraging ML techniquesto improve the modelling, observing, fundamentalunderstanding, and prediction of the ocean system.

ML applications fundamentally rely on the dataavailable for learning, and here the ocean presents aunique challenge for ML applications. The importanttimescales in the ocean range from seconds tomillennia, with strong interactions between processesacross those scales. For example, a wind gust cantrigger a phytoplankton bloom. Observations arelargely sparse, noisy, and unbalanced. Temporally,very few long-timescale observations exist that spanmore than a few decades. A general problemwith models of the ocean, either ML derived ormore conventional, is that the system is highly non-stationary. With climate change, the mean state andits variance are liable to change, and a model that istrained from today’s data may not be general enoughto accurately represent an ocean in a warmer climate.Other components of the Earth system such as landor atmospheric models, or GFD in general, also facesimilar challenges, but they are exsaserbated withinoceanography due to the lack of spatial and temporalobservational coverage.

ML offers many avenues with which the challenges

https://www.soos.aq/

https://www.soos.aq/

REFERENCES 20

listed above could be tackled. For example, withinstantaneous processes (such as radiative transfer)or small spatial scale problems (for example eddydetection), a cross-validation approach with anassociated independent test dataset could be fruitful.Indeed, cross-validation is widely advisable. On longertimescales, methods related to physical constraintswould likely offer better results. Hybrid approaches forcombining physics-driven models and ML models arebecoming increasingly useful to aid the developmentof ocean models and to increase their computationalefficiency on HPC platforms. Such ‘Neural EarthSystem Models’ (NESYM[124]) can, for example, useML for parameterization of sub-gridscale processes.Pairings of ML and conventional methods also showgreat promise for improving signal-to-noise ratiosduring training while also anchoring ML learning toa stronger physical foundation [233].

Both the field of oceanography and ML arequickly evolving, and the computational tools availableto implement ML techniques are also becomingincreasingly accessible. With ample enthusiasm forML applications to address oceanographic problems,it is also important to keep in mind that ML asa field is largely not concerned with the physicalsciences. Approaching ML applications with cautionand care is necessary to ensure meaningful results.The importance of increasing trust in ML methodsalso highlights a need for collaboration betweenoceanographers and ML domain experts. ML as a fieldis developing very swiftly, and promoting collaborationcan help develop methods that are tailored to also suitthe needs of oceanographers.

This review has outlined the recent advancesand some remaining challenges associated with MLadoption within oceanography. As with any promisingnew set of methods, while there is ample opportunity,it is also worth noting that ML adoption also comeswith risk. However, exploring the full potential andcharting the limits of ML within oceanography iscrucial and deserves considerable attention from theresearch community.

Acknowledgments

MS and VB acknowledge funding from the CooperativeInstitute for Modeling the Earth System, PrincetonUniversity, under Award NA18OAR4320123 from theNational Oceanic and Atmospheric Administration,U.S. Department of Commerce. RL and VBacknowledge funding from the French Government’sMake Our Planet Great Again program managedby the Agence National de Recherche under the“Investissements d’avenir” award ANR-17-MPGA-0010.

DJ acknowledges funding from a UKRI FutureLeaders Fellowship (reference MR/T020822/1).

PD gratefully acknowledges funding from theRoyal Society for his University Research Fellow-ship as well as the ESiWACE, MAELSTROM andAI4Copernicus under Horizon 2020 and the EuropeanHigh-Performance Computing Joint Undertaking (JU;grant agreement No 823988, 955513 and 101016798).The JU received funding from the European High-Performance Computing Joint Undertaking (JU) un-der grant agreement No 955513. The JU receives sup-port from the European Union’s Horizon 2020 researchand innovation programme and United Kingdom, Ger-many, Italy, Luxembourg, Switzerland, Norway.

JB acknowledges funding from the projectSFE(#2700733) of the Norwegian Research Council.Many thanks to Laurent Bertino (NERSC) for the in-sightful discussion about data assimilation.

The authors also wish to thank Youngrak Cho forinvaluable help with Fig. 1 and 3.

References

[1] Henry DI Abarbanel, Paul J Rozdeba, and SashaShirman. Machine learning: Deepest learningas statistical data assimilation problems. Neuralcomputation, 30(8):2025–2055, 2018.

[2] Cleveland Abbe. The physical basis of long-rangeweather forecasts. Monthly Weather Review,29(12):551–561, 1901.

[3] Jan Ackmann, Peter D. Dueben, Tim N. Palmer,and Piotr K. Smolarkiewicz. Machine-learnedpreconditioners for linear solvers in geophysicalfluid flows, 2020.

[4] Rilwan Adewoyin, Peter Dueben, Peter Watson,Yulan He, and Ritabrata Dutta. Tru-net: A deeplearning approach to high resolution prediction ofrainfall, 2021.

[5] N. Agarwal, D.Kondrashov, P. Dueben,E.Ryzhov, and P. Berloff. A comparison ofdata-driven approaches to build low-dimensionalocean models. Submitted to JAMES, 2021.

[6] Aida Alvera-Azcarate, Alexander Barth, GaelleParard, and Jean-Marie Beckers. Analysis ofsmos sea surface salinity data using dineof.Remote sensing of environment, 180:137–145,2016.

[7] Maddalena Amendola, Rossella Arcucci, LaetitiaMottet, Cesar Quilodran Casas, Shiwei Fan,Christopher Pain, Paul Linden, and Yi-Ke Guo.Data assimilation in the latent space of a neuralnetwork, 2020.

[8] S Aoki, Kaihe Yamazaki, Daisuke Hirano,K Katsumata, K Shimada, Y Kitade, H Sasaki,

REFERENCES 21

and H Murase. Reversal of freshening trendof antarctic bottom water in the australian-antarctic basin during 2010s. Scientific reports,10:14415, 09 2020.

[9] Sebastian Bach, Alexander Binder, GregoireMontavon, Frederick Klauschen, Klaus-RobertMuller, and Wojciech Samek. On pixel-wiseexplanations for non-linear classifier decisions bylayer-wise relevance propagation. PLOS ONE,10(7):1–46, 07 2015.

[10] V. Balaji. Climbing down Charney’s lad-der: machine learning and the post-Dennardera of computational climate science. Philo-sophical Transactions of the Royal Society A:Mathematical, Physical and Engineering Sci-ences, 379(2194):20200085, April 2021. Pub-lisher: Royal Society.

[11] V. Balaji, E. Maisonnave, N. Zadeh, B. N.Lawrence, J. Biercamp, U. Fladrich, G. Aloisio,R. Benson, A. Caubel, J. Durachta, M.-A. Fou-jols, G. Lister, S. Mocavero, S. Underwood, andG. Wright. CPMIP: measurements of real com-putational performance of Earth system mod-els in CMIP6. Geoscientific Model Development,10(1):19–34, 2017.

[12] V. Balaji, K. E. Taylor, M. Juckes, B. N.Lawrence, P. J. Durack, M. Lautenschlager,C. Blanton, L. Cinquini, S. Denvil, M. Elk-ington, F. Guglielmo, E. Guilyardi, D. Has-sell, S. Kharin, S. Kindermann, S. Nikonov,A. Radhakrishnan, M. Stockhause, T. Weigel,and D. Williams. Requirements for a global datainfrastructure in support of cmip6. GeoscientificModel Development, 11(9):3659–3680, 2018.

[13] Agnes Baran, Sebastian Lerch, Mehrez El Ayari,and Sandor Baran. Machine learning for totalcloud cover prediction. Neural Computing andApplications, 33(7):2605–2620, Jul 2020.

[14] Lorena A Barba and Rio Yokota. How will thefast multipole method fare in the exascale era.SIAM News, 46(6):1–3, 2013.

[15] Elizabeth A Barnes, Kirsten Mayer, BenjaminToms, Zane Martin, and Emily Gordon. Iden-tifying opportunities for skillful weather predic-tion with interpretable neural networks. arXivpreprint arXiv:2012.07830, 2020.

[16] Elizabeth A. Barnes, Benjamin Toms, James W.Hurrell, Imme Ebert-Uphoff, Chuck Anderson,and David Anderson. Indicator Patterns ofForced Change Learned by an Artificial NeuralNetwork. Journal of Advances in Modeling EarthSystems, 12(9), 2020.

[17] T. P. Barnett and R. Preisendorfer. Originsand Levels of Monthly and Seasonal Forecast

Skill for United States Surface Air TemperaturesDetermined by Canonical Correlation Analy-sis. Monthly Weather Review, 115(9):1825–1850,September 1987. Publisher: American Meteo-rological Society Section: Monthly Weather Re-view.

[18] Alexander Barth, Aida Alvera-Azcarate, MatjazLicer, and Jean-Marie Beckers. Dincae 1.0:a convolutional neural network with errorestimates to reconstruct sea surface temperaturesatellite observations. Geoscientific ModelDevelopment, 13(3):1609–1622, 2020.

[19] P. Bauer, P.D. Dueben, T. Hoefler, T. Quintino,T.C. Schulthess, and N.P. Wedi. The digitalrevolution of earth-system science. Nat ComputSci, 1:104 – 113, 2021.

[20] Peter Bauer, Tiago Quintino, Nils Wedi, An-tonino Bonanni, Marcin Chrust, Willem De-coninck, Michail Diamantakis, Peter Duben,Stephen English, Johannes Flemming, PaddyGillies, Ioan Hadade, James Hawkes, MikeHawkins, Olivier Iffrig, Christian Kuhnlein,Michael Lange, Peter Lean, Olivier Marsden,Andreas Muller, Sami Saarinen, Domokos Sar-many, Michael Sleigh, Simon Smart, Piotr Smo-larkiewicz, Daniel Thiemert, Giovanni Tumolo,Christian Weihrauch, Cristiano Zanna, and Pe-dro Maciel. The ecmwf scalability programme:Progress and plans. (857), 02 2020.

[21] Peter Bauer, Alan Thorpe, and Gilbert Brunet.The quiet revolution of numerical weatherprediction. Nature, 525(7567):47–55, 2015.

[22] Michael J. Bell, Michel Lefebvre, Pierre-Yves LeTraon, Neville Smith, and Kirsten Wilmer-Becker. Godae: The global ocean dataassimilation experiment. Oceanography, 22(3),September 2009.

[23] Rasmus E Benestad, Inger Hanssen-Bauer, andDeliang Chen. Empirical-Statistical Downscal-ing. World Scientific, 2008.

[24] Yoshua Bengio, Ian Goodfellow, and AaronCourville. Deep learning, volume 1. MIT pressMassachusetts, USA:, 2017.

[25] Tom Beucler, Michael Pritchard, Stephan Rasp,Jordan Ott, Pierre Baldi, and Pierre Gentine.Enforcing Analytic Constraints in Neural Net-works Emulating Physical Systems. Physical Re-view Letters, 126(9):098302, March 2021. Pub-lisher: American Physical Society.

[26] Emmanuel de Bezenac, Arthur Pajot, andPatrick Gallinari. Deep learning for physical pro-cesses: incorporating prior scientific knowledge.Journal of Statistical Mechanics: Theory and Ex-periment, 2019(12):124009, December 2019.

REFERENCES 22

[27] Gerard Biau and Erwan Scornet. A randomforest guided tour. Test, 25(2):197–227, 2016.

[28] Christopher M Bishop. Pattern recognition andmachine learning. springer, 2006.

[29] Vilhelm Bjerknes. Das problem derwettervorhers-age, betrachtet vom stand-punkte der mechanik und der physik. Meteor.Z., 21:1–7, 1904.

[30] Marc Bocquet, Julien Brajard, Alberto Carrassi,and Laurent Bertino. Data assimilation asa learning tool to infer ordinary differentialequation representations of dynamical models.Nonlinear Processes in Geophysics, 26(3):143–162, 2019.

[31] Lars Boehme and Isabella Rosso. Classify-ing oceanographic structures in the amundsensea, antarctica. Geophysical Research Letters,48(5):e2020GL089412, 2021. e2020GL0894122020GL089412.

[32] Thomas Bolton and Laure Zanna. Applicationsof deep learning to ocean data inference andsubgrid parameterization. Journal of Advancesin Modeling Earth Systems, 11(1):376–399, 2019.

[33] Thomas Bolton and Laure Zanna. Applicationsof deep learning to ocean data inference andsubgrid parameterization. Journal of Advancesin Modeling Earth Systems, 11(1):376–399, 2019.

[34] Massimo Bonavita and Patrick Laloyaux. Ma-chine learning for model error inference and cor-rection. Journal of Advances in Modeling EarthSystems, page e2020MS002232, 2020.

[35] Sandrine Bony, Bjorn Stevens, Isaac H Held,John F Mitchell, Jean-Louis Dufresne, Kerry AEmanuel, Pierre Friedlingstein, Stephen Griffies,and Catherine Senior. Carbon dioxide andclimate: perspectives on a scientific assessment.In Climate Science for Serving Society, pages391–413. Springer, 2013.

[36] Julien Brajard, Alberto Carrassi, Marc Bocquet,and Laurent Bertino. Connections between dataassimilation and machine learning to emulate anumerical model. In NCAR, editor, proceedingsof the 9th International Workshop on Climateinformatics. NCAR, 2019.

[37] Julien Brajard, Alberto Carrassi, Marc Bocquet,and Laurent Bertino. Combining data assimila-tion and machine learning to emulate a dynami-cal model from sparse and noisy observations: Acase study with the Lorenz 96 model. Journal ofComputational Science, 44:101171, 2020.

[38] Julien Brajard, Alberto Carrassi, Marc Bocquet,and Laurent Bertino. Combining data assimi-lation and machine learning to infer unresolved

scale parametrisation. Phil. Trans. R. Soc. A,379, 2021.

[39] Noah D. Brenowitz, Tom Beucler, MichaelPritchard, and Christopher S. Bretherton. In-terpreting and Stabilizing Machine-LearningParametrizations of Convection. Journal ofthe Atmospheric Sciences, 77(12):4357–4375, dec2020.

[40] Noah D Brenowitz and Christopher S Brether-ton. Prognostic validation of a neural networkunified physics parameterization. GeophysicalResearch Letters, 45(12):6289–6298, 2018.

[41] Christopher S. Bretherton, Catherine Smith,and John M. Wallace. An Intercomparisonof Methods for Finding Coupled Patterns inClimate Data. Journal of Climate, 5(6):541–560,June 1992. Publisher: American MeteorologicalSociety Section: Journal of Climate.

[42] Steven L. Brunton, Joshua L. Proctor, andJ. Nathan Kutz. Discovering governing equationsfrom data by sparse identification of nonlineardynamical systems. Proceedings of the NationalAcademy of Sciences, 113(15):3932–3937, April2016.

[43] Kirk Bryan. A numerical method for the studyof the circulation of the world ocean. Journal ofcomputational physics, 135(2):154–169, 1997.

[44] Kirk Bryan and Michael D Cox. A nonlinearmodel of an ocean driven by wind and differentialheating: Part i. description of the three-dimensional velocity and density fields. Journalof Atmospheric Sciences, 25(6):945–967, 1968.

[45] Jared L. Callaham, James V. Koch, Bingni W.Brunton, J. Nathan Kutz, and Steven L.Brunton. Learning dominant physical processeswith data-driven balance models. NatureCommunications, 12(1), Feb 2021.

[46] Alberto Carrassi, Marc Bocquet, LaurentBertino, and Geir Evensen. Data assimilationin the geosciences: An overview of methods, is-sues, and perspectives. Wiley InterdisciplinaryReviews: Climate Change, 9(5):e535, 2018.

[47] Alberto Carrassi and Stephane Vannitsem.Accounting for model error in variationaldata assimilation: A deterministic formulation.Monthly Weather Review, 138(9):3369–3386,2010.

[48] David Cartwright. On The Origins Of KnowledgeOf The Sea Tides From Antiquity To TheThirteenth Century. Earth Sciences History,20(2):105–126, November 2007.

[49] Christopher Chapman and Anastase AlexandreCharantonis. Reconstruction of Subsurface Ve-

REFERENCES 23

locities From Satellite Observations Using Iter-ative Self-Organizing Maps. IEEE Geoscienceand Remote Sensing Letters, 14(5):617–620, May2017.

[50] Christopher Chapman and Anastase AlexandreCharantonis. Reconstruction of subsurface ve-locities from satellite observations using iterativeself-organizing maps. IEEE Geoscience and Re-mote Sensing Letters, 14(5):617–620, 2017.

[51] Christopher Chapman, Mary-Anne Lea, AmelieMeyer, Jean-baptiste Sallee, and Mark Hindell.Defining southern ocean fronts and their influ-ence on biological and physical processes in achanging climate. Nature Climate Change, 10:1–11, 02 2020.

[52] Ricky TQ Chen, Yulia Rubanova, Jesse Bet-tencourt, and David Duvenaud. Neural or-dinary differential equations. arXiv preprintarXiv:1806.07366, 2018.

[53] Michael Chui, James Manyika, Mehdi Miremadi,Nicolaus Henke, Rita Chung, Pieter Nel, andSankalp Malhotra. Notes from the ai frontierinsights from hundreds of use cases. DiscussionPaper https://www.mckinsey.com/featured-insights/artificial-intelligence/notes-from-the-ai-frontier-applications-and-value-of-deep-learning,2018.

[54] Emmet Cleary, Alfredo Garbuno-Inigo, ShiweiLan, Tapio Schneider, and Andrew M. Stuart.Calibrate, emulate, sample. Journal of Compu-tational Physics, 424:109716, 2021.

[55] Tadeo J Cocucci, Manuel Pulido, MagdalenaLucini, and Pierre Tandeo. Model errorcovariance estimation in particle and ensemblekalman filters using an online expectation–maximization algorithm. Quarterly Journal ofthe Royal Meteorological Society, 147(734):526–543, 2021.

[56] Fenwick C. Cooper and Laure Zanna. Optimisa-tion of an idealised ocean model, stochastic pa-rameterisation of sub-grid eddies. Ocean Mod-elling, 88:38–53, 2015.

[57] Corinna Cortes and Vladimir Vapnik. Support-vector networks. Machine learning, 20(3):273–297, 1995.

[58] Philippe Courtier, J-N Thepaut, and AnthonyHollingsworth. A strategy for operational im-plementation of 4d-var, using an incremental ap-proach. Quarterly Journal of the Royal Meteoro-logical Society, 120(519):1367–1387, 1994.

[59] Fleur Couvreux, Frederic Hourdin, DannyWilliamson, Romain Roehrig, Victoria Volodina,Najda Villefranque, Catherine Rio, Olivier Au-douin, James Salter, eric bazile, Florent Brient,

Florence Favot, Rachel Honnert, Marie-PierreLefebvre, Jean-Baptiste Madeleine, QuentinRodier, and Wenzhe Xu. Process-based climatemodel development harnessing machine learn-ing: I. a calibration tool for parameterizationimprovement, July 2020. Archive Location:world Publisher: Earth and Space Science OpenArchive Section: Atmospheric Sciences.

[60] James Cummings, Laurent Bertino, PierreBrasseur, Ichiro Fukumori, Masafumi Kamachi,Matthew J Martin, Kristian Mogensen, PeterOke, Charles Emmanuel Testut, Jacques Verron,et al. Ocean data assimilation systems for godae.Oceanography, 22(3):96–109, 2009.

[61] George Cybenko. Approximation by superposi-tions of a sigmoidal function. Mathematics ofcontrol, signals and systems, 2(4):303–314, 1989.

[62] Alexander D’Amour, Katherine Heller, DanMoldovan, Ben Adlam, Babak Alipanahi, AlexBeutel, Christina Chen, Jonathan Deaton, Ja-cob Eisenstein, Matthew D. Hoffman, FarhadHormozdiari, Neil Houlsby, Shaobo Hou, Ghas-sen Jerfel, Alan Karthikesalingam, Mario Lucic,Yian Ma, Cory McLean, Diana Mincu, AkinoriMitani, Andrea Montanari, Zachary Nado, VivekNatarajan, Christopher Nielson, Thomas F. Os-borne, Rajiv Raman, Kim Ramasamy, RorySayres, Jessica Schrouff, Martin Seneviratne,Shannon Sequeira, Harini Suresh, Victor Veitch,Max Vladymyrov, Xuezhi Wang, Kellie Webster,Steve Yadlowsky, Taedong Yun, Xiaohua Zhai,and D. Sculley. Underspecification presents chal-lenges for credibility in modern machine learning,2020.

[63] Mike Davis. Late Victorian Holocausts. Verso,2001.

[64] Margaret Deacon. Scientists and the Sea,1650–1900: A Study of Marine Science. Rout-ledge, April 2018. Google-Books-ID: WRdWD-wAAQBAJ.

[65] Anna Denvil-Sommer, Marion Gehlen, MathieuVrac, and Carlos Mejia. Lsce-ffnn-v1: a two-step neural network model for the reconstructionof surface ocean pco 2 over the global ocean.Geoscientific Model Development, 12(5):2091–2105, 2019.

[66] Damien Desbruyeres, Leon Chafik, and Guil-laume Maze. A shift in the ocean circulation haswarmed the subpolar north atlantic ocean since2016. Communications Earth & Environment, 2,02 2021.

[67] Chris Ding and Xiaofeng He. ¡i¿k¡/i¿-meansclustering via principal component analysis. InProceedings of the Twenty-First International

REFERENCES 24

Conference on Machine Learning, ICML ’04,page 29, New York, NY, USA, 2004. Associationfor Computing Machinery.

[68] Hui Ding, Matthew Newman, Michael A.Alexander, and Andrew T. Wittenberg. Di-agnosing Secular Variations in RetrospectiveENSO Seasonal Forecast Skill Using CMIP5Model-Analogs. Geophysical Research Letters,46(3):1721–1730, 2019.

[69] Keith W. Dixon, John R. Lanzante, Mary JoNath, Katharine Hayhoe, Anne Stoner, AparnaRadhakrishnan, V. Balaji, and Carlos F.Gaitan. Evaluating the stationarity assumptionin statistically downscaled climate projections: ispast performance an indicator of future results?Climatic Change, pages 1–14, 2016.

[70] H. M. Van Den Dool. Searching for ana-logues, how long must we wait? Tel-lus A, 46(3):314–324, 1994. eprint:https://onlinelibrary.wiley.com/doi/pdf/10.1034/j.1600-0870.1994.t01-2-00006.x.

[71] Peter Duben, Umberto Modigliani, Alan Geer,Stephan Siemen, Florian Pappenberger, PeterBauer, Andy Brown, Martin Palkovic, BaudouinRaoult, Nils Wedi, and Vasileios Baousis.Machine learning at ecmwf: A roadmap for thenext 10 years, 01 2021.

[72] P. D. Dueben and P. Bauer. Challenges anddesign choices for global weather and climatemodels based on machine learning. GeoscientificModel Development, 11(10):3999–4009, 2018.

[73] Peter D. Dueben and T. N. Palmer. Bench-mark tests for numerical weather forecasts oninexact hardware. Monthly Weather Review,142(10):3809 – 3829, 01 Oct. 2014.

[74] Pierre Maurice Marie Duhem. La TheoriePhysique : Son Objet Et Sa Structure. Chevalier& Riviere, 1906.

[75] Oliver R. A. Dunbar, Alfredo Garbuno-Inigo,Tapio Schneider, and Andrew M. Stuart.Calibration and Uncertainty Quantification ofConvective Parameters in an Idealized GCM.arXiv:2012.13262 [math, stat], December 2020.arXiv: 2012.13262.

[76] D. I. Duncan, P. Eriksson, S. Pfreundschuh,C. Klepp, and D. C. Jones. On the distinctivenessof observed oceanic raindrop distributions. At-mospheric Chemistry and Physics, 19(10):6969–6984, 2019.

[77] Dale Durran. Numerical Methods for FluidDynamics With Applications to Geophysics,volume 32. Springer-Verlag, 01 2010.

[78] Imme Ebert-Uphoff and Kyle Hilburn. Evalu-ation, tuning and interpretation of neural net-works for working with images in meteorologicalapplications. Bulletin of the American Meteoro-logical Society, pages 1 – 47, 31 Aug. 2020.

[79] Carsten Eden and Dirk Olbers. Why westernboundary currents are diffusive: A link betweenbottom pressure torque and bolus velocity. OceanModelling, 32(1-2):14–24, 2010.

[80] Paul Edwards. A vast machine: computermodels, climate data, and the politics of globalwarming. The MIT Press, 2010.

[81] V.W. Ekman. On the influence of the earth’srotation on ocean currents. Arch. Math. Astron.Phys., 2 (11), 1905.

[82] N Benjamin Erichson, Michael Muehlebach,and Michael W Mahoney. Physics-informedautoencoders for lyapunov-stable fluid flowprediction. arXiv preprint arXiv:1905.10866,2019.

[83] Geir Evensen. Sequential data assimilation witha nonlinear quasi-geostrophic model using montecarlo methods to forecast error statistics. Journalof Geophysical Research: Oceans, 99(C5):10143–10162, 1994.

[84] Veronika Eyring, Sandrine Bony, Gerald Meehl,C. Senior, B. Stevens, Stouffer Ronald, andK. Taylor. Overview of the coupled model in-tercomparison project phase 6 (cmip6) experi-mental design and organisation. GeoscientificModel Development Discussions, 8:10539–10583,12 2015.

[85] Veronika Eyring, Peter Cox, Gregory Flato, Pe-ter Gleckler, Gab Abramowitz, Peter Caldwell,William Collins, Bettina Gier, Alex Hall, ForrestHoffman, George Hurtt, Alexandra Jahn, ChrisJones, Stephen Klein, John Krasting, LesterKwiatkowski, Ruth Lorenz, Eric Maloney, Ger-ald Meehl, and Mark Williamson. Taking climatemodel evaluation to the next level. Nature Cli-mate Change, 9:102–110, 02 2019.

[86] Veronika Eyring, Peter J Gleckler, ChristophHeinze, Ronald J Stouffer, Karl E Taylor, V Bal-aji, Eric Guilyardi, Sylvie Joussaume, StephanKindermann, Bryan N Lawrence, Gerald AMeehl, Mattia Righi, and Dean N Williams. To-wards improved and more routine Earth systemmodel evaluation in CMIP. Earth System Dy-namics, 7(4):813–830, 2016.

[87] Ronan Fablet, Bertrand Chapron, LucasDrumetz, Etienne Memin, Olivier Pannekoucke,and Francois Rousseau. Learning variationaldata assimilation models and solvers. arXivpreprint arXiv:2007.12941, 2020.

REFERENCES 25

[88] Ronan Fablet, Said Ouala, and Cedric Herzet.Bilinear residual neural network for the identi-fication and forecasting of dynamical systems.arXiv preprint arXiv:1712.07003, 2017.

[89] Raffaele Ferrari, James C McWilliams, Vitto-rio M Canuto, and Mikhail Dubovikov. Parame-terization of eddy fluxes near oceanic boundaries.Journal of Climate, 21(12):2770–2789, 2008.

[90] Eunice Foote. ircumstances affecting the heat ofthe sun’s rays: Art. xxxi. American Journal ofScience and Arts, XXII/no. LXVI:357–359, 1856.

[91] G Forget, J M Campin, P Heimbach, C N Hill,R M Ponte, and C Wunsch. ECCO version 4:an integrated framework for non-linear inversemodeling and global ocean state estimation.Geoscientific Model Development, 8(10):3071–3104, 2015.

[92] B Fox-Kemper, S Bachman, B Pearson, andS Reckinger. Principles and advances in subgridmodelling for eddy-rich simulations. ClivarExchanges, 19(2):42–46, 2014.

[93] Rachel Furner, Peter Haynes, Dave Munday,Brooks Paige, Daniel C. Jones, and EmilyShuckburgh. Sensitivity analysis of a data-drivenmodel of ocean temperature. Geoscientific ModelDevelopment, 2021.

[94] Alberto Garabato. A perspective on thefuture of physical oceanography. Philosophicaltransactions. Series A, Mathematical, physical,and engineering sciences, 370:5480–511, 12 2012.

[95] AJ Geer. Learning earth system modelsfrom observations: machine learning or dataassimilation? Philosophical Transactions of theRoyal Society A, 379(2194):20200089, 2021.

[96] Robert Geirhos, Jorn-Henrik Jacobsen, ClaudioMichaelis, Richard Zemel, Wieland Brendel,Matthias Bethge, and Felix A. Wichmann.Shortcut learning in deep neural networks.Nature Machine Intelligence, 2(11):665–673, Nov2020.

[97] Peter R. Gent and James C. Mcwilliams.Isopycnal mixing in ocean circulation models.Journal of Physical Oceanography, 20(1):150 –155, 01 Jan. 1990.

[98] P. Gentine, M. Pritchard, S. Rasp, G. Reinaudi,and G. Yacalis. Could machine learningbreak the convection parameterization deadlock?Geophysical Research Letters, 45(11):5742–5751,2018.

[99] Tom George, Georgy Manucharyan, and AndrewThompson. Deep learning to infer eddy heatfluxes from sea surface height patterns ofmesoscale turbulence, 11 2019.

[100] Michael Ghil and Paola Malanotte-Rizzoli. Dataassimilation in meteorology and oceanography.Advances in geophysics, 33:141–266, 1991.

[101] Subimal Ghosh and P. P. Mujumdar. Statisticaldownscaling of GCM simulations to streamflowusing relevance vector machine. Advances inWater Resources, 31(1):132–146, January 2008.

[102] Ralf Giering and Thomas Kaminski. Recipes foradjoint code construction. ACM Transactions onMathematical Software (TOMS), 24(4):437–474,1998.

[103] Ian J Goodfellow, Jean Pouget-Abadie, MehdiMirza, Bing Xu, David Warde-Farley, SherjilOzair, Aaron Courville, and Yoshua Bengio.Generative adversarial networks. arXiv preprintarXiv:1406.2661, 2014.

[104] Peter Groenquist, Chengyuan Yao, Tal Ben-Nun, Nikoli Dryden, Peter Dueben, ShigangLi, and Torsten Hoefler. Deep learningfor post-processing ensemble weather forecasts.Philosophical Transactions of the Royal SocietyA: Mathematical, Physical and EngineeringSciences, 379(2194):20200092, 2021.

[105] Ian Grooms. Analog ensemble data assimilationand a method for constructing analogs withvariational autoencoders. Quarterly Journal ofthe Royal Meteorological Society, 147(734):139–149, 2021.

[106] Arthur Guillaumin and Laure Zanna. StochasticDeep Learning parameterization of Ocean Mo-mentum Forcing, August 2021. Archive Loca-tion: world Publisher: Earth and Space ScienceOpen Archive Section: Oceanography.

[107] Sebastien Guimbard, Jerome Gourrion, MarcosPortabella, Antonio Turiel, Carolina Gabarro,and Jordi Font. Smos semi-empirical oceanforward model adjustment. IEEE transactionson geoscience and remote sensing, 50(5):1676–1687, 2012.

[108] Madec Gurvan, Romain Bourdalle-Badie,Jerome Chanut, Emanuela Clementi, AndrewCoward, Christian Ethe, Doroteaciro Iovino,Dan Lea, Claire Levy, Tomas Lovato, NicolasMartin, Sebastien Masson, Silvia Mocavero,Clement Rousset, Dave Storkey, Martin Van-coppenolle, Simon Mueller, George Nurser, MikeBell, and Guillaume Samson. Nemo ocean en-gine, October 2019. Add SI3 and TOP referencemanuals.

[109] T. Gysi, C. Osuna, O. Fuhrer, M. Bianco, andT. C. Schulthess. Stella: a domain-specifictool for structured grid methods in weather andclimate models. In SC ’15: Proceedings of the

REFERENCES 26

International Conference for High PerformanceComputing, Networking, Storage and Analysis,pages 1–12, 2015.

[110] Thomas Haine. What did the viking discoverersof america know of the north atlantic environ-ment? Weather, 63(3):60–65, 2008.

[111] Yoo-Geun Ham, Jeong-Hwan Kim, and Jing-JiaLuo. Deep learning for multi-year enso forecasts.Nature, 573(7775):568–572, 2019.

[112] K. Hanawa and L. Talley. Ocean circulationand climate, International Geophysics Series.International Geophysics Series (pp. 373–386).Cambridge, MA, USA: Academic Press, 2001.

[113] Sam Hatfield, Matthew Chantry, Peter Duben,and Tim Palmer. Accelerating high-resolutionweather models with deep-learning hardware.In Proceedings of the Platform for AdvancedScientific Computing Conference, PASC ’19,New York, NY, USA, 2019. Association forComputing Machinery.

[114] Samuel Edward Hatfield, Matthew Chantry, Pe-ter Dominik Dueben, Philippe Lopez, Alan JonGeer, and Tim N Palmer. Building tangent-linearand adjoint models for data assimilation withneural networks. Earth and Space Science OpenArchive, page 34, 2021.

[115] W Hazeleger and SS Drijfhout. Eddy subductionin a model of the subtropical gyre. Journal ofphysical oceanography, 30(4):677–695, 2000.

[116] Isaac Held. The gap between simulation andunderstanding in climate modeling. 86(11):1609–1614, September 2005.

[117] B Helland-Hansen. Nogen hydrografiskemetoder. forhandlinger ved de 16 skandinaviskenaturforsherermøte. Kristiania, 39:357–359,1916.

[118] Isabel A. Houghton and James D. Wilson. Elnino detection via unsupervised clustering ofargo temperature profiles. Journal of GeophysicalResearch: Oceans, 125(9):e2019JC015947, 2020.e2019JC015947 10.1029/2019JC015947.

[119] Frederic Hourdin, Thorsten Mauritsen, AndrewGettelman, Jean-Christophe Golaz, Venkatra-mani Balaji, Qingyun Duan, Doris Folini, Duoy-ing Ji, Daniel Klocke, Yun Qian, et al. The artand science of climate model tuning. Bulletin ofthe American Meteorological Society, 98(3):589–602, 2017.

[120] Frederic Hourdin, Danny Williamson, CatherineRio, Fleur Couvreux, Romain Roehrig, NajdaVillefranque, Ionela Musat, Fatoumata Bint Di-allo, Laurent Fairhead, and Victoria Volodina.

Process-based climate model development har-nessing machine learning: II. model calibrationfrom single column to global, May 2020. ArchiveLocation: world Publisher: Earth and Space Sci-ence Open Archive Section: Atmospheric Sci-ences.

[121] William W Hsieh. Machine learning methods inthe environmental sciences: Neural networks andkernels. Cambridge university press, 2009.

[122] William W Hsieh and Benyang Tang. Applyingneural network models to prediction and dataanalysis in meteorology and oceanography.Bulletin of the American Meteorological Society,79(9):1855–1870, 1998.

[123] Chris W Hughes and Beverly A De Cuevas. Whywestern boundary currents in realistic oceansare inviscid: A link between form stress andbottom pressure torques. Journal of PhysicalOceanography, 31(10):2871–2885, 2001.

[124] Christopher Irrgang, Niklas Boers, Maike Son-newald, Elizabeth A. Barnes, ChristopherKadow, Joanna Staneva, and Jan Saynisch-Wagner. Will artificial intelligence supersedeearth system and climate models?, 2021.

[125] Christopher Irrgang, Jan Saynisch, and MaikThomas. Estimating global ocean heat contentfrom tidal magnetic satellite observations. Sci-entific Reports, 9:7893, 2019.

[126] Xiaowei Jin, Shengze Cai, Hui Li, andGeorge Em Karniadakis. Nsfnets (navier-stokesflow nets): Physics-informed neural networks forthe incompressible navier-stokes equations. Jour-nal of Computational Physics, 426:109951, 2021.

[127] Gregory C Johnson and Harry L Bryden. On thesize of the antarctic circumpolar current. DeepSea Research Part A. Oceanographic ResearchPapers, 36(1):39–53, 1989.

[128] Daniel C. Jones, Harry J. Holt, Andrew J. S.Meijers, and Emily Shuckburgh. UnsupervisedClustering of Southern Ocean Argo FloatTemperature Profiles. Journal of GeophysicalResearch: Oceans, 124(1):390–402, 2019.

[129] Daniel C. Jones and Takamitsu Ito. Gaussianmixture modeling describes the geography of thesurface ocean carbon budget. In J. Brajard,A. Charantonis, C. Chen, and J. Runge, editors,Proceedings of the 9th International Workshopon Climate Informatics: CI 2019, pages 108–113. University Corporation for AtmosphericResearch (UCAR), 2019.

[130] Eugenia Kalnay, Hong Li, Takemasa Miyoshi,Shu-Chih Yang, and Joaquim Ballabrera-Poy.4-d-var or ensemble kalman filter? Tellus

REFERENCES 27

A: Dynamic Meteorology and Oceanography,59(5):758–773, 2007.

[131] Maria T. Kavanaugh, Matthew J. Oliver, Fran-cisco P. Chavez, Ricardo M. Letelier, Frank E.Muller-Karger, and Scott C. Doney. Seascapesas a new vernacular for pelagic ocean monitoring,management and conservation. ICES Journal ofMarine Science, 73(7):1839–1850, July 2016.

[132] Jeffrey R Key and Axel J Schweiger. Toolsfor atmospheric radiative transfer: Streamer andfluxnet. Computers & Geosciences, 24(5):443–451, 1998.

[133] Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprintarXiv:1312.6114, 2013.

[134] M. Kloewer, P. D. Dueben, and T. N. Palmer.Number formats, error mitigation, and scopefor 16-bit arithmetics in weather and climatemodeling analyzed with a shallow water model.Journal of Advances in Modeling Earth Systems,12(10):e2020MS002246, 2020. e2020MS00224610.1029/2020MS002246.

[135] Dmitrii Kochkov, Jamie A. Smith, Ayya Alieva,Qing Wang, Michael P. Brenner, and StephanHoyer. Machine learning accelerated computa-tional fluid dynamics, 2021.

[136] Teuvo Kohonen. Self-organized formation oftopologically correct feature maps. Biologicalcybernetics, 43(1):59–69, 1982.

[137] Nikola Borislavov Kovachki and Andrew MStuart. Ensemble Kalman Inversion: ADerivative-Free Technique For Machine LearningTasks. Inverse Problems, 2019.

[138] Mark A Kramer. Nonlinear principal componentanalysis using autoassociative neural networks.AIChE journal, 37(2):233–243, 1991.

[139] T. Kurth, S. Treichler, J. Romero,M. Mudigonda, N. Luehr, E. Phillips, A. Ma-hesh, M. Matheson, J. Deslippe, M. Fatica,P. Prabhat, and M. Houston. Exascale deeplearning for climate analytics. In SC18: In-ternational Conference for High PerformanceComputing, Networking, Storage and Analysis,pages 649–660, 2018.

[140] P. Laloyaux, M. Bonavita, M. Chrust, andS. Gurol. Exploring the potential and limitationsof weak-constraint 4d-var. Quarterly Journal ofthe Royal Meteorological Society, 146(733):4067–4082, 2020.

[141] PY Le Traon, F Nadal, and N Ducet. Animproved mapping method of multisatellitealtimeter data. Journal of atmospheric andoceanic technology, 15(2):522–534, 1998.

[142] Yann LeCun, Yoshua Bengio, and GeoffreyHinton. Deep learning. nature, 521(7553):436–444, 2015.

[143] J. Leinonen, D. Nerini, and A. Berne. Stochasticsuper-resolution for downscaling time-evolvingatmospheric fields with a generative adversarialnetwork. IEEE Transactions on Geoscience andRemote Sensing, pages 1–13, 2020.

[144] Jean-Michel Lellouche, Eric Greiner, OlivierLe Galloudec, Charly Regnier, Mounir Benkiran,Charles-Emmanuel Testut, Romain Bourdalle-Badie, Marie Drevillon, Gilles Garric, andYann Drillet. Mercator ocean global high-resolution monitoring and forecasting system.New Frontiers in Operational Oceanography,pages 563–592, 2018.

[145] Redouane Lguensat, Julien Le Sommer, SammyMetref, Emmanuel Cosme, and Ronan Fablet.Learning generalized quasi-geostrophic modelsusing deep neural numerical models. arXivpreprint arXiv:1911.08856, 2019.

[146] Redouane Lguensat, Pierre Tandeo, PierreAilliot, Manuel Pulido, and Ronan Fablet. Theanalog data assimilation. Monthly WeatherReview, 145(10):4093–4107, 2017.

[147] Julia Ling, Reese Jones, and Jeremy Templeton.Machine learning strategies for systems withinvariance properties. Journal of ComputationalPhysics, 318:22–35, August 2016.

[148] Edward N. Lorenz. The slow manifold—whatis it? Journal of Atmospheric Sciences,49(24):2449 – 2451, 15 Dec. 1992.

[149] Edward N Lorenz. Empirical orthogonalfunctions and statistical weather prediction,1956.

[150] Edward N Lorenz. Atmospheric predictabilityas revealed by naturally occurring analogues.Journal of Atmospheric Sciences, 26(4):636–646,1969.

[151] Edward N. Lorenz. Atmospheric predictabilityas revealed by naturally occurring analogues.Journal of the Atmospheric Sciences, 26(4):636–646, Jul 1969.

[152] Julian Mack, Rossella Arcucci, Miguel Molina-Solana, and Yi-Ke Guo. Attention-basedconvolutional autoencoders for 3d-variationaldata assimilation. Computer Methods in AppliedMechanics and Engineering, 372:113291, 2020.

[153] S. Manabe and K. Bryan. Climate calculationswith a combined ocean-atmosphere model. J.Atmos. Sci, 26(4):786–789, 1969.

[154] S. Manabe, R. J. Stouffer, M. J. Spelman,and K. Bryan. Transient Responses of a

REFERENCES 28

Coupled Ocean–Atmosphere Model to GradualChanges of Atmospheric CO2. Part I. AnnualMean Response. Journal of Climate, 4(8):785–818, August 1991. Publisher: AmericanMeteorological Society Section: Journal ofClimate.

[155] Georgy E. Manucharyan, Lia Siegelman, andPatrice Klein. A deep learning approach tospatiotemporal sea surface height interpolationand estimation of deep currents in geostrophicocean turbulence. Journal of Advances inModeling Earth Systems, 13(1):e2019MS001965,2021. e2019MS001965 2019MS001965.

[156] Elodie Martinez, Thomas Gorgues, MatthieuLengaigne, Clement Fontana, RaphaelleSauzede, Christophe Menkes, Julia Uitz,Emanuele Di Lorenzo, and Ronan Fablet.Reconstructing global chlorophyll-a variationsusing a non-linear statistical approach. Frontiersin Marine Science, 7:464, 2020.

[157] E. P. Maurer, H. G. Hidalgo, T. Das, M. D.Dettinger, and D. R. Cayan. The utility ofdaily large-scale climate data in the assessmentof climate change impacts on daily streamflowin California. Hydrology and Earth SystemSciences, 14(6):1125–1138, June 2010. Publisher:Copernicus GmbH.

[158] Guillaume Maze, Herle Mercier, Ronan Fa-blet, Pierre Tandeo, Manuel Lopez Radcenco,Philippe Lenca, Charlene Feucher, and le goffClement. Coherent heat patterns revealed byunsupervised classification of argo temperatureprofiles in the north atlantic ocean. Progress inOceanography, 151, 01 2017.

[159] Matthew R Mazloff, Patrick Heimbach, andCarl Wunsch. An Eddy-Permitting SouthernOcean State Estimate. Journal of PhysicalOceanography, 40(5):880–899, 2010.

[160] Amy McGovern, Ryan Lagerquist, DavidJohn Gagne, G. Eli Jergensen, Kimberly L. El-more, Cameron R. Homeyer, and Travis Smith.Making the Black Box More Transparent: Un-derstanding the Physical Implications of MachineLearning. Bulletin of the American Meteorolog-ical Society, 100(11):2175–2199, November 2019.Publisher: American Meteorological Society.

[161] Leland McInnes, John Healy, and JamesMelville. Umap: Uniform manifold approxi-mation and projection for dimension reduction.arXiv preprint arXiv:1802.03426, 2018.

[162] Geoffrey J McLachlan and Kaye E Basford.Mixture models: Inference and applications toclustering, volume 38. M. Dekker New York,1988.

[163] A. Merz and G Wust. Die atlantische vertikalzirkulation. 3 Beitrag. Zeitschr. D.G.F.E,Berlin, 1923.

[164] Adam Monahan, John Fyfe, Maarten Ambaum,David Stephenson, and Gerald North. Empiricalorthogonal functions: The medium is themessage. Journal of Climate, 22, 12 2009.

[165] Alvaro Montenegro, Richard T. Callaghan,and Scott M. Fitzpatrick. Using seafaringsimulations and shortest-hop trajectories tomodel the prehistoric colonization of remoteoceania. Proceedings of the National Academyof Sciences, 2016.

[166] Douglas C Montgomery, Elizabeth A Peck,and G Geoffrey Vining. Introduction to linearregression analysis. John Wiley & Sons, 2021.

[167] Walter H. Munk. On the wind-driven oceancirculation. Journal of Atmospheric Sciences,7(2):80 – 93, 01 Apr. 1950.

[168] Walter H Munk. On the wind-driven oceancirculation. Journal of meteorology, 7(2):80–93,1950.

[169] Walter Heinrich Munk and E Palmen. Noteon the dynamics of the antarctic circumpolarcurrent 1. Tellus, 3(1):53–55, 1951.

[170] Zied Ben Mustapha, Severine Alvain, CedricJamet, Hubert Loisel, and David Dessailly. Au-tomatic classification of water-leaving radianceanomalies from global seawifs imagery: applica-tion to the detection of phytoplankton groups inopen ocean waters. Remote Sensing of Environ-ment, 146:97–112, 2014.

[171] Jerome Namias. Recent seasonal interactionsbetween north pacific waters and the overlyingatmospheric circulation. Journal of GeophysicalResearch, 64(6):631–646, 1959.

[172] Frederik Nebeker. Calculating the weather:Meteorology in the 20th century. Elsevier, 1995.

[173] Philipp Neumann, Peter Duben, PanagiotisAdamidis, Peter Bauer, Matthias Bruck, LuisKornblueh, Daniel Klocke, Bjorn Stevens, NilsWedi, and Joachim Biercamp. Assessingthe scales in numerical weather and climatepredictions: will exascale be the rescue?Philosophical Transactions of the Royal SocietyA: Mathematical, Physical and EngineeringSciences, 377(2142):20180148, April 2019.

[174] An T Nguyen, Helen Pillar, Victor Ocana,Arash Bigdeli, Timothy A Smith, and PatrickHeimbach. The arctic subpolar gyre stateestimate (aste): Description and assessment of adata-constrained, dynamically consistent; ocean-sea ice estimate for 2002-2017. Earth and SpaceScience Open Archive, page 62, 2020.

REFERENCES 29

[175] Peer Nowack, Peter Braesicke, Joanna Haigh,Nathan Luke Abraham, John Pyle, and Aposto-los Voulgarakis. Using machine learning to buildtemperature-based ozone parameterizations forclimate sensitivity simulations. EnvironmentalResearch Letters, 13, 2018.

[176] AM Obukhov. Statistically homogeneous fieldson a sphere. Usp. Mat. Nauk, 2(2):196–198, 1947.

[177] Paul A O’Gorman and John G Dwyer. Usingmachine learning to parameterize moist convec-tion: Potential for modeling of climate, climatechange, and extreme events. Journal of Advancesin Modeling Earth Systems, 10(10):2548–2563,2018.

[178] Julian D Olden, Michael K Joy, and Russell GDeath. An accurate comparison of methodsfor quantifying variable importance in artificialneural networks using simulated data. EcologicalModelling, 178(3):389 – 397, 2004.

[179] Dhruvit Patel, Daniel Canaday, Michelle Girvan,Andrew Pomerance, and Edward Ott. Usingmachine learning to predict statistical propertiesof non-stationary dynamical processes: Systemclimate,regime transitions, and the effect ofstochasticity. Chaos: An InterdisciplinaryJournal of Nonlinear Science, 31(3):033149,2021.

[180] Etienne Pauthenet, Fabien Roquet, GurvanMadec, Jean-Baptiste Sallee, and David Nerini.The thermohaline modes of the global ocean.Journal of Physical Oceanography, 49(10):2535 –2552, 01 Oct. 2019.

[181] SG Penny, E Bach, K Bhargava, C-C Chang,C Da, L Sun, and T Yoshida. Stronglycoupled data assimilation in multiscale media:Experiments using a quasi-geostrophic coupledmodel. Journal of Advances in Modeling EarthSystems, 11(6):1803–1829, 2019.

[182] Norman A Phillips. The general circulationof the atmosphere: A numerical experiment.Quarterly Journal of the Royal MeteorologicalSociety, 82(352):123–164, 1956.

[183] Christopher Rackauckas, Yingbo Ma, JuliusMartensen, Collin Warner, Kirill Zubov, Ro-hit Supekar, Dominic Skinner, Ali Ramad-han, and Alan Edelman. Universal Differen-tial Equations for Scientific Machine Learning.arXiv:2001.04385 [cs, math, q-bio, stat], August2020. arXiv: 2001.04385.

[184] Maziar Raissi, Paris Perdikaris, and George EmKarniadakis. Physics informed deep learning(part i): Data-driven solutions of nonlinearpartial differential equations. arXiv preprintarXiv:1711.10561, 2017.

[185] Ali Ramadhan, Gregory LeClaire Wagner, ChrisHill, Jean-Michel Campin, Valentin Churavy,Tim Besard, Andre Souza, Alan Edelman, Raf-faele Ferrari, and John Marshall. Oceanani-gans.jl: Fast and friendly geophysical fluid dy-namics on gpus. Journal of Open Source Soft-ware, 5(53):2018, 2020.

[186] Stephan Rasp, Peter D. Dueben, SebastianScher, Jonathan A. Weyn, Soukayna Mouatadid,and Nils Thuerey. Weatherbench: A benchmarkdata set for data-driven weather forecasting.Journal of Advances in Modeling Earth Systems,12(11):e2020MS002203, 2020. e2020MS00220310.1029/2020MS002203.

[187] Stephan Rasp and Sebastian Lerch. Neural net-works for postprocessing ensemble weather fore-casts. Monthly Weather Review, 146(11):3885 –3900, 01 Nov. 2018.

[188] Stephan Rasp, Michael S Pritchard, and PierreGentine. Deep learning to represent subgridprocesses in climate models. Proceedings ofthe National Academy of Sciences, 115(39):9684–9689, 2018.

[189] Markus Reichstein, Gustau Camps-Valls, BjornStevens, Martin Jung, Joachim Denzler, NunoCarvalhais, and Prabhat. Deep learning andprocess understanding for data-driven Earthsystem science. Nature, 566(7743):195–204,February 2019. Number: 7743 Publisher: NaturePublishing Group.

[190] Lewis Fry Richardson. Weather prediction bynumerical process. Cambridge university press,2007.

[191] P.L. Richardson. On the history of merid-ional overturning circulation schematic dia-grams. Progr. Oceanogr., 76, 466e486, 2008.

[192] Allan R Robinson, Michael A Spall, Leonard JWalstad, and Wayne G Leslie. Data assimila-tion and dynamical interpolation in gulfcast ex-periments. Dynamics of atmospheres and oceans,13(3-4):301–316, 1989.

[193] Dean Roemmich, Gregory C. Johnson, StephenRiser, Russ Davis, John Gilson, W. BrechnerOwens, Silvia L. Garzoli, Claudia Schmid,and Mark Ignaszewski. The Argo Program:Observing the Global Ocean with ProfilingFloats. Oceanography, 22(2):34–43, 2009.Publisher: Oceanography Society.

[194] Isabella Rosso, Matthew R. Mazloff, Lynne D.Talley, Sarah G. Purkey, Natalie M. Freeman,and Guillaume Maze. Water mass and biogeo-chemical variability in the kerguelen sector of thesouthern ocean: A machine learning approach

REFERENCES 30

for a mixing hot spot. Journal of GeophysicalResearch: Oceans, 125(3):e2019JC015877, 2020.e2019JC015877 10.1029/2019JC015877.

[195] Cynthia Rudin. Stop explaining black boxmachine learning models for high stakes decisionsand use interpretable models instead. NatureMachine Intelligence, 1:206–215, 05 2019.

[196] E.A. Ryzhov, D. Kondrashov, N. Agarwal, J.C.McWilliams, and P. Berloff. On data-driveninduction of the low-frequency variability in acoarse-resolution ocean model. Ocean Modelling,153:101664, 2020.

[197] Pavel Sakov, F Counillon, L Bertino,KA Lisæter, PR Oke, and A Korablev. Topaz4:an ocean-sea ice data assimilation system forthe north atlantic and arctic. Ocean Science,8(4):633–656, 2012.

[198] J.W. Sandstrom and B. Helland-Hansen. Uberdie berechnung von meeresstromung. Reporton Norwegian fishery and marine investigations,Bergen Grieg, vol. 2, no. 4 = vol. 2, part 2., 1903.

[199] Helmut Schiller and Roland Doerffer. Neuralnetwork for emulation of an inverse modeloperational derivation of case ii water propertiesfrom meris data. International journal of remotesensing, 20(9):1735–1746, 1999.

[200] Jurgen Schmidhuber. Deep learning in neuralnetworks: An overview. Neural networks, 61:85–117, 2015.

[201] Tapio Schneider, Joao Teixeira, Christopher SBretherton, Florent Brient, Kyle G Pressel,Christoph Schar, and A Pier Siebesma. Climategoals and computing the future of clouds. NatureClimate Change, 7(1):3–5, 2017.

[202] Nina Schuhen, Thordis L. Thorarinsdottir, andTilmann Gneiting. Ensemble model outputstatistics for wind vectors. Monthly WeatherReview, 140(10):3204 – 3219, 01 Oct. 2012.

[203] Simon Shackley, James Risbey, Peter Stone, andBrian Wynne. Adjusting to policy expectationsin climate change modeling. Climatic Change,43(2):413–454, 1999.

[204] David Silver, Julian Schrittwieser, Karen Si-monyan, Ioannis Antonoglou, Aja Huang, ArthurGuez, Thomas Hubert, Lucas Baker, MatthewLai, Adrian Bolton, et al. Mastering thegame of go without human knowledge. nature,550(7676):354–359, 2017.

[205] Maike Sonnewald, Stephanie Dutkiewicz,Christopher Hill, and Gael Forget. Elucidatingecological complexity: Unsupervised learningdetermines global marine eco-provinces. ScienceAdvances, 6(22):1–12, 2020.

[206] Maike Sonnewald and Redouane Lguensat.Revealing the impact of global heating on northatlantic circulation using transparent machinelearning. Earth and Space Science Open Archive,page 27, 2021.

[207] Maike Sonnewald, Carl Wunsch, and Patrick He-imbach. Unsupervised learning reveals geographyof global ocean dynamical regions. Earth andSpace Science, 6(5):784–794, 2019.

[208] Hugo Steinhaus. Sur la division des corpsmateriels en parties. Bull. Acad. Polon. Sci,1(804):801, 1956.

[209] Henry Stommel. The westward intensification ofwind-driven ocean currents. Eos, TransactionsAmerican Geophysical Union, 29(2):202–206,1948.

[210] Richard S Sutton and Andrew G Barto.Reinforcement learning: An introduction. MITpress, 2018.

[211] H. U. Sverdrup. Oceanography for Meteorol-ogists. Daya Books, 1942. Google-Books-ID:3bBlcJfr2ogC.

[212] H. U. Sverdrup. Wind-driven currents in a baro-clinic ocean; with application to the equatorialcurrents of the eastern pacific. Proceedings of theNational Academy of Sciences, 33(11):318–326,1947.

[213] P. Swapna, R. Krishnan, N. Sandeep, A. G.Prajeesh, D. C. Ayantika, S. Manmeet, andR. Vellore. Long-Term Climate SimulationsUsing the IITM Earth System Model (IITM-ESMv2) With Focus on the South AsianMonsoon. Journal of Advances in ModelingEarth Systems, 10(5):1127–1149, 2018. eprint:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2017MS001262.

[214] P Swapna, MK Roxy, K Aparna, K Kulkarni,AG Prajeesh, K Ashok, R Krishnan, S Moorthi,A Kumar, and BN Goswami. The iitm earthsystem model: Transformation of a seasonalprediction model to a long term climate model.Bulletin of the American Meteorological Society,2014.

[215] Csaba Szepesvari. Algorithms for reinforcementlearning. Synthesis lectures on artificial intelli-gence and machine learning, 4(1):1–103, 2010.

[216] Pierre Tandeo, Pierre Ailliot, Marc Bocquet,Alberto Carrassi, Takemasa Miyoshi, ManuelPulido, and Yicun Zhen. A review ofinnovation-based methods to jointly estimatemodel and observation error covariance matricesin ensemble data assimilation. Monthly WeatherReview, 148(10):3973 – 3994, 01 Oct. 2020.

REFERENCES 31

[217] Jan-Erik Tesdal and Ryan P. Abernathey.Drivers of local ocean heat content variability ineccov4. Journal of Climate, 34(8):2941 – 2956,01 Apr. 2021.

[218] S Thiria, C Mejia, F Badran, and M Crepon. Aneural network approach for modeling nonlineartransfer functions: Application for wind retrievalfrom spaceborne scatterometer data. Journal ofGeophysical Research: Oceans, 98(C12):22827–22841, 1993.

[219] Leif N Thomas, Amit Tandon, Amala Mahade-van, M Hecht, and H Hasumi. Ocean modelingin an eddying regime. In Geophysical MonographSeries, volume 177, pages 17–38. American Geo-physical Union, 2008.

[220] O. Tinto Prims, M. C. Acosta, A. M. Moore,M. Castrillo, K. Serradell, A. Cortes, and F. J.Doblas-Reyes. How to use mixed precision inocean models: exploring a potential reduction ofnumerical precision in nemo 4.0 and roms 3.6.Geoscientific Model Development, 12(7):3135–3148, 2019.

[221] Benjamin A. Toms, Elizabeth A. Barnes, andImme Ebert-Uphoff. Physically InterpretableNeural Networks for the Geosciences: Appli-cations to Earth System Variability. Journalof Advances in Modeling Earth Systems, 12(9),September 2020.

[222] L. Tuppi, P. Ollinaho, M. Ekblom, V. She-myakin, and H. Jarvinen. Necessary condi-tions for algorithmic tuning of weather predic-tion models using openifs as an example. Geo-scientific Model Development, 13(11):5799–5812,2020.

[223] John Tyndall. Note on the transmission ofheat through gaseous bodies. Proceedings RoyalSociety of London, 10:37–39, 1859.

[224] Geoffrey Vallis. Geophysical fluid dynamics:Whence, whither and why? Proceedings.Mathematical, physical, and engineering sciences/ the Royal Society, 472:20160140, 08 2016.

[225] Laurens Van der Maaten and Geoffrey Hinton.Visualizing data using t-sne. Journal of machinelearning research, 9(11), 2008.

[226] Thomas Vandal, Evan Kodra, Jennifer Dy,Sangram Ganguly, Ramakrishna Nemani, andAuroop R. Ganguly. Quantifying uncertaintyin discrete-continuous and skewed data withbayesian deep learning. In Proceedings of the24th ACM SIGKDD International Conferenceon Knowledge Discovery & Data Mining, KDD’18, page 2377–2386, New York, NY, USA, 2018.Association for Computing Machinery.

[227] Thomas Vandal, Evan Kodra, and Auroop R.Ganguly. Intercomparison of machine learningmethods for statistical downscaling: the case ofdaily and extreme precipitation. Theoretical andApplied Climatology, 137(1):557–570, July 2019.

[228] GA Vecchi, T Delworth, R Gudgel, S Kapnick,A Rosati, AT Wittenberg, F Zeng, W Anderson,V Balaji, K Dixon, et al. On the seasonalforecasting of regional tropical cyclone activity.Journal of Climate, 27(21):7994–8016, 2014.

[229] A Verdy and M R Mazloff. A data assimilat-ing model for estimating Southern Ocean bio-geochemistry. JOURNAL OF GEOPHYSICALRESEARCH-OCEANS, 122(9):6968–6988, 2017.

[230] L. Zust, A. Fettich, M. Kristan, and M. Licer.Hidra 1.0: Deep-learning-based ensemble sealevel forecasting in the northern adriatic. Geosci-entific Model Development Discussions, 2020:1–25, 2020.

[231] Gilbert Walker. World weather. Quar-terly Journal of the Royal Meteorologi-cal Society, 54(226):79–87, 1928. eprint:https://rmets.onlinelibrary.wiley.com/doi/pdf/10.1002/qj.49705422601.

[232] Rui Wang, Karthik Kashinath, Mustafa Mustafa,Adrian Albert, and Rose Yu. Towards Physics-informed Deep Learning for Turbulent FlowPrediction. arXiv:1911.08655 [physics, stat],June 2020. arXiv: 1911.08655.

[233] Peter Watson. Applying machine learning toimprove simulations of a chaotic dynamicalsystem using empirical error correction. Journalof Advances in Modeling Earth Systems, May2019.

[234] Jonathan A. Weyn, Dale R. Durran, and RichCaruana. Improving data-driven global weatherprediction using deep convolutional neural net-works on a cubed sphere. Journal of Advances inModeling Earth Systems, 12(9):e2020MS002109,2020. e2020MS002109 10.1029/2020MS002109.

[235] Christopher K Wikle, Ralph F Milliff, RaduHerbei, and William B Leeds. Modern statis-tical methods in oceanography: A hierarchicalperspective. Statistical Science, pages 466–486,2013.

[236] Jared Willard, Xiaowei Jia, Shaoming Xu,Michael Steinbach, and Vipin Kumar. Inte-grating Physics-Based Modeling with MachineLearning: A Survey. arXiv:2003.04919 [physics,stat], July 2020. arXiv: 2003.04919.

[237] Christopher KI Williams and Carl EdwardRasmussen. Gaussian processes for regression,1996.

REFERENCES 32

[238] A. W. Wood, L. R. Leung, V. Sridhar, andD. P. Lettenmaier. Hydrologic Implicationsof Dynamical and Statistical Approaches toDownscaling Climate Model Outputs. ClimaticChange, 62(1):189–216, January 2004.

[239] Carl Wunsch. Ocean observations and the cli-mate forecast problem. International Geophysics,83:233–245, 12 2002.

[240] Carl Wunsch. Towards the world oceancirculation experiment and a bit of aftermath.Physical Oceanography: Developments Since1950, 09 2005.

[241] Janni Yuval, Paul A. O’Gorman, and Chris N.Hill. Use of Neural Networks for Stable,Accurate and Physically Consistent Pa-rameterization of Subgrid AtmosphericProcesses With Good Performance at Re-duced Precision. Geophysical Research Let-ters, 48(6):e2020GL091363, 2021. eprint:https://agupubs.onlinelibrary.wiley.com/doi/pdf/10.1029/2020GL091363.

[242] Laure Zanna and Thomas Bolton. Data-driven equation discovery of ocean mesoscaleclosures. Geophysical Research Letters,47(17):e2020GL088376, 2020. e2020GL08837610.1029/2020GL088376.

[243] Laure Zanna and Thomas Bolton. Data-driven Equation Discovery of Ocean MesoscaleClosures. Geophysical Research Letters, pages 1–13, 2020.

[244] Stephen E. Zebiak and Mark A. Cane. A ModelEl Ni&ntilde–Southern Oscillation. MonthlyWeather Review, 115(10):2262–2278, October1987. Publisher: American MeteorologicalSociety Section: Monthly Weather Review.

REFERENCES 33

Appendix A. List of acronyms

Abbreviation Description

4DVar 4-dimensional variational assimilationAE AutoEncoderAI Artificial IntelligenceAIC Akaike information criterionBIC Bayesian information criterionConvNet Convolutional Neural NetworkDBSCAN Density-Based Spatial Clustering of Applications with NoiseDA Data AssimilationDL Deep LearningDNN Deep Neural NetworkECCO Estimating the Circulation and Climate of the OceanEnKF Ensemble Kalman filterEOF Empirical Orthogonal FunctionsGAN Generative Adversarial NetworkGFD Geophysical Fluid DynamicsGMM Gaussian Mixture ModelGODAE Global Ocean Data Assimilation ExperimentGODAS Global Ocean Data Assimilation SystemGPR Gaussian Process RegressionGPU Graphical Processing Units (GPU)HPC High Performance ComputingIAI Interpretable Artificial IntelligenceKNN K Nearest NeighborsLR Linear RegressionMAE Mean Absolute ErrorML Machine LearningMLP Multi-Layer PerceptronMSE Mean Square ErrorNESYM Neural Earth System ModelsNN Neural NetworksPCA Principal Component AnalysisPINN Physics Informed Neural NetworksRF Random ForestRL Reinforcement LearningRNN Recurrent Neural NetworkRVM Relevance Vector MachinesSGD Stochastic Gradient DescentSOM Self Organizing MapsSVM Support Vector MachinesSVR Support Vector Regressiont-SNE t-distributed Stochastic Neighbor EmbeddingUMAP Uniform Manifold Approximation and ProjectionVAE Variational AutoencoderXAI Explainable Artificial IntelligenceWOCE World Ocean Circulation ExperimentWWII World War two

1 2 3 4 5 arXiv:2104.12506v1 [physics.ao-ph] 26 Apr 2021

Documents

Transcript of 1 2 3 4 5 arXiv:2104.12506v1 [physics.ao-ph] 26 Apr 2021