MAP estimators and their consistency in Bayesian nonparametric inverse problems

This paper considers questions from Bayesian statistics in an infinite-dimensional setting, for example in function spaces. We assume our state space to be a general separable Banach space $\bigl (X, \Vert \cdot \Vert _X\bigr )$. While in the finite-dimensional setting, the prior and posterior distribution of such statistical problems can typically be described by densities w.r.t. the Lebesgue measure, such a characterization is no longer possible in the infinite-dimensional spaces we consider here: it can be shown that no analogue of the Lebesgue measure exists in infinite-dimensional spaces. One way to work around this technical problem is to replace the Lebesgue measure with a Gaussian measure on X, i.e. with a Borel probability measure μ₀ on X such that all finite-dimensional marginals of μ₀ are (possibly degenerate) normal distributions. Using a fixed, centred (mean-zero) Gaussian measure $\mu _0 = \mathcal {N}(0,\mathcal {C}_0)$ as a reference measure, we then assume that the distribution of interest, μ, has a density with respect to μ₀:

$$\begin{equation} \frac{\mathrm{d}\mu }{\mathrm{d}\mu _0}(u) \propto \exp ({-}\Phi (u)). \end{equation} \tag{ 1.1 }$$

Measures μ of this form arise naturally in a number of applications, including the theory of conditioned diffusions [18] and the Bayesian approach to inverse problems [33]. In these settings, there are many applications where $\Phi :X\rightarrow \mathbb {R}$ is a locally Lipschitz continuous function, and it is in this setting that we work.

Our interest is in defining the concept of 'most likely' functions with respect to the measure μ, and in particular the maximum a posteriori (MAP) estimator in the Bayesian context. We will refer to such functions as MAP estimators throughout. We will define the concept precisely and link it to a problem in the calculus of variations, study posterior consistency of the MAP estimator in the Bayesian setting and compute it for a number of illustrative applications.

To motivate the form of MAP estimators considered here, we consider the case where $X=\mathbb {R}^d$ is finite dimensional and the prior μ₀ is Gaussian $\mathcal {N}(0,\mathcal {C}_0)$. This prior has density $\exp (-\frac{1}{2}|\mathcal {C}_0^{-1/2}u|^2)$ with respect to the Lebesgue measure, where | · | denotes the Euclidean norm. The probability density for μ with respect to the Lebesgue measure, given by (1.1), is maximized at minimizers of

$$\begin{equation} I(u):=\Phi (u)+\case{1}{2}\Vert u\Vert _E^2, \end{equation} \tag{ 1.2 }$$

where $\Vert \cdot \Vert _E=|\mathcal {C}_0^{-1/2}u|$. We would like to derive such a result in the infinite-dimensional setting.

The natural way to talk about MAP estimators in the infinite-dimensional setting is to seek the centre of a small ball with maximal probability and then study the limit of this centre as the radius of the ball shrinks to zero. To this end, let B^δ(z)⊂X be the open ball of radius δ centred at z ∈ X. If there is a functional I, defined on E, which satisfies

$$\begin{equation} \lim _{\delta \rightarrow 0}\frac{\mu (B^\delta (z_2))}{\mu (B^\delta (z_1))} = \exp (I(z_1)-I(z_2)), \end{equation} \tag{ 1.3 }$$

then I is termed the Onsager–Machlup functional [11, 21]. For any fixed z₁, the function z₂ for which the above limit is maximal is a natural candidate for the MAP estimator of μ and is clearly given by minimizers of the Onsager–Machlup function. In the finite-dimensional case, it is clear that I given by (1.2) is the Onsager–Machlup functional.

From the theory of infinite-dimensional Gaussian measures [5, 25], it is known that copies of the Gaussian measure μ₀ shifted by z are absolutely continuous w.r.t. μ₀ itself, if and only if z lies in the Cameron–Martin space $\bigl (E, \langle \cdot , \cdot \rangle _E, \Vert \cdot \Vert _E\bigr )$; furthermore, if the shift direction z is in E, then the shifted measure μ_z has density

$$\begin{equation} \frac{{\rm d}\mu _z}{{\rm d}\mu _0} = \exp \left(\langle z, u\rangle _E - \frac{1}{2} \Vert z\Vert _E^2\right). \end{equation} \tag{ 1.4 }$$

In the finite-dimensional example, above, the Cameron–Martin norm of the Gaussian measure μ₀ is the norm || · ||_E, and it is easy to verify that (1.4) holds for all $z\in \mathbb {R}^d$. In the infinite-dimensional case, it is important to keep in mind that (1.4) only holds for z ∈ E⊊X. Similarly, relation (1.3) only holds for z₁, z₂ ∈ E. In our application, the Cameron–Martin formula (1.4) is used to bound the probability of the shifted ball B^δ(z₂) from equation (1.3). (For an exposition of the standard results about small ball probabilities for Gaussian measures, we refer to [5, 25]; see also [24] for related material.) The main technical difficulty that is encountered stems from the fact that the Cameron–Martin space E, while being dense in X, has measure zero with respect to μ₀. An example where this problem can be explicitly seen is the case where μ₀ is the Wiener measure on L²; in this example, E corresponds to a subset of the Sobolev space H¹, which has indeed measure zero w.r.t. the Wiener measure.

Our theoretical results assert that despite these technical complications, the situation from the finite-dimensional example, above, carries over to the infinite-dimensional case essentially without change. In theorem 3.2, we show that the Onsager–Machlup functional in the infinite-dimensional setting still has the form (1.2), where || · ||_E is now the Cameron–Martin norm associated with μ (using ||z||_E = ∞ for z ∈ X∖E), and in corollary 3.10, we show that the MAP estimators for μ lie in the Cameron–Martin space E and coincide with the minimizers of the Onsager–Machlup functional I.

In the second part of the paper, we consider the inverse problem of estimating an unknown function u in a Banach space X, from a given observation $y \in \mathbb {R}^J$, where

$$\begin{equation} y=G(u)+\zeta ; \end{equation} \tag{ 1.5 }$$

here, $G:X\rightarrow \mathbb {R}^J$ is a possibly nonlinear operator, and ζ is a realization of an $\mathbb {R}^J$-valued centred Gaussian random variable with known covariance Σ. A prior probability measure μ₀(du) is put on u, and the distribution of y|u is given by (1.5), with ζ assumed to be independent of u. Under appropriate conditions on μ₀ and G, the Bayes theorem is interpreted as giving the following formula for the Radon–Nikodym derivative of the posterior distribution μ^y on u|y with respect to μ₀:

$$\begin{equation} \frac{\mathrm{d}\mu ^y}{\mathrm{d}\mu _0}(u) \propto \exp (-\Phi (u;y)), \end{equation} \tag{ 1.6 }$$

where

$$\begin{equation} \Phi (u;y) = \case{1}{2}|\Sigma ^{-\frac{1}{2}}(y-G(u))|^2. \end{equation} \tag{ 1.7 }$$

The derivation of the Bayes formula (1.6) for problems with finite-dimensional data, and ζ in this form, is discussed in [7]. Clearly, then, Bayesian inverse problems with Gaussian priors fall into the class of problems studied in this paper, for potentials Φ given by (1.7) which depend on the observed data y. When the probability measure μ arises from the Bayesian formulation of inverse problems, it is natural to ask whether the MAP estimator is close to the truth underlying the data, in either the small noise or large sample size limits. This is a form of Bayesian posterior consistency, here defined in terms of the MAP estimator only. We will study this question for finite observations of a nonlinear forward model, subject to Gaussian additive noise.

The paper is organized as follows:

• in section 2, we detail our assumptions on Φ and μ₀;
• in section 3, we give conditions for the existence of an Onsager–Machlup functional I and show that the MAP estimator is well defined as the minimizer of this functional;
• in section 4, we study the problem of Bayesian posterior consistency by studying the limits of Onsager–Machlup minimizers in the small noise and large sample size limits;
• in section 5, we study applications arising from data assimilation for the Navier–Stokes equation, as a model for what is done in weather prediction;
• in section 6, we study applications arising in the theory of conditioned diffusions.

We conclude the introduction with a brief literature review. We first note that MAP estimators are widely used in practice in the infinite-dimensional context [22, 30]. We also note that the functional I in (1.2) resembles a Tikhonov–Phillips regularization of the minimization problem for Φ [12], with the Cameron–Martin norm of the prior determining the regularization. In the theory of classical non-statistical inversion, formulation via the Tikhonov–Phillips regularization leads to an infinite-dimensional optimization problem and has led to deeper understanding and improved algorithms. Our aim is to achieve the same in a probabilistic context. One way of defining a MAP estimator for μ given by (1.1) is to consider the limit of parametric MAP estimators: first discretize the function space using n parameters, and then apply the finite-dimensional argument above to identify an Onsager–Machlup functional on $\mathbb {R}^n$. Passing to the limit n → ∞ in the functional provides a candidate for the limiting Onsager–Machlup functional. This approach is taken in [27, 28, 32] for problems arising in conditioned diffusions. Unfortunately, however, it does not necessarily lead to the correct identification of the Onsager–Machlup functional as defined by (1.3). The reason for this is that the space on which the Onsager–Machlup functional is defined is smoother than the space on which small ball probabilities are defined. Small ball probabilities are needed to properly define the Onsager–Machlup functional in the infinite-dimensional limit. This means that discretization and use of standard numerical analysis limit theorems can, if incorrectly applied, use more regularity than is admissible in identifying the limiting Onsager–Machlup functional. We study the problem directly in the infinite-dimensional setting, without using discretization, leading, we believe, to greater clarity. Adopting the infinite-dimensional perspective for MAP estimation has been widely studied for diffusion processes [9] and related stochastic PDEs [34]; see [35] for an overview. Our general setting is similar to that used to study the specific applications arising in [9, 34, 35]. By working with small ball properties of Gaussian measures, and assuming that Φ has natural continuity properties, we are able to derive results in considerable generality. There is a recent related definition of MAP estimators in [19], with application to density estimation in [16]. However, whilst the goal of minimizing I is also identified in [19], the proof in that paper is only valid in finite dimensions since it implicitly assumes that the Cameron–Martin norm is μ₀ -almost surely (a.s.) finite. In our specific application to fluid mechanics, our analysis demonstrates that widely used variational methods [2] may be interpreted as MAP estimators for an appropriate Bayesian inverse problem and, in particular, that this interpretation, which is understood in the atmospheric sciences community in the finite-dimensional context, is well defined in the limit of infinite spatial resolution.

Posterior consistency in Bayesian nonparametric statistics has a long history [15]. The study of posterior consistency for the Bayesian approach to inverse problems is starting to receive considerable attention. The papers [1, 23] are devoted to obtaining rates of convergence for linear inverse problems with conjugate Gaussian priors, whilst the papers [4, 29] study non-conjugate priors for linear inverse problems. Our analysis of posterior consistency concerns nonlinear problems, and finite data sets, so that multiple solutions are possible. We prove an appropriate weak form of posterior consistency, without rates, building on ideas appearing in [3].

Our form of posterior consistency is weaker than the general form of Bayesian posterior consistency since it does not concern fluctuations in the posterior, simply a point (MAP) estimator. However, we note that for linear Gaussian problems there are examples where the conditions which ensure convergence of the posterior mean (which coincides with the MAP estimator in the linear Gaussian case) also ensure posterior contraction of the entire measure [1, 23].

Throughout this paper, we assume that $\bigl ( X, \Vert \cdot \Vert _X\bigr )$ is a separable Banach space and that μ₀ is a centred Gaussian (probability) measure on X with Cameron–Martin space $\bigl (E, \langle \cdot , \cdot \rangle _E, \Vert \cdot \Vert _E\bigr )$. The measure μ of interest is given by (1.1) and we make the following assumptions concerning the potential Φ.

Assumption 2.1. The function $\Phi :X\rightarrow \mathbb {R}$ satisfies the following conditions.

• (i)  
  For every ε > 0, there is an $M\in \mathbb {R}$, such that for all u ∈ X,

  $$\begin{equation*} \Phi (u) \ge M -\varepsilon \Vert u\Vert _X^2. \end{equation*}$$
• (ii)  
  Φ is locally bounded from above, i.e. for every r > 0, there exists K = K(r) > 0 such that, for all u ∈ X with ||u||_X < r, we have

  $$\begin{equation*} \Phi (u) \le K. \end{equation*}$$
• (iii)  
  Φ is locally Lipschitz continuous, i.e. for every r > 0, there exists L = L(r) > 0, such that for all u₁, u₂ ∈ X with ||u₁||_X, ||u₂||_X < r, we have

  $$\begin{equation*} |\Phi (u_1)-\Phi (u_2)| \le L\Vert u_1-u_2\Vert _X. \end{equation*}$$

Assumption 2.1(i) ensures that expression (1.1) for the measure μ is indeed normalizable to give a probability measure; the specific form of the lower bound is designed to ensure that application of the Fernique theorem (see [5] or [25]) proves that the required normalization constant is finite. Assumption 2.1(ii) enables us to obtain explicit bounds from below on small ball probabilities and assumption 2.1(iii) allows us to use continuity to control the Onsager–Machlup functional. Numerous examples satisfying these conditions are given in [18, 33]. Finally, we define a function $I:X\rightarrow \mathbb {R}$ by

$$\begin{equation} I(u) = \left\lbrace \begin{array}{@{}l@{\quad }l@{}}\Phi (u) + \frac{1}{2}\Vert u\Vert _E^2 & \mbox{if $u \in E$, and} \\ +\infty & \mbox{else.} \end{array}\right. \end{equation} \tag{ 2.1 }$$

We will see in section 3 that I is the Onsager–Machlup functional.

Remark 2.2. We close with a brief remark concerning the definition of the Onsager–Machlup function in the case of a non-centred reference measure $\mu _0=\mathcal {N}(m,\mathcal {C}_0)$. Shifting coordinates by m, it is possible to apply the theory based on the centred Gaussian measure μ₀, and then undo the coordinate change. The relevant Onsager–Machlup functional can then be shown to be

$$\begin{equation*} I(u) = \left\lbrace \begin{array}{@{}l@{\quad }l@{}}\Phi (u) + \frac{1}{2}\Vert u-m\Vert _E^2 & \mbox{if $u-m \in E$, and} \\ +\infty & \mbox{else.} \end{array}\right. \end{equation*}$$

In this section, we prove two main results. The first, theorem 3.2, establishes that I given by (1.2) is indeed the Onsager–Machlup functional for the measure μ given by (1.1). Then theorem 3.5 and corollary 3.10 show that the MAP estimators, defined precisely in definition 3.1, are characterized by the minimizers of the Onsager–Machlup functional.

For z ∈ X, let B^δ(z)⊂X be the open ball centred at z ∈ X with radius δ in X. Let

$$\begin{equation*} J^{\delta }(z) = \mu \bigl ( B^\delta (z) \bigr ) \end{equation*}$$

be the mass of the ball B^δ(z). We first define the MAP estimator for μ as follows.

Definition 3.1. Let

$$\begin{equation*} z^\delta =\mathop{arg max}_{z \in X} J^{\delta }(z). \end{equation*}$$

Any point $\tilde{z}\in X$ satisfying $\lim _{\delta \rightarrow 0}(J^\delta (\tilde{z})/J^\delta (z^\delta ))=1$ is a MAP estimator for the measure μ given by (1.1).

We show later on (theorem 3.5) that a strongly convergent subsequence of {z^δ}_{δ > 0} exists, and its limit, that we prove to be in E, is a MAP estimator and also minimizes the Onsager–Machlup functional I. Corollary 3.10 then shows that any MAP estimator $\tilde{z}$ as given in definition 3.1 lives in E as well, and minimizers of I characterize all MAP estimators of μ.

One special case where it is easy to see that the MAP estimator is unique is the case where Φ is linear, but we note that, in general, the MAP estimator cannot be expected to be unique. To achieve uniqueness, stronger conditions on Φ would be required.

We first need to show that I is the Onsager–Machlup functional for our problem.

Theorem 3.2. Let assumption 2.1 hold. Then the function I defined by (2.1) is the Onsager–Machlup functional for μ, i.e. for any z₁, z₂ ∈ E, we have

$$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta (z_1)}{J^\delta (z_2)} =\exp (I(z_2)-I(z_1)). \end{equation*}$$

Proof. Note that J^δ(z) is finite and positive for any z ∈ E by assumptions 2.1(i) and (ii) together with the Fernique theorem and the positive mass of all balls in X, centred at points in E, under Gaussian measure [5]. The key estimate in the proof is the following consequence of proposition 3 in section 18 of [25]:

$$\begin{equation} \lim _{\delta \rightarrow 0} \frac{\mu _0( B^\delta (z_1))}{\mu _0(B^\delta (z_2))} = \exp \left( \frac{1}{2}\Vert z_2\Vert _E^2-\frac{1}{2}\Vert z_1\Vert _E^2 \right). \end{equation} \tag{ 3.1 }$$

This is the key estimate in the proof since it transfers questions about probability, naturally asked on the space X of full measure under μ₀, into statements concerning the Cameron–Martin norm of μ₀, which is almost surely infinite under μ₀.

We have

$$\begin{eqnarray*} \frac{J^\delta (z_1)}{J^\delta (z_2)} &=\frac{ \int _{B^\delta (z_1)}\exp (-\Phi (u)) \mu _0(\mathrm{d}u) }{ \int _{B^\delta (z_2)}\exp (-\Phi (v)) \mu _0(\mathrm{d}v) } \\ &=\frac{ \int _{B^\delta (z_1)}\exp (-\Phi (u)+\Phi (z_1))\exp (-\Phi (z_1)) \mu _0(\mathrm{d}u) }{ \int _{B^\delta (z_2)}\exp (-\Phi (v)+\Phi (z_2))\exp (-\Phi (z_2)) \mu _0(\mathrm{d}v) }. \end{eqnarray*}$$

By assumption 2.1(iii), for any u, v ∈ X,

$$\begin{equation*} -L \Vert u-v\Vert _X \le \Phi (u)-\Phi (v) \le L \Vert u-v\Vert _X, \end{equation*}$$

where L = L(r) with r > max {||u||_X, ||v||_X}. Therefore, setting L₁ = L(||z₁||_X + δ) and L₂ = L(||z₂||_X + δ), we can write

$$\begin{eqnarray*} \frac{J^\delta (z_1)}{J^\delta (z_2)} &\le \mathrm{e}^{\delta (L_1+L_2)}\frac{ \int _{B^\delta (z_1)}\exp (-\Phi (z_1)) \mu _0(\mathrm{d}u) }{ \int _{B^\delta (z_2)}\exp (-\Phi (z_2)) \mu _0(\mathrm{d}v) } \\ &=\mathrm{e}^{\delta (L_1+L_2)}\,\mathrm{e}^{-\Phi (z_1)+\Phi (z_2)} \frac{ \int _{B^\delta (z_1)} \mu _0(\mathrm{d}u) }{ \int _{B^\delta (z_2)} \mu _0(\mathrm{d}v) }. \end{eqnarray*}$$

Now, by (3.1), we have

$$\begin{equation*} \frac{J^\delta (z_1)}{J^\delta (z_2)} \le r_1(\delta ) \,\mathrm{e}^{\delta (L_2+L_1)}\,\mathrm{e}^{-I(z_1)+I(z_2)} \end{equation*}$$

with r₁(δ) → 1 as δ → 0. Thus,

$$\begin{equation} \limsup _{\delta \rightarrow 0}\frac{J^\delta (z_1)}{J^\delta (z_2)} \,\le \, \mathrm{e}^{-I(z_1)+I(z_2)}. \end{equation} \tag{ 3.2 }$$

Similarly, we obtain

$$\begin{equation*} \frac{J^\delta (z_1)}{J^\delta (z_2)} \ge \frac{1}{r_2(\delta )} \,\mathrm{e}^{-\delta (L_2+L_1)}\,\mathrm{e}^{-I(z_1)+I(z_2)} \end{equation*}$$

with r₂(δ) → 1 as δ → 0 and deduce that

$$\begin{equation} \liminf _{\delta \rightarrow 0}\frac{J^\delta (z_1)}{J^\delta (z_2)} \,\ge \, \mathrm{e}^{-I(z_1)+I(z_2)}. \end{equation} \tag{ 3.3 }$$

Inequalities (3.2) and (3.3) give the desired result. □

We note that similar methods of analysis show the following.

Corollary 3.3. Let the assumptions of theorem 3.2 hold. Then for any z ∈ E,

$$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta (z)}{\int _{B^{\delta }(0)}\mu _0(\mathrm{d}u)} = \frac{1}{Z} \,\mathrm{e}^{-I(z)}, \end{equation*}$$

where Z = ∫_Xexp ( − Φ(u)) μ₀(du).

Proof. Noting that we consider μ to be a probability measure and hence,

$$\begin{equation*} \frac{J^\delta (z)}{\int _{B^{\delta }(0)}\mu _0(\mathrm{d}u)} =\frac{\frac{1}{Z}\int _{B^{\delta }(z)}\exp (-\Phi (u))\mu _0(\mathrm{d}u)}{\int _{B^{\delta }(0)}\mu _0(\mathrm{d}u)}, \end{equation*}$$

with Z = ∫_Xexp ( − Φ(u)) μ₀(du), arguing along the lines of the proof of the above theorem gives

$$\begin{equation*} \frac{1}{Z}\frac{1}{r(\delta )}\,\mathrm{e}^{-\delta \hat{L}}\,\mathrm{e}^{-I(z)}\le \frac{J^\delta (z)}{\int _{B^{\delta }(0)}\mu _0(\mathrm{d}u)}\le \frac{1}{Z}r(\delta )\,\mathrm{e}^{\delta \hat{L}}\,\mathrm{e}^{-I(z)} \end{equation*}$$

with $\hat{L}=L(\Vert z\Vert _X+\delta )$ (where L( · ) is as in definition 2.1) and r(δ) → 1 as δ → 0. The result then follows by taking $\lim \sup$ and $\lim \inf$ as δ → 0. □

Proposition 3.4. Suppose assumptions 2.1 hold. Then the minimum of $I:E \rightarrow \mathbb {R}$ is attained for some element z* ∈ E.

Proof. The existence of a minimizer of I in E, under the given assumptions, is proved as theorem 5.4 in [33] (and as theorem 2.7 in [7] in the case where Φ is non-negative). □

The rest of this section is devoted to a proof of the result that MAP estimators can be characterized as minimizers of the Onsager–Machlup functional I (theorem 3.5 and corollary 3.10).

Theorem 3.5. Suppose that assumptions 2.1(ii) and (iii) hold. Assume also that there exists an $M\in \mathbb {R}$ such that Φ(u) ⩾ M for any u ∈ X.

• (i)  
  Let z^δ = argmax_{z ∈ X}J^δ(z). There is a $\bar{z}\in E$ and a subsequence of {z^δ}_{δ > 0} which converges to $\bar{z}$ strongly in X.
• (ii)  
  The limit $\bar{z}$ is a MAP estimator and a minimizer of I.

The proof of this theorem is based on several lemmas. We state and prove these lemmas first and defer the proof of theorem 3.5 to the end of the section where we also state and prove a corollary characterizing the MAP estimators as minimizers of the Onsager–Machlup functional.

Lemma 3.6. Let δ > 0. For any centred Gaussian measure μ₀ on a separable Banach space X, we have

$$\begin{equation*} \frac{J^\delta _0(z)}{J^\delta _0(0)}\le c \,\mathrm{e}^{-\frac{a_1}{2}(\Vert z\Vert _X-\delta )^2}, \end{equation*}$$

where $c=\exp (\frac{a_1}{2}\delta ^2)$ and a₁ is a constant independent of z and δ.

Proof. We first show that this is true for a centred Gaussian measure on $\mathbb {R}^n$ with the covariance matrix C = diag[λ₁, ..., λ_n] in basis {e₁, ..., e_n}, where λ₁ ⩾ λ₂ ⩾ ⋅⋅⋅ ⩾ λ_n. Let a_j = 1/λ_j, and $|z|^2=z_1^2+\cdots +z_n^2$. Define

$$\begin{equation} J^\delta _{0,n}(z) := \int _{B^\delta (z)} \,\mathrm{e}^{-\frac{1}{2}(a_1x_1^2+\cdots +a_nx_n^2)}\,\mathrm{d}x, \quad \mbox{for any } z\in \mathbb {R}^n, \end{equation} \tag{ 3.4 }$$

and with B^δ(z) the ball of radius δ and centre z in $\mathbb {R}^n$. We have

$$\begin{eqnarray*} \frac{J^\delta _{0,n}(z)}{J^\delta _{0,n}(0)} &=\frac{ \int _{B^\delta (z)} \,\mathrm{e}^{-\frac{1}{2}(a_1x_1^2+\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}\left(a_1x_1^2+\cdots +a_nx_n^2\right)} \,\mathrm{d}x }\\ &< \frac{\mathrm{e}^{-\frac{1}{2}(a_1-\varepsilon )(|z|-\delta )^2}}{\mathrm{e}^{-\frac{1}{2}(a_1-\varepsilon )\delta ^2}}\frac{ \int _{B^\delta (z)} \,\mathrm{e}^{-\frac{1}{2}\left(\varepsilon x_1^2+(a_2-a_1+\varepsilon )x_2^2+\cdots +(a_n-a_1+\varepsilon )x_n^2\right)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}(\varepsilon x_1^2+(a_2-a_1+\varepsilon )x_2^2+\cdots +(a_n-a_1+\varepsilon )x_n^2)} \,\mathrm{d}x }\\ &< c \,\mathrm{e}^{-\frac{1}{2}(a_1-\varepsilon )(|z|-\delta )^2}\frac{ \int _{B^\delta (z)}\hat{\mu }_0(\mathrm{d}x) }{ \int _{B^\delta (0)} \hat{\mu }_0(\mathrm{d}x) }, \end{eqnarray*}$$

for any ε < a₁ and where $\hat{\mu }_0$ is a centred Gaussian measure on $\mathbb {R}^n$ with the covariance matrix diag[1/ε, 1/(a₂ − a₁ + ε), ..., 1/(a_n − a₁ + ε)] (noting that a_n ⩾ a_{n − 1} ⩾ ⋅⋅⋅ ⩾ a₁). By Anderson's inequality for the infinite-dimensional spaces (see theorem 2.8.10 of [5]), we have $\hat{\mu }_0(B(z,\delta ))\le \hat{\mu }_0(B(0,\delta ))$ and therefore

$$\begin{equation*} \frac{J^\delta _{0,n}(z)}{J^\delta _{0,n}(0)}< c \,\mathrm{e}^{-\frac{1}{2}(a_1-\varepsilon )(|z|-\delta )^2}, \end{equation*}$$

and since ε is arbitrarily small, the result follows for the finite-dimensional case.

To show the result for an infinite-dimensional separable Banach space X, we first note that $\lbrace e_j\rbrace _{j=1}^\infty$, the orthogonal basis in the Cameron–Martin space of X for μ₀, separates the points in X; therefore, $T:u\rightarrow \lbrace e_j(u)\rbrace _{j=1}^\infty$ is an injective map from X to $\mathbb {R}^\infty$. Let u_j = e_j(u) and

$$\begin{equation*} P_nu=(u_1,u_2,\ldots ,u_n,0,0,\ldots ). \end{equation*}$$

Then, since μ₀ is a Radon measure, for the balls B(0, δ) and B(z, δ), for any ε₀ > 0, there exists large enough N such that the cylindrical sets $A_0=P_n^{-1}(P_n(B^\delta (0)))$ and $A_z=P_n^{-1}(P_n(B^\delta (z)))$ satisfy μ₀(B^δ(0)▵A₀) < ε₀ and μ₀(B^δ(z)▵A_z) < ε₀ for n > N [5], where ▵ denotes the symmetric difference. Let z_j = (z, e_j) and zⁿ = (z₁, z₂, ..., z_n, 0, ...), and for 0 < ε₁ < δ/2, n > N large enough so that ||z − zⁿ||_X ⩽ ε₁. With $\alpha =c\,\mathrm{e}^{-\frac{a_1}{2}(\Vert z\Vert _X-\varepsilon _1-\delta )^2}$, we have

$$\begin{eqnarray*} J_0^\delta (z) &\le J_{0,n}^\delta (z^n)+\varepsilon _0\\ &\le \alpha J_{0,n}^\delta (0)+\varepsilon _0\\ &\le \alpha J_0^\delta (0)+(1+\alpha )\varepsilon _0. \end{eqnarray*}$$

Since ε₀ and ε₁ converge to zero as n → ∞, the result follows. □

Lemma 3.7. Suppose that $\bar{z}\not\in E$, {z^δ}_{δ > 0}⊂X and z^δ converges weakly to z in X as δ → 0. Then for any ε > 0, there exists δ small enough such that

$$\begin{equation*} \frac{J^\delta _0(z^\delta )}{J^\delta _0(0)}<\varepsilon . \end{equation*}$$

Proof. Let $\mathcal {C}$ be the covariance operator of μ₀, and $\lbrace e_j\rbrace _{j\in \mathbb {N}}$ the eigenfunctions of $\mathcal {C}$ scaled with respect to the inner product of E, the Cameron–Martin space of μ₀, so that $\lbrace e_j\rbrace _{j\in \mathbb {N}}$ forms an orthonormal basis in E. Let {λ_j} be the corresponding eigenvalues and a_j = 1/λ_j. Since z^δ converges weakly to $\bar{z}$ in X as δ → 0,

$$\begin{equation} e_j(z^\delta )\rightarrow e_j(\bar{z})\quad \mbox{for any }j\in \mathbb {N}, \end{equation} \tag{ 3.5 }$$

and as $\bar{z}\not\in E$ for any A > 0, there exist N sufficiently large and $\tilde{\delta }>0$ sufficiently small, such that

$$\begin{equation*} \inf _{z\in B^{\tilde{\delta }}(\bar{z})}\left\lbrace \sum _{j=1}^N a_j x_j^2\right\rbrace >A^2, \end{equation*}$$

where x_j = e_j(z). By (3.5), for $\delta _1<\tilde{\delta }$ small enough, we have $B^{\delta _1}(z^{\delta _1})\subset B^{\tilde{\delta }}(\bar{z})$ and therefore

$$\begin{equation} \inf _{z\in B^{\delta _1}(z^{\delta _1})}\left\lbrace \sum _{j=1}^N a_j x_j^2\right\rbrace >A^2. \end{equation} \tag{ 3.6 }$$

Let $T_n:X\rightarrow \mathbb {R}^n$ map z to (e₁(z), ..., e_n(z)), and consider $J_{0,n}^\delta (z)$ to be defined as in (3.4). Having (3.6), and choosing δ ⩽ δ₁ such that $\mathrm{e}^{-\frac{1}{4}(a_1+\cdots +a_N)\delta ^2}>1/2$, for any n ⩾ N, we can write

$$\begin{eqnarray*} \frac{J^\delta _{0,n}(T_nz^\delta )}{J^\delta _{0,n}(0)} &=\frac{ \int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{1}{2}(a_1x_1^2+\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}\left(a_1x_1^2+\cdots +a_nx_n^2\right)} \,\mathrm{d}x }\\ &\le \frac{ \int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{1}{4}(a_1x_1^2+\cdots +a_Nx_N^2)}\,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{4}(a_1x_1^2+\cdots +a_Nx_N^2)}\,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }\\ &\le \frac{ \mathrm{e}^{-\frac{1}{4}A^2}\int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \frac{1}{2}\int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }\\ &\le 2\,\mathrm{e}^{-\frac{1}{4}A^2}. \end{eqnarray*}$$

As A > 0 was arbitrary, the constant in the last line of the above equation can be made arbitrarily small, by making δ sufficiently small and n sufficiently large. Having this and arguing in a similar way to the final paragraph of the proof of lemma 3.6, the result follows. □

Corollary 3.8. Suppose that $z\not\in E$. Then

$$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J_0^\delta (z)}{J_0^\delta (0)}=0. \end{equation*}$$

Lemma 3.9. Consider {z^δ}_{δ > 0}⊂X and suppose that z^δ converges weakly and not strongly to 0 in X as δ → 0. Then for any ε > 0, there exists δ small enough such that

$$\begin{equation*} \frac{J^\delta _0(z^\delta )}{J^\delta _0(0)}<\varepsilon . \end{equation*}$$

Proof. Since z^δ converges weakly and not strongly to 0, we have

$$\begin{equation*} \liminf _{\delta \rightarrow 0}\Vert z^\delta \Vert _X>0, \end{equation*}$$

and therefore, for δ₁ small enough, there exists α > 0 such that ||z^δ||_X > α for any δ < δ₁. Let λ_j, a_j and e_j, $j\in \mathbb {N}$, be defined as in the proof of lemma 3.7. Since z^δ⇀0 as δ → 0,

$$\begin{equation} e_j(z^\delta )\rightarrow 0,\quad \mbox{for any }j\in \mathbb {N}. \end{equation} \tag{ 3.7 }$$

Also, as for μ₀-almost every x ∈ X, $x=\sum _{j\in \mathbb {N}} e_j(x)\hat{e_j}$ and $\lbrace \hat{e}_j=e_j/\sqrt{\lambda _j}\rbrace$ is an orthonormal basis in $X^*_{\mu _0}$ (closure of X* in L²(μ₀)) [5], we have

$$\begin{equation} \sum _{j\in \mathbb {N}} (e_j(x))^2<\infty \quad \mbox{for $\mu _0$-almost every $x\in X$}. \end{equation} \tag{ 3.8 }$$

Now, for any A > 0, let N large enough such that a_N > A². Then, having (3.7) and (3.8), one can choose δ₂ < δ₁ small enough and N₁ > N large enough so that for δ < δ₂ and n > N₁,

$$\begin{equation*} \sum _{j=1}^N(e_j(z^\delta ))^2< \frac{C\alpha }{2}\quad \mbox{and}\quad \sum _{j=N+1}^n(e_j(z^\delta ))^2> \frac{C\alpha }{2}. \end{equation*}$$

Therefore, letting $J_{0,n}^\delta (z)$ and T_n be defined as in the proof of lemma 3.7, we can write

$$\begin{eqnarray*} \frac{J^\delta _{0,n}(T_nz^\delta )}{J^\delta _{0,n}(0)} &=\frac{ \int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{1}{2}(a_1x_1^2+\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}\left(a_1x_1^2+\cdots +a_nx_n^2\right)} \,\mathrm{d}x }\\ &\le \frac{ \int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{A^2}{2}(x_{N+1}^2+\cdots +x_n^2)}\,\mathrm{e}^{-\frac{1}{2}({a_1}x_1^2+\cdots +{a_N}x_N^2+(a_{N+1}-{A^2})x_{N+1}^2\cdots +(a_n-{A^2})x_n^2)} \,\mathrm{d}x }{ \int _{B^\delta (0)} \,\mathrm{e}^{-\frac{A^2}{2}(x_{N+1}^2+\cdots +x_n^2)}\,\mathrm{e}^{-\frac{1}{2}({a_1}x_1^2+\cdots +{a_N}x_N^2+(a_{N+1}-{A^2})x_{N+1}^2\cdots +(a_n-{A^2})x_n^2)} \,\mathrm{d}x } \\ &\le \frac{ \mathrm{e}^{-\frac{1}{2}A^2(\frac{C\alpha }{2}-\delta ^2)}\int _{B^\delta (T_nz^\delta )} \,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }{ \mathrm{e}^{-\frac{1}{2}A^2\delta ^2}\int _{B^\delta (0)} \,\mathrm{e}^{-\frac{1}{2}(\frac{a_1}{2}x_1^2+\cdots +\frac{a_N}{2}x_N^2+a_{N+1}x_{N+1}^2\cdots +a_nx_n^2)} \,\mathrm{d}x }\\ &\le 2\,\mathrm{e}^{-\frac{C\alpha }{4}A^2}, \end{eqnarray*}$$

if δ < δ₂ is small enough so that $\mathrm{e}^{A\delta ^2}<2$. Having this and arguing in a similar way to the final paragraph of the proof of lemma 3.6, the result follows. □

Having these preparations in place, we can give the proof of theorem 3.5.

Proof of theorem 3.5. (i) We first show that {z^δ} is bounded in X. By assumption 2.1(ii), for any r > 0, there exists K = K(r) > 0 such that

$$\begin{equation*} \Phi (u) \le K(r) \end{equation*}$$

for any u satisfying ||u||_X < r; thus, K may be assumed to be a non-decreasing function of r. This implies that

$$\begin{equation*} \max _{z\in E}\int _{B^{\delta }(z)}\,\mathrm{e}^{-\Phi (u)} \mu _0(\mathrm{d}u) \ge \int _{B^{\delta }(0)}\,\mathrm{e}^{-\Phi (u)} \mu _0(\mathrm{d}u) \ge \mathrm{e}^{-K(\delta )} \int _{B^{\delta }(0)}\mu _0(\mathrm{d}u). \end{equation*}$$

We assume that δ ⩽ 1, and then the inequality above shows that

$$\begin{equation} \frac{J^\delta (z^\delta )}{\int _{B^{\delta }(0)}\mu _0(\mathrm{d}u)} \ge \frac{1}{Z}\,\mathrm{e}^{-K(1)}=\varepsilon _1, \end{equation} \tag{ 3.9 }$$

noting that ε₁ is independent of δ.

We can also write

$$\begin{eqnarray*} ZJ^\delta (z) &=\int _{B^\delta (z)}\,\mathrm{e}^{-\Phi (u)}\mu _0(\mathrm{d}u)\\ &\le \mathrm{e}^{-M}\int _{B^\delta (z)}\mu _0(\mathrm{d}u)\\ &=: \mathrm{e}^{-M} J^\delta _0(z), \end{eqnarray*}$$

which implies that for any z ∈ X and δ > 0,

$$\begin{equation} J^\delta _0(z)\ge Z\,\mathrm{e}^M J^\delta (z). \end{equation} \tag{ 3.10 }$$

Now suppose {z^δ} is not bounded in X, so that for any R > 0, there exists δ_R such that $\Vert z^{\delta _R}\Vert _X>R$ (with δ_R → 0 as R → ∞). By (3.10), (3.9) and the definition of $z^{\delta _R}$, we have

$$\begin{equation*} J_0^{\delta _R}(z^{\delta _R})\ge Z\,\mathrm{e}^M J^{\delta _R}(z^{\delta _R}) \ge Z\,\mathrm{e}^M J^{\delta _R}(0) \ge \mathrm{e}^{M}\,\mathrm{e}^{-K(1)}J_0^{\delta _R}(0), \end{equation*}$$

implying that for any δ_R and corresponding $z^{\delta _R}$

$$\begin{equation*} \frac{J_0^{\delta _R}(z^{\delta _R})}{J_0^{\delta _R}(0)}\ge c=\mathrm{e}^{M}\,\mathrm{e}^{-K(1)}. \end{equation*}$$

This contradicts the result of lemma 3.6 (below) for δ_R small enough. Hence, there exists R, δ_R > 0 such that

$$\begin{equation*} \Vert z^\delta \Vert _X\le R\quad \mbox{ for any }\delta <\delta _R. \end{equation*}$$

Therefore, there exist a $\bar{z}\in X$ and a subsequence of $\lbrace z^\delta \rbrace _{0<\delta <\delta _R}$ which converges weakly in X to $\bar{z}\in X$ as δ → 0.

Now, suppose either

• (a)  
  there is no strongly convergent subsequence of {z^δ} in X, or
• (b)  
  if there is one, its limit $\bar{z}$ is not in E.

Let U_E = {u ∈ E: ||u||_E ⩽ 1}. Each of the above situations implies that for any positive $A\in \mathbb {R}$, there is a δ† such that for any δ ⩽ δ†,

$$\begin{equation} B^\delta (z^\delta )\cap \left(B^\delta (0)+AU_E\right)=\emptyset . \end{equation} \tag{ 3.11 }$$

We first show that $\bar{z}$ has to be in E. By the definition of z^δ, we have (for δ < 1)

$$\begin{equation} 1\le \frac{J^\delta (z^\delta )}{J^\delta (0)}\le \frac{\mathrm{e}^M}{\mathrm{e}^{-K(1)}}\,\frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (0)}\mu _0(\mathrm{d}u)}. \end{equation} \tag{ 3.12 }$$

Supposing $\bar{z}\not\in E$, in lemma 3.7, we show that for any ε > 0, there exists δ small enough such that

$$\begin{equation*} \frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (0)}\mu _0(\mathrm{d}u)}<\varepsilon . \end{equation*}$$

Hence, choosing A in (3.11) such that $\mathrm{e}^{-A^2/2}<\frac{1}{2}\,\mathrm{e}^{K(1)}\,\mathrm{e}^{-M}$, and setting $\varepsilon =\mathrm{e}^{-A^2/2}$, from (3.12), we obtain 1 ⩽ J^δ(z^δ)/J^δ(0) < 1, which is a contradiction. We therefore have $\bar{z}\in E$.

Now, knowing that $\bar{z}\in E$, we can show that z^δ converges strongly in X. Suppose it does not. Then for $z^\delta -\bar{z}$, the hypotheses of lemma 3.9 are satisfied. Again choosing A in (3.11) such that $\mathrm{e}^{-A^2/2}<\frac{1}{2}\,\mathrm{e}^{K(1)}\,\mathrm{e}^{-M}$, and setting $\varepsilon =\mathrm{e}^{-A^2/2}$, from lemma 3.9 and (3.12), we obtain 1 ⩽ J^δ(z^δ)/J^δ(0) < 1, which is a contradiction. Hence, there is a subsequence of {z^δ} converging strongly in X to $\bar{z}\in E$.

(ii) Let z* = arg minI(z) ∈ E; the existence is assured by theorem 3.2. By assumption 2.1(iii), we have

$$\begin{equation*} \frac{J^\delta (z^\delta )}{J^\delta (\bar{z})} \le \mathrm{e}^{-\Phi (z^\delta )+\Phi (\bar{z})}\,\mathrm{e}^{(L_1+L_2)\delta } \frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (\bar{z})}\mu _0(\mathrm{d}u)}, \end{equation*}$$

with L₁ = L(||z^δ||_X + δ) and $L_2=L(\Vert \bar{z}\Vert _X+\delta )$. Therefore, since Φ is continuous on X and $z^\delta \rightarrow \bar{z}$ in X,

$$\begin{equation*} \limsup _{\delta \rightarrow 0}\frac{J^\delta (z^\delta )}{J^\delta (\bar{z})} \le \limsup _{\delta \rightarrow 0} \frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (\bar{z})}\mu _0(\mathrm{d}u)}. \end{equation*}$$

Suppose {z^δ} is not bounded in E, or if it is, it only converges weakly (and not strongly) in E. Then, $\Vert \bar{z}\Vert _E < \liminf _{\delta \rightarrow 0} \Vert z^\delta \Vert _E$, and hence for small enough δ, $\Vert \bar{z}\Vert _E<\Vert z^\delta \Vert _E$. Therefore, for the centered Gaussian measure μ₀, since $\Vert z^\delta -\bar{z}\Vert _X\rightarrow 0$, we have

$$\begin{equation*} \limsup _{\delta \rightarrow 0}\frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (\bar{z})}\mu _0(\mathrm{d}u)}\le 1. \end{equation*}$$

Since by the definition of z^δ, $J^\delta (z^\delta )\ge J^\delta (\bar{z})$ and hence

$$\begin{equation*} \liminf _{\delta \rightarrow 0}\bigl (J^\delta (z^\delta )/ J^\delta (\bar{z})\bigr )\ge 1, \end{equation*}$$

this implies that

$$\begin{equation} \lim _{\delta \rightarrow 0}\frac{J^\delta (z^\delta )}{J^\delta (\bar{z})}=1. \end{equation} \tag{ 3.13 }$$

In the case where {z^δ} converges strongly to $\bar{z}$ in E, by the Cameron–Martin theorem we have

$$\begin{equation*} \frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (\bar{z})}\mu _0(\mathrm{d}u)} =\frac{ \mathrm{e}^{-\frac{1}{2}\Vert z^\delta \Vert _E^2} \int _{B^\delta (0)}\,\mathrm{e}^{\langle z^\delta ,u \rangle _E}\mu _0(\mathrm{d}u) }{ \mathrm{e}^{-\frac{1}{2}\Vert \bar{z}\Vert _E^2} \int _{B^\delta (0)}\,\mathrm{e}^{\langle \bar{z},u \rangle _E}\mu _0(\mathrm{d}u)}, \end{equation*}$$

and then by an argument very similar to the proof of theorem 18.3 of [25], one can show that

$$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{\int _{B^\delta (z^\delta )}\mu _0(\mathrm{d}u)}{\int _{B^\delta (\bar{z})}\mu _0(\mathrm{d}u)}=1, \end{equation*}$$

and (3.13) follows again in a similar way. Therefore, $\bar{z}$ is a MAP estimator of measure μ.

It remains to show that $\bar{z}$ is a minimizer of I. Suppose $\bar{z}$ is not a minimizer of I so that $I(\bar{z})-I(z^*)>0$. Let δ₁ be small enough so that in the equation before (3.2), $1<r_1(\delta )< \mathrm{e}^{I(\bar{z})-I(z^*)}$ for any δ < δ₁ and therefore,

$$\begin{equation} \frac{J^\delta (\bar{z})}{J^\delta (z^*)} \le r_1(\delta ) \,\mathrm{e}^{-I(\bar{z})+I(z^*)}<1. \end{equation} \tag{ 3.14 }$$

Let $\alpha =r_1(\delta ) \,\mathrm{e}^{-I(\bar{z})+I(z^*)}$. We have

$$\begin{equation*} \frac{J^\delta (z^\delta )}{J^\delta (z^*)} =\frac{J^\delta (z^\delta )}{J^\delta (\bar{z})} \frac{J^\delta (\bar{z})}{J^\delta (z^*)}, \end{equation*}$$

and this by (3.14) and (3.13) implies that

$$\begin{equation*} \limsup _{\delta \rightarrow 0} \frac{J^\delta (z^\delta )}{J^\delta (z^*)} \le \alpha \limsup _{\delta \rightarrow 0} \frac{J^\delta (z^\delta )}{J^\delta (\bar{z})}< 1, \end{equation*}$$

which is a contradiction, since by the definition of z^δ, J^δ(z^δ) ⩾ J^δ(z*) for any δ > 0. □

Corollary 3.10. Under the conditions of theorem 3.5, we have the following.

• (i)  
  Any MAP estimator, given by definition 3.1, minimizes the Onsager–Machlup functional I.
• (ii)  
  Any z* ∈ E which minimizes the Onsager–Machlup functional I is a MAP estimator for measure μ given by (1.1).

Proof. 

• (i)  
  Let $\tilde{z}$ be a MAP estimator. By theorem 3.5, we know that {z^δ} has a subsequence which strongly converges in X to $\bar{z}$. Let {z^α} be the said subsequence. Then by (3.13) one can show that

  $$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta (z^\delta )}{J^\delta (\bar{z})}=\lim _{\alpha \rightarrow 0}\frac{J^\alpha (z^\alpha )}{J^\alpha (\bar{z})}=1. \end{equation*}$$

  By the above equation and since $\tilde{z}$ is a MAP estimator, we can write

  $$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta (\tilde{z})}{J^\delta (\bar{z})} = \lim _{\delta \rightarrow 0}\frac{J^\delta (z^\delta )}{J^\delta (\bar{z})} \lim _{\delta \rightarrow 0}\frac{J^\delta (\tilde{z})}{J^\delta (z^\delta )} =1. \end{equation*}$$

  Then corollary 3.8 implies that $\tilde{z}\in E$, and supposing that $\tilde{z}$ is not a minimizer of I would result in a contradiction using an argument similar to that in the last paragraph of the proof of the above theorem.
• (ii)  
  Note that the assumptions of theorem 3.5 imply those of theorem 3.2. Since $\bar{z}$ is a minimizer of I as well, by theorem 3.2 we have

  $$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta (\bar{z})}{J^\delta (z^*)}=1. \end{equation*}$$

  Then we can write

  $$\begin{equation*} \lim _{\delta \rightarrow 0}\frac{J^\delta ( z^*)}{J^\delta (z^\delta )} = \lim _{\delta \rightarrow 0}\frac{J^\delta (\bar{z})}{J^\delta (z^\delta )} = \lim _{\delta \rightarrow 0}\frac{J^\delta (z^*)}{J^\delta (\bar{z})} =1. \end{equation*}$$

  The result follows by definition 3.1.

 □

The structure (1.1), where μ₀ is Gaussian, arises in the application of the Bayesian methodology to the solution of inverse problems. In that context, it is interesting to study posterior consistency: the idea that the posterior concentrates near the truth which gave rise to the data, in the small noise or large sample size limits; these two limits are intimately related and indeed there are theorems that quantify this connection for certain linear inverse problems [6].

In this section, we describe the Bayesian approach to nonlinear inverse problems, as outlined in the introduction. We assume that the data are found from the application of G to the truth u† with additional noise:

$$\begin{equation*} y=G(u^\dagger )+\zeta . \end{equation*}$$

The posterior distribution μ^y is then of the form (1.6) and in this case, it is convenient to extend the Onsager–Machlup functional I to a mapping from $X \times \mathbb {R}^J$ to $\mathbb {R}$, defined as

$$\begin{equation*} I(u;y) = \Phi (u;y)+\case{1}{2}\Vert u\Vert ^2_E. \end{equation*}$$

We study the posterior consistency of MAP estimators in both the small noise and large sample size limits. The corresponding results are presented in theorems 4.4 and 4.1, respectively. Specifically, we characterize the sense in which the MAP estimators concentrate on the truth underlying the data in the small noise and large sample size limits.

4.1. Large sample size limit

Let us denote the exact solution by u† and suppose that as data we have the following n random vectors:

$$\begin{equation*} y_j = \mathcal {G}(u^\dagger )+\eta _j, \quad j=1, \ldots ,n, \end{equation*}$$

with $y_j\in \mathbb {R}^K$ and $\eta _j\sim \mathcal {N}(0,\mathcal {C}_1)$ independent identically distributed random variables. Thus, in the general setting, we have J = nK, $G(\cdot )=(\mathcal {G}(\cdot ),\ldots ,\mathcal {G}(\cdot ))$ and Σ a block diagonal matrix with $\mathcal {C}_1$ in each block. We have n independent observations, each polluted by $\mathcal{O}(1)$ noise, and we study the limit n → ∞. Corresponding to this set of data and given the prior measure $\mu _0\sim \mathcal {N}(0,\mathcal {C}_0)$, we have the following formula for the posterior measure on u:

$$\begin{equation*} \frac{\mathrm{d}\mu ^{y_1, \ldots ,y_n}}{\mathrm{d}\mu _0}(u) \propto \exp \left( -\frac{1}{2}\sum _{j=1}^n|y_j-\mathcal {G}(u)|_{\mathcal {C}_1}^2 \right). \end{equation*}$$

Here, and in the following, we use the notations $\langle\cdot ,\cdot \rangle_{\mathcal {C}_1}=\langle\mathcal {C}_1^{-1/2}\cdot ,\mathcal {C}_1^{-1/2}\cdot \rangle$ and $|\cdot |_{\mathcal {C}_1}^2=\langle\cdot ,\cdot \rangle _{\mathcal {C}_1}$. By corollary 3.10, MAP estimators for this problem are minimizers of

$$\begin{equation} I_n:=\Vert u\Vert _E^2+\sum _{j=1}^n|y_j-\mathcal {G}(u)|_{\mathcal {C}_1}^2. \end{equation} \tag{ 4.1 }$$

Our interest is in studying the properties of the limits of minimizers u_n of I_n, namely the MAP estimators corresponding to the preceding family of posterior measures. We have the following theorem concerning the behaviour of u_n when n → ∞.

Theorem 4.1. Assume that $\mathcal {G}:X\rightarrow \mathbb {R}^K$ is Lipschitz on bounded sets and u† ∈ E. For every $n \in \mathbb {N}$, let u_n ∈ E be a minimizer of I_n given by (4.1). Then there exist a u* ∈ E and a subsequence of $\lbrace u_n\rbrace _{n \in \mathbb {N}}$ that converges weakly to u* in E, almost surely. For any such u*, we have $\mathcal {G}(u^*)=\mathcal {G}(u^\dagger )$.

We describe some preliminary calculations useful in the proof of this theorem, then give lemma 4.2, also useful in the proof, and finally give the proof itself.

We first observe that, under the assumption that $\mathcal {G}$ is Lipschitz on bounded sets, assumptions 2.1 hold for Φ. We note that

$$\begin{eqnarray*} I_n &=\Vert u\Vert _E^2+\sum _{j=1}^n|y_j-\mathcal {G}(u)|_{\mathcal {C}_1}^2\\ &=\Vert u\Vert _E^2+n|\mathcal {G}(u^\dagger )-\mathcal {G}(u)|_{\mathcal {C}_1}^2 +2\sum _{j=1}^n\langle \mathcal {G}(u^\dagger )-\mathcal {G}(u), \mathcal {C}_1^{-1} \eta _j\rangle . \end{eqnarray*}$$

Hence,

$$\begin{equation*} \mathop{\rm arg\ min}_{u} I_n=\mathop{\rm arg\ min}_{u}\left\lbrace \Vert u\Vert _E^2+n|\mathcal {G}(u^\dagger )-\mathcal {G}(u)|_{\mathcal {C}_1}^2 +2\sum _{j=1}^n\langle \mathcal {G}(u^\dagger )-\mathcal {G}(u), \mathcal {C}_1^{-1} \eta _j\rangle \right\rbrace . \end{equation*}$$

Define

$$\begin{equation*} J_n(u)=|\mathcal {G}(u^\dagger )-\mathcal {G}(u)|_{\mathcal {C}_1}^2 +\frac{1}{n}\Vert u\Vert _E^2 +\frac{2}{n}\sum _{j=1}^n \langle \mathcal {G}(u^\dagger )-\mathcal {G}(u), \mathcal {C}_1^{-1}\eta _j \rangle . \end{equation*}$$

We have

$$\begin{equation*} \mathop{\rm arg\ min}_{u} I_n = \mathop{\rm arg\ min}_{u} J_n. \end{equation*}$$

Lemma 4.2. Assume that $\mathcal {G}:X\rightarrow \mathbb {R}^K$ is Lipschitz on bounded sets. Then for fixed $n\in \mathbb {N}$ and almost surely, there exists u_n ∈ E such that

$$\begin{equation*} J_n(u_n) = \inf _{u\in E}J_n(u). \end{equation*}$$

Proof. We first observe that, under the assumption that $\mathcal {G}$ is Lipschitz on bounded sets and because for a given n and fixed realizations η₁, ..., η_n there exists an r > 0 such that max {|y₁|, ..., |y_n|} < r, assumptions 2.1 hold for Φ. Since $\mathop{\rm arg\ min}_{u} I_n= \mathop{\rm arg\ min}_{u} J_n$, the result follows by proposition 3.4. □

We may now prove the posterior consistency theorem. From (4.3) onwards, the proof is an adaptation of the proof of theorem 2 of [3]. We note that the assumptions on limiting behaviour of measurement noise in [3] are stronger: property (9) of [3] is not assumed here for our J_n. On the other hand, a frequentist approach is used in [3], while here, since J_n is coming from a Bayesian approach, the norm in the regularization term is stronger (it is related to the Cameron–Martin space of the Gaussian prior). That is why in our case, asking what if u† is not in E and only in X is relevant and is answered in corollary 4.3 below.

Proof of theorem 4.1 By the definition of u_n, we have

$$\begin{equation*} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n}\Vert u_n\Vert _E^2 +\frac{2}{n}\sum _{j=1}^n \langle \mathcal {G}(u^\dagger )-\mathcal {G}(u_n), \mathcal {C}_1^{-1}\eta _j \rangle \le \frac{1}{n}\Vert u^\dagger \Vert _E^2. \end{equation*}$$

Therefore,

$$\begin{equation*} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n}\Vert u_n\Vert _E^2 \le \frac{1}{n}\Vert u^\dagger \Vert _E^2 +\frac{2}{n} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1} |\sum _{j=1}^n \mathcal {C}_1^{-1/2}\eta _j|. \end{equation*}$$

Using Young's inequality (see lemma 1.8 of [31], for example) for the last term on the right-hand side, we obtain

$$\begin{equation*} \frac{1}{2}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n}\Vert u_n\Vert _E^2 \le \frac{1}{n}\Vert u^\dagger \Vert _E^2 +\frac{2}{n^2} \left(\sum _{j=1}^n \mathcal {C}_1^{-1/2}\eta _j\right)^2. \end{equation*}$$

Taking expectation and noting that the {η_j} are independent, we obtain

$$\begin{equation*} \frac{1}{2}\mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n}\mathbb {E}\Vert u_n\Vert _E^2 \le \frac{1}{n}\Vert u^\dagger \Vert _E^2 +\frac{2K}{n}, \end{equation*}$$

where $K=\mathbb {E}|\mathcal {C}_1^{-1/2}\eta _1|^2$. This implies that

$$\begin{equation} \mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2\rightarrow 0\quad \mbox{as}\quad n\rightarrow \infty \end{equation} \tag{ 4.2 }$$

and

$$\begin{equation} \mathbb {E}\Vert u_n\Vert _E^2\le \Vert u^\dagger \Vert _E^2+2K. \end{equation} \tag{ 4.3 }$$

• (1)  
  We first show using (4.3) that there exist u* ∈ E and a subsequence $\lbrace u_{n_k(k)}\rbrace _{k\in \mathbb {N}}$ of {u_n} such that

  $$\begin{equation} \mathbb {E}\langle u_{n_k(k)},v\rangle _E\rightarrow \mathbb {E}\langle u^*,v\rangle _E\quad \mbox{for any }v\in E. \end{equation} \tag{ 4.4 }$$

  Let $\lbrace \phi _j\rbrace _{j\in \mathbb {N}}$ be a complete orthonormal system for E. Then,

  $$\begin{equation*} \mathbb {E}\langle u_n,\phi _1 \rangle _E \le \mathbb {E}\Vert u_n\Vert _E \Vert \phi _1\Vert _E \le \Vert u^\dagger \Vert _E^2+2K. \end{equation*}$$

  Therefore, there exist $\xi _1\in \mathbb {R}$ and a subsequence $\lbrace u_{n_1(k)}\rbrace _{k\in \mathbb {N}}$ of $\lbrace u_n\rbrace _{n\in \mathbb {N}}$ such that $\mathbb {E}\langle u_{n_1(k)},\phi _1 \rangle \rightarrow \xi _1$. Now considering $\mathbb {E}\langle u_{n_1(k)},\phi _2\rangle$ and using the same argument, we conclude that there exist $\xi _2\in \mathbb {R}$ and a subsequence $\lbrace u_{n_2(k)}\rbrace _{k\in \mathbb {N}}$ of $\lbrace u_{n_1(k)}\rbrace _{k\in \mathbb {N}}$ such that $\mathbb {E}\langle u_{n_2(k)},\phi _2 \rangle \rightarrow \xi _2$. Continuing similarly, we can show that there exist $\lbrace \xi _j\rbrace \in \mathbb {R}^\infty$ and $\lbrace u_{n_1(k)}\rbrace _{k\in \mathbb {N}}\supset \lbrace u_{n_2(k)}\rbrace _{k\in \mathbb {N}} \supset \dots \supset \lbrace u_{n_j(k)}\rbrace _{k\in \mathbb {N}}$ such that $\mathbb {E}\langle u_{n_j(k)},\phi _j \rangle \rightarrow \xi _j$ for any $j\in \mathbb {N}$ as k → ∞. Therefore,

  $$\begin{equation*} \mathbb {E}\langle u_{n_k(k)},\phi _j \rangle _E\rightarrow \xi _j,\quad \mbox{as }k\rightarrow \infty \mbox{ for any }j\in \mathbb {N}. \end{equation*}$$

  We need to show that $\lbrace \xi _j\rbrace \in \ell ^2(\mathbb {R})$. We have, for any $N\in \mathbb {N}$,

  $$\begin{equation*} \sum _{j=1}^N \xi _j^2 \le \lim _{k\rightarrow \infty }\mathbb {E}\sum _{j=1}^N\langle u_{n_k(k)},\phi _j \rangle _E^2 \le \limsup _{k\rightarrow \infty }\mathbb {E}\Vert u_{n_k(k)}\Vert _E^2\le \Vert u^\dagger \Vert _E^2+2K. \end{equation*}$$

  Therefore, $\lbrace \xi _j\rbrace \in \ell ^2(\mathbb {R})$ and $u^*:=\sum _{j=1}^\infty \xi _j\phi _j \in E$. We can now write for any nonzero v ∈ E,

  $$\begin{eqnarray*} \mathbb {E}\langle u_{n_k(k)}-u^*,v\rangle _E &=&\mathbb {E}\sum _{j=1}^\infty \langle v,\phi _j \rangle _E\langle u_{n_k(k)}-u^*,\phi _j\rangle _E\\ &\le & N\Vert v\Vert _E \mathbb {E}\sup _{j\in \lbrace 1, \ldots ,N\rbrace }|\langle u_{n_k(k)} - u^*,\phi _j\rangle _E| \\ &&+\, (\Vert u^\dagger \Vert _E^2+2K)^{1/2}\sum _{j=N}^\infty |\langle v,\phi _j \rangle _E|. \end{eqnarray*}$$

  Now for any fixed ε > 0, we choose N large enough so that

  $$\begin{equation*} (\Vert u^\dagger \Vert _E^2+2K)^{1/2} \sum _{j=N}^\infty |\langle v,\phi _j \rangle _E|< \frac{1}{2} \varepsilon , \end{equation*}$$

  and then k large enough so that

  $$\begin{equation*} N\Vert v\Vert _E \mathbb {E}|\langle u_{n_k(k)}-u^*,\phi _j\rangle _E| < \case{1}{2} \varepsilon \quad \mbox{for any }1\le j\le N. \end{equation*}$$

  This demonstrates that $\mathbb {E}\langle u_{n_k(k)}-u^*,v\rangle _E\rightarrow 0$ as k → ∞.
• (2)  
  Now we show the almost sure existence of a convergent subsequence of $\lbrace u_{n_k(k)}\rbrace$. By (4.2), we have $|\mathcal {G}(u_{n_k(k)})-\mathcal {G}(u^\dagger )|_{\mathcal {C}_1}\rightarrow 0$ in probability as k → ∞. Therefore, there exists a subsequence {u_m(k)} of $\lbrace u_{n_k(k)}\rbrace$ such that

  $$\begin{equation*} \mathcal {G}(u_{m(k)})\rightarrow \mathcal {G}(u^\dagger )\quad \mbox{a.s. }\mbox{ as }k\rightarrow \infty . \end{equation*}$$

  Now by (4.4), we have 〈u_m(k) − u*, v〉_E → 0 in probability as k → ∞, and hence there exists a subsequence $\lbrace u_{\hat{m}(k)}\rbrace$ of {u_m(k)} such that $u_{\hat{m}(k)}$ converges weakly to u* in E almost surely as k → ∞. Since E is compactly embedded in X, this implies that $u_{\hat{m}(k)}\rightarrow u^*$ in X almost surely as k → ∞. The result now follows by continuity of $\mathcal {G}$.

 □

In the case where u† ∈ X (and not necessarily in E), we have the following weaker result.

Corollary 4.3. Suppose that $\mathcal {G}$ and u_n satisfy the assumptions of theorem 4.1, and that u† ∈ X. Then there exists a subsequence of $\lbrace \mathcal {G}(u_n)\rbrace _{n\in \mathbb {N}}$ converging to $\mathcal {G}(u^\dagger )$ almost surely.

Proof. For any ε > 0, by the density of E in X, there exists v ∈ E such that ||u† − v||_X ⩽ ε. Then by the definition of u_n, we can write

$$\begin{eqnarray*} &&|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n}\Vert u_n\Vert _E^2 +\frac{2}{n}\sum _{j=1}^n \langle \mathcal {G}(u^\dagger )-\mathcal {G}(u_n),\mathcal {C}_1^{-1}\eta _j \rangle \\ &&\le |\mathcal {G}(u^\dagger )-\mathcal {G}(v)|_{\mathcal {C}_1}^2+\frac{1}{n}\Vert v\Vert _E^2+\frac{2}{n}\sum _{j=1}^n \langle \mathcal {G}(u^\dagger )-\mathcal {G}(v),\mathcal {C}_1^{-1}\eta _j \rangle . \end{eqnarray*}$$

Therefore, dropping $\frac{1}{n}\Vert u_n\Vert _E^2$ on the left-hand side, and using Young's inequality, we obtain

$$\begin{equation*} \frac{1}{2} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 \le 2|\mathcal {G}(u^\dagger )-\mathcal {G}(v)|_{\mathcal {C}_1}^2+\frac{1}{n}\Vert v\Vert _E^2+\frac{3}{n^2}\sum _{j=1}^n |\mathcal {C}_1^{-1/2}\eta _j|^2. \end{equation*}$$

By the local Lipschitz continuity of $\mathcal {G}$, $|\mathcal {G}(u^\dagger )-\mathcal {G}(v)|_{\mathcal {C}_1}\le C\varepsilon ^2$, and therefore, taking the expectations and noting the independence of {η_j}, we obtain

$$\begin{equation*} \mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2\le 4C\varepsilon ^2+\frac{2C_\varepsilon }{n}+\frac{6K}{n}, \end{equation*}$$

implying that

$$\begin{equation*} \limsup _{n\rightarrow \infty }\mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2\le 4C\varepsilon ^2. \end{equation*}$$

Since the $\lim \inf$ is obviously positive and ε was arbitrary, we have $\lim _{n\rightarrow \infty }\mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2=0$. This implies that $|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}\rightarrow 0$ in probability. Therefore, there exists a subsequence of $\lbrace \mathcal {G}(u_n)\rbrace$ which converges to $\mathcal {G}(u^\dagger )$ almost surely. □

4.2. Small noise limit

Consider the case where, as data, we have the random vector

$$\begin{equation} y_n = \mathcal {G}(u^\dagger )+\frac{1}{n}\eta _n \end{equation} \tag{ 4.5 }$$

for $n \in \mathbb {N}$ and with u† again as the true solution and $\eta _j\sim \mathcal {N}(0,\mathcal {C}_1)$, $j\in \mathbb {N}$, Gaussian random vectors in $\mathbb {R}^K$. Thus, in the preceding general setting, we have $G=\mathcal {G}$ and J = K. Rather than having n independent observations, we have an observation noise scaled by small γ = 1/n converging to zero. For these data and given the prior measure μ₀ on u, we have the following formula for the posterior measure:

$$\begin{equation*} \frac{\mathrm{d}\mu ^{y_n}}{\mathrm{d}\mu _0}(u) \propto \exp \left( -\frac{n^2}{2}\left| y_n-\mathcal {G}(u) \right|_{\mathcal {C}_1}^2 \right). \end{equation*}$$

By the result of the previous section, the MAP estimators for the above measure are the minimizers of

$$\begin{equation} I_n(u):=\Vert u\Vert _E^2+n^2|y_n-\mathcal {G}(u)|_{\mathcal {C}_1}^2. \end{equation} \tag{ 4.6 }$$

Our interest is in studying the properties of the limits of minimizers of I_n as n → ∞. We have the following almost-sure convergence result.

Theorem 4.4. Assume that $\mathcal {G}:X\rightarrow \mathbb {R}^K$ is Lipschitz on bounded sets, and u† ∈ E. For every $n \in \mathbb {N}$, let u_n ∈ E be a minimizer of I_n(u) given by (4.6). Then, there exist a u* ∈ E and a subsequence of $\lbrace u_n\rbrace _{n \in \mathbb {N}}$ that converges weakly to u* in E, almost surely. For any such u*, we have $\mathcal {G}(u^*)=\mathcal {G}(u^\dagger )$.

Proof. The proof is very similar to that of theorem 4.1 and so we only sketch differences. We have

$$\begin{eqnarray*} I_n &=\Vert u\Vert _E^2+n^2|y_n-\mathcal {G}(u)|_{\mathcal {C}_1}^2\\ &=\Vert u\Vert _E^2+n^2|\mathcal {G}(u^\dagger )+\frac{1}{n}\eta _n-\mathcal {G}(u)|_{\mathcal {C}_1}^2\\ &=\Vert u\Vert _E^2+n^2|\mathcal {G}(u^\dagger )-\mathcal {G}(u)|_{\mathcal {C}_1}^2+|\eta _n|_{\mathcal {C}_1}^2 +2 n\langle \mathcal {G}(u^\dagger )-\mathcal {G}(u),\eta _n \rangle_{\mathcal {C}_1}. \end{eqnarray*}$$

Letting

$$\begin{equation*} J_n(u)=\frac{1}{n^2}\Vert u\Vert _E^2+|\mathcal {G}(u^\dagger )-\mathcal {G}(u)|_{\mathcal {C}_1}^2 +\frac{2}{n}\langle \mathcal {G}(u^\dagger )-\mathcal {G}(u),\eta _n \rangle_{\mathcal {C}_1}, \end{equation*}$$

we hence have arg min_uI_n = arg min_uJ_n. For this J_n, the result of lemma 4.2 holds true, using an argument similar to the large sample size case. The result of theorem 4.4 carries over as well. Indeed, by the definition of u_n, we have

$$\begin{equation*} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n^2}\Vert u_n\Vert _E^2 +\frac{2}{n}\langle \mathcal {G}(u^\dagger )-\mathcal {G}(u_n), \mathcal {C}_1^{-1}\eta _n \rangle \le \frac{1}{n^2}\Vert u^\dagger \Vert _E^2. \end{equation*}$$

Therefore,

$$\begin{equation*} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n^2}\Vert u_n\Vert _E^2 \le \frac{1}{n^2}\Vert u^\dagger \Vert _E^2 +\frac{2}{n} |\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1} |\mathcal {C}_1^{-1/2}\eta _n|. \end{equation*}$$

Using Young's inequality for the last term on the right-hand side, we obtain

$$\begin{equation*} \frac{1}{2}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n^2}\Vert u_n\Vert _E^2 \le \frac{1}{n^2}\Vert u^\dagger \Vert _E^2 +\frac{2}{n^2} |\mathcal {C}_1^{-1/2}\eta _n|^2. \end{equation*}$$

Taking expectation, we obtain

$$\begin{equation*} \mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2 +\frac{1}{n^2}\mathbb {E}\Vert u_n\Vert _E^2 \le \frac{1}{n^2}\Vert u^\dagger \Vert _E^2 +\frac{2K}{n^2}. \end{equation*}$$

This implies that

$$\begin{equation} \mathbb {E}|\mathcal {G}(u^\dagger )-\mathcal {G}(u_n)|_{\mathcal {C}_1}^2\rightarrow 0\quad \mbox{as}\quad n\rightarrow \infty \end{equation} \tag{ 4.7 }$$

and

$$\begin{equation} \mathbb {E}\Vert u_n\Vert _E^2\le \Vert u^\dagger \Vert _E^2+2K. \end{equation} \tag{ 4.8 }$$

Having (4.7) and (4.8), and with the same argument as in the proof of theorem 4.1, it follows that there exist a u* ∈ E and a subsequence of {u_n} that converges weakly to u* in E almost surely, and for any such u*, we have $\mathcal {G}(u^*)=\mathcal {G}(u^\dagger )$. □

As in the large sample size case, here also, if we have u† ∈ X and we do not restrict the true solution to be in the Cameron–Martin space E, one can prove, in a similar way to the argument of the proof of corollary 4.3, the following weaker convergence result.

Corollary 4.5. Suppose that $\mathcal {G}$ and u_n satisfy the assumptions of theorem 4.4, and that u† ∈ X. Then there exists a subsequence of $\lbrace \mathcal {G}(u_n)\rbrace _{n\in \mathbb {N}}$ converging to $\mathcal {G}(u^\dagger )$ almost surely.

In this section, we present an application of the methods presented above to filtering and smoothing in fluid dynamics, which is relevant to data assimilation applications in oceanography and meteorology. We link the MAP estimators introduced in this paper to the variational methods used in applications [2], and we demonstrate posterior consistency in this context.

We consider the 2D Navier–Stokes equation on the torus $\mathbb {T}^{2} := [-1,1) \times [-1,1)$ with the following periodic boundary conditions:

$$\begin{eqnarray*} \begin{array}{ll}\partial _tv - \nu \Delta v + v \cdot \nabla v + \nabla p = f & \mbox{for all $(x, t) \in \mathbb {T}^{2} \times (0, \infty )$,} \\ \nabla \cdot v = 0 & \mbox{for all $(x, t) \in \mathbb {T}^{2} \times (0, \infty )$,} \\ v = u & \mbox{for all $(x,t) \in \mathbb {T}^{2} \times \lbrace 0\rbrace $.} \end{array} \end{eqnarray*}$$

Here, $v :\mathbb {T}^{2} \times (0, \infty ) \rightarrow \mathbb {R}^{2}$ is a time-dependent vector field representing the velocity, $p :\mathbb {T}^{2} \times (0,\infty ) \rightarrow \mathbb {R}$ is a time-dependent scalar field representing the pressure, $f :\mathbb {T}^{2} \rightarrow \mathbb {R}^{2}$ is a vector field representing the forcing (which we assume to be time independent for simplicity) and ν is the viscosity. We are interested in the inverse problem of determining the initial velocity field u from pointwise measurements of the velocity field at later times. This is a model for the situation in weather forecasting where observations of the atmosphere are used to improve the initial condition used for forecasting. For simplicity, we assume that the initial velocity field is divergence free and integrates to zero over $\mathbb {T}^2$, noting that this property will be preserved in time.

Define

$$\begin{equation*} \mathcal {H}:= \left\lbrace \mbox{trigonometric polynomials } u:\mathbb {T}^2 \rightarrow {\mathbb {R}}^2 \Bigl | \nabla \cdot u = 0, \int _{\mathbb {T}^{2}} u(x) \,\mathrm{d}x = 0 \right\rbrace \end{equation*}$$

and H as the closure of $\mathcal {H}$ with respect to the $(L^{2}(\mathbb {T}^{2}))^{2}$ norm. We define the map $P:(L^{2}(\mathbb {T}^{2}))^{2} \rightarrow H$ to be the Leray–Helmholtz orthogonal projector (see [31]). Given k = (k₁, k₂)^T, define k^⊥ := (k₂, −k₁)^T. Then an orthonormal basis for H is given by $\psi _k :\mathbb {R}^{2} \rightarrow \mathbb {R}^{2}$, where

$$\begin{equation*} \psi _k (x) := \frac{k^{\perp }}{|k|} \exp (\pi {\rm i} k \cdot x) \end{equation*}$$

for $k \in \mathbb {Z}^{2} \setminus \lbrace 0\rbrace$. Thus, for u ∈ H, we may write

$$\begin{equation*} u = \sum _{k \in \mathbb {Z}^{2} \setminus \lbrace 0\rbrace } u_k(t) \psi _k(x), \end{equation*}$$

where, since u is a real-valued function, we have the reality constraint $u_{-k} = - \bar{u}_k$. Using the Fourier decomposition of u, we define the fractional Sobolev spaces

$$\begin{equation*} H^{s} := \left\lbrace u \in H \bigg | \sum _{k \in \mathbb {Z}^{2} \setminus \lbrace 0\rbrace } (\pi ^{2}| k |^{2})^{s}| u_k |^{2} < \infty \right\rbrace \end{equation*}$$

with the norm $\Vert u \Vert _s:= \bigl (\sum _k (\pi ^{2}| k |^{2})^{s} | u_k |^{2}\bigr )^{1/2}$, where $s \in \mathbb {R}$. If A = −PΔ, the Stokes operator, then H^s = D(A^s/2). We assume that f ∈ H^s for some s > 0.

Let t_ℓ = ℓh, for ℓ = 0, ..., L, and define $v_\ell \in \mathbb {R}^M$ to be the set of pointwise values of the velocity field given by $\lbrace v(x_m,t_\ell )\rbrace _{m \in \mathbb {M}}$, where $\mathbb {M}$ is some finite set of points in $\mathbb {T}^2$ with cardinality M/2. Note that each v_ℓ depends on u and we may define $\mathcal {G}_\ell :H \rightarrow \mathbb {R}^{M}$ by $\mathcal {G}_\ell (u)=v_\ell$. We let {η_ℓ}_{ℓ ∈ {1, ..., L}} be a set of random variables in $\mathbb {R}^{M}$ which perturbs the points {v_ℓ}_{ℓ ∈ {1, ..., L}} to generate the observations {y_ℓ}_{ℓ ∈ {1, ..., L}} in $\mathbb {R}^{M}$ given by

$$\begin{equation*} y_{\ell } := v_{\ell } + \gamma {\eta }_{\ell }, \quad \ell \in \lbrace 1,\ldots , L\rbrace . \end{equation*}$$

We let $y=\lbrace y_\ell \rbrace _{\ell =1}^L$, the accumulated data up to time T = Lh, with similar notation for η, and define $\mathcal {G}:H \rightarrow \mathbb {R}^{ML}$ by $\mathcal {G}(u)= (\mathcal {G}_1(u), \ldots ,\mathcal {G}_L(u))$. We now solve the inverse problem of finding u from $y=\mathcal {G}(u)+\gamma \eta$. We assume that the prior distribution on u is a Gaussian $\mu _0=N(0,\mathcal {C}_0)$, with the property that μ₀(H) = 1 and that the observational noise {η_ℓ}_{ℓ ∈ {1, ..., L}} is i.i.d. in $\mathbb {R}^M$, independent of u, with η₁ distributed according to a Gaussian measure N(0, I). If we define

$$\begin{equation*} \Phi (u)=\frac{1}{2\gamma ^2}\sum _{j=1}^L |y_j-\mathcal {G}_j(u)|^2, \end{equation*}$$

then under the preceding assumptions, the Bayesian inverse problem for the posterior measure μ^y for u|y is well defined and is Lipschitz in y with respect to the Hellinger metric (see [7]). The Onsager–Machlup functional in this case is given by

$$\begin{equation*} I_{\mathrm{NS}}(u) = \case{1}{2}\Vert u\Vert _{\mathcal {C}_0}^2+\Phi (u). \end{equation*}$$

We are in the setting of subsection 4.2, with γ = 1/n and K = ML. In the applied literature, approaches to assimilating data into mathematical models based on minimizing I_NS are known as variational methods, and sometimes as 4DVAR [2].

We now describe numerical experiments concerned with studying posterior consistency in the case γ → 0. We let $\mathcal {C}_0 = A^{-2}$ noting that if u ∼ μ₀, then u ∈ H^s almost surely for all s < 1; in particular, u ∈ H. Thus μ₀(H) = 1 as required. The forcing in f is taken to be f = ∇^⊥Ψ, where Ψ = cos (πk · x) and ∇^⊥ = J∇ with J the canonical skew-symmetric matrix and k = (5, 5). The dimension of the attractor is determined by the viscosity parameter ν. For the particular forcing used, there is an explicit steady state for all ν > 0, and for ν ⩾ 0.035, this solution is stable (see [26], chapter 2, for details). As ν decreases, the flow becomes increasingly complex, and we focus subsequent studies of the inverse problem on the mildly chaotic regime which arises for ν = 0.01. We use a time-step of δt = 0.005. The data are generated by computing a true signal solving the Navier–Stokes equation at the desired value of ν, and then adding Gaussian random noise to it at each observation time. Furthermore, we let h = 4δt = 0.02 and take L = 10, so that T = 0.2. We take M = 32² spatial observations at each observation time. The observations are made at the gridpoints; thus the observations include all numerically resolved, and hence observable, wavenumbers in the system. Since the noise is added in spectral space in practice, for convenience we define $\sigma = \gamma / \sqrt{M}$ and present the results in terms of σ. The same grid is used for computing the reference solution and the MAP estimator.

Figure 1 illustrates the posterior consistency which arises as the observational noise strength γ → 0. The three curves shown quantify (i) the relative error of the MAP estimator u* compared with the truth, u†, (ii) the relative error of $\mathcal {G}(u^*)$ compared with $\mathcal {G}(u^{\dagger })$ and (iii) the relative error of $\mathcal {G}(u^*)$ with respect to the observations y. The figure clearly illustrates theorem 4.4, via the dashed curve for (ii), and indeed shows that the map estimator itself is converging to the true initial condition, via the blue curve, as γ → 0. Recall that the observations approach the true value of the initial condition, mapped forward under $\mathcal {G}$, as γ → 0, and note that the dashed and dashed–dotted curves shows that the image of the MAP estimator under the forward operator $\mathcal {G}$, $\mathcal {G}(u^*)$, is closer to $\mathcal {G}(u^\dagger )$ than y, asymptotically as γ → 0.

Zoom In Zoom Out Reset image size

In this section, we consider the MAP estimator for conditioned diffusions, including bridge diffusions and an application to filtering/smoothing. We identify the Onsager–Machlup functional governing the MAP estimator in three different cases. We demonstrate numerically that this functional may have more than one minimizer. Furthermore, we illustrate the results of the consistency theory in section 4 using numerical experiments. Subsection 6.1 concerns the unconditioned case, and includes the assumptions made throughout. Subsections 6.2 and 6.3 describe bridge diffusions and the filtering/smoothing problem, respectively. Finally, subsection 6.4 is devoted to numerical experiments for an example in filtering/smoothing.

6.1. Unconditioned case

For simplicity, we restrict ourselves to scalar processes with additive noise, taking the form

$$\begin{equation} {\rm d}u = f(u) \,{\rm d}t + \sigma \,{\rm d}W, \quad u(0)=u^-. \end{equation} \tag{ 6.1 }$$

If we let ν denote the measure on $X := C([0,T];\mathbb {R})$ generated by the stochastic differential equation (SDE) given in (6.1), and ν₀ the same measure obtained in the case f ≡ 0, then the Girsanov theorem states that ν ≪ ν₀ with density

$$\begin{equation*} \frac{\mathrm{d}\nu }{\mathrm{d}\nu _0}(u) = \exp \left({-}\frac{1}{2\sigma ^2}\int _0^T |f(u(t))|^2\,{\rm d}t+\frac{1}{\sigma ^2}\int _0^T f(u(t)) {\rm d}u(t)\right). \end{equation*}$$

If we choose an $F:\mathbb {R}\rightarrow \mathbb {R}$ with F'(u) = f(u), then an application of Itô's formula gives

$$\begin{equation*} {\rm d}F(u(t)) = f(u(t))\, {\rm d}u(t) +\frac{\sigma ^2}{2} f^{\prime }(u(t)) \, {\rm d}t, \end{equation*}$$

and using this expression to remove the stochastic integral, we obtain

$$\begin{equation} \frac{\mathrm{d}\nu }{\mathrm{d}\nu _0}(u) \propto \exp \left({-}\frac{1}{2\sigma ^2}\int _0^T (|f(u(t))|^2+\sigma ^2 f^{\prime }(u(t))) \,{\rm d}t +\frac{1}{\sigma ^2}F(u(T))\right). \end{equation} \tag{ 6.2 }$$

Thus, the measure ν has a density with respect to the Gaussian measure ν₀ and (6.2) takes the form (1.1) with μ = ν and μ₀ = ν₀; we have

$$\begin{equation*} \frac{\mathrm{d}\nu }{\mathrm{d}\nu _0}(u) \propto \exp ({-}\Phi _1(u) ), \end{equation*}$$

where $\Phi _1:X \rightarrow \mathbb {R}$ is defined by

$$\begin{equation} \Phi _1(u) = \int _0^T \Psi (u(t)) \,{\rm d}t - \frac{1}{\sigma ^2}F(u(T)) \end{equation} \tag{ 6.3 }$$

and

$$\begin{equation*} \Psi (u)=\frac{1}{2\sigma ^2} (|f(u)|^2+\sigma ^2 f^{\prime }(u)). \end{equation*}$$

We make the following assumption concerning the vector field f driving the SDE.

Assumption 6.1. The function f = F' in (6.1) satisfies the following conditions.

• (1)  
  $F \in C^2(\mathbb {R},\mathbb {R})$ for all $u \in \mathbb {R}$.
• (2)  
  There is $M \in \mathbb {R}$ such that Ψ(u) ⩾ M for all $u \in \mathbb {R}$ and F(u) ⩽ M for all $u \in \mathbb {R}$.

Under these assumptions, we see that Φ₁ given by (6.3) satisfies assumptions 2.1 and, indeed, the slightly stronger assumptions made in theorem 3.5. Let H¹[0, T] denote the space of absolutely continuous functions on [0, T]. Then the Cameron–Martin space E₁ for ν₀ is

$$\begin{equation*} E_1=\bigg\lbrace v \in H^1[0,T]| \int _0^T |v^{\prime }(s)|^2\, {\rm d}s<\infty \mbox{ and } v(0)=0\bigg\rbrace \end{equation*}$$

and the Cameron–Martin norm is given by

$$\begin{equation*} \Vert v\Vert _{E_1} = \sigma ^{-1}\Vert v\Vert _{H^1}, \end{equation*}$$

where

$$\begin{equation*} \Vert v\Vert _{H^1} = \left(\int _0^T \bigl |v^{\prime }(s)\bigr |^2\, {\rm d}s\right)^{\frac{1}{2}}. \end{equation*}$$

The mean of ν₀ is the constant function m ≡ u⁻ and so, using remark 2.2, we see that the Onsager–Machlup functional for the unconditioned diffusion (6.1) is thus $I_1:E_1 \rightarrow \mathbb {R}$ given by

$$\begin{equation*} I_1(u) = \Phi _1(u) + \frac{1}{2\sigma ^2}\Vert u - u^- \Vert _{H^1}^2 = \Phi _1(u) + \frac{1}{2\sigma ^2}\Vert u \Vert _{H^1}^2. \end{equation*}$$

Together, theorems 3.2 and 3.5 tell us that this functional attains its minimum over $E_1^{\prime }$ defined by

$$\begin{equation*} E_1^{\prime } = \left\lbrace v \in H^1[0,T]| \int _0^T |v^{\prime }(s)|^2 {\rm d}s<\infty\quad \mbox{ and }\quad v(0)=u^-\right\rbrace . \end{equation*}$$

Furthermore, such minimizers define MAP estimators for the unconditioned diffusion (6.1), i.e. the most likely paths of the diffusion.

We note that the regularity of minimizers for I₁ implies that the MAP estimator is C², whilst sample paths of the SDE (6.1) are not even differentiable. This is because the MAP estimator defines the centre of a tube in X which contains the most likely paths. The centre itself is a smoother function than the paths. This is a generic feature of MAP estimators for measures defined via density with respect to a Gaussian in infinite dimensions.

6.2. Bridge diffusions

In this subsection, we study the probability measure generated by solutions of (6.1), conditioned to hit u⁺ at time 1 so that u(T) = u⁺, and denote this measure μ. Let μ₀ denote the Brownian bridge measure obtained in the case f ≡ 0. By applying the approach to determining bridge diffusion measures in [17], we obtain, from (6.2), the expression

$$\begin{equation} \frac{\mathrm{d}\mu }{\mathrm{d}\mu _0}(u) \propto \exp \left({-}\int _0^T \Psi (u(t)) \,{\rm d}t +\frac{1}{\sigma ^2}F(u^+)\right). \end{equation} \tag{ 6.4 }$$

Since u⁺ is fixed, we now define $\Phi _2:X \rightarrow \mathbb {R}$ by

$$\begin{equation*} \Phi _2(u) = \int _0^T \Psi (u(t))\, {\rm d}t \end{equation*}$$

and then (6.4) takes again the form (1.1). The Cameron–Martin space for the (zero mean) Brownian bridge is

$$\begin{equation*} E_2 = \left\lbrace v \in H^1[0,T]| \int _0^T |v^{\prime }(s)|^2\, {\rm d}s<\infty\ \mbox{ and }\quad v(0)=v(T)=0\right\rbrace , \end{equation*}$$

and the Cameron–Martin norm is again $\sigma ^{-1}\Vert \cdot \Vert _{H^1}$. The Onsager–Machlup function for the unconditioned diffusion (6.1) is thus $I_2:E_2^{\prime } \rightarrow \mathbb {R},$ given by

$$\begin{equation*} I_2(u) = \Phi _2(u) + \frac{1}{2\sigma ^2}\Vert u - m\Vert _{H^1}^2, \end{equation*}$$

where m, given by $m(t) = \frac{T-t}{T} u^- + \frac{t}{T} u^+$ for all t ∈ [0, T], is the mean of μ₀ and

$$\begin{equation*} E_2^{\prime } = \left\lbrace v \in H^1[0,T]| \int _0^T |v^{\prime }(s)|^2\, {\rm d}s<\infty \mbox{ and } v(0)=u^-, u(T)=u^+ \right\rbrace . \end{equation*}$$

The MAP estimators for μ are found by minimizing I₂ over $E_2^{\prime }$.

6.3. Filtering and smoothing

We now consider conditioning the measure ν on observations of the process u at discrete time points. Assume that we observe $y \in \mathbb {R}^J$ given by

$$\begin{equation} y_j=u(t_j)+\eta _j, \end{equation} \tag{ 6.5 }$$

where 0 < t₁ < ⋅⋅⋅ < t_J < T and the η_j are independent identically distributed random variables with η_j ∼ N(0, γ²). Let $\mathbb {Q}_0(\mathrm{d}y)$ denote the $\mathbb {R}^J$-valued Gaussian measure N(0, γ²I) and let $\mathbb {Q}(\mathrm{d}y|u)$ denote the $\mathbb {R}^J$-valued Gaussian measure $N(\mathcal {G}u,\gamma ^2 I)$, where $\mathcal {G}:X \rightarrow \mathbb {R}^J$ is defined by

$$\begin{equation*} \mathcal {G}u=(u(t_1),\ldots ,u(t_J)). \end{equation*}$$

Recall ν₀ and ν from the unconditioned case and define the measures $\mathbb {P}_0$ and $\mathbb {P}$ on $X\times \mathbb {R}^J$ as follows. The measure $\mathbb {P}_0(\mathrm{d}u,\mathrm{d}y)=\nu _0(\mathrm{d}u)\mathbb {Q}_0(\mathrm{d}y)$ is defined to be an independent product of ν₀ and $\mathbb {Q}_0$, whilst $\mathbb {P}(\mathrm{d}u,\mathrm{d}y)=\nu (\mathrm{d}u)\mathbb {Q}(\mathrm{d}y|u)$. Then,

$$\begin{equation*} \frac{\mathrm{d}\mathbb {P}}{\mathrm{d}\mathbb {P}_0}(u,y) \propto \exp \left({-}\int _0^T \Psi (u(t))\, {\rm d}t +\frac{1}{\sigma ^2}F(u(T) )-\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2\right) \end{equation*}$$

with the constant of proportionality depending only on y. Clearly, by continuity,

$$\begin{equation*} \inf _{\Vert u\Vert _X\le 1} \exp \left({-}\int _0^T \Psi (u(t))\, {\rm d}t +\frac{1}{\sigma ^2}F(u(T))-\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2\right)>0, \end{equation*}$$

and hence,

$$\begin{equation*} \int _{\Vert u\Vert _X \le 1} \exp \left(-\int _0^T \Psi (u(t))\, {\rm d}t +\frac{1}{\sigma ^2}F(u(T))-\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2\right) \nu _0({\rm d}u)>0. \end{equation*}$$

Applying the conditioning lemma 5.3 in [17] then gives

$$\begin{equation*} \frac{\mathrm{d}\mu ^y}{\mathrm{d}\nu _0}(u) \propto \exp \left({-}\int _0^T \Psi (u(t))\, {\rm d}t +\frac{1}{\sigma ^2}F(u(T))-\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2\right). \end{equation*}$$

Thus, we define

$$\begin{equation*} \Phi _3(u) =\int _0^T \Psi (u(t))\, {\rm d}t -\frac{1}{\sigma ^2}F(u(T)) +\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2. \end{equation*}$$

The Cameron–Martin space is again E₁ and the Onsager–Machlup functional is thus $I_3:E_1^{\prime } \rightarrow \mathbb {R}$, given by

$$\begin{equation} I_3(u) = \Phi _3(u) + \frac{1}{2\sigma ^2}\Vert u\Vert _{H^1}^2. \end{equation} \tag{ 6.6 }$$

The MAP estimator for this set-up is, again, found by minimizing the Onsager–Machlup functional I₃.

The only difference between the potentials Φ₁ and Φ₃, and thus between the functionals I₁ for the unconditioned case and I₃ for the case with discrete observations, is the presence of the term $\frac{1}{2\gamma ^2}\sum _{j=1}^J |y_j-u(t_j)|^2$. In the Euler–Lagrange equations describing the minima of I₃, this term leads to Dirac distributions at the observation points t₁, ..., t_J, and it transpires that, as a consequence, minimizers of I₃ have jumps in their first derivatives at t₁, ..., t_J. This effect can be clearly seen in the local minima of I₃ shown in figure 2.

Zoom In Zoom Out Reset image size

6.4. Numerical experiments

In this section, we perform three numerical experiments related to the MAP estimator for the filtering/smoothing problem presented in section 6.3.

For the experiments, we generate a random 'signal' by numerically solving the SDE (6.1), using the Euler–Maruyama method, for a double-well potential F given by

$$\begin{equation*} F(u) = - \frac{(1-u)^2(1+u)^2}{1+u^2}, \end{equation*}$$

with diffusion constant σ = 1 and initial value u⁻ = −1. From the resulting solution u(t), we generate random observations y₁, ..., y_J using (6.5). Then we implement the Onsager–Machlup functional I₃ from equation (6.6) and use numerical minimization, employing the Broyden–Fletcher–Goldfarb–Shanno method (see [13]; we use the implementation found in the GNU scientific library [14]), to find the minima of I₃. The same grid is used for numerically solving the SDE and for approximating the values of I₃.

The first experiment concerns the problem of local minima of I₃. For a small number of observations, we find multiple local minima; the minimization procedure can converge to different local minima, depending on the starting point of the optimization. This effect makes it difficult to find the MAP estimator, which is the global minimum of I₃, numerically. The problem is illustrated in figure 2, which shows four different local minima for the case of J = 2 observations. In the presence of local minima, some care is needed when numerically computing the MAP estimator. For example, one could start the minimization procedure with a collection of different starting points and take the best of the resulting local minima as the result. One would expect this problem to become less pronounced as the number of observations increases, since the observations will 'pull' the MAP estimator towards the correct solution, thus reducing the number of local minima. This effect is confirmed by experiments: for larger numbers of observations, our experiments found only one local minimum.

The second experiment concerns the posterior consistency of the MAP estimator in the small noise limit. Here, we use a fixed number J of observations of a fixed path of (6.1), but let the variance γ² of the observational noise η_j converge to 0. Noting that the exact path of the SDE, denoted by u† in (4.5), has the regularity of a Brownian motion and therefore the observed path is not contained in the Cameron–Martin space E₃, we are in the situation described in corollary 4.5. Our experiments indicate that we have $\mathcal {G}(u_\gamma ) \rightarrow \mathcal {G}(u^\dagger )$ as γ↓0, where u_γ denotes the MAP estimator corresponding to observational variance γ², confirming the result of corollary 4.5. As discussed above, for small values of γ, one would expect the minimum of I₃ to be unique, and indeed, experiments where different starting points of the optimization procedure were tried did not find different minima for small δ. The result of a simulation with J = 5 is shown in figure 3.

Zoom In Zoom Out Reset image size

Finally, we perform an experiment to illustrate posterior consistency in the large sample size limit: for this experiment, we still use one fixed path u† of the SDE (6.1). Then, for different values of J, we generate observations y₁, ..., y_J using (6.5) at equidistantly spaced times t₁, ..., t_J, for fixed γ = 1, and then determine the L² distance of the resulting MAP estimate u_J to the exact path u†. As discussed above, for large values of J, one would expect the minimum of I₃ to be unique, and indeed, experiments where different starting points of the optimization procedure were tried did not find different minima for large J. The situation considered here is not covered by the theoretical results from section 4, but the results of the numerical experiment, shown in figure 4, indicate that posterior consistency still holds.

Zoom In Zoom Out Reset image size