Variational regularization theory based on image space approximation rates

Assumption 2.1. Let τ be a topology such that $\left(\mathbb{X},\tau \right)$ is a locally convex Hausdorff space and $\mathcal{R}:\mathbb{X}\to \left(-\infty ,\infty \right]$ a proper, convex function. We assume that the sublevel set $\left\{x\in \mathbb{X}:\mathcal{R}\left(x\right){\leqslant}\lambda \right\}$ is τ-compact for all $\lambda \in \mathbb{R}$. Note that this implies that $\mathcal{R}$ is lower semi-continuous on $\left(\mathbb{X},\tau \right)$.

Let $\mathbb{Y}$ be a Hilbert space and $A:\mathbb{X}\to \mathbb{Y}$ a linear, τ-to-weak continuous operator.

Assumption 2.1 implies τ-compactness of the sublevel sets of the Tikhonov functional. Using the finite intersection property of these sets one can show that R_α (g) is nonempty for all $g\in \mathbb{Y}$. Furthermore, for every g ∈ im(A) there exist a (possibly not unique) $\mathcal{R}$-minimal $x\in \mathbb{X}$ with Ax = g, i.e. $\mathcal{R}\left(x\right){\leqslant}\mathcal{R}\left(z\right)$ for all $z\in \mathbb{X}$ with Az = g. Let $\nu \in \left[\right.0,\infty \left.\right)$ and ϱ > 0. We define

$$\begin{equation}\begin{gathered}{\varrho }_{\nu }:\mathbb{X}\to \left[0,\infty \right]\quad \text{by}\quad {\varrho }_{\nu }\left(x\right)=\mathrm{sup}\left\{{\alpha }^{-\nu }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\enspace :\alpha { >}0,\enspace {x}_{\alpha }\in {R}_{\alpha }\left(Ax\right)\right\},\\ {K}_{\nu }^{\varrho }=\left\{x\in \mathbb{X}:{\varrho }_{\nu }\left(x\right){\leqslant}\varrho \right\}\quad \text{and}\quad {K}_{\nu }{:=}\left\{x\in \mathbb{X}\enspace :{\varrho }_{\nu }\left(x\right){< }\infty \right\}.\end{gathered}\end{equation} \tag{ 2.1 }$$

Note that x ∈ K_ν if and only if a bound (1.2) holds true and ϱ_ν (x) is the smallest possible constant c > 0.

Let $L:\mathbb{X}{\times}\mathbb{X}\to \left[0,\infty \right]$ satisfy the triangle inequality. We use L to measure the reconstruction error.

The first and central result is a uniform bound in ${K}_{\nu }^{\varrho }$ on $L\left(x,{\hat{x}}_{\alpha }\right)$ with ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$ in terms of the modulus of continuity. Recall that the latter is given by

$$\begin{equation}{\Omega}\left(\delta ,K\right){:=}\mathrm{sup}\left\{L\left({x}_{1},{x}_{2}\right)\enspace :{x}_{1},{x}_{2}\in K\quad \;\text{with}\;\quad {\Vert}A{x}_{1}-A{x}_{2}{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta \right\}\end{equation} \tag{ 2.2 }$$

for a subset $K\subset \mathbb{X}$.

We consider two parameter choice rules for the regularization parameter α. An a priori rule requiring prior knowledge of the parameter ν in (1.2) characterizing the regularity of the unknown x, and the discrepancy principle as most well-known a-posteriori rule.

Theorem 1. Let $\nu \in \left(\right.0,1 \left.\right]$ and ϱ, α > 0. Suppose $x\in {K}_{\nu }^{\varrho }$. Let α > 0 and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$.

• (a)  
  (A priori rule) Let c_r ⩾ c_l > 0. If ${c}_{l}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}$, then

  $$\begin{equation*}L\left(x,{\hat{x}}_{\alpha }\right){\leqslant}{\Omega}\left({c}_{1}\delta ,{K}_{\nu }^{{c}_{2}\varrho }\right)\end{equation*}$$

  with ${c}_{1}{:=}1+{c}_{r}^{\nu }$ and ${c}_{2}{:=}2+{c}_{l}^{-\nu }$.
• (b)  
  (Discrepancy principle) Let C_D > c_D > 1. If ${c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta$, then

  $$\begin{equation*}L\left(x,{\hat{x}}_{\alpha }\right){\leqslant}{\Omega}\left({d}_{1}\delta ,{K}_{\nu }^{{d}_{2}\varrho }\right)\end{equation*}$$

  with d₁ := 1 + C_D and ${d}_{2}{:=}2+{\left({c}_{D}-1\right)}^{-1}$.

The proof of theorem 1 can be found in subsection 3.5. Under mild assumptions theorem 1 gives rise to an almost minimax result in the following manner. Recall that the worst case error of a reconstruction map $R:\mathbb{Y}\to \mathbb{X}$ on a set $K\subset \mathbb{X}$ is given by

$$\begin{equation*}{{\Delta}}_{R}\left(\delta ,K\right){:=}\mathrm{sup}\left\{L\left(x,R\left({g}^{\delta }\right)\right)\left.\right):x\in K,{g}^{\delta }\in \mathbb{Y}\quad \;\text{with}\;\quad {\Vert}{g}^{\delta }-Ax{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta \right\}.\end{equation*}$$

and satisfies the lower bound

$$\begin{equation}{{\Delta}}_{R}\left(\delta ,K\right){\geqslant}\frac{1}{2}{\Omega}\left(2\delta ,K\right)\end{equation} \tag{ 2.3 }$$

(see [5, remark 3.12], [28, lemma 3.11] or [4, 4.3.1. proposition 1]). Let ${\bar{R}}_{\alpha }:\mathbb{Y}\to \mathbb{X}$ satisfy ${\bar{R}}_{\alpha }\left({g}^{\delta }\right)\in {R}_{\alpha }\left({g}^{\delta }\right)$ for all ${g}^{\delta }\in \mathbb{Y}$ with either α = α(δ) satisfying the a priori parameter choice given in theorem 1(a) or α = α(δ, g^δ ) satisfying the discrepancy principle in theorem 1(b). In the case ${\Omega}\left(\delta ,{K}_{\nu }^{\varrho }\right)\sim {\varrho }^{e}{\delta }^{f}$ for some exponents e, f > 0 this yields a minimax result

$$\begin{equation*}{{\Delta}}_{{\bar{R}}_{\alpha }}\left(\delta ,{K}_{\nu }^{\varrho }\right){\leqslant}C\enspace {\mathrm{inf}}_{R}{{\Delta}}_{R}\left(\delta ,{K}_{\nu }^{\varrho }\right).\end{equation*}$$

This shows that up to a constant C no method can achieve a better approximation uniformly on ${K}_{\nu }^{\varrho }$.

Moreover, we would like to highlight the flexibility in the choice of the loss function L. Many recent works in Banach space or convex regularization theory are restricted to error bounds in the Bregman divergence (see e.g. [7, 16, 19, 29]). In some situations the meaning of the Bregman divergence is unclear and lower bounds on the Bregman distance are required to obtain more tangible statements. In [28] these lower bounds cause a restriction on the parameters s, p, q of the Besov scale. By applying theorem 1 to Besov space regularization we can overcome these restrictions.

Here we consider $\mathcal{R}:\mathbb{X}\to \left[\right.0,\infty \left.\right)$ given by $\mathcal{R}\left(x\right)=\frac{1}{u}{\Vert}x{{\Vert}}_{\mathbb{X}}^{u}$ for fixed u ∈ [1, ∞). We assume ${\mathbb{X}}_{A}$ to be a Banach space with a continuous, dense embedding $\mathbb{X}\subset {\mathbb{X}}_{A}$ such that A extends to a norm isomorphism $A:{\mathbb{X}}_{A}\to \mathbb{Y}$, i.e. there exists a constant M ⩾ 1 such that

$$\begin{equation}\frac{1}{M}{\Vert}x{{\Vert}}_{{\mathbb{X}}_{A}}{\leqslant}{\Vert}Ax{{\Vert}}_{\mathbb{Y}}{\leqslant}M{\Vert}x{{\Vert}}_{{\mathbb{X}}_{A}}\quad \text{for}\enspace \text{all}\enspace x\in {\mathbb{X}}_{A}.\end{equation} \tag{ 2.4 }$$

Note that injectivity is necessary for (2.4). On the other hand injectivity of $A:\mathbb{X}\to \mathbb{Y}$ suffices for the existence of a space ${\mathbb{X}}_{A}$ such that (2.4) holds with M = 1. (Take the Banach completion of $\mathbb{X}$ in the norm $x{\mapsto}{\Vert}Ax{{\Vert}}_{\mathbb{Y}}$).

For example, in Besov space settings we will assume ${\mathbb{X}}_{A}$ a space with negative smoothness index, and we consider spaces $\mathbb{X}$ with smoothness index 0.

Moreover we need the following assumption on K₁ and ϱ₁ defined in (2.1). Recall that a quasi-norm satisfies the properties of norm except that the triangle inequality is replaced by ${\Vert}x+y{\Vert}{\leqslant}c\left({\Vert}x{\Vert}+{\Vert}y{\Vert}\right)$ for a constant c > 0. A complete and quasi-normed vector space is called a quasi-Banach space.

Assumption 2.2. Let u ∈ (0, ∞). Suppose K₁ is a vector space and that there is a quasi-norm ||⋅||_lin on K₁ such that (K₁, ||⋅||_lin) is a quasi-Banach space. Moreover assume

$$\begin{equation*}\frac{1}{M}{\varrho }_{1}\left(x\right){\leqslant}{\Vert}x{{\Vert}}_{\text{lin}}^{u-1}{\leqslant}M{\varrho }_{1}\left(x\right)\quad \text{for}\enspace \text{all}\enspace x\in {K}_{1}.\end{equation*}$$

This assumption is motivated by the computation of K₁ for the examples below.

Recall that for a quasi-Banach space ${\mathbb{X}}_{S}$ with a continuous embedding ${\mathbb{X}}_{S}\subset {\mathbb{X}}_{A}$ and θ ∈ (0, 1) the real interpolation space ${\left({\mathbb{X}}_{A},{\mathbb{X}}_{S}\right)}_{\theta ,\infty }$ consists of all $x\in {\mathbb{X}}_{A}$ such that

$$\begin{equation*}{\Vert}x{{\Vert}}_{{\left({\mathbb{X}}_{A},{\mathbb{X}}_{S}\right)}_{\theta ,\infty }}{:=}{\mathrm{sup}}_{t{ >}0}\enspace {t}^{-\theta }K\left(x,t\right){< }\infty .\end{equation*}$$

Here the K-functional is given by

$$\begin{equation*}K\left(x,t\right){:=}{\mathrm{inf}}_{z\in {\mathbb{X}}_{S}}\left({\Vert}x-z{{\Vert}}_{{\mathbb{X}}_{A}}+t{\Vert}z{{\Vert}}_{{\mathbb{X}}_{S}}\right).\end{equation*}$$

For the definition of the real interpolation spaces ${\left({\mathbb{X}}_{A},{\mathbb{X}}_{S}\right)}_{\theta ,q}$ for q ∈ (0, ∞) we refer to [2].

Theorem 2 (Error bounds). Suppose (2.4) and assumption 2.2 hold true. If $\mathbb{X}$ is not reflexive, suppose assumption 2.1 holds true.

Let ${\mathbb{X}}_{L}$ be a Banach space with a continuous embedding ${\mathbb{X}}_{L}\subset {\mathbb{X}}_{A}$. Let 0 < ξ < θ < 1 and δ, ϱ, α > 0 and c_r ⩾ c_l > 0, C_D > c_D > 1. Suppose there is a continuous embedding ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,1}\subset {\mathbb{X}}_{L}$. Assume

$$\begin{equation*}x\in {\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }\quad \text{with}\quad {\Vert}x{{\Vert}}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}{\leqslant}\varrho .\end{equation*}$$

Let ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. There exists a constant C > 0 independent of x, δ and ϱ such that whenever α satisfies either

$$\begin{align*}\hfill {c}_{l}{\varrho }^{-\frac{u-1}{\theta }}{\delta }^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{\theta }}& {\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{u-1}{\theta }}{\delta }^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{\theta }}\quad \text{or}\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \hfill \end{align*}$$

the bound

$$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{{\mathbb{X}}_{L}}{\leqslant}C{\varrho }^{\frac{\xi }{\theta }}{\delta }^{1-\frac{\xi }{\theta }}\end{equation*}$$

holds true.

We refer to subsection 4.4 for the proof of theorem 2.

Remark 2.3. The statement of the theorem remains valid in the limiting case θ = 1 where the source condition in terms of ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }$ has to be replaced by simply x ∈ K₁ with ||x||_lin ⩽ ϱ. Here the a priori rule is α ∼ ϱ^−(u−1) δ.

We illustrate the impact of this result by applying it to three more concrete Banach space regularization setups.

Example 1:  Weighted p -norm penalization. Let Λ be a countable index set, p ∈ (0, ∞) and $\bar{\omega }={\left({\bar{\omega }}_{j}\right)}_{j\in {\Lambda}}$ a sequence of positive reals. We consider weighted sequence spaces ${\ell }_{\bar{\omega }}^{p}$ defined by

$$\begin{equation*}{\ell }_{\bar{\omega }}^{p}=\left\{x\in {\mathbb{R}}^{{\Lambda}}:{\Vert}x{{\Vert}}_{\bar{\omega },p}{< }\infty \right\}\quad \text{with}\;\quad {\Vert}x{{\Vert}}_{\bar{\omega },p}^{p}={\sum }_{j\in {\Lambda}}{\bar{\omega }}_{j}^{p}\vert {x}_{j}{\vert }^{p}.\end{equation*}$$

We assume that the forward operator maps a weighted ℓ²-space isomorphically to the image space $\mathbb{Y}$. More precisely, we suppose that (2.4) holds true with ${\mathbb{X}}_{A}={\ell }_{\bar{a}}^{2}$ for $\bar{a}={\left({\bar{a}}_{j}\right)}_{j\in {\Lambda}}$ a sequence of positive real numbers.

Moreover let p ∈ (1, 2) and $\bar{r}={\left({\bar{r}}_{j}\right)}_{j\in {\Lambda}}$ a sequence of weights such that $\bar{a}{\bar{r}}^{-1}$ is bounded. We consider $\mathbb{X}={\ell }_{\bar{r}}^{p}\subset {\ell }_{\bar{a}}^{2}$ (see [20, proposition A.1]) with $\mathcal{R}\left(x\right)=\frac{1}{p}{\Vert}x{{\Vert}}_{\bar{r},p}^{p}$.

Furthermore we introduce weighted weak ℓ^p -spaces. For $\mu ={\left({\mu }_{j}\right)}_{j\in {\Lambda}}$ and $\nu ={\left({\nu }_{j}\right)}_{j\in {\Lambda}}$ sequences of positive reals and t ∈ (0, ∞) those are defined by the following quasi-norms

$$\begin{equation*}{\ell }_{\mu ,\nu }^{t,\infty }=\left\{x\in {\mathbb{R}}^{{\Lambda}}:{\Vert}x{{\Vert}}_{\mu ,\nu ,t}{< }\infty \right\}\quad \text{with}\quad {\Vert}x{{\Vert}}_{\mu ,\nu ,t}^{t}=\underset{\tau { >}0}{\mathrm{sup}}\left({\tau }^{t}{\sum }_{j\in {\Lambda}}{\nu }_{j}{\mathbb{1}}_{\left\{{\mu }_{j}\vert {x}_{j}\vert { >}\tau \right\}}\right).\end{equation*}$$

We apply theorem 2 and obtain the following result.

Corollary 2.4 (Error bounds for weighted p-norm penalties). Let p ∈ (1, 2), t ∈ (2p − 2, p) and δ, ϱ, α > 0 and c_r ⩾ c_l > 0, C_D > c_D > 1 and $\mu {:=}{\left({\bar{a}}^{2}{\bar{r}}^{-p}\right)}^{\frac{1}{2-p}},\nu {:=}{\left({\bar{a}}^{-1}\bar{r}\right)}^{\frac{2p}{2-p}}$. Assume $x\in {\ell }_{\mu ,\nu }^{t,\infty }$ with ||x||_μ,ν,t ⩽ ϱ and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. There is a constant C > 0 independent of x, δ and ϱ such that whenever α satisfies either

$$\begin{equation*}{c}_{l}{\varrho }^{-\frac{t\left(2-p\right)}{2-t}}{\delta }^{\frac{2\left(2-p\right)}{2-t}}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{t\left(2-p\right)}{2-t}}{\delta }^{\frac{2\left(2-p\right)}{2-t}}\quad \;\text{or}\;\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \end{equation*}$$

the bound

$$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{\bar{r},p}{\leqslant}C{\varrho }^{\frac{t\left(2-p\right)}{p\left(2-t\right)}}{\delta }^{\frac{2\left(p-t\right)}{p\left(2-t\right)}}.\end{equation*}$$

holds true.

The proof of corollary 2.4 can be found in subsection 4.4.

Remark 2.5. In the limiting case t = 2p − 2 the statement remains valid if one replaces ${\ell }_{\mu ,\nu }^{t,\infty }$ by ${K}_{1}={\ell }_{\bar{s}}^{2p-2}$ with $\bar{s}={\bar{a}}^{-\frac{1}{p-1}}{\bar{r}}^{\frac{p}{p-1}}$. Here we obtain the rate ${\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{\bar{r},p}{\leqslant}C{\varrho }^{\frac{p-1}{p}}{\delta }^{\frac{1}{p}}$.

In [10] the rate $\mathcal{O}\left({\delta }^{\frac{1}{p}}\right)$ is already proven under a condition similar to (1.1). Here we obtain intermediate convergences rates between $\mathcal{O}\left({\delta }^{0}\right)$ and $\mathcal{O}\left({\delta }^{\frac{1}{p}}\right)$. This has the advantage that we obtain statements on the speed of convergences on larger sets.

Remark 2.6. Corollary 2.4 remains valid word by word in the case p = 1 (see [20, theorem 4.4]).

Example 2:  Besov 0, p, p-penalties. We introduce a scale of sequence spaces that allows to characterize Besov function spaces by decay properties of coefficients in wavelet expansions (see [26]).

Let ${\left({{\Lambda}}_{j}\right)}_{j\in {\mathbb{N}}_{0}}$ be a family of sets such that 2^jd ⩽ |Λ_j | ⩽ C_Λ2^jd for some constant C_Λ ⩾ 1 and all $j\in {\mathbb{N}}_{0}$. We consider the index set ${\Lambda}{:=}\left\{\left(j,k\right):j\in {\mathbb{N}}_{0},k\in {{\Lambda}}_{j}\right\}$.

For p, q ∈ (0, ∞) and $s\in \mathbb{R}$ we set ${b}_{p,q}^{s}=\left\{x\in {\mathbb{R}}^{{\Lambda}}:{\Vert}x{{\Vert}}_{s,p,q}{< }\infty \right\}$ with

$$\begin{equation*}{\Vert}x{{\Vert}}_{s,p,q}^{q}{:=}{\sum }_{j\in {\mathbb{N}}_{0}}{2}^{jq\left(s+\frac{d}{2}-\frac{d}{p}\right)}{\left({\sum }_{k\in {{\Lambda}}_{j}}\vert {x}_{j,k}{\vert }^{p}\right)}^{q/p}.\end{equation*}$$

with the usual replacements for p = ∞ or q = ∞.

Let a > 0 and assume that the forward operator $A:{b}_{2,2}^{-a}\to \mathbb{Y}$ satisfies (2.4) with ${\mathbb{X}}_{A}={b}_{2,2}^{-a}$. Let p ∈ (1, ∞) (for p = 1 we refer to [20] again) with $\frac{d}{p}-\frac{d}{2}{\leqslant}a$. Then we have a continuous embedding ${b}_{p,p}^{0}\subset {b}_{2,2}^{-a}$ (see [27, 3.3.1.(6),(7), 3.2.4.(1)]).

We use $\mathbb{X}={b}_{p,p}^{0}$ with

$$\begin{equation*}\mathcal{R}\left(x\right)=\frac{1}{p}{\Vert}x{{\Vert}}_{0,p,p}^{p}=\frac{1}{p}{\sum }_{\left(j,k\right)\in {\Lambda}}{2}^{jp\left(\frac{d}{2}-\frac{d}{p}\right)}\vert {x}_{j,k}{\vert }^{p}\quad \text{for}\enspace x\in {b}_{p,p}^{0}.\end{equation*}$$

Note that we have

$$\begin{equation}{b}_{p,p}^{s}={\ell }_{{\bar{\omega }}_{s,p}}^{p}\quad \text{with}\;\text{equal}\;\text{norm}\;\text{for}\quad {\left({\bar{\omega }}_{s,p}\right)}_{\left(j,k\right)}={2}^{j\left(s+\frac{d}{2}-\frac{d}{p}\right)}.\end{equation} \tag{ 2.5 }$$

Hence for p < 2, this example is a special case of example 1.

Let $\tilde {s}=\frac{a}{p-1}$ and $\tilde {t}=2p-2$. For $0{< }s{< }\tilde {s}$ we set

$$\begin{equation}{k}_{s}{:=}{\left({b}_{2,2}^{-a},{b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\right)}_{\theta ,\infty }\quad \text{with}\quad \theta =\frac{p-1}{p}\frac{s+a}{a}.\end{equation} \tag{ 2.6 }$$

Here the application of theorem 2 yields the following error bound.

Corollary 2.7 (Error bounds for 0, p, p-penalties). Let $0{< }s{< }\tilde {s}$ and δ, ϱ, α > 0, c_r ⩾ c_l > 0, C_D > c_D > 1. Assume x ∈ k_s with ${\Vert}x{{\Vert}}_{{k}_{s}}{\leqslant}\varrho$ and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. There is a constant C > 0 independent of x, δ and ϱ such that whenever α satisfies either

$$\begin{equation*}{c}_{l}{\varrho }^{-\frac{pa}{s+a}}{\delta }^{\frac{\left(2-p\right)s+2a}{s+a}}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{pa}{s+a}}{\delta }^{\frac{\left(2-p\right)s+2a}{s+a}}\quad \;\text{or}\;\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \end{equation*}$$

the bound

$$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,p,p}{\leqslant}C{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}.\end{equation*}$$

holds true.

The proof of corollary 2.7 can be found in subsection 4.4.

Remark 2.8. In the limiting case $s=\tilde {s}$ the result remains valid if one replaces k_s by ${K}_{1}={b}_{\tilde {t},\tilde {t}}^{\tilde {s}}$ and we obtain the bound ${\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,p,p}{\leqslant}C{\varrho }^{\frac{p-1}{p}}{\delta }^{\frac{1}{p}}$.

For p = 2 we have ${k}_{s}={b}_{2,\infty }^{s}$ (see [27, 3.3.6.(9)]). The following proposition provides a nesting of k_s for p ≠ 2 by Besov sequence spaces.

Proposition 2.9. Let $0{< }s{< }\tilde {s}$ and $t=\frac{2pa}{\left(2-p\right)s+2a}$.

• (a)  
  For p < 2 we have continuous embeddings

  $$\begin{equation*}{b}_{t,t}^{s}\subset {k}_{s}\subset {b}_{t-\varepsilon ,\infty }^{s}\quad \text{for}\enspace \text{all}\enspace 0{< }\varepsilon {< }t.\end{equation*}$$
• (b)  
  For p > 2 we have continuous embeddings ${b}_{t,t}^{s}\subset {k}_{s}\subset {b}_{2,\infty }^{s}$.

We refer to subsection 5.4 for a proof of proposition 2.9. For p < 2 the same argument as in [20, example 6.7] shows that describing the regularity of functions with jumps or kinks via their wavelet expansion in terms of k_s allows for a higher value of s then using ${B}_{\mathrm{s},\infty }^{\mathrm{p}}\left({\Omega}\right)$ as in [28]. Therefore we obtain a faster convergence rate for this class of functions.

For p > 2 we measure the error in a stronger norm than the ℓ²-norm. On the other hand the set on which we obtain convergence rates is smaller than ${b}_{2,\infty }^{s}$.

Example 3:  Besov 0, 2, q-penalties. Again we consider a > 0 and ${\mathbb{X}}_{A}={b}_{2,2}^{-a}$ with A satisfying (2.4). Let q ∈ (1, ∞). Then there is a continuous embedding $\mathbb{X}{:=}{b}_{2,q}^{0}\subset {b}_{2,2}^{-a}$ (see [27, 3.3.1.(7)]) and we choose

$$\begin{equation*}\mathcal{R}\left(x\right)=\frac{1}{q}{\Vert}x{{\Vert}}_{0,2,q}^{q}=\frac{1}{q}{\sum }_{j\in {\mathbb{N}}_{0}}{\left({\sum }_{k\in {{\Lambda}}_{j}}\vert {x}_{j,k}{\vert }^{2}\right)}^{q/2}.\end{equation*}$$

For a convergence analysis in the case q = 1 we refer to [17]. The application of theorem 2 provides:

Corollary 2.10 (Error bounds for 0, 2, q-penalties). Let $0{< }s{< }\frac{a}{q-1}$ and δ, ϱ, α > 0, c_r ⩾ c_l > 0, C_D > c_D > 1. Assume $x\in {b}_{2,\infty }^{s}$ with ||x||_s,2,∞ ⩽ ϱ and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. There is a constant C > 0 independent of x, δ and ϱ such that whenever α satisfies either

$$\begin{equation*}{c}_{l}{\varrho }^{-\frac{qa}{s+a}}{\delta }^{\frac{\left(2-q\right)s+2a}{s+a}}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{qa}{s+a}}{\delta }^{\frac{\left(2-q\right)s+2a}{s+a}}\quad \;\text{or}\;\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \end{equation*}$$

the bound

$$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,2,2}{\leqslant}C{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}.\end{equation*}$$

holds true.

The proof of corollary 2.10 can be found in subsection 4.4.

Remark 2.11. In the limiting case $s=\frac{a}{q-1}$ the result remains valid if one replaces ${b}_{2,\infty }^{s}$ by ${K}_{1}={b}_{2,\tilde {q}}^{\tilde {s}}$ with $\tilde {q}=2q-2$. Here we obtain ${\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,2,2}{\leqslant}C{\varrho }^{\frac{q-1}{q}}{\delta }^{\frac{1}{q}}$.

In contrast to the analysis in [17] we measure the error in the ℓ²-norm independent of the value of q, i.e. the error norm is not dictated by the penalty term.

The smaller q the larger is the region $0{< }s{< }\frac{a}{q-1}$ of regularity parameters for which we guarantee upper bounds. Furthermore we see that changing the fine index q while keeping p = 2 does not change the set where convergence rates are guaranteed, but it influences the parameter choice rule.

Example 4:  Radon transform. To give a more concrete example we discuss the Radon transform which appears as forward operator in computed tomography and positron emission tomography. This example also shows how our results apply to operators initially defined on function spaces.

Let $d\in \mathbb{N}$ with d ⩾ 2, ${\Omega}{:=}\left\{x\in {\mathbb{R}}^{d}:\vert x\vert {\leqslant}1\right\}$, ${S}^{d-1}{:=}\left\{x\in {\mathbb{R}}^{d}:\vert x\vert =1\right\}$ and $\mathbb{Y}{:=}{L}^{2}\left({S}^{d-1}{\times}\left[-1,1\right]\right)$. Then the Radon transform $R:{L}^{2}\left({\Omega}\right)\to {L}^{2}\left({S}^{d-1}{\times}\mathbb{R}\right)$ is given by

$$\begin{equation*}\left(Rf\right)\left(\theta ,t\right){:=}{\int }_{x\cdot \theta =t}f\left(x\right)\enspace \mathrm{d}x,\quad \theta \in {S}^{d-1},\enspace t\in \mathbb{R}.\end{equation*}$$

With $a=\frac{d-1}{2}$ it follows from [15, theorem 3.1] that R is a norm isomorphism from ${B}_{2,2}^{-a}\left({\Omega}\right)$ to $\mathbb{Y}$. Here ${B}_{2,2}^{-a}\left({\Omega}\right)$ denotes a Besov function space. We refer to the book [9] for an introduction to this scale of function spaces.

Furthermore, with Λ and the scale of spaces ${b}_{p,q}^{s}$ as introduced in example 2 and s_max > a we consider a s_max-regular wavelet system ${\left({\psi }_{\lambda }\right)}_{\lambda \in {\Lambda}}$ on Ω such that the synthesis operator

$$\begin{equation*}\mathcal{S}:{b}_{p,q}^{s}\to {B}_{p,q}^{s}\left({\Omega}\right)\quad \;\text{given}\;\text{by}\;\quad x{\mapsto}\sum\limits _{\lambda \in {\Lambda}}{x}_{\lambda }{\psi }_{\lambda }\end{equation*}$$

is well defined and a norm isomorphism for all $s\in \mathbb{R}$ and p, q ∈ (0, ∞] satisfying s ∈ (σ_p − s_max, s_max) with ${\sigma }_{p}=\mathrm{max}\left\{d\left(\frac{1}{p}-1\right),0\right\}$ (see [26]). Now for $\mathcal{R}$ as in example 2 or example 3 we consider

$$\begin{equation}{S}_{\alpha }\left(g\right)=\mathcal{S}{\hat{x}}_{\alpha }\quad \text{with}\quad {\hat{x}}_{\alpha }\in \underset{x\in \mathrm{dom}\left(\mathcal{R}\right)}{\text{argmin}}\left(\frac{1}{2\alpha }{\Vert}{g}^{\mathrm{o}\mathrm{b}\mathrm{s}}-R\mathcal{S}x{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left(x\right)\right)\end{equation} \tag{ 2.7 }$$

and obtain the following convergence rate results.

Corollary2.12 (Convergence rates for wavelet regularization of the Radon transform).

• (a)  
  Let p ∈ (1, ∞). With the notation of example 2 suppose $0{< }s{< }\mathrm{min}\left\{\bar{s},{s}_{\text{max}}\right\}$ and δ, ϱ, α > 0, c_r ⩾ c_l > 0, C_D > c_D > 1. Assume $f\in {B}_{t,t}^{s}\left({\Omega}\right)$ with ${\Vert}f{{\Vert}}_{{B}_{t,t}^{s}\left({\Omega}\right)}{\leqslant}\varrho$ and ${\hat{f}}_{\alpha }\in {S}_{\alpha }\left({g}^{\delta }\right)$ with $\mathcal{R}$ as in example 2. Then there is a constant C > 0 independent of f, δ and ϱ such that whenever α satisfies either

  $$\begin{equation*}{c}_{l}{\varrho }^{-\frac{pa}{s+a}}{\delta }^{\frac{\left(2-p\right)s+2a}{s+a}}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{pa}{s+a}}{\delta }^{\frac{\left(2-p\right)s+2a}{s+a}}\quad \text{or}\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-R{\hat{f}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \end{equation*}$$

  the bound

  $$\begin{equation*}{\Vert}f-{\hat{f}}_{\alpha }{{\Vert}}_{{B}_{p,p}^{0}\left({\Omega}\right)}{\leqslant}C{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}.\end{equation*}$$

  holds true. If p ⩽ 2 we also obtain the bound

  $$\begin{equation*}{\Vert}f-{\hat{f}}_{\alpha }{{\Vert}}_{{L}^{p}\left({\Omega}\right)}{\leqslant}C{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}.\end{equation*}$$
• (b)  
  Let q ∈ (0, ∞). Suppose $0{< }s{< }\mathrm{min}\left\{\frac{a}{q-1},{s}_{\mathrm{max}}\right\}$ and δ, ϱ, α > 0, c_r ⩾ c_l > 0, C_D > c_D > 1. Assume $f\in {B}_{2,\infty }^{s}\left({\Omega}\right)$ with ${\Vert}f{{\Vert}}_{{B}_{2,\infty }^{s}\left({\Omega}\right)}{\leqslant}\varrho$ and ${\hat{f}}_{\alpha }\in {S}_{\alpha }\left({g}^{\delta }\right)$ with $\mathcal{R}$ as in example 3. Then there is a constant C > 0 independent of f, δ and ϱ such that whenever α satisfies either

  $$\begin{equation*}{c}_{l}{\varrho }^{-\frac{qa}{s+a}}{\delta }^{\frac{\left(2-q\right)s+2a}{s+a}}{\leqslant}\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{qa}{s+a}}{\delta }^{\frac{\left(2-q\right)s+2a}{s+a}}\quad \text{or}\quad {c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-R{\hat{f}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{C}_{D}\delta \end{equation*}$$

  the bound

  $$\begin{equation*}{\Vert}f-{\hat{f}}_{\alpha }{{\Vert}}_{{L}^{2}\left({\Omega}\right)}{\leqslant}C{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}.\end{equation*}$$

  holds true.

The proof of corollary 2.12 can be found in subsection 4.4. In corollary 2.12(a) it would also be sufficient to require $f\in \mathcal{S}{k}_{s}$ instead of $f\in {B}_{t,t}^{s}\left({\Omega}\right)$. Transferring the interpolation identity in (2.6) to function spaces shows that $\mathcal{S}{k}_{s}$ is independent of the chosen wavelet system (see [20, section 6.2] for a similar discussion).

In the same manner the presented theory can be applied to other linear, finitely smoothing forward operators as inverses of elliptic differential operators with smooth, periodic coefficients or specific periodic convolution operators (see [17, example 2.5] for more details).

Assuming only assumption 2.1 we compare (1.2) to source conditions used in the literature. For a concave and upper semi-continuous function $\phi :\left[\right.0,\infty \left.\right)\to \left[\right.0,\infty \left.\right)$ we consider variational source conditions of the form

$$\begin{equation}\mathcal{R}\left(x\right)-\mathcal{R}\left(z\right){\leqslant}\phi \left({\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}\right)\quad \text{for}\enspace \text{all}\enspace z\in \mathbb{X}.\end{equation} \tag{ 2.8 }$$

In [19] this condition is used to prove convergence rates with respect to the twisted Bregman distance of $\mathcal{R}$ and it is shown that the source condition (1.1) implies (2.8) with $\phi \sim \sqrt{\cdot }$. In [7] necessity of (2.8) for convergence rates with respect to the twisted Bregman distance under a fixed parameter choice rule is proven.

Inspired by [16] we also study the defect of the Tikhonov functional

$$\begin{equation*}{\sigma }_{x}\left(\alpha \right){:=}{T}_{\alpha }\left(x,Ax\right)-{T}_{\alpha }\left({x}_{\alpha },Ax\right).\end{equation*}$$

The following result shows that Hölder-type variational source conditions, Hölder-type bounds on the defect of the Tikhonov functional and Hölder type image space approximation rates are equivalent.

Theorem 3. Let $\nu \in \left(\frac{1}{2},1\right]$. Assume $x\in \mathrm{dom}\left(\mathcal{R}\right)$ is $\mathcal{R}$-minimal in A⁻¹({Ax}) and x_α ∈ R_α (Ax) for α > 0 is any selection of a minimizers for exact data. The following statements are equivalent:

• (a)  
  There exists a constant c₁ > 0 with ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{c}_{1}{\alpha }^{\nu }$ for all α > 0.
• (b)  
  There exists a constant c₂ > 0 such that σ_x (α) ⩽ c₂ α^2ν−1.
• (c)  
  There exists a constant c₃ > 0 with (2.8) holds true for $\phi \left(t\right)={c}_{3}{t}^{\frac{2\nu -1}{2\nu }}$.

More precisely (a) implies (b) with ${c}_{2}=\frac{{c}_{1}^{2}}{4\nu -2}$, (b) implies (c) with ${c}_{3}=2{c}_{2}^{\frac{1}{2\nu }}$ and (c) implies (a) with ${c}_{1}={c}_{3}^{\nu }$.

We provide a proof of theorem 3 in subsection 4.4. The result allows the following representation of K_ν in terms of variational source conditions:

$$\begin{align}\hfill {K}_{\nu }& =\left\{x\in \mathbb{X}:\text{There}\;\text{exists}\;c{ >}0\;\text{such}\;\text{that}\;\left(2.8\right)\;\text{with}\;\right.\hfill \\ \hfill & \quad \left.\phi \left(t\right)=c{t}^{\frac{2\nu -1}{2\nu }}\text{holds}\;\text{true.}\right\}\hfill \end{align} \tag{ 2.9 }$$

for all $\nu \in \nu \in \left(\frac{1}{2},1\right]$. Note that since the map $\left(\frac{1}{2},1\right]\to \left(0,\frac{1}{2}\right]$ given by $\nu {\mapsto}\frac{2\nu -1}{2\nu }$ is bijective this characterization grasps all Hölder type functions $\phi \left(t\right)=\mathcal{O}\left({t}^{\mu }\right)$ for $\mu \in \left(0,\frac{1}{2}\right]$. Due to [16, proposition 3] the largest meaningful exponent is $\mu =\frac{1}{2}$. Furthermore, (2.8) implies $x\in \mathrm{dom}\left(\mathcal{R}\right)$ and this in turn yields (a) with $\nu =\frac{1}{2}$. Therefore, we cannot expect a characterization of (a) for $\nu {< }\frac{1}{2}$ by variational source conditions. Hence all meaningful Hölder type variational source conditions of the form (2.8) are covered in theorem 3 and (2.9). In other words it is not possible to extend theorem 3 to a larger set of exponents.

Together with theorem 1 we see that Hölder-type variational source conditions imply upper bounds on the reconstruction error for any loss function given by the modulus of continuity. In contrast as far as the author knows all upper bounds in the literature derived from (2.8) are restricted to the twisted Bregman distance.

This subsection provides an important preliminary that we use in several places throughout the paper. We introduce a convex function $\mathcal{Q}$ on $\mathbb{Y}$ that can be seen as a push forward of $\mathcal{R}$ through the linear operator A. We show that the proximity mapping of $\alpha \mathcal{Q}$ equals A ◦ R_α . Recall that for a convex, proper and lower semi-continuous function $\mathcal{Q}:\mathbb{Y}\to \left(-\infty ,\infty \right]$ and $g\in \mathbb{Y}$ there is a unique minimizer ${\text{Prox}}_{\mathcal{Q}}\left(g\right)$ of the function $y{\mapsto}\frac{1}{2}{\Vert}g-y{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left(y\right)$. The single-valued mapping

$$\begin{equation*}{\text{Prox}}_{\mathcal{Q}}:\mathbb{Y}\to \mathbb{Y}\quad \text{given}\;\text{by}\quad g{\mapsto}{\text{Prox}}_{\mathcal{Q}}\left(g\right){:=}\underset{y\in \mathbb{Y}}{\text{argmin}}\left(\frac{1}{2}{\Vert}g-y{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left(y\right)\right)\end{equation*}$$

is called proximity mapping of Q (see [1, 11.4, definition 12.23]).

Lemma 3.1. We define

$$\begin{equation*}\mathcal{Q}:\mathbb{Y}\to \left(-\infty ,\infty \right]\quad \text{by}\quad \mathcal{Q}\left(g\right){:=}\mathrm{inf}\left\{\mathcal{R}\left(x\right)\enspace :\enspace x\in \mathbb{X}\enspace \text{with}\enspace Ax=g\right\}\end{equation*}$$

with inf ∅ = ∞. Then $\mathcal{Q}$ is convex, proper and lower semi-continuous, and we have $\mathrm{dom}\left(\mathcal{Q}\right)=A\left(\mathrm{dom}\left(\mathcal{R}\right)\right)$.

Proof. Let $\lambda \in \mathbb{R}$. First we prove that ${L}_{\lambda }{:=}\left\{g\in \mathbb{Y}\enspace :\enspace \mathcal{Q}\left(g\right){\leqslant}\lambda \right\}$ satisfies

$$\begin{equation*}{L}_{\lambda }=A\left(\left\{x\in \mathbb{X}\enspace :\enspace \mathcal{R}\left(x\right){\leqslant}\lambda \right\}\right).\end{equation*}$$

To this end let g ∈ L_λ . There exists $x\in \mathbb{X}$ with Ax = g and $\mathcal{R}\left(x\right){\leqslant}\mathcal{R}\left(z\right)$ for all $z\in \mathbb{X}$ with Az = g. Then $\mathcal{R}\left(x\right)=\mathcal{Q}\left(g\right){\leqslant}\lambda$. On the other hand if $x\in \mathbb{X}$ with $\mathcal{R}\left(x\right){\leqslant}\lambda$ then $\mathcal{Q}\left(Ax\right){\leqslant}\mathcal{R}\left(x\right){\leqslant}\lambda$.

Taking union over $\lambda \in \mathbb{R}$ yields $\mathrm{dom}\left(\mathcal{Q}\right)=A\left(\mathrm{dom}\left(\mathcal{R}\right)\right)$. Hence $\mathcal{Q}$ is proper as $\mathcal{R}$ is proper. The sublevel sets L_λ are convex as the image of a convex set under a linear map and closed as the image of a τ-compact set under a τ-to-weak continuous map. Hence $\mathcal{Q}$ is convex and lower semi-continuous.□

Remark 3.2. Note that in the case of an injective forward operator A, the map $\mathcal{Q}$ is given by $\mathcal{Q}\left(g\right)=\mathcal{R}\left({A}^{-1}g\right)$ if g ∈ im(A) and $\mathcal{Q}\left(g\right)=\infty$ if $g\in \mathbb{Y}{\backslash}\text{im}\left(A\right)$ where ${A}^{-1}:\text{im}\left(A\right)\to \mathbb{X}$ denotes the inverse map of A.

Proposition 3.3. Let $g\in \mathbb{Y}$ and α > 0. Then

$$\begin{equation*}A{\hat{x}}_{\alpha }={\mathrm{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\quad \text{and}\quad \mathcal{R}\left({\hat{x}}_{\alpha }\right)=\mathcal{Q}\left({\mathrm{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\right)\quad \text{for}\enspace \text{all}\enspace {\hat{x}}_{\alpha }\in {R}_{\alpha }\left(g\right).\end{equation*}$$

In particular $A{\circ}{R}_{\alpha }={\mathrm{Prox}}_{\alpha \mathcal{Q}}$ is single-valued. Hence $A{\hat{x}}_{\alpha }$ and $\mathcal{R}\left({\hat{x}}_{\alpha }\right)$ do not depend on the particular choice of ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left(g\right)$.

Proof. Let $v\in \mathrm{dom}\left(\mathcal{Q}\right)$. By lemma 3.1 we have v ∈ im(A). There exists $z\in \mathbb{X}$ with Az = v and $\mathcal{R}\left(z\right){\leqslant}\mathcal{R}\left(y\right)$ for all $y\in \mathbb{X}$ with Ay = v. By definition of $\mathcal{Q}$ that is $\mathcal{R}\left(z\right)=\mathcal{Q}\left(v\right)$. The first identity follows from

$$\begin{align*}\hfill \frac{1}{2\alpha }{\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left(A{\hat{x}}_{\alpha }\right)& {\leqslant}\frac{1}{2\alpha }{\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left({\hat{x}}_{\alpha }\right)\hfill \\ \hfill & {\leqslant}\frac{1}{2\alpha }{\Vert}g-Az{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left(z\right)\hfill \\ \hfill & =\frac{1}{2\alpha }{\Vert}g-v{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left(v\right).\hfill \end{align*}$$

Inserting $v=A{\hat{x}}_{\alpha }$ yields $\mathcal{R}\left({\hat{x}}_{\alpha }\right)=\mathcal{Q}\left(A{\hat{x}}_{\alpha }\right)=\mathcal{Q}\left({\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\right)$.□

The statement in proposition 3.3 can be read as follows: the function $\mathcal{Q}$ on $\mathbb{Y}$ stores all relevant information on $\mathcal{R}$ and A to recover the mapping A ◦ R_α in one object. Note that the definition of K_ν can be rephrased only in terms of $\mathcal{Q}$.

Remark 3.4. Suppose $x\in \mathrm{dom}\left(\mathcal{R}\right)$, α > 0 and x_α ∈ R_α (Ax). In [16] the authors study upper bounds on $\mathcal{R}\left(x\right)-\mathcal{R}\left({x}_{\alpha }\right)$ (defect for penalty) and on σ_x (α) (defect for Tikhonov functional) in terms of α. The first quantity bounds the second and it is bounded by the double of the second (see [16, proposition 2.4]). In [16, remark 2.5] the authors rely on this nesting to argue that changing the selection of minimizers changes the defect for penalty at most by a factor of 2. Proposition 3.3 actually shows that the defect for penalty is independent of the choice of x_α ∈ R_α (Ax).

Exploiting firm non-expansiveness (see [1, definition 4.1]) of proximal operators we draw a further conclusion of proposition 3.3.

Corollary 3.5 (Firm non-expansiveness). Let $g,h\in \mathbb{Y}$, α > 0, ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left(g\right)$ and ${\hat{z}}_{\alpha }\in {R}_{\alpha }\left(h\right)$. Then

$$\begin{equation*}{\Vert}\left(g-A{\hat{x}}_{\alpha }\right)-\left(h-A{\hat{z}}_{\alpha }\right){{\Vert}}_{\mathbb{Y}}^{2}+{\Vert}A{\hat{x}}_{\alpha }-A{\hat{z}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}{\leqslant}{\Vert}g-h{{\Vert}}_{\mathbb{Y}}^{2}.\end{equation*}$$

Proof. By [1, proposition 12.27] the proximity operator ${\text{Prox}}_{\alpha \mathcal{Q}}$ satisfies

$$\begin{align*}\hfill & {\Vert}\left(g-{\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\right)-\left(h-{\text{Prox}}_{\alpha \mathcal{Q}}\left(h\right)\right){{\Vert}}_{\mathbb{Y}}^{2}+{\Vert}{\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right)-{\text{Prox}}_{\alpha \mathcal{Q}}\left(h\right){{\Vert}}_{\mathbb{Y}}^{2}\hfill \\ \hfill & \qquad {\leqslant}{\Vert}g-h{{\Vert}}_{\mathbb{Y}}^{2}\hfill \end{align*}$$

for all $g,h\in \mathbb{Y}$. Inserting the first identity in proposition 3.3 yields the claim.□

The following proposition captures properties of the sets K_ν . In particular, we show that K_ν is nontrivial for $\nu \in \left(\right.0,1 \left.\right]$.

Lemma 3.6. We have

• (a)  
  ${K}_{0}=\mathbb{X}$.
• (b)  
  ${K}_{{\nu }_{2}}\subset {K}_{{\nu }_{1}}$ for 0 ⩽ ν₁ ⩽ ν₂.
• (c)  
  ${K}_{\nu }=\underset{z\in \mathbb{X}}{\mathrm{argmin}}\enspace \mathcal{R}\left(z\right)+\mathrm{ker}\left(A\right)$ for all ν > 1.
• (d)  
  $\mathrm{dom}\left(\mathcal{R}\right)+\mathrm{ker}\left(A\right)\subseteq {K}_{1/2}$.

Proof. 

• (a)  
  Let $x\in \mathbb{X}$. We set ${D}_{x}{:=}\mathrm{inf}\left\{{\Vert}Ax-Ay{{\Vert}}_{\mathbb{Y}}:y\in \underset{z\in \mathbb{X}}{\text{argmin}}\enspace \mathcal{R}\left(z\right)\right\}$. Let $y\in {\text{argmin}}_{z\in \mathbb{X}}\enspace \mathcal{R}\left(z\right)$, α > 0 and x_α ∈ R_α (Ax). Then

  $$\begin{equation*}\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left({x}_{\alpha }\right){\leqslant}\frac{1}{2\alpha }{\Vert}Ax-Ay{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left(y\right).\end{equation*}$$

  As $\mathcal{R}\left(y\right){\leqslant}\mathcal{R}\left({x}_{\alpha }\right)$ this implies ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{\Vert}Ax-Ay{{\Vert}}_{\mathbb{Y}}$. Hence ϱ₀(x) ⩽ D_x < ∞.
• (b)  
  Suppose $x\in {K}_{{\nu }_{2}}$. Then

  $$\begin{equation*}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}={\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{\frac{{\nu }_{1}}{{\nu }_{2}}}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{1-\frac{{\nu }_{1}}{{\nu }_{2}}}{\leqslant}{\varrho }_{{\nu }_{2}}{\left(x\right)}^{\frac{{\nu }_{1}}{{\nu }_{2}}}{\varrho }_{0}{\left(x\right)}^{1-\frac{{\nu }_{1}}{{\nu }_{2}}}{\alpha }^{{\nu }_{1}}.\end{equation*}$$

  implies ${\varrho }_{{\nu }_{1}}\left(x\right){\leqslant}{\varrho }_{{\nu }_{2}}{\left(x\right)}^{\frac{{\nu }_{1}}{{\nu }_{2}}}{\varrho }_{0}{\left(x\right)}^{1-\frac{{\nu }_{1}}{{\nu }_{2}}}$.
• (c)  
  Let ν > 1. Suppose x ∈ K_ν . From [1, proposition 16.34] and proposition 3.3 we obtain

  $$\begin{equation*}{\eta }_{\alpha }{:=}\frac{1}{\alpha }\left(Ax-A{x}_{\alpha }\right)=\frac{1}{\alpha }\left(Ax-{\text{Prox}}_{\alpha \mathcal{Q}}\left(Ax\right)\right)\in \partial \mathcal{Q}\left(A{x}_{\alpha }\right).\end{equation*}$$

  Since η_α → 0 and Ax_α → Ax for α → 0 in the norm topology of $\mathbb{Y}$ this implies $0\in \partial \mathcal{Q}\left(Ax\right)$. Hence $Ax\in {\text{argmin}}_{g\in \mathbb{Y}}\enspace \mathcal{Q}\left(g\right)$. Let $y\in \mathbb{X}$ be $\mathcal{R}$-minimal with Ay = Ax. Then

  $$\begin{equation*}\mathcal{R}\left(y\right)=\mathcal{Q}\left(Ax\right){\leqslant}\mathcal{Q}\left(Az\right){\leqslant}\mathcal{R}\left(z\right)\quad \text{for}\enspace \text{all}\enspace z\in \mathbb{X}.\end{equation*}$$

  Hence

  $$\begin{equation*}x=y+x-y\in \underset{z\in \mathbb{X}}{\text{argmin}}\enspace \mathcal{R}\left(z\right)+\mathrm{ker}\left(A\right).\end{equation*}$$

  On the other hand assume $x=y+k\in {\text{argmin}}_{z\in \mathbb{X}}\enspace \mathcal{R}\left(z\right)+\mathrm{ker}\left(A\right)$. Then

  $$\begin{equation*}\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left({x}_{\alpha }\right){\leqslant}{T}_{\alpha }\left(y,Ax\right)=\mathcal{R}\left(y\right){\leqslant}\mathcal{R}\left({x}_{\alpha }\right)\end{equation*}$$

  yields ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}=0$. Hence x ∈ K_ν .
• (d)  
  Let $x=y+k\in \mathrm{dom}\left(\mathcal{R}\right)+\mathrm{ker}\left(A\right)$. From

  $$\begin{equation*}\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}{\leqslant}{T}_{\alpha }\left({x}_{\alpha },Ax\right){\leqslant}{T}_{\alpha }\left(y,Ax\right)=\mathcal{R}\left(y\right)\end{equation*}$$

  we obtain ${\varrho }_{1/2}\left(x\right){\leqslant}\sqrt{2\mathcal{R}\left(y\right)}$.□

The set K_ν does not change for ν > 1. As announced in subsection 2.2 we will see that K₁ is the set of elements satisfying source condition (1.1).

Moreover note that the last inequality in the proof of lemma 3.6(b) resembles an interpolation inequality. This gives a first hint to a connection to interpolation theory in the case of Banach space regularization.

This subsection is devoted to error bounds in the image space $\mathbb{Y}$ in terms of the deterministic noise level and the image space approximation error for exact data. Let δ ⩾ 0, $x\in \mathbb{X}$ and ${g}^{\delta }\in \mathbb{Y}$ with ${\Vert}{g}^{\delta }-Ax{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta$.

Lemma 3.7. The following inequalities

$$\begin{equation}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}},\end{equation} \tag{ 3.1 }$$

$$\begin{equation}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\end{equation} \tag{ 3.2 }$$

hold true for all α > 0, ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$, x_α ∈ R_α (Ax).

Proof. Corollary 3.5 with g = g^δ and h = Ax yields

$$\begin{equation*}{\Vert}\left(Ax-A{x}_{\alpha }\right)-\left({g}^{\delta }-A{\hat{x}}_{\alpha }\right){{\Vert}}_{\mathbb{Y}}^{2}+{\Vert}A{x}_{\alpha }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}{\leqslant}{\delta }^{2}.\end{equation*}$$

We neglect the first summand on the left-hand side and obtain

$$\begin{equation*}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}+{\Vert}A{x}_{\alpha }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\end{equation*}$$

and the second for

$$\begin{align*}\hfill {\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}& {\leqslant}{\Vert}\left(Ax-A{x}_{\alpha }\right)-\left({g}^{\delta }-A{\hat{x}}_{\alpha }\right){{\Vert}}_{\mathbb{Y}}+{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \\ \hfill & {\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}.\hfill \end{align*}$$

□

Proposition 3.8. Let $\nu \in \left(\right.0,1 \left.\right]$ and α, ϱ > 0. Suppose $x\in {K}_{\nu }^{\varrho }$.

• (a)  
  Let c_r > 0. If $\alpha {\leqslant}{c}_{r}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}$ then

  $$\begin{equation*}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\left(1+{c}_{r}^{\nu }\right)\delta \quad \text{for}\enspace \text{all}\enspace {\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right).\end{equation*}$$
• (b)  
  Let c_D > 1. If ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$ satisfies ${c}_{D}\delta {\leqslant}{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{\Vert}$, then

  $$\begin{equation*}{\left({c}_{D}-1\right)}^{\frac{1}{\nu }}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}{\leqslant}\alpha .\end{equation*}$$

Proof. Let x_α ∈ R_α (Ax).

• (a)  
  By (3.1) and the definition of ϱ_ν we obtain

  $$\begin{equation*}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +\enspace \varrho {\alpha }^{\nu }{\leqslant}\left(1+{c}_{r}^{\nu }\right)\delta .\end{equation*}$$
• (b)  
  The bound (3.2) implies

  $$\begin{equation*}{c}_{D}\delta {\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +\varrho {\alpha }^{\nu }.\end{equation*}$$

  Subtracting δ and rearranging yields the claim.□

First we recall the well-known fact that the source condition (1.1) implies a linear convergence rate in the image space (see e.g. [12, lemma 3.5]).

Lemma 3.9. Let $z\in \mathbb{X}$ and assume $\omega \in \mathbb{Y}$ with ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left(z\right)$. Then

$$\begin{equation*}{\Vert}Az-A{z}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{\Vert}\omega {{\Vert}}_{\mathbb{Y}}\alpha \quad \text{for}\enspace \text{all}\enspace \alpha { >}0\enspace \text{and}\enspace {z}_{\alpha }\in {R}_{\alpha }\left(Az\right).\end{equation*}$$

Proof. The first order optimality condition yields ${\xi }_{\alpha }{:=}\frac{1}{\alpha }{A}^{{\ast}}A\left(z-{z}_{\alpha }\right)\in \partial \mathcal{R}\left({z}_{\alpha }\right)$. Solving the inequality

$$\begin{align*}\hfill \frac{1}{\alpha }{\Vert}Az-A{z}_{\alpha }{{\Vert}}^{2}& =\langle {\xi }_{\alpha },z-{z}_{\alpha }\rangle {\leqslant}\mathcal{R}\left(z\right)-\mathcal{R}\left({z}_{\alpha }\right){\leqslant}\langle {A}^{{\ast}}\omega ,z-{z}_{\alpha }\rangle \hfill \\ \hfill & {\leqslant}{\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\Vert}Az-A{z}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \end{align*}$$

for ${\Vert}Az-A{z}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ proves the claim.□

Lemma 3.10. Let α > 0, ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. Furthermore let β > 0, ${\left({\hat{x}}_{\alpha }\right)}_{\beta }\in {R}_{\beta }\left(A{\hat{x}}_{\alpha }\right)$ and x_β ∈ R_β (Ax).

• (a)  
  If β ∈ (0, α] then

  $$\begin{equation*}{\Vert}A{\hat{x}}_{\alpha }-A{\left({\hat{x}}_{\alpha }\right)}_{\beta }{{\Vert}}_{\mathbb{Y}}{\leqslant}\frac{\beta \delta }{\alpha }+{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$
• (b)  
  If β ∈ [α, ∞) then

  $$\begin{equation*}{\Vert}A{\hat{x}}_{\alpha }-A{\left({\hat{x}}_{\alpha }\right)}_{\beta }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +2{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

Proof. 

• (a)  
  By the first order optimality condition the element ${\hat{x}}_{\alpha }$ satisfies the prerequisite ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left({\hat{x}}_{\alpha }\right)$ of lemma 3.9 with $\omega =\frac{1}{\alpha }\left({g}^{\delta }-A{\hat{x}}_{\alpha }\right)$.By lemma A.1 the map $\alpha {\mapsto}\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ is non increasing. Together with (3.2) we obtain

  $$\begin{equation*}{\Vert}\omega {{\Vert}}_{\mathbb{Y}}=\frac{1}{\alpha }{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\frac{\delta }{\alpha }+\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\frac{\delta }{\alpha }+\frac{1}{\beta }{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

  Hence lemma 3.9 implies the claim.
• (b)  
  We use first corollary 3.5 with g = Ax and $h=A{\hat{x}}_{\alpha }$ then (3.1) and finally non decreasingness of $\alpha {\mapsto}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ (see lemma A.1) to estimate

  $$\begin{align*}\hfill {\Vert}\left(Ax-A{x}_{\beta }\right)-\left(A{\hat{x}}_{\alpha }-A{\left({\hat{x}}_{\alpha }\right)}_{\beta }\right){{\Vert}}_{\mathbb{Y}}& {\leqslant}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \\ \hfill & {\leqslant}\delta +{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \\ \hfill & {\leqslant}\delta +{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}.\hfill \end{align*}$$

  The triangle inequality finishes the proof.□

Proposition 3.11. Let $\nu \in \left(\right.0,1 \left.\right]$ and ϱ, c_l , α > 0. Suppose $x\in {K}_{\nu }^{\varrho }$ and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left({g}^{\delta }\right)$. If ${c}_{l}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}{\leqslant}\alpha$, then ${\varrho }_{\nu }\left({\hat{x}}_{\alpha }\right){\leqslant}\left(2+{c}_{l}^{-\nu }\right)\varrho$.

Proof. Let β ⩽ α. With $\delta {\leqslant}{c}_{l}^{-\nu }\varrho {\alpha }^{\nu }$ we estimate

$$\begin{equation*}\frac{\delta \beta }{\alpha }{\leqslant}{c}_{l}^{-\nu }\varrho {\alpha }^{\nu -1}\beta {\leqslant}{c}_{l}^{-\nu }\varrho {\beta }^{\nu }.\end{equation*}$$

Furthermore

$$\begin{equation*}\delta {\leqslant}{c}_{l}^{-\nu }\varrho {\alpha }^{\nu }{\leqslant}{c}_{l}^{-\nu }\varrho {\beta }^{\nu }\quad \text{for}\enspace \text{all}\enspace \beta {\geqslant}\alpha .\end{equation*}$$

Together with ${\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}{\leqslant}\varrho {\beta }^{\nu }$ for all β > 0 and x_β ∈ R_β (Ax) the result follows from lemma 3.10.□

Now we are in position to give the proof of theorem 1.

Proof of theorem  1

• (a)  
  By proposition 3.8 we have ${\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{c}_{1}\delta$ and proposition 3.11 yields $x,{\hat{x}}_{\alpha }\in {K}_{\nu }^{{c}_{2}\varrho }$.
• (b)  
  Using the triangle inequality we obtain

  $$\begin{equation*}{\Vert}Ax-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta +{\Vert}{g}^{\delta }-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{d}_{1}\delta .\end{equation*}$$

  Proposition 3.8 provides ${\left({c}_{D}-1\right)}^{\frac{1}{\nu }}{\varrho }^{-\frac{1}{\nu }}{\delta }^{\frac{1}{\nu }}{\leqslant}\alpha$. Therefore proposition 3.11 yields $x,{\hat{x}}_{\alpha }\in {K}_{\nu }^{{d}_{2}\varrho }$.In both cases the claim follows from the definition of the modulus Ω.□

We start with a converse to lemma 3.9: a linear bound ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}=\mathcal{O}\left(\alpha \right)$ implies the source condition (1.1) and the minimal $\mathcal{O}$-constant ϱ₁(x) agrees with the minimal norm ${\Vert}\omega {{\Vert}}_{\mathbb{Y}}$ attended by a source element ω. Similar results can be found in [12, lemma 4.1] and [22, proposition 4.1]. For sake of self-containedness we include a proof.

Proposition 4.1. Let $x\in \mathbb{X}$ with $\mathcal{R}\left(x\right)=\mathrm{inf}\left\{\mathcal{R}\left(z\right):z\in \mathbb{X}\enspace \;\text{with}\;\enspace Az=Ax\right\}$. Then

$$\begin{equation*}{\varrho }_{1}\left(x\right)=\mathrm{inf}\left\{{\Vert}\omega {{\Vert}}_{\mathbb{Y}}:{A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)\right\}.\end{equation*}$$

If this quantity is finite and x_α ∈ R_α (Ax), α > 0 is any selection, then the net ${\left(\frac{1}{\alpha }\left(Ax-A{x}_{\alpha }\right)\right)}_{\alpha { >}0}$ convergences weakly for α ↘ 0 to the unique $\omega \in \mathbb{Y}$ with ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)$ and ${\Vert}\omega {{\Vert}}_{\mathbb{Y}}={\varrho }_{1}\left(x\right)$.

Proof. Taking the infimum over ω in lemma 3.9 yields

$$\begin{equation*}{\varrho }_{1}\left(x\right){\leqslant}\mathrm{inf}\left\{{\Vert}\omega {{\Vert}}_{\mathbb{Y}}:{A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)\right\}.\end{equation*}$$

To prove the remaining inequality let $x\in \mathbb{X}$ with ϱ₁(x) < ∞. Then the net ${\left(\frac{1}{\alpha }\left(Ax-A{x}_{\alpha }\right)\right)}_{\alpha { >}0}$ is norm bounded in the Hilbert space $\mathbb{Y}$. By the Banach–Alaoglu theorem every null sequence of positive numbers has a subsequence α_n > 0 such that $\frac{1}{{\alpha }_{n}}\left(Ax-A{x}_{{\alpha }_{n}}\right)$ converges weakly to some $\omega \in \mathbb{Y}$ with ${\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\leqslant}{\varrho }_{1}\left(x\right)$. Lemma A.2 and the minimality assumption yield

$$\begin{equation*}\frac{1}{2{\alpha }_{n}}{\Vert}Ax-A{x}_{{\alpha }_{n}}{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left({x}_{{\alpha }_{n}}\right)\to \mathcal{R}\left(x\right).\end{equation*}$$

Together with ${\Vert}Ax-A{x}_{{\alpha }_{n}}{{\Vert}}_{\mathbb{Y}}{\leqslant}{\varrho }_{1}\left(x\right){\alpha }_{n}$ we obtain $\mathcal{R}\left({x}_{{\alpha }_{n}}\right)\to \mathcal{R}\left(x\right)$. The first order optimality condition yields $\frac{1}{{\alpha }_{n}}{A}^{{\ast}}A\left(x-{x}_{{\alpha }_{n}}\right)\in \partial \mathcal{R}\left({x}_{{\alpha }_{n}}\right)$. Hence for $z\in \mathbb{X}$ we obtain

$$\begin{align*}\hfill \mathcal{R}\left(x\right)+\langle {A}^{{\ast}}\omega ,z-x\rangle & =\mathcal{R}\left(x\right)+\langle \omega ,A\left(z-x\right)\rangle \hfill \\ \hfill & =\underset{n\to \infty }{\mathrm{lim}}\enspace \mathcal{R}\left({x}_{{\alpha }_{n}}\right)+\left\langle \frac{1}{{\alpha }_{n}},A,\left(x-{x}_{{\alpha }_{n}}\right),,,A,\left(z-{x}_{{\alpha }_{n}}\right)\right\rangle \hfill \\ \hfill & =\underset{n\to \infty }{\mathrm{lim}}\enspace \mathcal{R}\left({x}_{{\alpha }_{n}}\right)+\left\langle \frac{1}{{\alpha }_{n}},{A}^{{\ast}},A,\left(x-{x}_{{\alpha }_{n}}\right),,,z,-,{x}_{{\alpha }_{n}}\right\rangle {\leqslant}\mathcal{R}\left(z\right).\hfill \end{align*}$$

This shows ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)$. Therefore the stated identity is proven.

Being the preimage of the convex set $\partial \mathcal{R}\left(x\right)$ under the linear map A* the set $\left\{\omega \in \mathbb{Y}:{A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)\right\}$ is convex. Strict convexity of ${\Vert}\cdot {{\Vert}}_{\mathbb{Y}}$ yields uniqueness of ω. In particular this implies the convergence of the net.

Corollary 4.2. We have ϱ₁(x) = 0 if and only if $x\in {\mathrm{argmin}}_{z\in \mathbb{X}}\enspace \mathcal{R}\left(z\right)$.□

Proof. By the second statement in proposition 4.1 we have ρ₁(x) = 0 if and only if $0\in \partial \mathcal{R}\left(x\right)$. Hence the first order optimality condition $x\in {\text{argmin}}_{z\in \mathbb{X}}\enspace \mathcal{R}\left(z\right)$ if and only if $0\in \partial \mathcal{R}\left(x\right)$ yields the claim.

Example 4.3. Let p ∈ [1, 2], $\mathbb{X}={\ell }^{p}{:=}{\ell }^{p}\left(\mathbb{N}\right)$, $\mathbb{Y}={\ell }^{2}{:=}{\ell }^{2}\left(\mathbb{N}\right)$, A: ℓ^p → ℓ² the embedding operator given by x ↦ x and $\mathcal{R}$ given by $\mathcal{R}\left(x\right)=\frac{1}{p}{\Vert}x{{\Vert}}_{{\ell }^{p}}$. Let x ∈ ℓ^p .

If p > 1 then $\partial \mathcal{R}\left(x\right)=\left\{\xi \right\}$ with |ξ_j | = |x_j |^p−1. The adjoint A* identifies with the embedding operator ℓ² → ℓp' with p' the Hölder conjugate of p. Hence x ∈ K₁ if and only if ${\Vert}\xi {{\Vert}}_{{\ell }^{2}}{< }\infty$, and we have

$$\begin{equation*}{\varrho }_{1}\left(x\right)={\Vert}\xi {{\Vert}}_{{\ell }^{2}}={\left({\sum }_{j\in \mathbb{N}}\vert {x}_{j}{\vert }^{2p-2}\right)}^{1/2}={\Vert}x{{\Vert}}_{2p-2}^{p-1}.\end{equation*}$$

Therefore assumption 2.2 is satisfied in this case.

For p = 1 we have $\xi \in \partial \mathcal{R}\left(x\right)$ if and only if ξ_j = 1 for x_j > 0, ξ_j = −1 for x_j < 0 and |ξ_j | ⩽ 1 for x_j = 0. Hence K₁ consists of all elements with finitely many non vanishing coefficients. We have ${\varrho }_{1}\left(x\right)=\#{\left\{j\in \mathbb{N}:{x}_{j}\ne 0\right\}}^{1/2}$ and assumption 2.2 is not fulfilled.

In this subsection we assume ${\mathbb{X}}_{A}$ is a Banach space with a dense, continuous embedding $\mathbb{X}\subset {\mathbb{X}}_{A}$ and that A extends to ${\mathbb{X}}_{A}$ such that (2.4) is satisfied. Let u ∈ [1, ∞) and consider the penalty given by $\mathcal{R}\left(x\right)=\frac{1}{u}{\Vert}x{{\Vert}}_{\mathbb{X}}^{u}$.

If $\mathbb{X}$ is reflexive we choose τ to be the weak topology on $\mathbb{X}$. Then the sublevel sets of $\mathcal{R}$ are τ-compact by the Banach–Alaoglu theorem. Moreover A is weak-to-weak continuous as it is bounded. Therefore, assumption 2.1 is automatically satisfied in this case.

In this subsection we compute K₁ for the three penalties covered in the examples in subsection 2.2. We start with a tool that helps computing the function ϱ₁ up to equivalence. Note that the density $\mathbb{X}\subset {\mathbb{X}}_{A}$ allows us to view the adjoint of the embedding as an embedding ${\mathbb{X}}_{A}^{\prime }\subset {\mathbb{X}}^{\prime }$.

Proposition 4.4. We have x ∈ K₁ if and only if $\partial \mathcal{R}\left(x\right)\cap {\mathbb{X}}_{A}^{\prime }\ne \varnothing$. The function

$$\begin{equation*}{\bar{\varrho }}_{1}:{\mathbb{X}}_{A}\to \left[0,\infty \right]\quad \text{given}\;\text{by}\quad {\bar{\varrho }}_{1}\left(x\right)=\mathrm{inf}\left\{{\Vert}\xi {{\Vert}}_{{\mathbb{X}}_{A}^{\prime }}:\xi \in \partial \mathcal{R}\left(x\right)\cap {\mathbb{X}}_{A}^{\prime }\right\}\end{equation*}$$

satisfies

$$\begin{equation*}\frac{1}{M}{\varrho }_{1}\left(x\right){\leqslant}{\bar{\varrho }}_{1}\left(x\right){\leqslant}M{\varrho }_{1}\left(x\right)\quad \text{for}\enspace \text{all}\enspace x\in \mathbb{X}.\end{equation*}$$

Proof. Suppose $\xi \in \partial \mathcal{R}\left(x\right)\cap {\mathbb{X}}_{A}^{\prime }$. Let $z\in {\mathbb{X}}_{A}$, then

$$\begin{equation*}\langle \xi ,z\rangle {\leqslant}{\Vert}\xi {{\Vert}}_{{\mathbb{X}}_{A}^{\prime }}{\Vert}z{{\Vert}}_{{\mathbb{X}}_{A}}{\leqslant}M{\Vert}\xi {{\Vert}}_{{\mathbb{X}}_{A}^{\prime }}{\Vert}Az{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

Proposition B.2 provides $\omega \in \mathbb{Y}$ with ${\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\leqslant}M{\Vert}\xi {{\Vert}}_{{{\mathbb{X}}_{A}}^{\prime }}$ and ${A}^{{\ast}}\omega =\xi \in \partial \mathcal{R}\left(x\right)$.

Together with proposition 4.1 this yields the first inequality.

Let $\omega \in \mathbb{Y}$, such that ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left(x\right)$. Then

$$\begin{equation*}\langle {A}^{{\ast}}\omega ,z\rangle =\langle \omega ,Az\rangle {\leqslant}{\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\Vert}Az{{\Vert}}_{\mathbb{Y}}{\leqslant}M{\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\Vert}z{{\Vert}}_{{\mathbb{X}}_{A}}\end{equation*}$$

for all $z\in \mathbb{X}$. Hence ${\Vert}{A}^{{\ast}}\omega {{\Vert}}_{{\mathbb{X}}_{A}^{\prime }}{\leqslant}M{\Vert}\omega {{\Vert}}_{\mathbb{Y}}$. This proves the second inequality.□

Computation of K₁ for weighted p -norm penalization. We revisit the first example in subsection 2.2. Recall ${\mathbb{X}}_{A}={\ell }_{\bar{a}}^{2}$ and $\mathbb{X}={\ell }_{\bar{r}}^{p}$ with p ∈ (1, 2).

Proposition 4.5. Let $\bar{s}={\bar{a}}^{-\frac{1}{p-1}}{\bar{r}}^{\frac{p}{p-1}}$. Then ${K}_{1}={\ell }_{\bar{s}}^{2p-2}$ with

$$\begin{equation*}\frac{1}{M}{\varrho }_{1}\left(x\right){\leqslant}{\Vert}x{{\Vert}}_{s,2p-2}^{p-1}{\leqslant}M{\varrho }_{1}\left(x\right)\quad \text{for}\enspace \text{all}\enspace x\in {\ell }_{\bar{a}}^{2}.\end{equation*}$$

Proof. Let $x\in {\ell }_{\bar{r}}^{p}$. Then $\partial \mathcal{R}\left(x\right)=\left\{\xi \right\}$ with $\vert {\xi }_{j}\vert ={r}_{j}^{p}\vert {x}_{j}{\vert }^{p-1}$. With ${\bar{\varrho }}_{1}$ as in proposition 4.4 and in view of proposition B.1 we obtain

$$\begin{equation*}{\bar{\varrho }}_{1}\left(x\right)={\Vert}\xi {{\Vert}}_{{a}^{-1},2}={\left({\sum }_{j\in {\Lambda}}{a}_{j}^{-2}{r}_{j}^{2p}\vert {x}_{j}{\vert }^{2p-2}\right)}^{1/2}={\Vert}x{{\Vert}}_{s,2p-2}^{p-1}.\end{equation*}$$

Proposition 4.4 yields the result.□

Computation of K₁ for Besov 0, p, p -penalties. Next we characterize K₁ for example 2. Recall ${\mathbb{X}}_{A}={b}_{2,2}^{-a}$ and $\mathbb{X}={b}_{p,p}^{0}$ with p ∈ (1, ∞).

Proposition 4.6. Let $\tilde {s}=\frac{a}{p-1}$ and $\tilde {t}=2p-2$. Then ${K}_{1}={b}_{\tilde {t},\tilde {t}}^{\tilde {s}}$ with

$$\begin{equation*}\frac{1}{M}{\varrho }_{1}\left(x\right){\leqslant}{\Vert}x{{\Vert}}_{\tilde {s},\tilde {t},\tilde {t}}^{p-1}{\leqslant}M{\varrho }_{1}\left(x\right)\quad \text{for}\enspace \text{all}\enspace x\in {b}_{2,2}^{-a}.\end{equation*}$$

Proof. The proof works along the lines of the proof of proposition 4.5 by identifying the expression for ||ξ||_a,2,2 with ${\Vert}x{{\Vert}}_{\tilde {s},\tilde {t},\tilde {t}}$.□

Computation of K₁ for Besov 0, 2, q -penalties. Finally we compute K₁ for example 3 with ${\mathbb{X}}_{A}={b}_{2,2}^{-a}$ and $\mathbb{X}={b}_{2,q}^{0}$ with q ∈ (1, ∞).

Proposition 4.7. Let $\tilde {s}=\frac{a}{q-1}$ and $\tilde {q}=2q-2$. Then ${K}_{1}={b}_{2,\tilde {q}}^{\tilde {s}}$ with

$$\begin{equation*}\frac{1}{M}{\varrho }_{1}\left(x\right){\leqslant}{\Vert}x{{\Vert}}_{\tilde {s},2,\tilde {q}}^{q-1}{\leqslant}M{\varrho }_{1}\left(x\right)\quad \text{for}\enspace \text{all}\enspace x\in {b}_{2,2}^{-a}.\end{equation*}$$

Proof. If $x\in {b}_{2,q}^{0}$, then $\partial \mathcal{R}\left(x\right)=\left\{\xi \right\}$ with ${\xi }_{j,k}={\left({\sum }_{{k}^{\prime }}\vert {x}_{j,{k}^{\prime }}{\vert }^{2}\right)}^{\frac{q}{2}-1}\vert {x}_{j,k}\vert$. With ${\bar{\varrho }}_{1}$ as in proposition 4.4 and using proposition B.1 we obtain ${\bar{\varrho }}_{1}\left(x\right)={\Vert}\xi {{\Vert}}_{a,2,2}={\Vert}x{{\Vert}}_{\tilde {s},2,\tilde {q}}^{q-1}.$ proposition 4.4 yields the result.

Note that assumption 2.2 holds true for all three examples.

K_ν via approximation by elements of K₁. In [3, proposition 1] the authors point out that the set of elements satisfying the source condition (1.1) is the set of possible minimizers of the Tikhonov functional. Therefore one might suggest that the approximation error of $x\in \mathbb{X}$ by x_α ∈ R_α (Ax) is determined by the best approximation from the family of sets

$$\begin{equation*}{B}_{r}{:=}\left\{x\in \mathbb{X}:{\varrho }_{1}\left(x\right){\leqslant}r\right\}\quad \;\text{with}\enspace r{\geqslant}0.\end{equation*}$$

We consider the best approximation error

$$\begin{equation*}{\gamma }_{x}:\enspace \left[\right.\enspace 0,\infty \left.\right)\to \left[\right.0,\infty \left.\right)\quad \text{given}\;\text{by}\quad {\gamma }_{x}\left(r\right)=\underset{z\in {B}_{r}}{\mathrm{inf}}{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

The function γ_x is well defined as corollary 4.2 yields $\varnothing \ne {\text{argmin}}_{z\in \mathbb{X}}\enspace \mathcal{R}\left(z\right)\subset {B}_{r}$ for all r ⩾ 0. Moreover it is non increasing as ${B}_{{r}_{1}}\subseteq {B}_{{r}_{2}}$ for r₁ ⩽ r₂.

The following proposition is the starting point to prove equivalence of Hölder-type bounds on γ_x and on ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$.

Proposition 4.8. Let $x\in \mathbb{X}$, α > 0 and x_α ∈ R_α (Ax). Then

$$\begin{equation*}{\gamma }_{x}\left(\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\right){\leqslant}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}4{\gamma }_{x}\left(\frac{1}{4\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\right).\end{equation*}$$

Proof. Proposition 4.1 and the first order optimality condition $\frac{1}{\alpha }{A}^{{\ast}}A\left(x-{x}_{\alpha }\right)\in \partial \mathcal{R}\left({x}_{\alpha }\right)$ provide ${\varrho }_{1}\left({x}_{\alpha }\right){\leqslant}\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$. This proves the first inequality by definition of γ_g .

To show the second inequality let z ∈ B_r . By proposition 4.1 there is $\omega \in \mathbb{Y}$ with ${\Vert}\omega {{\Vert}}_{\mathbb{Y}}{\leqslant}r$ and ${A}^{{\ast}}\omega \in \partial \mathcal{R}\left(z\right)$ hence

$$\begin{equation*}\mathcal{R}\left(z\right)-\mathcal{R}\left({x}_{\alpha }\right){\leqslant}\langle {A}^{{\ast}}\omega ,z-{x}_{\alpha }\rangle {\leqslant}r{\Vert}Az-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

From 2αT_α (x_α , Ax) ⩽ 2αT_α (z, Ax) and the last inequality we deduce

$$\begin{align*}\hfill {\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}& {\leqslant}{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}+2\alpha r{\Vert}Az-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \\ \hfill & {\leqslant}{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}+2\alpha r{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}+2\alpha r{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\hfill \end{align*}$$

Taking the infimum over z ∈ B_r and estimating the third summand using $ab{\leqslant}\frac{1}{2}{a}^{2}+\frac{1}{2}{b}^{2}$ we obtain

$$\begin{align*}\hfill {\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}& {\leqslant}{\gamma }_{x}{\left(r\right)}^{2}+2\alpha r{\gamma }_{x}\left(r\right)+2{\alpha }^{2}{r}^{2}+\frac{1}{2}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\hfill \\ \hfill & {\leqslant}2{\left({\gamma }_{x}\left(r\right)+\alpha r\right)}^{2}+\frac{1}{2}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}.\hfill \end{align*}$$

Hence ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}2{\gamma }_{x}\left(r\right)+2\alpha r$ and the choice $r=\frac{1}{4\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ yields the second inequality.□

As announced we see equivalence of Hölder-type bounds on γ_x and on ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ as a consequence.

Proposition 4.9. Let ν ∈ (0, ∞) and $x\in \mathbb{X}$. The following statements are equivalent:

• (a)  
  There exists a constant c₁ > 0 such that γ_x (r) ⩽ c₁ r^−ν for all r > 0.
• (b)  
  There exists a constant c₂ > 0 such ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{c}_{2}{\alpha }^{\frac{\nu }{1+\nu }}$ for all α > 0 and x_α ∈ R_α (Ax).

More precisely (a) implies (b) with ${c}_{2}=4{c}_{1}^{\frac{1}{1+\nu }}$ and (b) implies (a) with ${c}_{1}={c}_{2}^{1+\nu }$.

Proof. (a) ⇒ (b): the second inequality in proposition 4.8 yields

$$\begin{equation*}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{4}^{1+\nu }{c}_{1}{\alpha }^{\nu }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{-\nu }\end{equation*}$$

Multiplying by ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{\nu }$ and taking the power $\frac{1}{1+\nu }$ yields

$$\begin{equation*}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}4{c}_{1}^{\frac{1}{1+\nu }}{\alpha }^{\frac{\nu }{1+\nu }}\end{equation*}$$

(b) ⇒ (a): let r > 0. For $\alpha ={c}_{2}^{1+\nu }{r}^{-\left(1+\nu \right)}$ we obtain $\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{c}_{2}{\alpha }^{-\frac{1}{1+\nu }}=r$. Hence the first inequality in proposition 4.8 yields

$$\begin{equation*}{\gamma }_{x}\left(r\right){\leqslant}{\gamma }_{x}\left(\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\right){\leqslant}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}{c}_{2}{\alpha }^{\frac{\nu }{1+\nu }}={c}_{2}^{1+\nu }{r}^{-\nu }.\end{equation*}$$

□

K_ν via real interpolation. Again we assume ${\mathbb{X}}_{A}$ is a Banach space such that (2.4) holds true. The next lemma shows that under assumption 2.2 the spaces ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }$ classify the image space approximation precision.

Proposition 4.10 (K_ν as a real interpolation space). Suppose assumption 2.2 holds true. Let θ ∈ (0, 1) and $\nu {:=}\frac{\theta }{\left(1-\theta \right)\left(u-1\right)+\theta }$. We have ${K}_{\nu }={\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }$ with

$$\begin{equation*}\begin{gathered}{C}_{1}{\Vert}x{{\Vert}}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}{\leqslant}{\varrho }_{\nu }{\left(x\right)}^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{u-1}}{\leqslant}{C}_{2}{\Vert}x{{\Vert}}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}\\ \quad \text{for}\enspace \text{all}\enspace x\in {\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }\end{gathered}\end{equation*}$$

with constants C₁, C₂ > 0 depending only on u, θ and M.

Proof. Assume ϱ := ϱ_ν (x) < ∞. Proposition 4.9 provides the bound

$$\begin{equation*}{\gamma }_{x}\left(r\right){\leqslant}{\varrho }^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{\left(1-\theta \right)\left(u-1\right)}}{r}^{-\frac{\theta }{\left(1-\theta \right)\left(u-1\right)}}\end{equation*}$$

Let t > 0. We choose r := ϱ^{(1−θ)(u−1)+θ} t^{−(1−θ)(u−1)}. If ɛ > 0 then there exists z ∈ K₁ with ϱ₁(z) ⩽ r and ${\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}{\leqslant}{\gamma }_{x}\left(r\right)+\varepsilon$. Therefore we obtain

$$\begin{equation*}K\left(x,t\right){\leqslant}{\Vert}x-z{{\Vert}}_{{\mathbb{X}}_{A}}+t{\Vert}z{{\Vert}}_{\text{lin}}{\leqslant}M\left({\gamma }_{x}\left(r\right)+\varepsilon \right)+t{M}^{\frac{1}{u-1}}{r}^{\frac{1}{u-1}}.\end{equation*}$$

For ɛ → 0 we obtain

$$\begin{align*}\hfill K\left(x,t\right)& {\leqslant}M{\varrho }^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{\left(1-\theta \right)\left(u-1\right)}}{r}^{-\frac{\theta }{\left(1-\theta \right)\left(u-1\right)}}+t{M}^{\frac{1}{u-1}}{r}^{\frac{1}{u-1}}\hfill \\ \hfill & =\left(M+{M}^{\frac{1}{u-1}}\right){\varrho }^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{u-1}}{t}^{\theta }.\hfill \end{align*}$$

This proves the first inequality.

Assume $n{:=}{\Vert}x{{\Vert}}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}{< }\infty$. We prove a bound on γ_x and apply proposition 4.9. Let r > 0. We choose $t{:=}2{M}^{\frac{1}{\left(1-\theta \right)\left(u-1\right)}}{n}^{\frac{1}{1-\theta }}{r}^{-\frac{1}{\left(1-\theta \right)\left(1-u\right)}}$. Since 2^1−θ > 1 there exists $z\in {\mathbb{X}}_{A}$ such that

$$\begin{align*}\hfill {M}^{-1}{\Vert}Ax-Az{\Vert}+t{M}^{-\frac{1}{u-1}}{\varrho }_{1}{\left(z\right)}^{\frac{1}{u-1}}& {\leqslant}{\Vert}x-z{{\Vert}}_{{\mathbb{X}}_{A}}+t{\Vert}z{{\Vert}}_{\text{lin}}\hfill \\ \hfill & \quad {\leqslant}{2}^{1-\theta }K\left(x,t\right){\leqslant}{2}^{1-\theta }n{t}^{\theta }=t{M}^{-\frac{1}{u-1}}{r}^{\frac{1}{u-1}}\hfill \end{align*}$$

Neglecting the first summand on the left-hand side we obtain ϱ₁(z) ⩽ r. Therefore

$$\begin{equation*}{\gamma }_{x}\left(r\right){\leqslant}{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}{\leqslant}M{2}^{1-\theta }n{t}^{\theta }=2{M}^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{\left(1-\theta \right)\left(u-1\right)}}{n}^{\frac{1}{1-\theta }}{r}^{-\frac{\theta }{\left(1-\theta \right)\left(u-1\right)}}.\end{equation*}$$

Proposition 4.9 yields ${\varrho }_{\nu }\left(x\right){\leqslant}8M{n}^{\frac{u-1}{\left(1-\theta \right)\left(u-1\right)+\theta }}$.□

Remark 4.11. As already exposed in example 4.3 we cannot expect assumption 2.2 to hold true for ℓ¹-type norms like Besov 0, 1, 1 or 0, 2, 1-norms. Nevertheless one may use proposition 4.9 directly to characterize the sets K_ν in this case. Applying theorem 1 then reproduces the convergence rates results for the 0, 2, 1-penalty in [17] and for weighed ℓ¹-penalties in [20] in the case of linear operators.

We apply theorem 1 to obtain error bounds measured in the norm of certain Banach spaces ${\mathbb{X}}_{L}$ with a continuous embedding ${\mathbb{X}}_{L}\subset {\mathbb{X}}_{A}$.

To this end we consider the loss function $L:\mathbb{X}{\times}\mathbb{X}\to \left[0,\infty \right]$ given by $L\left({x}_{1},{x}_{2}\right)={\Vert}{x}_{1}-{x}_{2}{{\Vert}}_{{\mathbb{X}}_{L}}$ if ${x}_{1}-{x}_{2}\in {\mathbb{X}}_{L}$ and L(x₁, x₂) = ∞ if ${x}_{1}-{x}_{2}\notin {\mathbb{X}}_{L}$. Before we prove theorem 2 we state a proposition that characterizes for which spaces ${\mathbb{X}}_{L}$ Hölder-type bounds on the modulus of continuity on balls of a given quasi-Banach space ${\mathbb{X}}_{S}\subset \mathbb{X}$ are satisfied.

Proposition 4.12 (Bound on the modulus). Let ${\mathbb{X}}_{S}\subset \mathbb{X}$ be a quasi-Banach space and ${\mathbb{X}}_{L}$ a Banach space with continuous embeddings ${\mathbb{X}}_{S}\subset {\mathbb{X}}_{L}\subset {\mathbb{X}}_{A}$ and e ∈ (0, 1). For ϱ > 0 we denote

$$\begin{equation*}{K}_{{\mathbb{X}}_{S}}^{\varrho }{:=}\left\{x\in {\mathbb{X}}_{S}:{\Vert}x{{\Vert}}_{{\mathbb{X}}_{S}}{\leqslant}\varrho \right\}.\end{equation*}$$

The following statements are equivalent:

• (a)  
  There is a continuous embedding ${\left({\mathbb{X}}_{A},{\mathbb{X}}_{S}\right)}_{e,1}\subset {\mathbb{X}}_{L}$.
• (b)  
  There exists a constant c > 0 with ${\Omega}\left(\delta ,{K}_{{\mathbb{X}}_{S}}^{\varrho }\right){\leqslant}c{\varrho }^{e}{\delta }^{1-e}$ for all δ, ϱ > 0.

Proof. By [2, section 3.5, theorem 3.11.4] statement (a) is equivalent to an interpolation inequality

$$\begin{equation}{\Vert}z{{\Vert}}_{{\mathbb{X}}_{L}}{\leqslant}C{\Vert}z{{\Vert}}_{{\mathbb{X}}_{A}}^{1-e}{\Vert}z{{\Vert}}_{{\mathbb{X}}_{S}}^{e}\quad \text{for}\enspace \text{all}\enspace z\in {\mathbb{X}}_{S}.\end{equation} \tag{ 4.1 }$$

Let ${x}_{1},{x}_{2}\in {K}_{{\mathbb{X}}_{S}}^{\varrho }$ with ${\Vert}A{x}_{1}-A{x}_{2}{{\Vert}}_{\mathbb{Y}}{\leqslant}\delta$. The quasi-triangle inequality yields ${\Vert}{x}_{1}-{x}_{2}{{\Vert}}_{{\mathbb{X}}_{S}}{\leqslant}2c\varrho$ and from (2.4) we obtain ${\Vert}{x}_{1}-{x}_{2}{{\Vert}}_{{\mathbb{X}}_{A}}{\leqslant}M\delta$. Hence (4.1) with z = x₁ − x₂ yields ${\Vert}{x}_{1}-{x}_{2}{{\Vert}}_{{\mathbb{X}}_{L}}{\leqslant}C{M}^{1-e}{\left(2c\right)}^{e}{\varrho }^{e}{\delta }^{1-e}$. Taking the supremum over x₁, x₂ yields (b).

Assuming a bound on the modulus we obtain (4.1) from

$$\begin{equation*}{\Vert}z{{\Vert}}_{{\mathbb{X}}_{L}}{\leqslant}{\Omega}\left(M{\Vert}z{{\Vert}}_{{\mathbb{X}}_{A}},{K}_{{\mathbb{X}}_{S}}^{{\Vert}z{{\Vert}}_{{\mathbb{X}}_{S}}}\right).\end{equation*}$$

□

Next we give the proof of theorem 2.

Proof of theorem  2.For ν as in proposition 4.10 the second inequality therein yields

$$\begin{equation*}x\in {K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{\varrho }\subset {K}_{\nu }^{\bar{\varrho }}\end{equation*}$$

with $\bar{\varrho }={\left({C}_{2}\varrho \right)}^{\frac{u-1}{\left(1-\theta \right)\left(u-1\right)+\theta }}$.

In view of theorem 1 it remains to prove an upper bound on ${\Omega}\left({c}_{1}\delta ,{K}_{\nu }^{{c}_{2}\bar{\varrho }}\right){\leqslant}C{\varrho }^{\frac{\xi }{\theta }}{\delta }^{1-\frac{\xi }{\delta }}$ for constants c₁, c₂ > 0 given therein. The first inequality in proposition 4.10 provides

$$\begin{equation*}{K}_{\nu }^{{c}_{2}\bar{\varrho }}\subset {K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{{c}_{3}\varrho }\quad \text{with}\quad {c}_{3}={C}_{1}^{-1}{C}_{2}{c}_{2}^{\frac{\left(1-\theta \right)\left(u-1\right)+\theta }{u-1}}.\end{equation*}$$

The reiteration theorem (see [2, theorem 3.11.5]) yields

$$\begin{equation}{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,1}={\left({\mathbb{X}}_{A},{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }\right)}_{\frac{\xi }{\theta },1}\end{equation} \tag{ 4.2 }$$

with equivalent quasi-norms. In particular ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }\subset {\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,1}\subset {\mathbb{X}}_{L}\subset {\mathbb{X}}_{A}$. Hence proposition 4.12 with ${\mathbb{X}}_{S}={\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }$ yields a constant c₄ with

$$\begin{equation*}{\Omega}\left({c}_{1}\delta ,{K}_{\nu }^{{c}_{2}\bar{\varrho }}\right){\leqslant}{\Omega}\left({c}_{1}\delta ,{K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{{c}_{3}\varrho }\right){\leqslant}C{\varrho }^{\frac{\xi }{\theta }}{\delta }^{1-\frac{\xi }{\theta }}\end{equation*}$$

with $C={c}_{4}{c}_{3}^{\frac{\xi }{\theta }}{c}_{1}^{1-\frac{\xi }{\theta }}$.

For the discrepancy principle the bound ${\Omega}\left({d}_{1}\delta ,{K}_{\nu }^{{d}_{2}\bar{\varrho }}\right){\leqslant}C{\varrho }^{\frac{\xi }{\theta }}{\delta }^{1-\frac{\xi }{\delta }}$ follows by replacing c₁ by d₁ and c₂ by d₂.

Remark 4.13. The statement in remark 2.3 for the limiting case θ = 1 follows along the same lines leaving out the step involving the reiteration theorem.□

Remark 4.14. The relation ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,1}\subset {\mathbb{X}}_{L}$ is necessary to obtain error bounds as in theorem 2 in the following sense: assuming ${\mathbb{X}}_{L}$ satisfies an error bound

$$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{{\mathbb{X}}_{L}}{\leqslant}C{\varrho }^{e}{\delta }^{1-e}\end{equation*}$$

for some e ∈ (0, 1) and all $x\in {K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{\varrho }$ under some a priori parameter choice α = α(δ), then the lower bound (2.3) yields

$$\begin{equation*}\frac{1}{2}{\Omega}\left(2\delta ,{K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{\varrho }\right){\leqslant}{{\Delta}}_{{R}_{\alpha \left(\delta \right)}}\left(\delta ,{K}_{{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }}^{\varrho }\right){\leqslant}C{\varrho }^{e}{\delta }^{1-e}.\end{equation*}$$

Thus the converse implication in proposition 4.12 and the identity (4.2) provides

$$\begin{equation*}{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta e,1}={\left({\mathbb{X}}_{A},{\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }\right)}_{e,1}\subset {\mathbb{X}}_{L}.\end{equation*}$$

Error bounds for weighted p -norm penalization. To prove corollary 2.4 we return to the setting of example 1.

Proof of corollary  2.4.First note that assumption 2.2 holds true by proposition 4.5. By [8, theorem 2, remark] we have

$$\begin{equation*}{\ell }_{\bar{r}}^{p}={\left({\ell }_{\bar{a}}^{2},{\ell }_{\bar{s}}^{2p-2}\right)}_{\xi ,p}={\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,p}\quad \text{with}\quad \xi {:=}\frac{p-1}{p}.\end{equation*}$$

Hence by [2, theorem 3.4.1(b); section 3.11] there is a continuous embedding ${\left({\ell }_{\bar{a}}^{2},{\ell }_{\bar{s}}^{2p-2}\right)}_{\xi ,1}\subset {\ell }_{\bar{r}}^{p}$. Hence the choice ${\mathbb{X}}_{L}={\ell }_{\bar{r}}^{p}$ satisfies the assumption of theorem 2.

The interpolation spaces ${\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\theta ,\infty }={\left({\ell }_{\bar{a}}^{2},{\ell }_{\bar{s}}^{2p-2}\right)}_{\theta ,\infty }$ are characterized by weighted weak ℓ^p -spaces ${\ell }_{\mu ,\nu }^{t,\infty }$ in the following manner:

$$\begin{equation*}\begin{gathered}{\ell }_{\mu ,\nu }^{t,\infty }={\left({\ell }_{\bar{a}}^{2},{\ell }_{\bar{s}}^{2p-2}\right)}_{\theta ,\infty }\quad \;\text{with}\;\quad \frac{1}{t}=\frac{1-\theta }{2}+\frac{\theta }{2p-2},\qquad \mu {:=}{\left({\bar{a}}^{2}{\bar{r}}^{-p}\right)}^{\frac{1}{2-p}},\enspace \\ \nu {:=}{\left({\bar{a}}^{-1}\bar{r}\right)}^{\frac{2p}{2-p}}\end{gathered}\end{equation*}$$

with equivalent quasi-norms (see [8, theorem 2]).

The application of theorem 2 yields corollary 2.4 and remark 2.5 follows from remark 2.3.□

Error bounds for Besov 0, p, p -penalties. Next we revisit example 2.

Proof of corollary  2.7.Here assumption 2.2 holds true by proposition 4.6. The identification [8, theorem 2, remark] for p ≠ 2 and [27, 3.3.6.(9)] for p = 2 yield

$$\begin{equation}{b}_{p,p}^{0}={\left({b}_{2,2}^{-a},{b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\right)}_{\xi ,p}={\left({\mathbb{X}}_{A},{K}_{1}\right)}_{\xi ,p}\quad \text{with}\quad \xi =\frac{p-1}{p}.\end{equation} \tag{ 4.3 }$$

Hence the choice ${\mathbb{X}}_{L}={b}_{p,p}^{0}$ satisfies the assumption in theorem 2.

We apply theorem 2 to obtain corollary 2.7. Remark 2.8 follows from remark 2.3.□

Furthermore we prove the nestings given in proposition 2.9.

Proof of proposition 2.9. Let $\theta =\frac{p-1}{p}\frac{s+a}{a}$. Then $\frac{1}{t}=\frac{1-\theta }{2}+\frac{\theta }{\tilde {t}}$. With [8, theorem 2, remark] and [2, theorem 3.4.1(b)] we obtain

$$\begin{equation*}{b}_{t,t}^{s}={\left({b}_{2,2}^{-a},{b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\right)}_{\theta ,t}\subset {\left({b}_{2,2}^{-a},{b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\right)}_{\theta ,\infty }={k}_{s}\end{equation*}$$

in both cases.

Suppose p < 2. Then $\tilde {t}{< }2$ and $t\in \left(\tilde {t},2\right)$. Let ɛ > 0 such that $t-\varepsilon \in \left(\tilde {t},2\right)$. There are $s{< }{s}^{\prime }{< }\tilde {s}$ and θ < θ' < 1 such that ${b}_{t-\varepsilon ,t-\varepsilon }^{{s}^{\prime }}={\left({b}_{2,2}^{-a},{b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\right)}_{{\theta }^{\prime },t-\varepsilon }$. The reiteration theorem (see [2, theorem 3.11.5]) yields ${k}_{s}={\left({b}_{2,2}^{-a},{b}_{t-\varepsilon ,t-\varepsilon }^{{s}^{\prime }}\right)}_{\frac{\theta }{{\theta }^{\prime }},\infty }$. From t − ɛ < 2 we obtain the continuous embeddings ${b}_{2,2}^{-a}\subset {b}_{2,\infty }^{-a}\subset {b}_{t-\varepsilon ,\infty }^{-a}$ (see [27, 3.2.4(1), 3.3.1(9)]). Together with the interpolation result ${b}_{t-\varepsilon ,\infty }^{s}={\left({b}_{t-\varepsilon ,\infty }^{-a},{b}_{t-\varepsilon ,\infty }^{r}\right)}_{\theta ,\infty }$ (see [27, 3.3.6 (9)]) we obtain the second inclusion using [27, 2.4.1 remark 4]. By [27, 3.3.1(9)]) therefore obtain the second inclusion for all 0 < < t.

For p > 2 we have ${b}_{\tilde {t},\tilde {t}}^{\tilde {s}}\subset {b}_{2,\tilde {t}}^{\tilde {s}}$ (see [27, 3.3.1(9)]). Hence [27, 3.3.6 (9)] and [27, 2.4.1 remark 4] yield ${k}_{s}\subset {\left({b}_{2,2}^{-a},{b}_{2,\tilde {t}}^{\tilde {s}}\right)}_{\theta ,\infty }={b}_{2,\infty }^{s}$.□

Error bounds for Besov 0, 2, q -penalties. Next we treat example 3.

Proof of corollary  2.10.Due to proposition 4.7 the assumption 2.2 is satisfied. By [27, 3.3.6.(9)] we have

$$\begin{equation*}{b}_{2,2}^{0}={\left({b}_{2,2}^{-a},{b}_{2,\tilde {q}}^{\tilde {s}}\right)}_{\xi ,2}\quad \text{with}\quad \xi =\frac{q-1}{q}.\end{equation*}$$

Therefore the choice ${\mathbb{X}}_{L}={b}_{2,2}^{0}$ satisfies the assumption on ${\mathbb{X}}_{L}$ in theorem 2.

Moreover for $0{< }s{< }\frac{a}{q-1}$ we have

$$\begin{equation*}{b}_{2,\infty }^{s}={\left({b}_{2,2}^{-a},{b}_{2,\tilde {q}}^{\tilde {s}}\right)}_{\theta ,2}\quad \text{with}\quad \theta =\frac{q-1}{q}\frac{s+a}{a}.\end{equation*}$$

□

Hence the application of theorem 2 yields corollary 2.10 and remark 2.3 yields remark 2.11.

Error bounds for the Radon transform. Finally we turn to proof of the convergence rate result with the Radon transform as forward operator.

Proof of corollary  2.12.

• (a)  
  Since a < s_max the synthesis operator $\mathcal{S}$ is a norm isomorphism ${b}_{2,2}^{-a}\to {B}_{2,2}^{-a}\left({\Omega}\right)$. Hence the operator $R{\circ}\mathcal{S}$ satisfies (2.4) with ${\mathbb{X}}_{A}={b}_{2,2}^{-a}$.The inequality $\frac{d}{p}-\frac{d}{2}{\leqslant}a$ implies σ_t − s_max ⩽ σ_t ⩽ s. Hence $\mathcal{S}:{b}_{t,t}^{s}\to {B}_{t,t}^{s}\left({\Omega}\right)$ is a norm isomorphism. Let c₁ be the operator norm of the inverse of $\mathcal{S}$. Then $f=\mathcal{S}x$ with $x\in {b}_{t,t}^{s}$ and ||x||_s,t,t ⩽ c₁ ϱ. Let c₂ be the embedding constant of ${b}_{t,t}^{s}\subset {k}_{s}$ (see proposition 2.9). Then we obtain x ∈ k_s with ${\Vert}x{{\Vert}}_{{k}_{s}}{\leqslant}{c}_{1}{c}_{2}\varrho$.With ${\hat{x}}_{\alpha }$ given by corollary 2.7 we obtain the bound

  $$\begin{equation*}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,p,p}{\leqslant}\tilde {C}{\varrho }^{\frac{a}{s+a}}{\delta }^{\frac{s}{s+a}}\end{equation*}$$

  with a constant $\tilde {C}{ >}0$ independent of f, δ and ϱ. Hence the first bound in corollary 2.4 implies

  $$\begin{equation*}{\Vert}f-{\hat{f}}_{\alpha }{{\Vert}}_{{B}_{p,p}^{0}\left({\Omega}\right)}={\Vert}\mathcal{S}\left(x-{\hat{x}}_{\alpha }\right){{\Vert}}_{{B}_{p,p}^{0}\left({\Omega}\right)}{\leqslant}{c}_{3}{\Vert}x-{\hat{x}}_{\alpha }{{\Vert}}_{0,p,p}\end{equation*}$$

  with c₃ the operator norm of $\mathcal{S}:{b}_{p,p}^{0}\to {B}_{p,p}^{0}\left({\Omega}\right)$. The bound in the L^p -norm for p ⩽ 2 follows from the continuity of the embedding ${B}_{p,p}^{0}\left({\Omega}\right)\subset {L}^{p}\left({\Omega}\right)$ (see [27]).
• (b)  
  This follows along the lines of the proof of (a) using corollary 2.10 instead of corollary 2.7.

□

Definition 5.1 (Minimal value function). For $g\in \mathbb{Y}$ we define

$$\begin{equation*}{\vartheta }_{g}:\left(0,\infty \right)\to \mathbb{R}\quad \;\text{by}\quad {\vartheta }_{g}\left(\alpha \right)={\mathrm{inf}}_{x\in \mathrm{dom}\left(\mathcal{R}\right)}\enspace {T}_{\alpha }\left(x,g\right)=\frac{1}{2\alpha }{\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left({\hat{x}}_{\alpha }\right).\end{equation*}$$

independent of the choice ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left(g\right)$.

The main result of this subsection is the differentiability of the minimal value function. The approximation error ${\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ is represented by calculus rules of ϑ_g .

Recall that the Moreau envelope function of some function $\mathcal{Q}:\enspace \mathbb{Y}\to \left(-\infty ,\infty \right]$ for α > 0 is given by

$$\begin{equation*}{\mathcal{Q}}_{\alpha }\left(g\right)={\mathrm{inf}}_{y\in \mathbb{Y}}\left(\frac{1}{2\alpha }{\Vert}g-y{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left(y\right)\right)\end{equation*}$$

and the infimum is uniquely attained at ${\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\in \mathbb{Y}$. The key ingredient is the following result by T. Strömberg:

Lemma 5.2 (See [25, proposition 3(iii)]). Let $\mathcal{Q}:\enspace \mathbb{Y}\to \left(-\infty ,\infty \right]$ be convex, proper and lower semi-continuous. The family of Moreau envelope functions ${\mathcal{Q}}_{\alpha }:\mathbb{Y}\to \mathbb{R}$, α > 0 satisfies

$$\begin{equation*}\frac{\partial }{\partial \alpha }{\mathcal{Q}}_{\alpha }\left(g\right)=-\frac{1}{2}{\Vert}\left(\nabla {\mathcal{Q}}_{\alpha }\right)\left(g\right){{\Vert}}_{\mathbb{Y}}^{2}.\end{equation*}$$

We apply lemma 5.2 to the function $\mathcal{Q}$ defined in lemma 3.1. Note that due to proposition 3.3 we have

$$\begin{equation}{\mathcal{Q}}_{\alpha }\left(g\right)=\frac{1}{2\alpha }{\Vert}g-{\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right){{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{Q}\left({\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right)\right)={\vartheta }_{g}\left(\alpha \right).\end{equation} \tag{ 5.1 }$$

Proposition 5.3. Let $g\in \mathbb{Y}$ and ${\hat{x}}_{\alpha }\in {R}_{\alpha }\left(g\right),\alpha { >}0$ any selection. The function ϑ_g is convex, non-increasing and continuously differentiable with

$$\begin{equation*}{\vartheta }_{g}^{\prime }\left(\alpha \right)=-\frac{1}{2{\alpha }^{2}}{\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}.\end{equation*}$$

Proof. The Moreau envelope function ${\mathcal{Q}}_{\alpha }$ is convex, real valued and continuous with the Fenchel conjugate ${\left({\mathcal{Q}}_{\alpha }\right)}^{{\ast}}={\mathcal{Q}}^{{\ast}}+\frac{\alpha }{2}{\Vert}\cdot {{\Vert}}_{\mathbb{Y}}^{2}$ (see [1, propositions 12.15 and 13.21]). The biconjugation theorem implies

$$\begin{equation*}{\vartheta }_{g}\left(\alpha \right)={\mathcal{Q}}_{\alpha }\left(g\right)={\left({\mathcal{Q}}^{{\ast}}+\frac{\alpha }{2}{\Vert}\cdot {{\Vert}}_{\mathbb{Y}}\right)}^{{\ast}}\left(g\right)=\underset{v\in \mathbb{Y}}{\mathrm{sup}}\left(\langle g,v\rangle -{\mathcal{Q}}^{{\ast}}\left(v\right)-\frac{\alpha }{2}{\Vert}v{{\Vert}}_{\mathbb{Y}}^{2}\right).\end{equation*}$$

Hence ϑ_g is convex and non-increasing being the supremum of affine non-increasing functions.

By [1, proposition 12.29] ${\mathcal{Q}}_{\alpha }$ is Fréchet differentiable with $\nabla {\mathcal{Q}}_{\alpha }=\frac{1}{\alpha }\left({\text{Id}}_{\mathbb{Y}}-{\text{Prox}}_{\alpha \mathcal{Q}}\right)$. Lemma 5.2 yields differentiability of $\alpha {\mapsto}{\mathcal{Q}}_{\alpha }\left(g\right)$ with derivative $-\frac{1}{2}{\Vert}\left(\nabla {\mathcal{Q}}_{\alpha }\right)\left(g\right){{\Vert}}^{2}$ for all $g\in \mathbb{Y}$. Therefore, ϑ_g is differentiable and we conclude with proposition 3.3

$$\begin{equation*}{\vartheta }_{g}^{\prime }\left(\alpha \right)=-\frac{1}{2}{\Vert}\left(\nabla {\mathcal{Q}}_{\alpha }\right)\left(g\right){{\Vert}}^{2}=-\frac{1}{2{\alpha }^{2}}{\Vert}g-{\text{Prox}}_{\alpha \mathcal{Q}}\left(g\right){{\Vert}}_{\mathbb{Y}}=-\frac{1}{2{\alpha }^{2}}{\Vert}g-A{\hat{x}}_{\alpha }{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

Finally, ϑ_g ' is continuous as ϑ_g is convex and differentiable.□

For the rest of this paper we always assume $x\in \mathrm{dom}\left(\mathcal{R}\right)$ is $\mathcal{R}$-minimal in A⁻¹({Ax}) and x_α ∈ R_α (Ax) for α > 0 is any selection of a minimizer for exact data.

If A is injective then the minimality is trivially satisfied for all $x\in \mathrm{dom}\left(\mathcal{R}\right)$.

As already mentioned we consider the defect of the Tikhonov functional ${\sigma }_{x}:\left(0,\infty \right)\to \left[\right.0,\infty \left.\right)$ given by

$$\begin{equation*}{\sigma }_{x}\left(\alpha \right)={T}_{\alpha }\left(x,Ax\right)-{T}_{\alpha }\left({x}_{\alpha },Ax\right)=\mathcal{R}\left(x\right)-\mathcal{R}\left({x}_{\alpha }\right)-\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}.\end{equation*}$$

The next proposition collects properties of the defect function.

Lemma 5.4. 

• (a)  
  σ_x is concave, non-decreasing and continuously differentiable with ${\sigma }_{x}^{\prime }\left(\alpha \right)=\frac{1}{2{\alpha }^{2}}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}$.
• (b)  
  We have lim_α↘0  σ_x (α) = 0.
• (c)  
  The function $\left(0,\infty \right)\to \left[\right.0,\infty \left.\right)$ given by $\alpha {\mapsto}{\sigma }_{x}\left(\frac{1}{\alpha }\right)$ is convex and continuous.

Proof. We have ${\sigma }_{x}\left(\alpha \right)=\mathcal{R}\left(x\right)-{\vartheta }_{Ax}\left(\alpha \right)$ with the minimal value function ϑ_Ax from definition 5.1. Hence (a) follows from proposition 5.3. Lemma A.2 yields (b) because of the $\mathcal{R}$-minimality assumption on x.

Let h be the function given in (c) Then h is differentiable and (a) yields

$$\begin{equation*}{h}^{\prime }\left(\alpha \right)=-\frac{1}{{\alpha }^{2}}{\sigma }_{x}^{\prime }\left(\frac{1}{\alpha }\right)=-\frac{1}{2}{\Vert}Ax-A{x}_{\frac{1}{\alpha }}{{\Vert}}_{\mathbb{Y}}.\end{equation*}$$

By lemma A.1(b) the function $\alpha {\mapsto}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ is non-decreasing. Hence h' is non-decreasing. Therefore h is convex. Continuity follows from the first statement.□

Let α > 0. We write

$$\begin{equation*}{\sigma }_{x}\left(\alpha \right)={\mathrm{sup}}_{z\in \mathbb{X}}\left(\mathcal{R}\left(x\right)-\mathcal{R}\left(z\right)-\frac{1}{2\alpha }{\Vert}Ax-Az{{\Vert}}^{2}\right)\end{equation*}$$

to note a similarity to the distance function in [7, (3.1)] and [6, chapter 12] and [16, chapter 3] used to derive variational source conditions of the form (2.8). In [16, proposition 4] its shown that a variational source condition (2.8) implies bounds on the defect function σ_x . The next result provides a sharp connection between bounds on the defect function and variational source conditions. We introduce two partially ordered sets of functions

$$\begin{align*}\hfill {\Sigma}& =\left\{\sigma :\left(0,\infty \right)\to \left[0,\infty \right]\enspace :\sigma \;\text{is}\;\text{proper,}\;\text{non}-\text{decreasing}\;\text{and}\;\sigma \left(1/\cdot \right)\right.\hfill \\ \hfill & \quad \left.\;\text{is}\;\text{convex}\;\text{l.s.c.}\right\}\hfill \\ \hfill {\Phi}& =\left\{\phi :\left[\right.0,\infty \left.\right)\to \left[\right.0,\infty \left.\right)\enspace :\phi \;\text{concave}\;\text{and}\;\text{upper}\;\text{semi}-\text{continuous}\right\}\hfill \end{align*}$$

with pointwise ordering. Here l.s.c. is an abbreviation for lower semi-continuous. Moreover, we consider the map $\mathcal{F}:{\Sigma}\to {\Phi}$ given by

$$\begin{equation}\left(\mathcal{F}\left(\sigma \right)\right)\left(t\right){:=}{\mathrm{inf}}_{\alpha { >}0}\left(\sigma \left(\alpha \right)+\frac{1}{2\alpha }t\right)\quad \text{for}\enspace t{\geqslant}0.\end{equation} \tag{ 5.2 }$$

In lemma C.2 we prove that $\mathcal{F}$ is well-defined, order preserving and bijective. The order preserving inverse ${\mathcal{F}}^{-1}:{\Phi}\to {\Sigma}$ is given by

$$\begin{equation}\left({\mathcal{F}}^{-1}\left(\phi \right)\right)\left(\alpha \right)={\mathrm{sup}}_{t{\geqslant}0}\left(\phi \left(t\right)-\frac{1}{2\alpha }t\right)\quad \text{for}\enspace \alpha { >}0.\end{equation} \tag{ 5.3 }$$

By lemma 5.4 we have σ_x ∈ Σ. It turns out that $\phi =\mathcal{F}\left({\sigma }_{x}\right)$ is the minimal function in Φ satisfying (2.8).

Lemma 5.5. Let ϕ ∈ Φ. Then the following statements are equivalent:

• (a)  
  $\mathcal{F}\left({\sigma }_{x}\right){\leqslant}\phi$.
• (b)  
  ${\sigma }_{x}{\leqslant}{\mathcal{F}}^{-1}\left(\phi \right)$.
• (c)  
  $\mathcal{R}\left(x\right)-\mathcal{R}\left(z\right){\leqslant}\phi \left({\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}\right)\quad \text{for}\enspace \text{all}\enspace z\in \mathbb{X}$.

In particular, we always have

$$\begin{equation}\mathcal{R}\left(x\right)-\mathcal{R}\left(z\right){\leqslant}\left(\mathcal{F}\left({\sigma }_{x}\right)\right)\left({\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}\right)\quad \text{for}\enspace \text{all}\enspace z\in \mathbb{X}\end{equation} \tag{ 5.4 }$$

Proof. The equivalence of (a) and (b) is immediate by lemma C.2. Next we prove (5.4). To this end let $z\in \mathbb{X}$ and α > 0. Then

$$\begin{equation*}{T}_{\alpha }\left({x}_{\alpha },Ax\right){\leqslant}\frac{1}{2\alpha }{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}+\mathcal{R}\left(z\right)\end{equation*}$$

and $\mathcal{R}\left(x\right)={T}_{\alpha }\left(x,Ax\right)$. We obtain

$$\begin{align*}\hfill \mathcal{R}\left(x\right)-\mathcal{R}\left(z\right)& ={T}_{\alpha }\left(x,Ax\right)-{T}_{\alpha }\left({x}_{\alpha },Ax\right)+{T}_{\alpha }\left({x}_{\alpha },Ax\right)-\mathcal{R}\left(z\right)\hfill \\ \hfill & {\leqslant}{\sigma }_{x}\left(\alpha \right)+\frac{1}{2\alpha }{\Vert}Ax-Az{{\Vert}}_{\mathbb{Y}}^{2}.\hfill \end{align*}$$

Taking the infimum over α on the right-hand side yields (5.4).

Hence (a) implies (c). Assuming (c) we estimate

$$\begin{align*}\hfill {\sigma }_{x}\left(\alpha \right)& {\leqslant}\phi \left({\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\right)-\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\hfill \\ \hfill & {\leqslant}{\mathrm{sup}}_{t{\geqslant}0}\left(\phi \left(t\right)-\frac{1}{2\alpha }t\right)=\left({\mathcal{F}}^{-1}\left(\phi \right)\right)\left(\alpha \right).\hfill \end{align*}$$

Hence ${\sigma }_{x}{\leqslant}{\mathcal{F}}^{-1}\left(\phi \right)$. This yields (a) as $\mathcal{F}$ is order preserving.□

Remark 5.6. Inequality (5.4) is sharp for z = x_α ∈ R_α (g) for all α > 0. To see this note that by definition $\left(\mathcal{F}\left({\sigma }_{x}\right)\right)\left(t\right){\leqslant}{\sigma }_{x}\left(\alpha \right)+\frac{1}{2\alpha }t$ for all t ⩾ 0 and α > 0. By (5.4) we have

$$\begin{align*}\hfill \mathcal{R}\left(x\right)-\mathcal{R}\left({x}_{\alpha }\right)& {\leqslant}\left(\mathcal{F}\left({\sigma }_{x}\right)\right)\left({\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\right)\hfill \\ \hfill & {\leqslant}{\sigma }_{x}\left(\alpha \right)+\frac{1}{2\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\hfill \\ \hfill & =\mathcal{R}\left(x\right)-\mathcal{R}\left({x}_{\alpha }\right).\hfill \end{align*}$$

The result of this subsection is a that σ_x and hence also the smallest index function ϕ allowing for a variational source condition (2.8) depend only on the net ${\left({\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}\right)}_{\alpha { >}0}$. Further we will exploit a condition when a bound ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\psi \left(\alpha \right)$ implies a bound on the defect function σ_x .

Lemma 5.7. We have

$$\begin{equation}{\sigma }_{x}\left(\alpha \right)={\int }_{0}^{\alpha }\frac{1}{2{\beta }^{2}}{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}^{2}\enspace \mathrm{d}\beta \quad \text{for}\enspace \text{all}\enspace \alpha { >}0.\end{equation} \tag{ 5.5 }$$

Proof. Let 0 < ɛ < α. Lemma 5.4(a) yields

$$\begin{equation*}{\sigma }_{x}\left(\alpha \right)-{\sigma }_{x}\left(\varepsilon \right)={\int }_{{\epsilon}}^{\alpha }{\sigma }_{x}^{\prime }\left(\beta \right)\enspace \mathrm{d}\beta ={\int }_{{\epsilon}}^{\alpha }\frac{1}{2{\beta }^{2}}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}\enspace \mathrm{d}\beta .\end{equation*}$$

In view of lemma 5.4(b) the expression for σ_x follows by taking the limit ɛ → 0.□

Proposition 5.8 (Image space approximation).

• (a)  
  We have

  $$\begin{equation*}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\sqrt{2\alpha {\sigma }_{x}\left(\alpha \right)}\quad \text{for}\enspace \text{all}\enspace \alpha { >}0.\end{equation*}$$
• (b)  
  Let $\psi :\left[\right.0,\infty \left.\right)\to \left[\right.0,\infty \left.\right)$ be continuous. Assume that there is a constant C_ψ > 0 with

  $$\begin{equation}{\int }_{0}^{\alpha }\frac{1}{\beta }\psi \left(\beta \right)\enspace \mathrm{d}\beta {\leqslant}{C}_{\psi }\psi \left(\alpha \right)\quad \text{for}\enspace \text{all}\enspace \alpha { >}0.\end{equation} \tag{ 5.6 }$$

  Then a bound ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}{\leqslant}\sqrt{2\alpha \psi \left(\alpha \right)}$ for all α > 0 implies σ_x (α) ⩽ C_ψ ψ(α) for all α > 0.

Proof. 

• (a)  
  By lemma 5.4 the continuous extension of σ_x to $\left[\right.0,\infty \left.\right)$ is concave. Hence the claim follows from

  $$\begin{equation*}\frac{1}{2{\alpha }^{2}}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}={\sigma }_{x}^{\prime }\left(\alpha \right){\leqslant}\frac{1}{\alpha }{\sigma }_{x}\left(\alpha \right).\end{equation*}$$
• (b)  
  Using (5.5) and (5.6) we obtain

  $$\begin{equation*}{\sigma }_{x}\left(\alpha \right)={\int }_{0}^{\alpha }\frac{1}{2{\beta }^{2}}{\Vert}Ax-A{x}_{\beta }{{\Vert}}_{\mathbb{Y}}^{2}\enspace \mathrm{d}\beta {\leqslant}{\int }_{0}^{\alpha }\frac{1}{\beta }\psi \left(\beta \right)\enspace \mathrm{d}\beta {\leqslant}{C}_{\psi }\psi \left(\alpha \right).\end{equation*}$$

□

Proof of theorem 3. (a) ⇒ (b). Consider the continuous function

$$\begin{equation*}\psi :\left[\right.0,\infty \left.\right)\to \left[\right.0,\infty \left.\right)\quad \;\text{given}\;\text{by}\enspace \psi \left(\alpha \right)=\frac{1}{2}{c}_{1}^{2}{\alpha }^{2\nu -1}.\end{equation*}$$

Then ${c}_{1}{\alpha }^{\nu }=\sqrt{2\alpha \psi \left(\alpha \right)}$ for all α > 0. We have

$$\begin{equation*}{\int }_{0}^{\alpha }\frac{1}{\beta }\psi \left(\beta \right)\enspace \mathrm{d}\beta =\frac{1}{2}{c}_{1}^{2}{\int }_{0}^{\alpha }{\beta }^{2\nu -2}\enspace \mathrm{d}\beta =\frac{1}{2\nu -1}\psi \left(\alpha \right).\end{equation*}$$

Hence (5.6) is satisfied with ${C}_{\psi }=\frac{1}{2\nu -1}$. Proposition 5.8 implies ${\sigma }_{x}\left(\alpha \right){\leqslant}\frac{{c}_{1}^{2}}{4\nu -2}{\alpha }^{2\nu -1}$.

(b) ⇒ (c): for σ(α) := c₂ α^2ν−1 inserting $\alpha ={\left(\frac{t}{2c}\right)}^{\frac{1}{2\nu }}$ yields

$$\begin{equation*}\left(\mathcal{F}\left(\sigma \right)\right)\left(t\right)={\mathrm{inf}}_{\alpha { >}0}\left({c}_{2}{\alpha }^{2\nu -1}+\frac{1}{2\alpha }t\right){\leqslant}{2}^{\frac{1}{2\nu }}{c}_{2}^{\frac{1}{2\nu }}{t}^{\frac{2\nu -1}{2\nu }}{\leqslant}2{c}_{2}^{\frac{1}{2\nu }}{t}^{\frac{2\nu -1}{2\nu }}.\end{equation*}$$

Lemma 5.5 with $\phi =\mathcal{F}\left(\sigma \right)$ yields the claim.

(c) ⇒ (a): (see also [16, proof of proposition 4]) the first order condition

$$\begin{equation*}{\xi }_{\alpha }{:=}\frac{1}{\alpha }{A}^{{\ast}}A\left(x-{x}_{\alpha }\right)\in \partial \mathcal{R}\left({x}_{\alpha }\right)\end{equation*}$$

provides

$$\begin{equation*}\frac{1}{\alpha }{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{2}=\langle {\xi }_{\alpha },x-{x}_{\alpha }\rangle {\leqslant}\mathcal{R}\left(x\right)-\mathcal{R}\left({x}_{\alpha }\right){\leqslant}{c}_{3}{\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}^{\frac{2\nu -1}{\nu }}.\end{equation*}$$

Solving for ${\Vert}Ax-A{x}_{\alpha }{{\Vert}}_{\mathbb{Y}}$ yields the claim.□