Higher-order total variation approaches and generalisations

Generally, when solving a specific ill-posed inverse problem with, for instance, Tikhonov regularisation, one usually has many choices regarding the regularisation functional. Now, while functionals associated with Hilbertian norms or seminorms possess several advantages such as smoothness and allow, in addition, for regularisation strategies that can be computed by solving a linear equation, they are often not able to provide a good model for piecewise smooth functions. This can, for instance, be illustrated as follows.

Example 2.1. Classical Sobolev spaces cannot contain non-trivial piecewise constant functions. Let Ω ⊂ R^d be a domain and Ω' ⊂ Ω be non-empty, open with ∂Ω' a null set. Then, the characteristic function u = χ_Ω', i.e., u(x) = 1 if x ∈ Ω' and 0 otherwise, is not contained in H^1,p (Ω) for any p ∈ [1, ∞]. To see this, suppose that v ∈ L^p (Ω, R^d ) is the weak derivative of u. Let $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({{\Omega}}^{\prime }\right)$ be a test function. Clearly,

$$\begin{equation*}{\int }_{{\Omega}}v\cdot \varphi \enspace \mathrm{d}x=-{\int }_{{\Omega}}u\enspace \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x=-{\int }_{{{\Omega}}^{\prime }}\mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x=0.\end{equation*}$$

Hence, v = 0 on Ω'. Likewise, one sees that also v = 0 on ${\Omega}{\backslash}\overline{{{\Omega}}^{\prime }}$. In total, v = 0 almost everywhere and as v is the weak derivative of u, u must be constant which is a contradiction.

The defect which is responsible for the failure of characteristic functions being (classical) Sobolev functions can, however, be remedied by allowing weak derivatives to be Radon measures. These are in particular able to concentrate on Lebesgue null-sets; a property that is necessary as the previous example just showed. In the following, we introduce some basic notions and results about vector-valued Radon measures, in particular, with an eye of embedding them into a functional-analytic framework. Moreover, we would like to have these notions readily available when dealing with higher-order derivatives and the associated higher-order total variation.

Throughout this section, let Ω ⊂ R^d be a domain and H a non-trivial finite-dimensional real Hilbert space with ⋅ and $\left\vert \cdot \right\vert$ denoting the associated scalar product and norm, respectively. As usual, the case H = R corresponds to the scalar case and H = R^d to the vector-field case, but, as we will see later, H could also be a space of higher-order tensors. The following definitions and statements regarding basic measure theory and can, for instance, be found in [7].

Definition 2.2. A vector-valued Radon measure or H -valued Radon measure on Ω is a function $\mu :\mathcal{B}\left({\Omega}\right)\to H$ on the Borel σ-algebra $\mathcal{B}\left({\Omega}\right)$ associated with the standard topology on Ω satisfying the following properties:

• (a)  
  It holds that μ(Ø) = 0,
• (b)  
  For each pairwise disjoint countable collection A₁, A₂, .... in $\mathcal{B}\left({\Omega}\right)$ it holds that $\mu \left({\bigcup }_{i\in \mathbf{N}}{A}_{i}\right)={\sum }_{i=1}^{\infty }\mu \left({A}_{i}\right)$ in H.

A positive Radon measure is a function $\mu :\mathcal{B}\left({\Omega}\right)\to \left[0,\infty \right]$ satisfying (a), (b) (with H replaced by [0, ∞]) as well as μ(K) < ∞ for each compact K ⊂⊂ Ω. It is called finite, if μ(Ω) < ∞.

Naturally, vector-valued Radon measures can be associated to an integral. For μ an H-valued Radon measure and step functions $u={\sum }_{j=1}^{N}{c}_{j}{\chi }_{{A}_{j}}$, $v={\sum }_{j=1}^{N}{v}_{j}{\chi }_{{A}_{j}}$ with c₁, ..., c_N ∈ R, v₁, ..., v_N ∈ H and ${A}_{1},\dots ,{A}_{N}\in \mathcal{B}\left({\Omega}\right)$, the following integrals make sense:

$$\begin{equation*}{\int }_{{\Omega}}u\enspace \mathrm{d}\mu =\sum _{j=1}^{N}{c}_{j}\mu \left({A}_{j}\right)\in H,\quad {\int }_{{\Omega}}v\cdot \enspace \mathrm{d}\mu =\sum _{j=1}^{N}{v}_{j}\cdot \mu \left({A}_{j}\right)\in \mathbf{R}.\end{equation*}$$

For uniformly continuous functions $u:\overline{{\Omega}}\to \mathbf{R}$ and $v:\overline{{\Omega}}\to H$, the integrals are given as

$$\begin{equation*}{\int }_{{\Omega}}u\enspace \mathrm{d}\mu =\underset{n\to \infty }{\mathrm{lim}}{\int }_{{\Omega}}{u}^{n}\enspace \mathrm{d}\mu ,\quad {\int }_{{\Omega}}v\cdot \enspace \mathrm{d}\mu =\underset{n\to \infty }{\mathrm{lim}}{\int }_{{\Omega}}{v}^{n}\cdot \enspace \mathrm{d}\mu \end{equation*}$$

where {uⁿ } and {vⁿ } are sequences of step functions converging uniformly to u and v, respectively. Of course, the above integrals are well-defined, meaning that there are approximating sequences as stated and the above limits exist independently of the specific choice of the approximating sequences. The following definition is the basis for introducing a norm for H-valued Radon measures.

Definition 2.3. For a vector-valued Radon measure μ on Ω the positive Radon measure $\left\vert \mu \right\vert$ given by

$$\begin{equation*}\left\vert \mu \right\vert \left(A\right)=\mathrm{sup}\enspace \left\{\sum _{i=1}^{\infty }\left\vert \mu \left({A}_{i}\right)\right\vert \enspace \vert \enspace {A}_{1},{A}_{2},\enspace \dots \in \mathcal{B}\left({\Omega}\right)\enspace \text{pairwise}\;\text{disjoint},\enspace {A}_{i}\subset A\enspace \text{for}\;\text{all}\enspace i\in \mathbf{N}\right\}\end{equation*}$$

is called the total-variation measure of μ.

The total-variation measure is always positive and finite, i.e., $0{\leqslant}\left\vert \mu \right\vert \left(A\right){< }\infty$ for all $A\in \mathcal{B}\left({\Omega}\right)$. By construction, μ is absolutely continuous with respect to $\left\vert \mu \right\vert$, i.e., μ(A) = 0 whenever $\left\vert \mu \right\vert \left(A\right)=0$ for a $A\in \mathcal{B}\left({\Omega}\right)$. By Radon–Nikodým's theorem, we thus have that each H-valued Radon measure μ can be written as $\mu ={\sigma }_{\mu }\left\vert \mu \right\vert$ with ${\sigma }_{\mu }\in {L}_{\left\vert \mu \right\vert }^{\infty }\left({\Omega},H\right)$ such that ${{\Vert}{\sigma }_{\mu }{\Vert}}_{\infty }{\leqslant}1$ and $\left\vert {\sigma }_{\mu }\right\vert =1$ almost everywhere with respect to $\left\vert \mu \right\vert$. In this light, integration can also be phrased as

$$\begin{equation*}{\int }_{{\Omega}}u\enspace \mathrm{d}\mu ={\int }_{{\Omega}}u{\sigma }_{\mu }\enspace \mathrm{d}\left\vert \mu \right\vert ,\quad {\int }_{{\Omega}}v\cdot \enspace \mathrm{d}\mu ={\int }_{{\Omega}}v\cdot {\sigma }_{\mu }\enspace \mathrm{d}\left\vert \mu \right\vert \end{equation*}$$

for $u:\overline{{\Omega}}\to \mathbf{R}$, $v:\overline{{\Omega}}\to H$ uniformly continuous. The following theorem, which is a direct consequence of [163, theorem 6.19], provides a useful characterisation of the space of vector valued measures as the dual of a separable space.

Proposition 2.4. The space $\mathcal{M}\left({\Omega},H\right)$ of all vector-valued Radon measures equipped with the norm ${{\Vert}\mu {\Vert}}_{\mathcal{M}}=\left\vert \mu \right\vert \left({\Omega}\right)$ for $\mu \in \mathcal{M}\left({\Omega},H\right)$ is a Banach space.

It can be identified with the dual space ${\mathcal{C}}_{0}{\left({\Omega},H\right)}^{{\ast}}$ as follows. For each $T\in {\mathcal{C}}_{0}{\left({\Omega},H\right)}^{{\ast}}$ there exists a unique $\mu \in \mathcal{M}\left({\Omega},H\right)$ such that

$$\begin{equation*}{{\Vert}T{\Vert}}_{{\mathcal{C}}_{0}^{{\ast}}}={{\Vert}\mu {\Vert}}_{\mathcal{M}},\quad T\left(\varphi \right)={\int }_{{\Omega}}\varphi \cdot \enspace \mathrm{d}\mu \quad \text{for}\enspace \text{all}\quad \varphi \in {\mathcal{C}}_{0}\left({\Omega},H\right).\end{equation*}$$

In particular, one has a notion of weak^*-convergence of Radon measures. For a sequence {μⁿ } and an element μ* in $\mathcal{M}\left({\Omega},H\right)$ we have that ${\mu }^{n}{\ast}{\rightharpoonup }{\mu }^{{\ast}}$ in $\mathcal{M}\left({\Omega},H\right)$ if

$$\begin{equation*}\text{for}\;\text{all}\quad \varphi \in {\mathcal{C}}_{0}\left({\Omega},H\right):\quad {\int }_{{\Omega}}\varphi \cdot \enspace \mathrm{d}{\mu }^{n}\to {\int }_{{\Omega}}\varphi \cdot \enspace \mathrm{d}{\mu }^{{\ast}}\quad \text{as}\quad n\to \infty .\end{equation*}$$

As the predual space ${\mathcal{C}}_{0}\left({\Omega},H\right)$ is separable, the Banach–Alaoglu theorem yields in particular the sequential relative weak^*-compactness of bounded sets. That means for instance that a bounded sequence always admits a weakly^*-convergent subsequence, a property that may compensate for the lack of reflexivity of $\mathcal{M}\left({\Omega},H\right)$.

The interpretation as a dual space as well as the density of test functions in ${\mathcal{C}}_{0}\left({\Omega},H\right)$ also allows to conclude that in order for a linear functional T defining a Radon measure, it suffices to test against $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},H\right)$ and to establish $\left\vert T\left(\varphi \right)\right\vert {\leqslant}C{{\Vert}\varphi {\Vert}}_{\infty }$ for all $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},H\right)$ and C > 0 independent of φ. This is useful for derivatives, i.e., the derivative of a $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},H\right)$ defines a Radon measure in $\mathcal{M}\left({\Omega},{H}^{d}\right)$ if

$$\begin{equation}\left\vert {\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x\right\vert {\leqslant}C{{\Vert}\varphi {\Vert}}_{\infty }\quad \text{for}\enspace \text{all}\quad \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{H}^{d}\right).\end{equation} \tag{ 1 }$$

In this case, we denote by $\nabla u\in \mathcal{M}\left({\Omega},{H}^{d}\right)$ the unique H^d -valued Radon measure for which ${\int }_{{\Omega}}$ φ ⋅ d∇u = −${\int }_{{\Omega}}$ u  div  φ dx for all $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{H}^{d}\right)$. Here, H^d is equipped with the scalar product $x\cdot y={\sum }_{i=1}^{d}{x}_{i}\cdot {y}_{i}$ for x, y ∈ H^d . In the case where (1) fails, there exists a sequence {φⁿ } in ${\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathbf{R}}^{d}\right)$ with ${{\Vert}{\varphi }^{n}{\Vert}}_{\infty }=1$ and $\left\vert {\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x\right\vert \to \infty$ as n → ∞. Thus, allowing the supremum to take the value ∞, this yields following definition.

Definition 2.5. The total variation of a $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},H\right)$ is the value

$$\begin{equation*}\mathrm{T}\mathrm{V}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{H}^{d}\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty }{\leqslant}1\right\}.\end{equation*}$$

Clearly, in case TV(u) < ∞, we have $\nabla u\in \mathcal{M}\left({\Omega},{H}^{d}\right)$ with ${{\Vert}\nabla u{\Vert}}_{\mathcal{M}}=\mathrm{T}\mathrm{V}\left(u\right)$. Trivially, for scalar functions, i.e., H = R, one recovers the well-known definition [7, 162]. Also, one immediately sees that TV is invariant to translations and rotations, or, more generally, to Euclidean-distance preserving transformations. This is the reason that this definition is also referred to as the isotropic total variation.

Example 2.6. Piecewise constant functions may have a Radon measure as derivative. Let Ω' ⊂ Ω be a subdomain such that ∂Ω' ∩ Ω can be parameterised by finitely many Lipschitz mappings. Then, the outer normal ν exists almost everywhere in ∂Ω' ∩ Ω with respect to the Hausdorff ${\mathcal{H}}^{d-1}$ measure and one can employ the divergence theorem. This yields, for u = χ_Ω' and $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathbf{R}}^{d}\right)$ with ||φ||_∞ ⩽ 1 that

$$\begin{equation*}{\int }_{{\Omega}}u\enspace \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x={\int }_{\partial {{\Omega}}^{\prime }\cap {\Omega}}\varphi \cdot \nu \enspace \mathrm{d}{\mathcal{H}}^{d-1}={\int }_{{\Omega}}\varphi \cdot \enspace \mathrm{d}\nu {\mathcal{H}}^{d-1}\enspace \llcorner \enspace \left(\partial {{\Omega}}^{\prime }\cap {\Omega}\right){\leqslant}{\mathcal{H}}^{d-1}\left(\partial {{\Omega}}^{\prime }\cap {\Omega}\right)\end{equation*}$$

so $\nabla u=-\nu {\mathcal{H}}^{d-1}\enspace \llcorner \enspace \left(\partial {{\Omega}}^{\prime }\cap {\Omega}\right)$ is a Radon measure. One sees, for instance via approximation, that ${{\Vert}\nabla u{\Vert}}_{\mathcal{M}}={\mathcal{H}}^{d-1}\left(\partial {{\Omega}}^{\prime }\cap {\Omega}\right)$.

The class of sets Ω' ⊂ Ω for which χ_Ω' possesses a Radon measure as weak derivative is actually much greater than the class of bounded Lipschitz domains. These are the sets of finite perimeter, denoted by $\mathrm{P}\mathrm{e}\mathrm{r}\left({{\Omega}}^{\prime }\right)={{\Vert}\nabla {\chi }_{{{\Omega}}^{\prime }}{\Vert}}_{\mathcal{M}}$. One the other hand, for u ∈ H^1,1(Ω), we have $\mathrm{T}\mathrm{V}\left(u\right)={\int }_{{\Omega}}\left\vert \nabla u\right\vert \enspace \mathrm{d}x$ and the weak derivative as Radon measure is just $\nabla u{\mathcal{L}}^{d}$, i.e., the Sobolev derivative interpreted as a weight on the Lebesgue measure. Collecting all functions whose weak derivative is a Radon measure, we arrive at the following space.

Definition 2.7. The space

$$\begin{equation*}\mathrm{B}\mathrm{V}\left({\Omega},H\right)=\left\{u\in {L}^{1}\left({\Omega},H\right)\enspace \vert \enspace \mathrm{T}\mathrm{V}\left(u\right){< }\infty \right\},\quad {{\Vert}u{\Vert}}_{\mathrm{B}\mathrm{V}}={{\Vert}u{\Vert}}_{1}+\mathrm{T}\mathrm{V}\left(u\right)\end{equation*}$$

is the space of H-valued functions of bounded variation. In case H = R, we denote by BV(Ω) = BV(Ω, R) and just refer to functions of bounded variation.

Proposition 2.8. The space BV(Ω, H) with the associated norm is a Banach space. The total variation functional TV is a continuous seminorm on BV(Ω, H) which vanishes exactly at the constant functions, i.e., ker(TV) = H 1, with H 1 being the set of constant, H-valued functions.

The total variation functional is just designed to possess many convenient properties [7].

Proposition 2.9. 

• The functional TV is proper, convex and lower semi-continuous on each L^p (Ω, H), i.e., for 1 ⩽ p ⩽ ∞.
• For 1 ⩽ p < ∞, each u ∈ BV(Ω, H) ∩ L^p (Ω, H) can smoothly be approximated as follows: for ɛ > 0, there exists ${u}^{\varepsilon }\in {\mathcal{C}}^{\infty }\left({\Omega},H\right)\cap \mathrm{B}\mathrm{V}\left({\Omega},H\right)\cap {L}^{p}\left({\Omega},H\right)$ such that
  $$\begin{equation*}{{\Vert}u-{u}^{\varepsilon }{\Vert}}_{p}{\leqslant}\varepsilon ,\quad \left\vert \mathrm{T}\mathrm{V}\left(u\right)-\mathrm{T}\mathrm{V}\left({u}^{\varepsilon }\right)\right\vert {\leqslant}\varepsilon .\end{equation*}$$
• If Ω is a bounded Lipschitz domain, then there exists a constant C > 0 such that for each u ∈ BV(Ω, H) with ${\int }_{{\Omega}}$ u  dx = 0, the Poincaré–Wirtinger estimate
  $$\begin{equation*}{{\Vert}u{\Vert}}_{d/\left(d-1\right)}{\leqslant}C\mathrm{T}\mathrm{V}\left(u\right)\end{equation*}$$
  holds.

From the regularisation-theoretic point of view, the fact that TV is proper, convex and lower semi-continuous on Lebesgue spaces is relevant, a property that fails for the Sobolev-seminorm ||∇ ⋅ ||₁. The Poincaré–Wirtinger estimate can be interpreted as a coercivity property on a subspace with codimension 1. Also note that this estimate is the same as for H^1,1(Ω, H)-functions and the respective constants C coincide. Consequently, the embedding properties of the latter space transfer immediately.

Proposition 2.10. Let Ω is a bounded Lipschitz domain. Then,

• The embedding BV(Ω, H) ↪ L^d/(d−1)(Ω, H) (with d/(d − 1) = ∞ for d = 1) exists and is continuous.
• The embedding BV(Ω, H) ↪ L^p (Ω, H) is compact for each 1 ⩽ p < d/(d − 1).
• Each bounded sequence {uⁿ } in BV(Ω, H) possesses a subsequence $\left\{{u}^{{n}_{k}}\right\}$ which converges to a u ∈ BV(Ω, H) weak^* in BV(Ω, H), which we define as ${u}^{{n}_{k}}\to u$ in L¹(Ω, H), $\nabla {u}^{{n}_{k}}{\ast}{\rightharpoonup }\nabla u$ in $\mathcal{M}\left({\Omega},{H}^{d}\right)$ as k → ∞.

Consequently, the total variation is suitable for regularising ill-posed inverse problems in certain L^p -spaces.

Let us now turn to solving ill-posed inverse problems with Tikhonov regularisation and BV-based penalty, i.e., solving

$$\begin{equation*}Ku=f\end{equation*}$$

for some data f in a Banach space Y. As mentioned in the introduction, since the focus of this review is on regularisation terms rather than tackling inverse problems in the most possible generality, we restrict ourselves here to linear and continuous forward operators K : L^d/(d−1)(Ω) → Y. Nevertheless we note that, building on the results developed here for the linear setting, an extension to non-linear operators typically boils down to ensuring additional requirements on the non-linear forward model rather than the regularisation term, see for instance [84, 106, 182].

Measuring the discrepancy in terms of the norm in Y, the problem is then to solve

$$\begin{equation*}\underset{u\in \mathrm{B}\mathrm{V}\left({\Omega}\right)}{\mathrm{min}}\enspace \frac{{{\Vert}Ku-f{\Vert}}_{Y}^{q}}{q}+\alpha {\int }_{{\Omega}}\enspace \mathrm{d}\left\vert \nabla u\right\vert \end{equation*}$$

for some exponent q ⩾ 1. Usually, Y is some Hilbert space and q = 2, resulting in a quadratic discrepancy, which is often used in case of Gaussian noise. For impulsive noise (or salt-and-pepper noise), the space Y = L¹(Ω'), with Ω' a domain, turns out to be useful. In case of Poisson noise, however, it is not advisable to take the norm but rather the Kullback–Leibler divergence between Ku and f, i.e. KL(Ku, f), where KL is given, for f ∈ L¹(Ω') with f ⩾ 0 almost everywhere, according to the non-negative integral

$$\begin{equation}\mathrm{K}\mathrm{L}\left(v,f\right)={\int }_{{{\Omega}}^{\prime }}f\left(\frac{v}{f}-\mathrm{log}\left(\frac{v}{f}\right)-1\right)\enspace \mathrm{d}x\end{equation} \tag{ 2 }$$

provided that v ⩾ 0 a.e., and ∞ else. In particular, in this context, we agree to set the integrand to v where f = 0 and to ∞ where v = 0 and f > 0.

In the following, we assume to have given a discrepancy functional S_f : Y → [0, ∞] that is proper, convex, lower semi-continuous and coercive. This is not the most general case but will be sufficient for us in order to ensure existence of minimisers of the Tikhonov functional.

Theorem 2.11. Let Ω be a bounded Lipschitz domain, Y be a Banach space, K : L^d/(d−1)(Ω) → Y linear and continuous (weak^*-to-weak-continuous in case d = 1), S_f : Y → [0, ∞] a proper, convex, lower semi-continuous and coercive discrepancy functional associated with some data f and α > 0. Then, there exist solutions of

$$\begin{equation}\underset{u\in {L}^{d/\left(d-1\right)}\left({\Omega}\right)}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+\alpha \mathrm{T}\mathrm{V}\left(u\right).\end{equation} \tag{ 3 }$$

If S_f is strictly convex and K is injective, the solution is unique whenever the minimum is finite.

We provide the proof for the sake of completeness and as a prototype for the generalisation to higher-order functionals.

Proof. Assume that the objective functional in (3) is proper, otherwise, there is nothing to show. For a minimising sequence {uⁿ }, the Poincaré–Wirtinger inequality gives boundedness of $\left\{{u}^{n}-{\left\vert {\Omega}\right\vert }^{-1}{\int }_{{\Omega}}{u}^{n}\enspace \mathrm{d}x\right\}$ in L^d/(d−1)(Ω) while the coercivity of S_f yields the boundedness of {Kuⁿ }. By continuity, $\left\{K\left({u}^{n}-{\left\vert {\Omega}\right\vert }^{-1}{\int }_{{\Omega}}{u}^{n}\enspace \mathrm{d}x\right)\right\}$ must be bounded, so if K 1 ≠ 0, then {${\int }_{{\Omega}}$ uⁿ   dx} is bounded as otherwise, {Kuⁿ } would be unbounded. In the case that K 1 = 0, we can without loss of generality assume that ${\int }_{{\Omega}}$ uⁿ   dx = 0 for all n as shifting along constants does not change the functional value. In each case, {${\int }_{{\Omega}}$ uⁿ   dx} is bounded, so {uⁿ } must be bounded in L^d/(d−1)(Ω). Hence, by compact embedding (proposition 2.10) we have ${u}^{{n}_{k}}\to {u}^{{\ast}}$ in L¹(Ω) as k → ∞ for a subsequence $\left\{{u}^{{n}_{k}}\right\}$ and u* ∈ BV(Ω). Reflexivity and continuity of K (weak^* sequential compactness and weak^*-to-weak continuity in case d = 1) give $K{u}^{{n}_{k}}\rightharpoonup K{u}^{{\ast}}$ in Y for another subsequence (not relabelled). By lower semi-continuity, u* has to be a solution to (3).

Finally, if S_f is strictly convex and K is injective, then S_f ◦ K is already strictly convex, so minimisers have to be unique. □

Example 2.12. 

• The discrepancy functional ${S}_{f}\left(v\right)=\frac{1}{q}{{\Vert}v-f{\Vert}}_{Y}^{q}$ for some f ∈ Y is obviously proper, convex, lower semi-continuous and coercive.
• It follows from lemma A.1 in the appendix that the discrepancy S_f (v) = KL(v, f) defined on Y = L¹(Ω') for f ∈ L¹(Ω') with f ⩾ 0 almost everywhere is proper, convex and coercive in L¹(Ω'). Lower semi-continuity in turn follows as special case of lemma A.2.

Remark 2.13. Note that if the inversion of K : L^p (Ω) → Y is well-posed for some p ∈ [1, ∞], then solutions of (3) still exist (even for α = 0). Clearly, the TV penalty is not necessary for obtaining a regularising effect for these problems. In this case, minimising the Tikhonov function with TV penalty may the interpreted as denoising. The most prominent example might be the Rudin–Osher–Fatemi problem [162] which reads as

$$\begin{equation*}\underset{u\in {L}^{2}\left({\Omega}\right)}{\mathrm{min}}\enspace \frac{1}{2}{\int }_{{\Omega}}{\left\vert u-f\right\vert }^{2}\enspace \mathrm{d}x+\alpha \mathrm{T}\mathrm{V}\left(u\right)\end{equation*}$$

for f ∈ L²(Ω). Here, as the identity is 'inverted', the effect of total-variation regularisation can be studied in detail. Minimisation problems of this type with other regularisation functionals are thus a good benchmark test for the properties of this functional.

The stability of solutions in case of varying f depends, of course, on the dependence of S_f on f. The appropriate notion here is the convergence of the discrepancy functional, i.e., for a sequence {fⁿ } and limit f, we say that ${S}_{{f}^{n}}$ converges to S_f if

$$\begin{equation}\left\{\begin{array}{cccc}{S}_{f}\left(v\right)\hfill & {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace {S}_{{f}^{n}}\left({v}^{n}\right)\hfill & \quad \text{whenever}\quad \hfill & {v}^{n}\rightharpoonup v\enspace \text{in}\enspace Y,\hfill \\ {S}_{f}\left(v\right)\hfill & {\geqslant}\underset{n\to \infty }{\text{lim sup}}\enspace {S}_{{f}^{n}}\left(v\right)\hfill & \quad \text{for}\;\text{each} \hfill & v\in Y.\hfill \end{array}\right.\end{equation} \tag{ 4 }$$

Moreover, we say that $\left\{{S}_{{f}^{n}}\right\}$ is equi-coercive if there is a coercive function S₀ : Y → [0, ∞] such that ${S}_{{f}^{n}}{\geqslant}{S}_{0}$ in Y for each n.

Theorem 2.14. In the situation of theorem 2.11, assume that ${S}_{{f}^{n}}$ converges to S_f in the sense of (4) and $\left\{{S}_{{f}^{n}}\right\}$ is equi-coercive. Then, for each sequence of minimisers {uⁿ } of (3) with discrepancy ${S}_{{f}^{n}}$,

• Either ${S}_{{f}^{n}}\left(K{u}^{n}\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{n}\right)\to \infty$ as n → ∞ and (3) with discrepancy f does not admit a finite solution.
• Or ${S}_{{f}^{n}}\left(K{u}^{n}\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{n}\right)\to {\mathrm{min}}_{u\in {L}^{d/\left(d-1\right)}\left({\Omega}\right)}{S}_{f}\left(u\right)+\alpha \mathrm{T}\mathrm{V}\left(u\right)$ as n → ∞ and there is, possibly up to constant shifts, a weak accumulation point u ∈ L^d/(d−1)(Ω) (weak^* accumulation point for d = 1) that minimises (3) with discrepancy S_f .

For each subsequence $\left\{{u}^{{n}_{k}}\right\}$ weakly converging to some u in L^d/(d−1)(Ω) (${u}^{{n}_{k}}{\ast}{\rightharpoonup }u$ in case d = 1), it holds that $\mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right)\to \mathrm{T}\mathrm{V}\left(u\right)$ as k → ∞ and u solves (3) with discrepancy S_f . If solutions to the latter are unique, we have uⁿ ⇀ u in L^d/(d−1)(Ω) (${u}^{n}{\ast}{\rightharpoonup }u$ in case d = 1).

Proof. Let, in the following ${\int }_{{\Omega}}$ uⁿ   dx = 0 for all n if K 1 = 0 and denote by F = S_f ◦ K + αTV as well as ${F}_{n}={S}_{{f}^{n}}\enspace {\circ}\enspace K+\alpha \mathrm{T}\mathrm{V}$. First of all, suppose that {F_n (uⁿ )} is bounded. As $\left\{{S}_{{f}^{n}}\right\}$ is equi-coercive, we can conclude as in the proof of theorem 2.11 that {uⁿ } is bounded. Therefore, a weak accumulation point (weak^* in case d = 1) exists.

Suppose that ${u}^{{n}_{k}}\rightharpoonup u$ as k → ∞. Then,

$$\begin{equation*}{S}_{f}\left(Ku\right){\leqslant}\underset{k\to \infty }{\text{lim inf}}\enspace {S}_{{f}^{{n}_{k}}}\left(K{u}^{{n}_{k}}\right),\quad \mathrm{T}\mathrm{V}\left(u\right){\leqslant}\underset{k\to \infty }{\text{lim inf}}\enspace \mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right)\end{equation*}$$

as well as, for each u' ∈ L^d/(d−1)(Ω)

$$\begin{align}\hfill F\left(u\right)& {\leqslant}\underset{k\to \infty }{\text{lim inf}}{S}_{{f}^{{n}_{k}}}\left(K{u}^{{n}_{k}}\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right)\\ \hfill & \hfill {\leqslant}\underset{k\to \infty }{\text{lim sup}}{S}_{{f}^{{n}_{k}}}\left(K{u}^{\prime }\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{\prime }\right){\leqslant}F\left({u}^{\prime }\right).\end{align}$$

Thus, u is a minimiser for F and plugging in u' = u we see that ${\mathrm{lim}}_{k\to \infty }{F}_{{n}_{k}}\left({u}^{{n}_{k}}\right)=F\left(u\right)$. In order to obtain ${\mathrm{lim}}_{k\to \infty }\mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right)=\mathrm{T}\mathrm{V}\left(u\right)$, suppose that ${\text{lim sup}}_{k\to \infty }\mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right){ >}\mathrm{T}\mathrm{V}\left(u\right)$, such that

$$\begin{equation*}\underset{k\to \infty }{\text{lim inf}}\enspace {S}_{{f}^{{n}_{k}}}\left(K{u}^{{n}_{k}}\right){\leqslant}\underset{k\to \infty }{\,\mathrm{lim}}{F}_{{n}_{k}}\left({u}^{{n}_{k}}\right)-\alpha \underset{k\to \infty }{\text{lim sup}}\enspace \mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right){< }{S}_{f}\left(Ku\right)\end{equation*}$$

which is a contradiction. Thus, ${\mathrm{lim}}_{k\to \infty }\mathrm{T}\mathrm{V}\left({u}^{{n}_{k}}\right)=\mathrm{T}\mathrm{V}\left(u\right)$. Finally, if u is the unique minimiser for (3) with discrepancy S_f , then uⁿ ⇀ u as n → ∞ for the whole sequence (${u}^{n}{\ast}{\rightharpoonup }u$ in case d = 1) as any subsequence has to contain another subsequence that converges weakly (weakly^*) to u.

In order to conclude the proof, suppose that lim inf_n→∞ F_n (uⁿ ) < ∞. In that case, the above arguments yield an accumulation point as stated as well as a minimiser u ∈ BV(Ω) of F with F(u) ⩽ liminf_n→∞ F_n (uⁿ ). In particular, F is proper. By convergence of ${S}_{{f}^{n}}$ to S_f and minimality, we have

$$\begin{equation*}F\left(u\right){\geqslant}\underset{n\to \infty }{\text{lim sup}}\enspace {F}_{n}\left(u\right){\geqslant}\underset{n\to \infty }{\text{lim sup}}\enspace {F}_{n}\left({u}^{n}\right){\geqslant}F\left(u\right)\end{equation*}$$

so the whole sequence of functional values converges.

Finally, in case F_n (uⁿ ) → ∞ as n → ∞, F cannot be proper: otherwise, we obtain analogously to the above that ∞ > F(u) ⩾ lim sup_n→∞ F_n (u) ⩾ liminf_n→∞ F_n (uⁿ ) for some u ∈ BV(Ω) which is a contradiction. □

Remark 2.15. The convergence of discrepancies as in (4) is related to gamma-convergence. Indeed, the difference is that, for the latter, on the right-hand side of the lim sup inequality, an arbitrary sequence converging to v is allowed (instead of the constant sequence). In this context, as can be seen in the proof of the stability result above, one could still weaken the lim sup-assumption in (4) by allowing not only the constant recovery sequence but any sequence for which the regularisation functional converges. However, in order to maintain an assumption on the discrepancy term that is independent of the choice of regularisation, we chose the slightly stronger condition.

Example 2.16. 

• A typical discrepancy is some power of the norm-distance in Y, i.e., ${S}_{f}\left(v\right)=\frac{1}{q}{{\Vert}v-f{\Vert}}_{Y}^{q}$ for some q ⩾ 1. It is easy to show that whenever fⁿ → f in Y, ${S}_{{f}^{n}}$ converges to S_f in the above sense. Also, the equi-coercivity of $\left\{{S}_{{f}^{n}}\right\}$ is immediate.
• For the Kullback–Leibler divergence, let Y = L¹(Ω') for some Ω' and assume that {fⁿ }, f in L¹(Ω') are such that fⁿ ⩽ Cf a.e. in Ω' for some C > 0 and KL(f, fⁿ ) → 0 as n → ∞. Then, it follows from lemma A.2 in the appendix that ${S}_{{f}^{n}}=\mathrm{K}\mathrm{L}\left(\cdot ,{f}^{n}\right)$ converges to S_f = KL(⋅, f), and also that ||fⁿ − f||₁ → 0. The latter in particular implies boundedness of {fⁿ } in L¹(Ω') which, together with the coercivity estimate of lemma A.1 shows that $\left\{{S}_{{f}^{n}}\right\}$ is equi-coercive.

In addition to well-posedness of the Tikhonov-functional minimisation, one is of course interested in regularisation results, i.e., the convergence of solutions to a minimum-TV-solution provided that the data converges and α → 0 in some sense. For this purpose, let u^† ∈ BV(Ω) be a minimum-TV-solution of Ku^† = f^† for some data f^† in Y, i.e., TV(u^†) ⩽ TV(u) for each Ku = f^†, suppose that for each δ > 0 one has given a f^δ ∈ Y such that ${S}_{{f}^{\delta }}\left({f}^{{\dagger}}\right){\leqslant}\delta$, and denote by u^α,δ a solution of (3) for parameter α > 0 and data f^δ .

Theorem 2.17. In the situation of theorem 2.11, let the discrepancy functionals $\left\{{S}_{{f}^{\delta }}\right\}$ be equi-coercive and converge to ${S}_{{f}^{{\dagger}}}$ in the sense of (4) for some data f^† ∈ Y with ${S}_{{f}^{{\dagger}}}\left(v\right)=0$ if and only if v = f^†. Choose for each δ > 0 the parameter α > 0 such that

$$\begin{equation*}\alpha \to 0,\quad \frac{\delta }{\alpha }\to 0\quad \text{as}\quad \delta \to 0.\end{equation*}$$

Then, again up to constant shifts, {u^α,δ } has at least one weak accumulation point in L^d/(d−1)(Ω) (weak^* in case d = 1). Each such accumulation point is a minimum-TV-solution of Ku = f^† and lim_δ→0TV(u^α,δ ) = TV(u^†).

Proof. Again we assume that ${\int }_{{\Omega}}$ u^α,δ dx = 0 for all (α, δ) if K 1 = 0. Using the optimality of u^α,δ for (3) compared to u^† gives

$$\begin{equation*}{S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{\alpha ,\delta }\right){\leqslant}\delta +\alpha \mathrm{T}\mathrm{V}\left({u}^{{\dagger}}\right).\end{equation*}$$

Since α → 0 as δ → 0, we have that ${S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)\to 0$ as δ → 0. Moreover, as also δ/α → 0, it follows that lim sup_δ→0TV(u^α,δ ) ⩽ TV(u^†). This allows to conclude that {u^α,δ } is bounded in BV(Ω) and, by embedding, admits a weak accumulation point in L^d/(d−1)(Ω) (weak^* in case d = 1).

Next, let u* be such an accumulation point associated with {δ_n }, δ_n → 0 as well as the corresponding parameters {α_n }. Then, ${S}_{{f}^{{\dagger}}}\left(K{u}^{{\ast}}\right){\leqslant}{\text{lim inf}}_{n\to \infty }{S}_{{f}^{{\delta }_{n}}}\left(K{u}^{{\alpha }_{n},{\delta }_{n}}\right)=0$, so Ku* = f^†. Moreover, $\mathrm{T}\mathrm{V}\left({u}^{{\ast}}\right){\leqslant}{\text{lim inf}}_{n\to \infty }\mathrm{T}\mathrm{V}\left({u}^{{\alpha }_{n},{\delta }_{n}}\right){\leqslant}\mathrm{T}\mathrm{V}\left({u}^{{\dagger}}\right)$, hence u* is a minimum-TV-solution. In particular, TV(u*) = TV(u^†), so ${\mathrm{lim}}_{n\to \infty }\mathrm{T}\mathrm{V}\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)=\mathrm{T}\mathrm{V}\left({u}^{{\dagger}}\right)$.

Finally, each sequence of {δ_n }, δ_n → 0 contains another subsequence $\left\{{u}^{{\delta }_{n}}\right\}$ for which $\mathrm{T}\mathrm{V}\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)\to \mathrm{T}\mathrm{V}\left({u}^{{\dagger}}\right)$ as n → ∞, so TV(u^α,δ ) → TV(u^†) as δ → 0. □

Finally, if a respective source condition is satisfied, we can, under some circumstances, give rates for some Bregman distance with respect to TV associated with respect to a particular subgradient element [48]. Recall that the Bregman distance ${D}_{{x}^{{\ast}}}^{F}\left(y,x\right)$ of x, y ∈ X for a convex functional $F:X\to \enspace \left.\right]- \infty ,\infty \left.\right]$ and subgradient element x* ∈ ∂F(x) is given by

$$\begin{equation*}{D}_{{x}^{{\ast}}}^{F}\left(y,x\right)=F\left(y\right)-F\left(x\right)-\langle {x}^{{\ast}},\enspace y-x\rangle .\end{equation*}$$

The convergence rate results are then a consequence of the following proposition.

Proposition 2.18. In the situation of theorem 2.17, let K*w^† ∈ ∂TV(u^†) for some w^† ∈ Y*. Then,

$$\begin{equation}{D}_{{K}^{{\ast}}{w}^{{\dagger}}}^{\mathrm{T}\mathrm{V}}\left({u}^{\alpha ,\delta },{u}^{{\dagger}}\right){\leqslant}\frac{1}{\alpha }\left({S}_{{f}^{\delta }}^{{\ast}}\left(\alpha {w}^{{\dagger}}\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-\alpha {w}^{{\dagger}}\right)+2\delta \right).\end{equation} \tag{ 5 }$$

Proof. Using the minimality of u^α,δ yields ${S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\alpha \mathrm{T}\mathrm{V}\left({u}^{\alpha ,\delta }\right){\leqslant}\alpha \mathrm{T}\mathrm{V}\left({u}^{{\dagger}}\right)+\delta$. Rearranging, adding ⟨K*w^†, u^† − u^α,δ ⟩ on both sides as well as using Fenchel's inequality twice yields

$$\begin{align*}\hfill {S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\alpha {D}_{{K}^{{\ast}}{w}^{{\dagger}}}^{\mathrm{T}\mathrm{V}}\left({u}^{\delta ,\alpha },{u}^{{\dagger}}\right)& {\leqslant}\alpha \langle {K}^{{\ast}}{w}^{{\dagger}},\enspace {u}^{{\dagger}}-{u}^{\alpha ,\delta }\rangle +\delta \hfill \\ \hfill & =\langle \alpha {w}^{{\dagger}},\enspace {f}^{{\dagger}}\rangle -\langle \alpha {w}^{{\dagger}},\enspace K{u}^{\alpha ,\delta }\rangle +\delta \hfill \\ \hfill & {\leqslant}{S}_{{f}^{\delta }}^{{\ast}}\left(\alpha {w}^{{\dagger}}\right)-\langle \alpha {w}^{{\dagger}},\enspace K{u}^{\alpha ,\delta }\rangle +2\delta \hfill \\ \hfill & {\leqslant}{S}_{{f}^{\delta }}^{{\ast}}\left(\alpha {w}^{{\dagger}}\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-\alpha {w}^{{\dagger}}\right)+{S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+2\delta .\hfill \end{align*}$$

Subtracting ${S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)$ and dividing by α gives the result. □

For well-known discrepancy terms, one easily gets parameter choice rules that lead to rates for ${D}_{{K}^{{\ast}}{w}^{{\dagger}}}^{\mathrm{T}\mathrm{V}}\left({u}^{\alpha ,\delta }\right)$.

Example 2.19. 

• For ${S}_{{f}^{\delta }}\left(v\right)=\frac{1}{q}{{\Vert}v-{f}^{\delta }{\Vert}}_{Y}^{q}$ with q > 1, ${S}_{{f}^{\delta }}^{{\ast}}\left(w\right)=\frac{1}{{q}^{{\ast}}}{{\Vert}w{\Vert}}_{{Y}^{{\ast}}}^{{q}^{{\ast}}}+\langle {f}^{\delta },\enspace w\rangle$ where 1/q + 1/q* = 1, hence (5) reads as
  $$\begin{equation*}{D}_{{K}^{{\ast}}{w}^{{\dagger}}}^{\mathrm{T}\mathrm{V}}\left({u}^{\alpha ,\delta },{u}^{{\dagger}}\right){\leqslant}\frac{2{\alpha }^{{q}^{{\ast}}-1}}{{q}^{{\ast}}}{{\Vert}{w}^{{\dagger}}{\Vert}}_{{Y}^{{\ast}}}^{{q}^{{\ast}}}+\frac{2\delta }{\alpha }.\end{equation*}$$
  In the non-trivial case of w^† ≠ 0, the right-hand side becomes minimal for $\alpha ={{\Vert}{w}^{{\dagger}}{\Vert}}_{{Y}^{{\ast}}}^{-1}{\left(\frac{{q}^{{\ast}}}{{q}^{{\ast}}-1}\right)}^{1/{q}^{{\ast}}}{\delta }^{1/{q}^{{\ast}}}$ giving the well-known rate of $\mathcal{O}\left({\delta }^{1/q}\right)=\mathcal{O}\left({{\Vert}{f}^{\delta }-{f}^{{\dagger}}{\Vert}}_{Y}\right)$ for the Bregman distance.
• For the Kullback–Leibler discrepancy, i.e., ${S}_{{f}^{\delta }}\left(v\right)=\mathrm{K}\mathrm{L}\left(v,{f}^{\delta }\right)$ on L¹(Ω'), a direct, pointwise computation shows that the dual functional obeys ${S}_{{f}^{\delta }}^{{\ast}}\left(w\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-w\right)={\int }_{{{\Omega}}^{\prime }}-{f}^{\delta }\enspace \mathrm{log}\left(1-{w}^{2}\right)\enspace \mathrm{d}x$ if $\left\vert w\right\vert {\leqslant}1$ almost everywhere, setting −t  log(0) = ∞ for t > 0 and −0  log(0) = 0, and ${S}_{{f}^{\delta }}^{{\ast}}\left(w\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-w\right)=\infty$ else. As w^† ∈ L^∞(Ω'), we may choose α > 0 such that $\alpha {{\Vert}{w}^{{\dagger}}{\Vert}}_{\infty }{\leqslant}\frac{1}{\sqrt{2}}$. Then, the equivalence
  $$\begin{equation*}{\alpha }^{2}{\int }_{{{\Omega}}^{\prime }}{f}^{\delta }{\left({w}^{{\dagger}}\right)}^{2}\enspace \mathrm{d}x{\leqslant}-{\int }_{{{\Omega}}^{\prime }}{f}^{\delta }\enspace \mathrm{log}\left(1-{\alpha }^{2}{\left({w}^{{\dagger}}\right)}^{2}\right)\enspace \mathrm{d}x{\leqslant}{\alpha }^{2}2\enspace \mathrm{log}\left(2\right){\int }_{{{\Omega}}^{\prime }}{f}^{\delta }{\left({w}^{{\dagger}}\right)}^{2}\enspace \mathrm{d}x\end{equation*}$$
  holds. Assuming ${\int }_{{{\Omega}}^{\prime }}{f}^{{\dagger}}{\left({w}^{{\dagger}}\right)}^{2}\enspace \mathrm{d}x{ >}0$, the weak convergence f^δ ⇀ f in L¹(Ω') (see lemma A.2) implies ${S}_{{f}^{\delta }}^{{\ast}}\left(\alpha {w}^{{\dagger}}\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-\alpha {w}^{{\dagger}}\right)\sim {\alpha }^{2}$ independent from δ. Hence, choosing $\alpha \sim \sqrt{\delta }$ yields the rate $\mathcal{O}\left(\sqrt{\delta }\right)$ for the Bregman distance as δ → 0.

Besides these functional-analytic properties, functions of bounded variation admit interesting structural and fine properties. Let us briefly discuss the structure of the gradient ∇u for a u ∈ BV(Ω). By Lebesgue's decomposition theorem, ∇u can be split into an absolutely continuous part ∇^a u with respect to the Lebesgue measure and a singular part ∇^s u. We tacitly identify ∇^a u with the Radon–Nikodým derivative, i.e., ∇^a u ∈ L¹(Ω, R^d ) via the measure ${\nabla }^{a}u{\mathcal{L}}^{d}$.

The singular part ∇^s u therefore has to capture the jump discontinuities of u. Indeed, introducing the jump set, it can further be decomposed. Recall that a u ∈ L¹(Ω) is almost everywhere approximately continuous, i.e., for almost every x ∈ Ω there exists a z ∈ R such that

$$\begin{equation*}\underset{r\to 0}{\mathrm{lim}}- {\int }_{{B}_{r}\left(x\right)}\left\vert u\left(y\right)-z\right\vert \enspace \mathrm{d}y=0.\end{equation*}$$

The collection of all points S_u for which u is not approximately continuous is called the discontinuity set of u.

Definition 2.20. Let $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega}\right)$ and x ∈ Ω.

• (a)  
  The function u is called approximately differentiable in x if there exists a v ∈ R^d such that

  $$\begin{equation*}\underset{r\to 0}{\mathrm{lim}}\frac{1}{r}- {\int }_{{\Omega}}\left\vert u\left(y\right)-u\left(x\right)-v\cdot \left(y-x\right)\right\vert \enspace \mathrm{d}y=0.\end{equation*}$$

  The vector ∇^≈ u(x) = v is called the approximate gradient of u at x.
• (b)  
  The point x is an approximate jump point of u if there exist u⁺(x) > u⁻(x) and a ν ∈ R^d , $\left\vert \nu \right\vert =1$ such that

  $$\begin{equation*}\underset{r\to 0}{\mathrm{lim}}- {\int }_{{B}_{r}^{+}\left(x,\nu \right)}\left\vert u\left(y\right)-{u}^{+}\left(x\right)\right\vert \enspace \mathrm{d}y=0,\quad \underset{r\to 0}{\mathrm{lim}}- {\int }_{{B}_{r}^{-}\left(x,\nu \right)}\left\vert u\left(y\right)-{u}^{-}\left(x\right)\right\vert \enspace \mathrm{d}y=0\end{equation*}$$

  where ${B}_{r}^{+}\left(x,\nu \right)$ and ${B}_{r}^{-}\left(x,\nu \right)$ are balls cut by the hyperplane perpendicular to ν and containing x, i.e.,

  $$\begin{equation*}\begin{aligned}\hfill {B}_{r}^{+}\left(x,\nu \right)& =\left\{y\in {\mathbf{R}}^{d}\enspace \vert \enspace \left\vert y-x\right\vert {< }r,\enspace \left(y-x\right)\cdot \nu { >}0\right\},\hfill \\ \hfill {B}_{r}^{-}\left(x,\nu \right)& =\left\{y\in {\mathbf{R}}^{d}\enspace \vert \enspace \left\vert y-x\right\vert {< }r,\enspace \left(y-x\right)\cdot \nu {< }0\right\}.\hfill \end{aligned}\end{equation*}$$

  The set J_u of all approximate jump points is called the jump set of u.

Theorem 2.21. ([7]). Let u ∈ BV(Ω). Then,

• (a)  
  u is almost everywhere approximately differentiable with ∇^a u = ∇^≈ u in L¹(Ω, R^d ).
• (b)  
  The jump set satisfies ${\mathcal{H}}^{d-1}\left({S}_{u}{\backslash}{J}_{u}\right)=0$ and we have $\nabla u\enspace \llcorner \enspace {J}_{u}=\left({u}^{+}-{u}^{-}\right){\nu }_{u}{\mathcal{H}}^{d-1}$.
• (c)  
  The restriction ∇u ⌞ (Ω\S_u ) is absolutely continuous with respect to ${\mathcal{H}}^{d-1}$.

In particular, the involved sets and functions are Borel sets and functions, respectively.

Denoting by

$$\begin{equation*}{\nabla }^{j}u={\nabla }^{s}u\enspace \llcorner \enspace {J}_{u},\quad {\nabla }^{c}u={\nabla }^{s}u\enspace \llcorner \enspace \left({\Omega}{\backslash}{S}_{u}\right)\end{equation*}$$

where ∇^j u and ∇^c u is the jump and Cantor part of ∇u, respectively, the gradient of a u ∈ BV(Ω) can be decomposed into

$$\begin{equation}\nabla u={\nabla }^{a}u{\mathcal{L}}^{d}+\left({u}^{+}-{u}^{-}\right){\nu }_{u}{\mathcal{H}}^{d-1}\enspace \llcorner \enspace {J}_{u}+{\nabla }^{c}u\end{equation} \tag{ 6 }$$

with ∇^c u being singular with respect to ${\mathcal{L}}^{d}$ and absolutely continuous with respect to ${\mathcal{H}}^{d-1}$.

This construction allows in particular to define penalties beyond the total variation seminorm (see, for instance [7, section 5.5]). Letting g : R^d → [0, ∞] a proper, convex and lower semi-continuous function and g_∞ be given according to

$$\begin{equation*}{g}_{\infty }\left(x\right)=\underset{t\to \infty }{\mathrm{lim}}\frac{g\left(tx\right)}{t}\end{equation*}$$

with ∞ allowed, then the functional

$$\begin{equation}{\mathcal{R}}_{g}\left(u\right)={\int }_{{\Omega}}g\left({\nabla }^{a}u\right)\enspace \mathrm{d}x+{\int }_{{J}_{u}}\left({u}^{+}-{u}^{-}\right){g}_{\infty }\left({\nu }_{u}\right)\enspace \mathrm{d}{\mathcal{H}}^{d-1}+{\int }_{{\Omega}}{g}_{\infty }\left({\sigma }_{{\nabla }^{c}u}\right)\enspace \mathrm{d}\left\vert {\nabla }^{c}u\right\vert \end{equation} \tag{ 7 }$$

where ${\sigma }_{{\nabla }^{c}u}$ is the sign of ∇^c u, i.e., ${\nabla }^{c}u={\sigma }_{{\nabla }^{c}u}\left\vert {\nabla }^{c}u\right\vert$, is proper, convex and lower semi-continuous on BV(Ω). With the Fenchel-dual functional, i.e., ${g}^{{\ast}}\left(y\right)=\underset{x\in {\mathbf{R}}^{d}}{\mathrm{sup}}\enspace x\cdot y-g\left(x\right)$, it can also be expressed in (pre-)dual form as

$$\begin{equation*}{\mathcal{R}}_{g}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\enspace \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi -{g}^{{\ast}}\left(\varphi \right)\enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathbf{R}}^{d}\right)\right\}.\end{equation*}$$

Obviously, the usual TV-case corresponds to g being the Euclidean norm on R^d . Also, g_∞(x) = ∞ for some $\left\vert x\right\vert =1$ does not allow jumps in the direction of x, so one usually assumes that g_∞(x) < ∞ for each $\left\vert x\right\vert =1$ in order to obtain a genuine penalty in BV(Ω). In addition, if there are c₀ > 0 and R > 0 such that $g\left(x\right){\geqslant}{c}_{0}\left\vert x\right\vert$ for each $\left\vert x\right\vert {\geqslant}R$, then there is a constant C > 0 such that

$$\begin{equation*}{\mathcal{R}}_{g}\left(u\right){\geqslant}{c}_{0}\mathrm{T}\mathrm{V}\left(u\right)-C\end{equation*}$$

for all u ∈ BV(Ω), i.e., ${\mathcal{R}}_{g}$ is as coercive as TV. Consequently, the well-posedness and convergence statements in theorems 2.11, 2.14 and 2.17 as well as in proposition 2.18 can be adapted to ${\mathcal{R}}_{g}$ in a straightforward manner with the proofs following the same line of argumentation.

Example 2.22. There are several possibilities for replacing the non-differentiable norm function $\left\vert \cdot \right\vert$ in the TV-functional by a smooth approximation in 0.

Choosing a ɛ > 0, consider

$$\begin{equation*}{g}_{\varepsilon }^{1}\left(x\right)=\left\{\begin{array}{cc}\frac{1}{2\varepsilon }{\left\vert x\right\vert }^{2}\hfill & \text{for}\;\left\vert x\right\vert {\leqslant}\varepsilon ,\hfill \\ \left\vert x\right\vert -\frac{\varepsilon }{2}\hfill & \text{else},\hfill \end{array}\right.\quad {g}_{\varepsilon }^{2}\left(x\right)=\sqrt{{\left\vert x\right\vert }^{2}+{\varepsilon }^{2}}-\varepsilon ,\end{equation*}$$

both being continuously differentiable in R^d and approximating $\left\vert \cdot \right\vert$ for ɛ → 0.

The associated penalties ${\mathcal{R}}_{{g}_{\varepsilon }^{1}}$ and ${\mathcal{R}}_{{g}_{\varepsilon }^{2}}$ are often referred to as Huber-TV and smooth TV, respectively.

Example 2.23. Taking g as a non-Euclidean norm on R^d yields functionals of anisotropic total-variation type. The common choice is g = |⋅|₁ which is also often referred to as anisotropic TV.

Remark 2.24. It is worth noting that g as above can also be made spatially dependent, which has applications for instance in the context of regularisation for inverse problems involving multiple modalities or multiple spectra. Under some assumptions, functionals ${\mathcal{R}}_{g}$ as in (7) with spatially dependent g are again lower semi-continuous on BV [6] and well-posedness results for TV apply [105].

Colour and multichannel images are usually represented by functions mapping into a vector-space. Total-variation functionals and regularisation approaches can easily be extended to such vector-valued functions; definition 2.5 already contains an isotropic variant for functions with values in a finite-dimensional space H, where we used the Hilbert-space norm $\vert x\vert ={\left({\sum }_{i=1}^{d}{x}_{i}\cdot {x}_{i}\right)}^{1/2}$ as pointwise norm on H^d for the test functions $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{H}^{d}\right)$.

However, in contrast to the scalar case, this is not the only choice yielding TV-functionals that are invariant under distance-preserving transformations. The essential property for a norm |⋅|_◦ on H^d needed for the latter is

$$\begin{equation*}\vert Ox{\vert }_{{\circ}}=\vert x{\vert }_{{\circ}}\quad \text{for}\;\text{all}\quad x\in {H}^{d}\quad \text{and}\quad O\in {\mathbf{R}}^{d{\times}d},\quad {O}^{{\ast}}O=\mathrm{i}\mathrm{d}\end{equation*}$$

where ${\left(Ox\right)}_{i}={\sum }_{j=1}^{d}{o}_{ij}{x}_{j}$. We call such norms unitarily left invariant. Denoting by |⋅|_* the dual norm, the associated total variation for a $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},H\right)$ is given by

$$\begin{equation*}\mathrm{T}\mathrm{V}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{H}^{d}\right),\enspace \vert \varphi \left(x\right){\vert }_{{\ast}}{\leqslant}1\quad \forall x\in {\Omega}\right\}\end{equation*}$$

and invariant to distance-preserving transformations. If the norm |⋅|_◦ is moreover unitarily right invariant, i.e.,

$$\begin{equation*}\vert xO{\vert }_{{\circ}}=\vert x{\vert }_{{\circ}}\quad \text{for}\enspace \text{all}\quad x\in {H}^{d},\quad O:H\to H\quad \text{unitary}\end{equation*}$$

where ${\left(xO\right)}_{i}={\left(O\left(x\right)\right)}_{i}$, then it can be written as a unitarily invariant matrix norm and hence |x|_◦ only depends on the singular values of the mapping associated with x in a permutation- and sign-invariant manner. More precisely, there exists a norm |⋅|_Σ on R^d with |Pσ|_Σ = |σ|_Σ for all σ ∈ R^d and P ∈ R^d×d with $\left\vert P\right\vert$ being a permutation matrix, such that |x|_◦ = |σ|_Σ for all x ∈ H^d , where σ are the singular values of the mapping H^d → R^d given by $y{\mapsto}{\left({x}_{i}\cdot y\right)}_{i}$. Conversely, any such norm on R^d induces a unitarily invariant matrix norm. A common choice are the norms generated by the p-vector norm, the Schatten- p -norms. For p = 1, p = 2 and p = ∞, they correspond to the nuclear norm, the Frobenius norm and the usual spectral norm, respectively, all of which have been proposed in the existing literature to use in conjunction with TV, see, e.g., [79, 164]. Among those possibilities, the nuclear norm appears particularly attractive as it provides a relaxation of the rank functional [156]. Hence, solutions with low-rank gradients and more pronounced edges can be expected from nuclear-norm-TV regularisation.

Also here, the well-posedness and convergence results in theorems 2.11, 2.14 and 2.17 as well as in proposition 2.18 are transferable to the vector-valued case, as can be seen from equivalence of norms.

Moreover, functionals of the type (7) are possible with g : H^d → [0, ∞] proper, convex and lower semi-continuous such that g_∞ exists. However, u takes values in H which calls for some adaptations which we briefly describe in the following. First, concerning definition 2.20(a), we are able to generalise in a straightforward way by considering v ∈ H^d , the norm in H and the scalar product in H^d such that the approximate gradient of u at x is ∇^≈ u(x) ∈ H^d . For jump points according to (b), we are no longer able to require u⁺(x) > u⁻(x) such that we have to replace this by u⁺(x) ≠ u⁻(x) and arrive at a meaningful definition replacing the absolute value by the norm in H. However, u⁺, u⁻ and ν are then only unique up to a sign. Nevertheless, (u⁺ − u⁻) ⊗ ν according to ${\left(\left({u}^{+}-{u}^{-}\right)\otimes \nu \right)}_{i}=\left({u}^{+}-{u}^{-}\right){\nu }_{i}$ is still unique. The analogue of theorem 2.21 and (6) holds with these notions, with the following adaptation:

$$\begin{equation*}\nabla u={\nabla }^{a}u{\mathcal{L}}^{d}+\left({u}^{+}-{u}^{-}\right)\otimes {\nu }_{u}{\mathcal{H}}^{d-1}\enspace \llcorner \enspace {J}_{u}+{\nabla }^{c}u\end{equation*}$$

with the Cantor part being of rank one, i.e., ${\nabla }^{c}u={\sigma }_{{\nabla }^{c}u}\left\vert {\nabla }^{c}u\right\vert$ where ${\sigma }_{{\nabla }^{c}u}$ is rank one $\left\vert {\nabla }^{c}u\right\vert$-almost everywhere [7, theorem 3.94]. The functional ${\mathcal{R}}_{g}$ according to

$$\begin{equation*}{\mathcal{R}}_{g}\left(u\right)={\int }_{{\Omega}}g\left({\nabla }^{a}u\right)\enspace \mathrm{d}x+{\int }_{{J}_{u}}{g}_{\infty }\left(\left({u}^{+}-{u}^{-}\right)\otimes {\nu }_{u}\right)\enspace \mathrm{d}{\mathcal{H}}^{d-1}+{\int }_{{\Omega}}{g}_{\infty }\left({\sigma }_{{\nabla }^{c}u}\right)\enspace \mathrm{d}\left\vert {\nabla }^{c}u\right\vert \end{equation*}$$

then realises a regulariser with the same regularisation properties as its counterpart for scalar functions.

For smooth functions, higher-order derivatives can be represented as tensor fields, i.e., the derivative represents a tensor in each point. As the order of partial differentiation might be interchanged, these tensors turn out to be symmetric. Symmetric tensors are therefore a suitable tool for representing these objects independent from indices. There are several ways to introduce and motivate tensors and vector spaces of tensors. For our purposes, the following definition will be sufficient. Note that there and throughout this chapter, l ⩾ 0 will always be a tensor order.

Zoom In Zoom Out Reset image size

Definition 3.1. We define

$$\begin{equation*}\begin{matrix}\hfill {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)=\left\{\xi :\underset{l\enspace \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}}{\underbrace{{\mathbf{R}}^{d}{\times}\cdots {\times}{\mathbf{R}}^{d}}}\to \mathbf{R}\enspace \vert \enspace \xi \enspace l-\text{linear}\right\},\hfill \\ \hfill {\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)=\left\{\xi :\underset{l\enspace \mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}}{\underbrace{{\mathbf{R}}^{d}{\times}\cdots {\times}{\mathbf{R}}^{d}}}\to \mathbf{R}\enspace \vert \enspace \xi \enspace l-\text{linear}\;\text{and}\;\text{symmetric}\right\},\hfill \end{matrix}\end{equation*}$$

as the vector space of l -tensors and symmetric l -tensors, respectively.

Here, $\xi \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ is called symmetric, if ξ(a₁, ..., a_l ) = ξ(a_π(1), ..., a_π(l)) for all a₁, ..., a_l ∈ R^d and π ∈ S_l , where S_l denotes the permutation group of {1, ..., l}.

For $\xi \in {\mathcal{T}}^{k}\left({\mathbf{R}}^{d}\right)$, k ⩾ 0 and $\eta \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ the tensor product is defined as the element $\xi \otimes \eta \in {\mathcal{T}}^{k+l}\left({\mathbf{R}}^{d}\right)$ obeying

$$\begin{equation*}\left(\xi \otimes \eta \right)\left({a}_{1},\dots ,{a}_{k+l}\right)=\xi \left({a}_{1},\dots ,{a}_{k}\right)\eta \left({a}_{k+1},\dots ,{a}_{k+l}\right)\end{equation*}$$

for all a₁, ..., a_k+l ∈ R^d .

Note that the space of l-tensors is actually the space of (0, l)-covariant tensors, however, we will not need to distinguish between co- and contravariant tensors. We have

$$\begin{equation*}{\mathcal{T}}^{0}\left({\mathbf{R}}^{d}\right)\equiv \mathbf{R},\quad {\mathcal{T}}^{1}\left({\mathbf{R}}^{d}\right)\equiv {\mathbf{R}}^{d},\dots ,{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\equiv {\mathbf{R}}^{d{\times}\cdots {\times}d},\end{equation*}$$

while for low orders, the symmetric tensor spaces coincide with well-known spaces Sym⁰(R^d ) ≡ R, Sym¹(R^d ) ≡ R^d and Sym²(R^d ) ≡ S^d×d , the space of symmetric d × d matrices.

In the following, we give a brief overview of the tensor operations that are the most relevant to define regularisation functionals on higher-order derivatives.

Remark 3.2. The space ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ can be associated with a unit basis. Indexed by p ∈ {1, ..., d}^l , its elements are given by ${e}_{p}\left({a}_{1},\dots ,{a}_{l}\right)={\prod }_{i=1}^{l}{a}_{i,{p}_{i}}$ while the respective coefficient for a $\xi \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ is given by ${\xi }_{p}=\xi \left({e}_{{p}_{1}},\dots ,{e}_{{p}_{l}}\right)$. Each $\xi \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ thus has the representation

$$\begin{equation*}\xi \left({a}_{1},\dots ,{a}_{l}\right)=\sum _{p\in {\left\{1,\dots ,d\right\}}^{l}}{\xi }_{p}{e}_{p}\left({a}_{1},\dots ,{a}_{l}\right).\end{equation*}$$

The identity of vector spaces ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)={\mathbf{R}}^{d{\times}\cdots {\times}d}$ is evident from that.

The space Sym^l (R^d ) is obviously a (generally proper) subspace of ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$. A (non-symmetric) tensor $\xi \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ can be symmetrised by averaging over all permuted arguments, i.e.,

$$\begin{equation*}\left(\vert \vert \vert \xi \right)\left({a}_{1},\dots ,{a}_{l}\right)=\frac{1}{l!}\sum _{\pi \in {S}_{l}}\xi \left({a}_{\pi \left(1\right)},\dots ,{a}_{\pi \left(l\right)}\right).\end{equation*}$$

The symmetrisation operator $\vert \vert \vert :{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\to {\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)$ obviously defines a projection. A basis for Sym^l (R^d ) is given by ${\mathrm{e}}_{p}^{\mathrm{S}\mathrm{y}\mathrm{m}}=\vert \vert \vert {e}_{p}$ for p ranging over all tuples in {1, ..., d}^l with non-decreasing entries. The coefficients ξ_p can still be obtained by ${\xi }_{p}=\xi \left({e}_{{p}_{1}},\dots ,{e}_{{p}_{l}}\right)$.

We would like to equip the spaces with a Hilbert space structure.

Definition 3.3. For $\xi ,\eta \in {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$, the scalar product and Frobenius norm are defined as

$$\begin{equation*}\xi \cdot \eta =\sum _{p\in {\left\{1,\dots ,d\right\}}^{l}}\xi \left({e}_{{p}_{1}},\dots ,{e}_{{p}_{l}}\right)\eta \left({e}_{{p}_{1}},\dots ,{e}_{{p}_{l}}\right),\quad \left\vert \xi \right\vert =\sqrt{\xi \cdot \xi }.\end{equation*}$$

Example 3.4. For ξ ∈ Sym^l (R^d ), the norm corresponds to the absolute value for l = 0, the Euclidean norm in R^d for l = 1 and in case l = 2, we can identify ξ ∈ Sym²(R^d ) with

$$\begin{equation*}\xi =\left(\begin{matrix}\hfill {\xi }_{11}\hfill & \hfill \cdots \hfill & \hfill {\xi }_{1d}\hfill \\ \hfill {\vdots}\hfill & \hfill \ddots \hfill & \hfill {\vdots}\hfill \\ \hfill {\xi }_{1d}\hfill & \hfill \cdots \hfill & \hfill {\xi }_{dd}\hfill \end{matrix}\right),\quad \left\vert \xi \right\vert ={\left(\sum _{i=1}^{d}{\xi }_{ii}^{2}+2\sum _{i{< }j}{\xi }_{ij}^{2}\right)}^{1/2}.\end{equation*}$$

With the Frobenius norm, tensor spaces become Hilbert spaces of finite dimension and the symmetrisation becomes an orthogonal projection, see, e.g., [99].

Proposition 3.5. 

• (a)  
  With the above scalar-product and norm, the spaces ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$, Sym^l (R^d ) are finite-dimensional Hilbert spaces with $\mathrm{dim}{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)={d}^{l}$ and $\mathrm{dim}{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)=\left(\genfrac{}{}{0pt}{}{d+l-1}{l}\right)$.
• (b)  
  The symmetrisation | | | is the orthogonal projection in ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ onto Sym^l (R^d ).

Tensor-valued mappings ${\Omega}\to {\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ on the domain Ω ⊂ R^d are called tensor fields. The tensor-field spaces $\mathcal{C}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, ${\mathcal{C}}_{\mathrm{c}}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ and ${\mathcal{C}}_{0}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ as well as the Lebesgue spaces ${L}^{p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ are then given in the usual manner. Also, measures can be tensor-valued, giving $\mathcal{M}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, the space of l-tensor-valued Radon measures. Duality according to proposition 2.4 holds, i.e., $\mathcal{M}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)={\mathcal{C}}_{0}{\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)}^{{\ast}}$. Note that for all spaces, the Frobenius norm is used as pointwise norm in the respective definitions of the tensor-field norm. Furthermore, all the above applies analogously to symmetric tensor fields, i.e., mappings between Ω → Sym^l (R^d ).

Turning to differentiation, the kth Fréchet derivative of a sufficiently smooth l-tensor field, where from now on k ⩾ 1 will always denote an order of differentiation, is naturally a (k + l)-tensor field which we denote by ${\nabla }^{k}\otimes u:{\Omega}\to {\mathcal{T}}^{k+l}\left({\mathbf{R}}^{d}\right)$ according to

$$\begin{equation*}\left({\nabla }^{k}\otimes u\right)\left(x\right)\left({a}_{1},\dots ,{a}_{k+l}\right)=\left({\mathrm{D}}^{k}u\left(x\right)\left({a}_{1},\dots ,{a}_{k}\right)\right)\left({a}_{k+1},\dots ,{a}_{k+l}\right).\end{equation*}$$

The fact that gradient tensor-fields are not symmetric in general gives rise to consider the k th symmetrised derivative given by ${\mathcal{E}}^{k}u=\vert \vert \vert {\nabla }^{k}\otimes u$. This definition is consistent as ${\mathcal{E}}^{{k}_{2}}{\mathcal{E}}^{{k}_{1}}={\mathcal{E}}^{{k}_{1}+{k}_{2}}$ for k₁, k₂ ⩾ 0. Divergence operators are then, up to the sign, formal adjoints of these differentiation operators. They are given as follows. Introducing the trace of a tensor $\xi \in {\mathcal{T}}^{l+2}\left({\mathbf{R}}^{d}\right)$ according to

$$\begin{equation*}\mathrm{t}\mathrm{r}\left(\xi \right)\left({a}_{1},\dots ,{a}_{l}\right)=\sum _{i=1}^{d}\xi \left({e}_{i},{a}_{1},\dots ,{a}_{l},{e}_{i}\right)\end{equation*}$$

gives an l-tensor. It can be interpreted as the tensor contraction of the first and the last component of the tensor. As for the vector-field case, the divergence is now the trace of the derivative. For k-times differentiable $v:{\Omega}\to {\mathcal{T}}^{k+l}\left({\mathbf{R}}^{d}\right)$, the kth divergence is thus given by

$$\begin{equation*}{\mathrm{d}\mathrm{i}\mathrm{v}}^{k}v={\mathrm{t}\mathrm{r}}^{k}\left({\nabla }^{k}\otimes v\right).\end{equation*}$$

Again, this is consistent with repeated application, i.e., ${\mathrm{d}\mathrm{i}\mathrm{v}}^{{k}_{1}+{k}_{2}}={\mathrm{d}\mathrm{i}\mathrm{v}}^{{k}_{2}}{\mathrm{d}\mathrm{i}\mathrm{v}}^{{k}_{1}}$. Note that there might be other choices of the divergence, such as contracting the derivative with any other than the last components of the tensor. This affects, however, only non-symmetric tensor fields. For symmetric tensor fields, the result is independent from the choice of the contraction components and always a symmetric tensor field.

Example 3.6. The symmetrised gradient of scalar functions Ω → Sym⁰(R^d ) coincides with the usual gradient while the divergence for mappings Ω → Sym¹(R^d ) coincides with the usual divergence.

The cases ${\mathcal{E}}^{2}{u}^{0}$ and $\mathcal{E}{u}^{1}$ for u⁰ : Ω → Sym⁰(R^d ) and u¹ : Ω → Sym¹(R^d ) can be handled with the identification of Sym²(R^d ) and symmetric matrices S^d×d :

$$\begin{equation*}{\left({\mathcal{E}}^{2}{u}^{0}\right)}_{ij}=\frac{{\partial }^{2}{u}^{0}}{\partial {x}_{i}\partial {x}_{j}},\quad {\left(\mathcal{E}{u}^{1}\right)}_{ij}=\frac{1}{2}\left(\frac{\partial {u}_{i}^{1}}{\partial {x}_{j}}+\frac{\partial {u}_{j}^{1}}{\partial {x}_{i}}\right).\end{equation*}$$

Analogously, for the divergence of a v : Ω → Sym²(R^d ), we have that

$$\begin{equation*}{\left(\mathrm{d}\mathrm{i}\mathrm{v}\enspace v\right)}_{i}=\sum _{j=1}^{d}\frac{\partial {v}_{ij}}{\partial {x}_{j}},\quad {\mathrm{d}\mathrm{i}\mathrm{v}}^{2}v=\sum _{i=1}^{d}\frac{{\partial }^{2}{v}_{ii}}{\partial {x}_{i}^{2}}+\sum _{i{< }j}2\frac{{\partial }^{2}{v}_{ij}}{\partial {x}_{i}\partial {x}_{j}}.\end{equation*}$$

In particular, for k ⩾ 1, there are the usual spaces of continuously differentiable tensor fields which are denoted by ${\mathcal{C}}^{k}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ and equipped with the usual norm ${{\Vert}u{\Vert}}_{k,\infty }={\mathrm{max}}_{0{\leqslant}m{\leqslant}k}\enspace {{\Vert}{\nabla }^{m}\otimes u{\Vert}}_{\infty }$. Likewise, we consider k-times continuously differentiable tensor fields with compact support ${\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ where k = ∞ leads to the space of test tensor fields. Also, for finite k, the space ${\mathcal{C}}_{0}^{k}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ is given as the closure of ${\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ in ${\mathcal{C}}^{k}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$. Of course, the analogous constructions apply to symmetric tensor fields, leading to the spaces ${\mathcal{C}}^{k}\left(\overline{{\Omega}},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, ${\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ and ${\mathcal{C}}_{0}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ as well as the space of test symmetric tensor fields ${\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$.

As Ω is assumed to be a connected set, we are able to describe the kernels of ∇^k and ${\mathcal{E}}^{k}$ for (symmetric) tensor fields in terms of finite-dimensional spaces of polynomials.

Proposition 3.7. Let $u\in {\mathcal{C}}^{k}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ such that ∇^k ⊗ u = 0. Then, u is a ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$-valued polynomial of maximal order k − 1, i.e., there are ${\xi }^{m}\in {\mathcal{T}}^{l+m}\left({\mathbf{R}}^{d}\right)$, m = 0, ..., k − 1 such that

$$\begin{equation}u\left(x\right)=\sum _{m=0}^{k-1}{\mathrm{t}\mathrm{r}}^{m}\left({\xi }^{m}\otimes \left(\underset{m\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}}{\underbrace{x\otimes \cdots \otimes x}}\right)\right)\quad \text{for}\enspace \text{each}\quad x\in {\Omega}.\end{equation} \tag{ 8 }$$

If ${\mathcal{E}}^{k}u=0$ for $u\in {\mathcal{C}}^{k+l}\left(\overline{{\Omega}},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, then u is a Sym^l (R^d )-valued polynomial of maximal order k + l − 1, i.e., the above representation holds for ξ^m ∈ Sym^l+m (R^d ), m = 0, ..., k + l − 1 with the sum ranging from 0 to k + l − 1.

Proof. At first we note that any ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$- and Sym^l (R^d )-valued polynomial of maximal order k − 1 and k + l − 1, respectively, admits a representation as claimed. In case ∇^k ⊗ u = 0 for $u\in {\mathcal{C}}^{k}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ it follows directly from a basis representation of u(x) that u is a ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$-valued polynomial of maximal order k − 1.

Now in case ${\mathcal{E}}^{k}u=0$ for $u\in {\mathcal{C}}^{k+l}\left(\overline{{\Omega}},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, we get that ∇^k+l ⊗ u = 0, see lemma A.3. This implies that u is a Sym^l (R^d )-valued polynomial of maximal degree k + l − 1 as claimed. □

Next, we would like to introduce and discuss weak forms of differentiation for (symmetric) tensor fields. Starting point for this is a version of the well-known Gauss–Green theorem for smooth (symmetric) tensor fields [28].

Proposition 3.8. Let Ω ⊂ R^d be a bounded Lipschitz domain, $u\in \mathcal{C}\left(\overline{{\Omega}},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, $v\in {\mathcal{C}}^{1}\left(\overline{{\Omega}},{\mathcal{T}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$. Then, a Gauss–Green theorem holds in the following form:

$$\begin{equation*}{\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace v\enspace \mathrm{d}x={\int }_{\partial {\Omega}}\left(u\otimes \nu \right)\cdot v\enspace \mathrm{d}{\mathcal{H}}^{d-1}-{\int }_{{\Omega}}\left(\nabla \otimes u\right)\cdot v\enspace \mathrm{d}x\end{equation*}$$

with ν being the outward unit normal on ∂Ω.

If $u\in \mathcal{C}\left(\overline{{\Omega}},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, $v\in {\mathcal{C}}^{1}\left(\overline{{\Omega}},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$, the identity reads as

$$\begin{equation*}{\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace v\enspace \mathrm{d}x={\int }_{\partial {\Omega}}\vert \vert \vert \left(u\otimes \nu \right)\cdot v\enspace \mathrm{d}{\mathcal{H}}^{d-1}-{\int }_{{\Omega}}\mathcal{E}u\cdot v\enspace \mathrm{d}x.\end{equation*}$$

If one of the tensor fields u or v have compact support in Ω the boundary term does not appear and the identities are valid for arbitrary domains Ω.

As usual, being able to express integrals of the form ${\int }_{{\Omega}}$(∇ ⊗ u) ⋅ v  dx and ${\int }_{{\Omega}}\mathcal{E}u\cdot v\enspace \mathrm{d}x$ for test tensor fields without the derivative of u allows to introduce a weak notion of ∇ ⊗ u and $\mathcal{E}u$, respectively, as well as associated Sobolev spaces.

Definition 3.9. For $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, $w\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathcal{T}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$ is the weak derivative of u, denoted w = ∇ ⊗ u, if for all $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathcal{T}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$, it holds that

$$\begin{equation*}{\int }_{{\Omega}}u\cdot \mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \enspace \mathrm{d}x=-{\int }_{{\Omega}}w\cdot \varphi \enspace \mathrm{d}x.\end{equation*}$$

Likewise, for $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, w ∈ L¹ _loc(Ω, Sym^l+1(R^d )) is the weak symmetrised derivative of u, denoted $w=\mathcal{E}u$, if the above identity holds for all $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$.

Like the scalar versions, ∇ and $\mathcal{E}$ are well-defined and constitute closed operators between the respective Lebesgue spaces with dense domain.

Definition 3.10. The Sobolev space of tensor fields of order l of differentiation order k and exponent p ∈ [1, ∞] is defined as

$$\begin{equation*}\begin{aligned}\hfill {H}^{k,p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)& =\left\{u\in {L}^{p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)\enspace \vert \enspace {{\Vert}u{\Vert}}_{k,p}{< }\infty \right\},\hfill \\ \hfill {{\Vert}u{\Vert}}_{k,p}& ={\left(\sum _{m=0}^{k}{{\Vert}{\nabla }^{m}\otimes u{\Vert}}_{p}^{p}\right)}^{1/p}\quad \text{if}\enspace p{< }\infty ,\hfill \\ \hfill {{\Vert}u{\Vert}}_{k,\infty }& =\underset{m=0,\dots ,k}{\mathrm{max}}\enspace {{\Vert}{\nabla }^{m}\otimes u{\Vert}}_{\infty },\hfill \end{aligned}\end{equation*}$$

while ${H}_{0}^{k,p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ is the closure of the subspace ${\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ with respect to the ||⋅||_k,p -norm.

Replacing ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ by Sym^l (R^d ) and letting

$$\begin{equation*}{{\Vert}u{\Vert}}_{k,p}={\left(\sum _{m=0}^{k}{{\Vert}{\mathcal{E}}^{m}u{\Vert}}_{p}^{p}\right)}^{1/p}\quad \text{if}\enspace p{< }\infty ,\quad {{\Vert}u{\Vert}}_{\infty ,k}=\underset{m=0,\dots ,k}{\mathrm{max}}\enspace {{\Vert}{\mathcal{E}}^{m}u{\Vert}}_{\infty },\end{equation*}$$

defines the Sobolev space of symmetric tensor fields, denoted by H^k,p (Ω, Sym^l (R^d )). The space ${H}_{0}^{k,p}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ is again the closure ${\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ with respect to the corresponding norm.

By closedness of the differential operators, the Sobolev spaces are Banach spaces. Also, since weak derivatives are symmetric, we have that ${H}^{k,p}\left({\Omega},{\mathcal{T}}^{0}\left({\mathbf{R}}^{d}\right)\right)={H}^{k,p}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{0}\left({\mathbf{R}}^{d}\right)\right)$ in the sense of Banach space isometry, as well as coincidence with the usual Sobolev spaces. For l ⩾ 1, the space ${H}^{k,p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ corresponds to the space where all components of u are in H^k,p (Ω). However, generally, for l ⩾ 1, the norm of H^k,p (Ω, Sym^l (R^d )) is weaker than the norm in ${H}^{k,p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, such that only ${H}^{k,p}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)\hookrightarrow {H}^{k,p}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ in the sense of continuous embedding and the latter is a strictly larger space.

Nevertheless, equality holds if some kind of Korn's inequality can be established which is, for instance, the case for the spaces ${H}_{0}^{1,p}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{1}\left({\mathbf{R}}^{d}\right)\right)$ for 1 < p < ∞ [120, section 5.6] as well as and the spaces ${H}_{0}^{1,2}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ for l ⩾ 1 (which follows from [28, proposition 3.6] via smooth approximation).

Finally, let us briefly discuss (symmetric) tensor-valued distributions and the distributional forms of ∇^k and ${\mathcal{E}}^{k}$.

Definition 3.11. A ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ -valued distribution on Ω is a linear mapping $u:{\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)\to \mathbf{R}$ that satisfies the following continuity estimate: for each K ⊂⊂ Ω, there is an m ∈ N and a C > 0 such that

$$\begin{equation*}\left\vert u\left(\varphi \right)\right\vert {\leqslant}C{{\Vert}\varphi {\Vert}}_{m,\infty }\quad \text{for}\enspace \text{all}\quad \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left(K,{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right).\end{equation*}$$

The distribution u is regular if there is a $\bar{u}\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ such that

$$\begin{equation*}u\left(\varphi \right)={\int }_{{\Omega}}\bar{u}\cdot \varphi \enspace \mathrm{d}x\quad \text{for}\enspace \text{all}\quad \varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)\right).\end{equation*}$$

A Sym^l (R^d )-valued distribution on Ω and its regularity is analogously defined by replacing ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$ by Sym^l (R^d ) in the above definition.

Then, the distributional (symmetrised) derivatives are given by (∇^k ⊗ u)(φ) = (−1)^k u(div^k φ), $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathcal{T}}^{k+1}\left({\mathbf{R}}^{d}\right)\right)$ and $\left({\mathcal{E}}^{k}u\right)\left(\varphi \right)={\left(-1\right)}^{k}u\left({\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \right)$, $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+1}\left({\mathbf{R}}^{d}\right)\right)$ which makes them a ${\mathcal{T}}^{k+l}\left({\mathbf{R}}^{d}\right)$- and Sym^k+l (R^d )-valued distribution, respectively. We then have the following generalisation of theorem 3.7 which will be useful for analysing functionals that depend on (symmetrised) distributional derivatives.

Proposition 3.12. If ∇^k ⊗ u = 0 for a ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$-valued distribution, then u is regular and a ${\mathcal{T}}^{l}\left({\mathbf{R}}^{d}\right)$-valued polynomial of maximal degree k − 1.

If ${\mathcal{E}}^{k}u=0$ for a Sym^l (R^d )-valued distribution, then u is regular and a Sym^l (R^d )-valued polynomial of maximal degree k + l − 1.

Proof. This can be deduced from proposition 3.7 via mollification arguments similar as in [28, proposition 3.3]. □

In the following, we discuss functions whose derivative is a Radon measure for a fixed order of differentiation. As higher-order derivatives of scalar functions are always symmetric, it suffices to consider only the symmetrised higher-order derivative ${\mathcal{E}}^{k}$ in this case as well as symmetric tensor fields. However, as we are also interested in intermediate differentiation orders, we moreover discuss spaces of symmetric tensors for which the symmetrised derivative of some order is a Radon measure.

In the following, recall that k ⩾ 1 denotes a differentiation order and l ⩾ 0 denotes a tensor order.

Definition 3.13. Let Ω ⊂ R^d be a domain.

• (a)  
  In the case l = 0, for $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega}\right)$, the total variation of order k is defined as

  $$\begin{equation*}{\mathrm{T}\mathrm{V}}^{k}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\enspace {\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k}\left({\mathbf{R}}^{d}\right)\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty }{\leqslant}1\right\}.\end{equation*}$$

  For general l ⩾ 0 and $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$, the total deformation of order k is

  $$\begin{equation*}{\mathrm{T}\mathrm{D}}^{k}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\cdot {\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l}\left({\mathbf{R}}^{d}\right)\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty }{\leqslant}1\right\}.\end{equation*}$$
• (b)  
  The normed space according to

  $$\begin{equation*}\begin{aligned}\hfill {\mathrm{B}\mathrm{D}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)& =\left\{u\in {L}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\right)\left({\mathbf{R}}^{d}\right)\enspace \vert \enspace {\mathrm{T}\mathrm{D}}^{k}\left(u\right){< }\infty \right\},\hfill \\ \hfill {{\Vert}u{\Vert}}_{{\mathrm{B}\mathrm{D}}^{k}}& ={{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{D}}^{k}\left(u\right)\hfill \end{aligned}\end{equation*}$$

  is called the space of symmetric tensor fields of bounded deformation of order k. The scalar case, i.e., l = 0, is referred to as the space of functions of bounded variation of order k. The latter spaces are denoted by BV^k (Ω).

We note that the Hilbert-space norm on the tensor space for the definition of TV^k leads to a corresponding pointwise norm on the derivatives. While this choice is rather natural, and does not require to distinguish primal and dual norms, also other choices are possible for which we refer to [128] in the second-order case.

Let us analyse some of the basic properties of these spaces.

Proposition 3.14. Let Ω ⊂ R^d be a domain, p ∈ [1, ∞]. Then:

• (a)  
  TD^k is proper, convex and a lower semi-continuous seminorm on L^p (Ω, Sym^l (R^d )).
• (b)  
  TD^k (u) = 0 if and only if ${\mathcal{E}}^{k}u=0$. In particular, TD^k (u) = 0 implies that u is a Sym^l (R^d )-valued polynomial of maximal degree k + l − 1.

Proof. With p* being the dual exponent to p, each test tensor field obeys ${\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \in {L}^{{p}^{{\ast}}}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)$ for $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l}\left({\mathbf{R}}^{d}\right)\right)$. The functional TD^k is thus a pointwise supremum over a set of continuous linear functionals and, consequently, convex and lower semi-continuous. By definition, it is obviously proper and positively homogeneous since if div^k φ is a test vector field, then also −div^k φ is.

By definition of TD^k we see that TD^k (u) = 0 if and only if ${\int }_{{\Omega}}$ u ⋅ div^k φ dx = 0 for each $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l}\left({\mathbf{R}}^{d}\right)\right)$. But this is equivalent to ${\mathcal{E}}^{k}u=0$ in the distributional sense such that in particular, the polynomial representation follows from proposition 3.12. □

In order to show more properties, for instance, that BD^k (Ω, Sym^l (R^d )) is a Banach space, let us adopt a more abstract viewpoint. We say that a function $\left\vert \cdot \right\vert :X\to \left[0,\infty \right]$ for X a Banach space is a lower semi-continuous seminorm on X if $\left\vert \cdot \right\vert$ is positive homogeneous, satisfies the triangle inequality and is lower semi-continuous. The kernel of $\left\vert \cdot \right\vert$, denoted $\mathrm{ker}\left(\left\vert \cdot \right\vert \right)$, is the set $\left\{x\in X\enspace \vert \enspace \left\vert x\right\vert =0\right\}$ which is a closed linear subspace of X.

Lemma 3.15. Let $\left\vert \cdot \right\vert$ be a lower semi-continuous seminorm on the Banach space X with norm ||⋅||_X . Then,

$$\begin{equation*}Y=\left\{x\in X\enspace \vert \enspace \left\vert x\right\vert {< }\infty \right\},\quad {{\Vert}x{\Vert}}_{Y}={{\Vert}x{\Vert}}_{X}+\left\vert x\right\vert \end{equation*}$$

is a Banach space. The seminorm $\left\vert \cdot \right\vert$ is continuous in Y.

Proof. It is immediate that Y is a normed space. Let {xⁿ } be a Cauchy sequence in Y which is obviously a Cauchy sequence in X. Hence, a limit x ∈ X exists for which the lower semi-continuity yields $\left\vert x\right\vert {\leqslant}{\text{lim inf}}_{n\to \infty }\left\vert {x}^{n}\right\vert {< }\infty$, the latter since $\left\{\left\vert {x}^{n}\right\vert \right\}$ is a real Cauchy sequence. In particular, x ∈ Y.

To obtain convergence with respect to $\left\vert \cdot \right\vert$, choose, for ɛ > 0, an n such that for all m ⩾ n, $\left\vert {x}^{n}-{x}^{m}\right\vert {\leqslant}\varepsilon$. Letting m → ∞ gives, as xⁿ − x^m → xⁿ − x in X,

$$\begin{equation*}\left\vert {x}^{n}-x\right\vert {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace \left\vert {x}^{n}-{x}^{m}\right\vert {\leqslant}\varepsilon .\end{equation*}$$

This implies xⁿ → x in Y which is what we intended to show.

Finally, the continuity of $\left\vert \cdot \right\vert$ follows from the standard estimate $\left\vert \left\vert {x}^{1}\right\vert -\left\vert {x}^{2}\right\vert \right\vert {\leqslant}\left\vert {x}^{1}-{x}^{2}\right\vert {\leqslant}{{\Vert}{x}^{1}-{x}^{2}{\Vert}}_{Y}$ for x¹, x² ∈ Y. □

It is then obvious from proposition 3.14 and lemma 3.15 that BD^k (Ω, Sym^l (R^d )) is a Banach space. In order to examine the structure of these spaces, it is crucial to understand the case k = 1, i.e., BD(Ω, Sym^l (R^d )) = BD¹(Ω, Sym^l (R^d )), where the symmetrised derivative is only a measure. For l ⩾ 1, these spaces are strictly greater that BV(Ω, Sym^l (Ω)) as a consequence of the failure of Korn's inequality. Important properties of these spaces are summarised as follows.

Theorem 3.16. ([32, theorem 2.6]).If u is a Sym^l (R^d )-valued distribution on Ω a bounded Lipschitz domain with $\mathcal{E}u\in \mathcal{M}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l+1}\left({\mathbf{R}}^{d}\right)\right)$, then u ∈ BD(Ω, Sym^l (R^d )).

Theorem 3.17. ([28, theorems 4.16 and 4.17]). For Ω a bounded Lipschitz domain and, 1 ⩽ p ⩽ d/(d − 1), the space BD(Ω, Sym^l (R^d )) is continuously embedded in L^p (Ω, Sym^l (R^d )). Moreover, for p < d/(d − 1), the embedding is compact.

Theorem 3.18. (Sobolev–Korn inequality [28, corollary 4.20]). For Ω a bounded Lipschitz domain and ${R}_{l}:{L}^{d/\left(d-1\right)}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\to \mathrm{ker}\left(\mathcal{E}\right)$ a linear and continuous projection onto the kernel of $\mathcal{E}$, there exists a constant C > 0 such that for each u ∈ BD(Ω, Sym^l (R^d )) it follows that

$$\begin{equation}{{\Vert}u-{R}_{l}u{\Vert}}_{d/\left(d-1\right)}{\leqslant}C{{\Vert}\mathcal{E}u{\Vert}}_{\mathcal{M}}.\end{equation} \tag{ 9 }$$

Note that the projection R_l as stated always exists as $\mathrm{ker}\left(\mathcal{E}\right)$ is finite-dimensional (see proposition 3.12).

Now, for general k and u ∈ BD^k (Ω, Sym^l (R^d )) fixed, $w={\mathcal{E}}^{k-1}u$ is a Sym^l+k−1(R^d )-valued distribution with the property

$$\begin{equation*}\left(\mathcal{E}w\right)\left(\varphi \right)=-w\left(\mathrm{d}\mathrm{i}\mathrm{v}\enspace \varphi \right)={\left(-1\right)}^{k}u\left({\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \right)={\left(-1\right)}^{k}{\int }_{{\Omega}}u\cdot {\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \enspace \mathrm{d}x={\int }_{{\Omega}}\varphi \cdot \enspace \mathrm{d}{\mathcal{E}}^{k}u\end{equation*}$$

for $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{\infty }\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l}\left({\mathbf{R}}^{d}\right)\right)$. In other words, $\mathcal{E}w={\mathcal{E}}^{k}u\in \mathcal{M}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l}\left({\mathbf{R}}^{d}\right)\right)$, thus theorem 3.16 implies that ${\mathcal{E}}^{k-1}u=w\in \mathrm{B}\mathrm{D}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l-1}\left({\mathbf{R}}^{d}\right)\right)$ and, in particular, we have u ∈ BD^k−1(Ω, Sym^l (R^d )). Hence, the spaces are nested:

$$\begin{equation*}{\mathrm{B}\mathrm{D}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\subset {\mathrm{B}\mathrm{D}}^{k-1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\subset \cdots \subset \mathrm{B}\mathrm{D}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right).\end{equation*}$$

Let us look at the norms: by the Sobolev–Korn inequality (9), for some linear projection ${R}_{k+l-1}:\mathrm{B}\mathrm{D}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k+l-1}\left({\mathbf{R}}^{d}\right)\right)\to \mathrm{ker}\left(\mathcal{E}\right)$, we see

$$\begin{equation*}{{\Vert}{\mathcal{E}}^{k-1}u-{R}_{k+l-1}{\mathcal{E}}^{k-1}u{\Vert}}_{1}{\leqslant}C{{\Vert}{\mathcal{E}}^{k}u{\Vert}}_{\mathcal{M}}\end{equation*}$$

which implies

$$\begin{equation*}{{\Vert}{\mathcal{E}}^{k-1}u{\Vert}}_{\mathcal{M}}{\leqslant}C\left({{\Vert}{\mathcal{E}}^{k}u{\Vert}}_{\mathcal{M}}+{{\Vert}{R}_{k+l-1}{\mathcal{E}}^{k-1}u{\Vert}}_{1}\right).\end{equation*}$$

Now, $u{\mapsto}{R}_{k+l-1}{\mathcal{E}}^{k-1}u$ is well-defined on BD^k (Ω, Sym^l (R^d )), linear, has finite-dimensional image and is hence continuous. We may therefore estimate

$$\begin{equation*}{{\Vert}{\mathcal{E}}^{k-1}u{\Vert}}_{\mathcal{M}}{\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{{\Vert}{\mathcal{E}}^{k}u{\Vert}}_{\mathcal{M}}\right).\end{equation*}$$

Proceeding inductively, we arrive at the estimate

$$\begin{equation}\sum _{m=0}^{k}{{\Vert}{\mathcal{E}}^{m}u{\Vert}}_{\mathcal{M}}{\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{{\Vert}{\mathcal{E}}^{k}u{\Vert}}_{\mathcal{M}}\right)\end{equation} \tag{ 10 }$$

for some C > 0 independent of u. Therefore, we obtain the following theorem.

Theorem 3.19. If Ω ⊂ R^d is a bounded Lipschitz domain, then the norm equivalence

$$\begin{equation}{{\Vert}u{\Vert}}_{1}+{{\Vert}{\mathcal{E}}^{k}u{\Vert}}_{\mathcal{M}}\sim \sum _{m=0}^{k}{{\Vert}{\mathcal{E}}^{m}u{\Vert}}_{\mathcal{M}}\end{equation} \tag{ 11 }$$

holds on BD^k (Ω, Sym^l (R^d )). The embeddings

$$\begin{equation*}{\mathrm{B}\mathrm{D}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\hookrightarrow {\mathrm{B}\mathrm{D}}^{k-1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\hookrightarrow \cdots \hookrightarrow \mathrm{B}\mathrm{D}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right)\end{equation*}$$

are continuous.

Proof. The nontrivial estimate to establish norm equivalence has just been shown in (10). The continuity of the embedding follows from the fact that the norm on the right-hand side in (11) is increasing with respect to k. □

In the scalar case, we can furthermore establish Sobolev embeddings.

Theorem 3.20. Let Ω be a bounded Lipschitz domain and 0 ⩽ m < k.

For k − m ⩽ d : the space BV^k (Ω) is continuously embedded in H^m,p (Ω) for $1{\leqslant}p{\leqslant}\frac{d}{d-\left(k-m\right)}$, where we set $\frac{d}{d-\left(k-m\right)}=\infty$ for k − m = d.

If $p{< }\frac{d}{d-\left(k-m\right)}$, then the embedding is compact.

For k − m > d : the space BV^k (Ω) is compactly embedded in ${\mathcal{C}}^{m,\alpha }\left(\overline{{\Omega}}\right)$ for each $\alpha \in \left.\right]0,1 \left[\right.$.

Proof. In the scalar case, ${{\Vert}u{\Vert}}_{1}+{\sum }_{\left\vert \beta \right\vert {\leqslant}k-1}{{\Vert}\nabla {\partial }^{\beta }u{\Vert}}_{\mathcal{M}}$ for β ∈ N^d a multiindex and u ∈ BV^k (Ω) constitutes an equivalent norm on BV^k (Ω), as a consequence of theorem 3.19. By the Poincaré inequality in BV(Ω),

$$\begin{equation*}{{\Vert}{\partial }^{\beta }u{\Vert}}_{d/\left(d-1\right)}{\leqslant}C\left({{\Vert}\nabla {\partial }^{\beta }u{\Vert}}_{\mathcal{M}}+{{\Vert}u{\Vert}}_{1}\right)\end{equation*}$$

for each $\left\vert \beta \right\vert {\leqslant}k-1$. This establishes the continuous embedding BV^k (Ω) → W^{k−1,d/(d−1)}(Ω). Application of the well-known embedding theorems for Sobolev spaces (see [1, theorems 5.4 and 6.2]) then give the results for the cases k − m < d and k − m > d as well as for the case k − m = d and p < ∞.

For the case k − m = d and p = ∞ we note that again by Sobolev embeddings [1, theorem 5.4] we get for a constant C > 0 and all u ∈ H^k,1(Ω) that

$$\begin{equation*}\sum _{i=0}^{m}{{\Vert}{\nabla }^{i}u{\Vert}}_{\infty }{\leqslant}C\sum _{i=0}^{k}{{\Vert}{\nabla }^{i}u{\Vert}}_{1}.\end{equation*}$$

Approximating u ∈ BV^k (Ω) with a sequence {uⁿ } in C^∞(Ω) ∩ BV^k (Ω) strictly converging to u in BV^k (Ω) as in  Lemma A.4, the result follows from applying this estimate to each uⁿ and using lower semi-continuity of the L^∞-norm with respect to convergence in L¹. □

We would like to employ TD^k as a regulariser and first characterise its kernel. For that purpose, we note that TD^k (u) = 0 for some u ∈ BD^k (Ω, Sym^l (R^d )) implies that ${\mathcal{E}}^{k}u=0$ in the distributional sense, hence proposition 3.12 implies that u is a Sym^l (R^d )-valued polynomial of maximal degree k + l − 1. This yields the following result.

Proposition 3.21. The space ker(TD^k ) is a subspace of polynomials of degree less than k + l. If l = 0, then ker(TV^k ) = P^k−1 = {u : Ω → R|u polynomial of degree ⩽ k − 1}.

Next, we like to discuss coercivity of the higher-order total variation functionals.

Proposition 3.22. Let k ⩾ 1, l ⩾ 0 and Ω be a bounded Lipschitz domain. Then, TD^k is coercive in the following sense: for each linear and continuous projection R : L^d/(d−1)(Ω, Sym^l (R^d )) → ker(TD^k ), there is a C > 0 such that

$$\begin{equation*}{{\Vert}u-Ru{\Vert}}_{d/\left(d-1\right)}{\leqslant}C{\mathrm{T}\mathrm{D}}^{k}\left(u\right)\quad \text{for}\enspace \text{all}\quad u\in {\mathrm{B}\mathrm{D}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{l}\left({\mathbf{R}}^{d}\right)\right).\end{equation*}$$

Proof. At first note that by the embeddings BD^k (Ω, Sym^l (R^d )) ↪ BD(Ω, Sym^l (R^d )) ↪ L^d/(d−1)(Ω, Sym^l (R^d )) the left-hand side of the claimed inequality is well defined and finite.

We use a contradiction argument in conjunction with compactness. Suppose for R as stated above there is a sequence {uⁿ } such that ${{\Vert}{u}^{n}-R{u}^{n}{\Vert}}_{d/\left(d-1\right)}=1$ and TD^k (uⁿ ) → 0 as n → ∞. This implies $\left\{{{\Vert}{u}^{n}-R{u}^{n}{\Vert}}_{1}\right\}$ being bounded, TD^k (uⁿ − Ruⁿ ) → 0 and by theorems 3.19 and 3.17 {uⁿ − Ruⁿ } has to be precompact in L¹(Ω, Sym^l (R^d )), i.e., without loss of generality, we may assume that uⁿ − Ruⁿ → u in L¹(Ω, Sym^l (R^d )). By lower semi-continuity,

$$\begin{equation*}{\mathrm{T}\mathrm{D}}^{k}\left(u\right){\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace {\mathrm{T}\mathrm{D}}^{k}\left({u}^{n}\right)=0,\end{equation*}$$

hence u ∈ ker(TD^k ) = rg(R). On the other hand, R(uⁿ − Ruⁿ ) = 0 for each n as R is a projection, thus, Ru = 0 and, consequently, u = 0. In total, we have lim_n→∞(uⁿ − Ruⁿ ) = 0 in BD^k (Ω), and again by continuous embedding, also in L^d/(d−1)(Ω) which is a contradiction to ${{\Vert}{u}^{n}-R{u}^{n}{\Vert}}_{d/\left(d-1\right)}=1$ for all n. Consequently, coercivity has to hold. □

Corollary 3.23. In the scalar case, for p ∈ [1, ∞] with $p{\leqslant}\frac{d}{d-k}$ if k < d, we also have

$$\begin{equation*}{{\Vert}u-Ru{\Vert}}_{p}{\leqslant}C{\mathrm{T}\mathrm{V}}^{k}\left(u\right).\end{equation*}$$

Proof. This follows with the embedding theorem 3.20:

$$\begin{equation*}{{\Vert}u-Ru{\Vert}}_{p}{\leqslant}C\left({{\Vert}u-Ru{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{k}\left(u\right)\right){\leqslant}C{\mathrm{T}\mathrm{V}}^{k}\left(u\right).\end{equation*}$$

□

Remark 3.24. The above coercivity estimate also implies that the Fenchel conjugate of TV^k is the indicator functional of closed convex set in ${L}^{{p}^{{\ast}}}\left({\Omega}\right)\cap \mathrm{ker}{\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{\perp }$ with non-empty interior. Indeed, for $\xi \in {L}^{{p}^{{\ast}}}\left({\Omega}\right)\cap \mathrm{ker}{\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{\perp }$ such that ${{\Vert}\xi {\Vert}}_{{p}^{{\ast}}}{\leqslant}{C}^{-1}$ it follows for any u ∈ L^p (Ω) that

$$\begin{equation*}\langle \xi ,\enspace u\rangle =\langle \xi ,\enspace u-Ru\rangle {\leqslant}{{\Vert}\xi {\Vert}}_{{p}^{{\ast}}}{{\Vert}u-Ru{\Vert}}_{p}{\leqslant}{\mathrm{T}\mathrm{V}}^{k}\left(u\right)\end{equation*}$$

which means that ${\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{{\ast}}\left(\xi \right)=0$. On the other hand, if $\xi \in {L}^{{p}^{{\ast}}}\left({\Omega}\right){\backslash}\mathrm{ker}{\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{\perp }$, then ⟨ξ, u⟩ > 0 for some u ∈ ker(TV^k ). Thus, ⟨ξ, u⟩ > TV^k (u) so ${\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{{\ast}}\left(\xi \right)=\infty$.

It is interesting to note that a coercivity estimate similar to the one of corollary 3.23 also holds between two higher-order TV functionals of different order.

Lemma 3.25. Let Ω be a bounded Lipschitz domain, 1 ⩽ k₁ < k₂ be two orders of differentiation, $p\in \left[\right.1,\infty \left[\right.$ with p ⩽ d/(d − k₂) if k₂ < d and $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ be a continuous, linear projection. Then there exists a constant C > 0 such that

$$\begin{equation}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u-Ru\right){\leqslant}C{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)\end{equation} \tag{ 12 }$$

holds for each $u\in {\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$.

Proof. Assume the opposite, i.e., the existence of {uⁿ } such that ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{n}-R{u}^{n}\right)=1$ and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{n}\right)\to 0$ as n → ∞. Then, by compact embedding ${\mathrm{B}\mathrm{D}}^{{k}_{2}-{k}_{1}}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)\to {L}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$, we have ${\nabla }^{{k}_{1}}\left({u}^{n}-R{u}^{n}\right)\to v$ as n → ∞ for some $v\in {L}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$ for a subsequence (not relabelled). On the other hand, the Poincaré estimate gives ${{\Vert}{u}^{n}-R{u}^{n}{\Vert}}_{p}{\leqslant}C{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{n}\right)$, so uⁿ − Ruⁿ → 0 as n → ∞ in L¹(Ω). By closedness of ${\nabla }^{{k}_{1}}$ this yields v = 0. By convergence in ${L}^{1}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$, this gives the contradiction ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{n}-R{u}^{n}\right)\to 0$ as n → ∞. □

The coercivity which has just been established can be regarded as the most important step towards existence for variational problems with TV^k -regularisation. Here, we first prove an existence result for linear inverse problems in a general abstract version.

Theorem 3.26. Let X be a reflexive Banach space, Y be a Banach space, K : X → Y be linear and continuous, S_f : Y → [0, ∞] a proper, convex, lower semi-continuous and coercive discrepancy functional associated with some data f, $\left\vert \cdot \right\vert :X\to \left[0,\infty \right]$ a lower semi-continuous seminorm and α > 0. Assume that there exists a linear and continuous projection $R:X\to \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ and a C > 0 such that

$$\begin{equation*}{{\Vert}u-Ru{\Vert}}_{X}{\leqslant}C\left\vert u\right\vert \quad \text{for}\enspace \text{all}\quad u\in X,\end{equation*}$$

and either

• (a)  
  $\mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ is finite-dimensional or, more generally,
• (b)  
  $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ admits a complement Z in $\mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ and ||u||_X ⩽ C||Ku||_Y for some C > 0 and all u ∈ Z.

Then, the Tikhonov minimisation problem

$$\begin{equation}\underset{u\in X}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+\alpha \left\vert u\right\vert \end{equation} \tag{ 13 }$$

is well-posed, i.e., there exists a solution and the solution mapping is stable in sense that, if ${S}_{{f}^{n}}$ converges to S_f as in (4) and $\left\{{S}_{{f}^{n}}\right\}$ is equi-coercive, then for each sequence of minimisers {uⁿ } of (13) with discrepancy ${S}_{{f}^{n}}$,

• Either ${S}_{{f}^{n}}\left(K{u}^{n}\right)+\alpha \left\vert {u}^{n}\right\vert \to \infty$ as n → ∞ and (13) with discrepancy S_f does not admit a finite solution,
• Or ${S}_{{f}^{n}}\left(K{u}^{n}\right)+\alpha \left\vert {u}^{n}\right\vert \to {\mathrm{min}}_{u\in X}{S}_{f}\left(u\right)+\alpha \left\vert u\right\vert$ as n → ∞ and there is, possibly up to shifts by functions in $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$, a weak accumulation point u ∈ X that minimises (13) with discrepancy S_f .

Further, in case (13) with discrepancy S_f admits a finite solution, for each subsequence $\left\{{u}^{{n}_{k}}\right\}$ weakly converging to some u ∈ X, it holds that $\left\vert {u}^{{n}_{k}}\right\vert \to \left\vert u\right\vert$ as k → ∞. Also, if S_f is strictly convex and K is injective, finite solutions u of (13) are unique and uⁿ ⇀ u in X.

The same result is true if, for instance, instead of being reflexive, X is the dual of a separable space, and we replace weak convergence by weak^* convergence in the (lower semi-) continuity assumptions on K, $\left\vert \cdot \right\vert$, S_f and in (4).

Proof. At first note that $\mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ being finite-dimensional implies condition (b) above, hence we can assume that (b) holds. We start with existence. Assume that the objective functional in (13) is proper as otherwise, there is nothing to show. For a minimising sequence {uⁿ }, by the coercivity assumption, {uⁿ − Ruⁿ } is bounded in X. Now, (b) implies the existence of a linear and continuous projection ${P}_{Z}:\mathrm{ker}\left(\left\vert \cdot \right\vert \right)\to Z$ such that id − P_Z projects $\mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ onto $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$. With vⁿ = P_Z Ruⁿ , we see that also {uⁿ − Ruⁿ + vⁿ } is a minimising sequence and it suffices to show boundedness of {vⁿ } to obtain a convergent subsequence. But the latter holds true since by assumption ${{\Vert}{v}^{n}{\Vert}}_{p}{\leqslant}C{{\Vert}K{v}^{n}{\Vert}}_{Y}$, such that ${{\Vert}K{v}^{n}{\Vert}}_{Y}{\leqslant}{{\Vert}K\left({u}^{n}-R{u}^{n}+{v}^{n}\right){\Vert}}_{Y}+{\Vert}K{\Vert}{{\Vert}{u}^{n}-R{u}^{n}{\Vert}}_{X}$, with the right-hand side being bounded as a consequence of the coercivity of S_f and the boundedness of {uⁿ − Ruⁿ }. Hence, as X is reflexive, a subsequence of {uⁿ − Ruⁿ + vⁿ } converges weakly to a limit u ∈ X. By continuity of K and lower semi-continuity of both S_f and $\left\vert \cdot \right\vert$ it follows that u is a solution to (13). In case S_f is strictly convex and K is injective, S_f ◦ K is already strictly convex, so finite minimisers of (13) have to be unique.

Now let {uⁿ } be a sequence of minimisers of (13) with discrepancy ${S}_{{f}^{n}}$. We denote by $F={S}_{f}\enspace {\circ}\enspace K+\alpha \left\vert \cdot \right\vert$ as well as ${F}_{n}={S}_{{f}^{n}}\enspace {\circ}\enspace K+\alpha \left\vert \cdot \right\vert$ and first suppose that {F_n (uⁿ )} is bounded. We can then add ${v}^{n}-R{u}^{n}\in \mathrm{ker}\left(K\right)\cap \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ to uⁿ , with vⁿ = P_Z Ruⁿ , and from equi-coercivity of $\left\{{S}_{{f}^{n}}\right\}$ obtain boundedness of {uⁿ − Ruⁿ + vⁿ } as before.

This shows that by shifting the minimisers within $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left(\left\vert \cdot \right\vert \right)$ always leads to a bounded sequence, i.e., we may assume without loss of generality that {uⁿ } is bounded such that a weak accumulation point exists. Suppose that ${u}^{{n}_{k}}\rightharpoonup u$ as k → ∞. Then, estimating as in the proof of theorem 2.14, we can obtain that u is a minimiser for F and that ${\mathrm{lim}}_{k\to \infty }{F}_{{n}_{k}}\left({u}^{{n}_{k}}\right)=F\left(u\right)$ as well as ${\mathrm{lim}}_{k\to \infty }\left\vert {u}^{{n}_{k}}\right\vert =\left\vert u\right\vert$. Also, if u is the unique minimiser for (13) with discrepancy S_f , uⁿ ⇀ u as n → ∞ follows since any subsequence has to contain another subsequence that converges weakly to u.

The result for the two remaining cases lim inf_n→∞ F_n (uⁿ ) < ∞ and F_n (uⁿ ) → ∞, respectively, finally follows analogously to theorem 2.14. □

Given that ker(TV^k ) is finite dimensional, the above result immediately implies well-posedness for $\left\vert \cdot \right\vert ={\mathrm{T}\mathrm{V}}^{k}$ with X = L^p (Ω), as stated in the following corollary. The crucial ingredient here is the estimate ||u − Ru||_p ⩽ CTV^k (u), which restricts the exponent of the underlying L^p -space to p ⩽ d/(d − k) if k < d. This shows that, the higher the order of differentiation used in the regularisation, the weaker are the requirements on the underlying spaces and, consequently, on the continuity of the operator K.

Corollary 3.27. With X = L^p (Ω), Ω being a bounded Lipschitz domain, and S_f and K as in theorem 3.26,

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+\alpha {\mathrm{T}\mathrm{V}}^{k}\left(u\right).\end{equation} \tag{ 14 }$$

is well-posed in the sense of theorem 3.26 whenever $p\in \left.\right]1,\infty \left[\right.$ with p ⩽ d/(d − k) if k < d.

As can be easily seen from the respective proofs, also the convergence result of theorem 2.17 and the result on convergence rates as in proposition 2.18 transfer to TV^k regularisation.

Theorem 3.28. With the assumptions of corollary 3.27, let u^† ∈ BV(Ω) be a minimum-TV^k -solution of Ku^† = f^† for some data f^† in Y and for each δ > 0 let f^δ be such that ${S}_{{f}^{\delta }}\left({f}^{{\dagger}}\right){\leqslant}\delta$ and denote by u^α,δ a finite solution of (14) for parameter α > 0 and data f^δ . Let the discrepancy functionals $\left\{{S}_{{f}^{\delta }}\right\}$ be equi-coercive and converge to ${S}_{{f}^{{\dagger}}}$ in the sense of (4) and ${S}_{{f}^{{\dagger}}}\left(v\right)=0$ if and only if v = f^†. Choose for each δ > 0 the parameter α > 0 such that

$$\begin{equation*}\alpha \to 0,\quad \frac{\delta }{\alpha }\to 0\quad \text{as}\enspace \delta \to 0.\end{equation*}$$

Then, up to shifts by functions in ker(K) ∩ P^k−1, {u^α,δ } has at least one L^p -weak accumulation point. Each L^p -weak accumulation point is a minimum-TV^k -solution of Ku = f^† and lim_δ→0TV^k (u^α,δ ) = TV^k (u^†).

Proposition 3.29. In the situation of theorem 3.28, let K*w^† ∈ ∂TV^k (u^†) for some w^† ∈ Y*. Then,

$$\begin{equation}{D}_{{K}^{{\ast}}{w}^{{\dagger}}}^{{\mathrm{T}\mathrm{V}}^{k}}\left({u}^{\alpha ,\delta },{u}^{{\dagger}}\right){\leqslant}\frac{1}{\alpha }\left({S}_{{f}^{\delta }}^{{\ast}}\left(\alpha {w}^{{\dagger}}\right)+{S}_{{f}^{\delta }}^{{\ast}}\left(-\alpha {w}^{{\dagger}}\right)+2\delta \right).\end{equation} \tag{ 15 }$$

The last result in particular guarantees convergence rates for the settings of example 2.19. Note also that the above results remain true in case p = 1 or in case p = d/(d − k) = ∞ and K is weak^*-to-weak continuous.

Let us finally note some first-order optimality conditions. For this purpose, recall that for X a Banach space, the normal cone ${\mathcal{N}}_{K}\left(u\right)$ of a set K ⊂ X at u ∈ K is given by the collection of all w ∈ X* for which ${\langle w,\enspace v-u\rangle }_{{X}^{{\ast}}{\times}X}{\leqslant}0$ for all v ∈ K. If we set ${\mathcal{N}}_{K}\left(u\right)=\varnothing$ for u ∉ K, we have that ${\mathcal{N}}_{K}=\partial {\mathcal{I}}_{K}$ where ${\mathcal{I}}_{K}$ is the indicator function of K, i.e., ${\mathcal{I}}_{K}\left(u\right)=0$ if u ∈ K and ∞ otherwise.

Proposition 3.30. In the situation of corollary 3.27, if ${S}_{f}\left(v\right)=\frac{1}{2}{\Vert}v-f{{\Vert}}_{Y}^{2}$ and Y is a Hilbert space, u* ∈ L^p (Ω) is a solution of

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}\enspace \frac{1}{2}{{\Vert}Ku-f{\Vert}}_{Y}^{2}+\alpha {\mathrm{T}\mathrm{V}}^{k}\left(u\right)\end{equation} \tag{ 16 }$$

if and only if

$$\begin{equation*}{u}^{{\ast}}\in {\mathcal{N}}_{{\mathrm{T}\mathrm{V}}^{k}}\left(\frac{{K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)}{\alpha }\right)\end{equation*}$$

where ${\mathcal{N}}_{{\mathrm{T}\mathrm{V}}^{k}}$ is the normal cone associated with the set $\overline{{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{k}}}$ where

$$\begin{equation*}{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{k}}=\left\{w\in {L}^{{p}^{{\ast}}}\left({\Omega}\right)\enspace \vert \enspace w={\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi ,\enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k}\left({\mathbf{R}}^{d}\right)\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty }{\leqslant}1\right\}.\end{equation*}$$

Proof. As $u{\mapsto}\frac{1}{2}{{\Vert}Ku-f{\Vert}}_{Y}^{2}$ is Gâteaux differentiable, it is continuous with unique subgradient, so, by subdifferential calculus, optimality of u* is equivalent to K*(f − Ku*) ∈ α∂TV^k (u*) which can also be expressed as

$$\begin{equation*}{u}^{{\ast}}\in \partial {\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{{\ast}}\left(\frac{{K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)}{\alpha }\right).\end{equation*}$$

Now since ${\mathrm{T}\mathrm{V}}^{k}={\mathcal{I}}_{{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{k}}}^{{\ast}}$, it follows that ${\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{{\ast}}={\mathcal{I}}_{{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{k}}}^{{\ast}{\ast}}={\mathcal{I}}_{\overline{{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{k}}}}$, so $\partial {\left({\mathrm{T}\mathrm{V}}^{k}\right)}^{{\ast}}={\mathcal{N}}_{{\mathrm{T}\mathrm{V}}^{k}}$. □

Remark 3.31. In the situation of proposition 3.30, it is also possible to give an a-priori estimate for the solutions of (16) in case K is injective on P^k−1. Indeed, with R : L^p (Ω) → P^k−1 the continuous projection operator on the kernel of TV^k and C > 0 the coercivity constant, i.e., ||u − Ru||_p ⩽ CTV^k (u) for all u ∈ L^p (Ω), by optimality, a solution u* satisfies $\alpha {\mathrm{T}\mathrm{V}}^{k}\left({u}^{{\ast}}\right){\leqslant}\frac{1}{2}{{\Vert}f{\Vert}}_{Y}^{2}$ and consequently, ${{\Vert}u-Ru{\Vert}}_{p}{\leqslant}\frac{1}{2\alpha }C{{\Vert}f{\Vert}}_{Y}^{2}$. Likewise, comparing with u* − Ru*, optimality also gives ${{\Vert}K{u}^{{\ast}}-f{\Vert}}_{Y}^{2}{\leqslant}{{\Vert}K\left({u}^{{\ast}}-R{u}^{{\ast}}\right)-f{\Vert}}_{Y}^{2}$, which is equivalent to ${{\Vert}KR{u}^{{\ast}}{\Vert}}_{Y}^{2}{\leqslant}2\langle KR{u}^{{\ast}},\enspace f-K\left({u}^{{\ast}}-R{u}^{{\ast}}\right)\rangle$. Using $ab{\leqslant}\frac{1}{4}{a}^{2}+{b}^{2}$, the latter leads to ${{\Vert}KR{u}^{{\ast}}{\Vert}}_{Y}^{2}{\leqslant}4{{\Vert}f-K\left({u}^{{\ast}}-R{u}^{{\ast}}\right){\Vert}}_{Y}^{2}$, where the right-hand side can further be estimated, using ${\left(a+b\right)}^{2}{\leqslant}\left(1+\varepsilon \right)\left({a}^{2}+\frac{1}{\varepsilon }{b}^{2}\right)$ with $\varepsilon =\frac{1}{4{\alpha }^{2}}{C}^{2}{{\Vert}K{\Vert}}^{2}$ to give

$$\begin{equation*}{{\Vert}KR{u}^{{\ast}}{\Vert}}_{Y}^{2}{\leqslant}4\left(1+\frac{{C}^{2}{{\Vert}K{\Vert}}^{2}}{4{\alpha }^{2}}\right)\left(1+{{\Vert}f{\Vert}}_{Y}^{2}\right){{\Vert}f{\Vert}}_{Y}^{2}.\end{equation*}$$

Now, as K is injective on P^k−1 = rg(R), there is a c > 0 such that c||Ru||_p ⩽ ||KRu||_Y for all u ∈ L^p (Ω). Consequently, employing the triangle inequality and estimating yields

$$\begin{equation}{{\Vert}{u}^{{\ast}}{\Vert}}_{p}{\leqslant}\frac{1}{2\alpha }\left(\frac{2}{c}\sqrt{4{\alpha }^{2}+{C}^{2}{{\Vert}K{\Vert}}^{2}}+C\right)\sqrt{1+{{\Vert}f{\Vert}}_{Y}^{2}}{{\Vert}f{\Vert}}_{Y},\end{equation} \tag{ 17 }$$

which is an a priori bound that only requires the knowledge of the Poincaré–Wirtinger-type constant C, the constant c in the inverse estimate for K on P^k+1, as well as an estimate on ||K||. Beyond being of theoretical interest, such a bound can for instance be used in numerical algorithms, see section 6, example 6.24.

If the Kullback–Leibler divergence is used instead of the quadratic Hilbert space discrepancy, i.e., S_f (v) = KL(v, f), Y = L¹(Ω'), and data f ⩾ 0 a.e., then one has to choose a u⁰ ∈ BV^k (Ω) such that KL(Ku⁰, f) < ∞. Set C_f = KL(Ku⁰, f) + αTV^k (u⁰). Then, an optimal solution u* will satisfy ${\mathrm{T}\mathrm{V}}^{k}\left({u}^{{\ast}}\right){\leqslant}\frac{{C}_{f}}{\alpha }$. Further, we have ||v||₁ ⩽ 2KL(v, f) + 2||f||₁ for v ∈ L¹(Ω') with v ⩾ 0 a.e., see lemma A.1, such that, if c > 0 is a constant with c||Ru||_p ⩽ ||KRu||₁ for all u ∈ L^p (Ω), we get

$$\begin{equation*}{{\Vert}R{u}^{{\ast}}{\Vert}}_{p}{\leqslant}\frac{1}{c}\left({{\Vert}K{u}^{{\ast}}{\Vert}}_{1}+{\Vert}K{\Vert}{{\Vert}{u}^{{\ast}}-R{u}^{{\ast}}{\Vert}}_{p}\right){\leqslant}\frac{1}{c}\left(\frac{2\alpha +C{\Vert}K{\Vert}}{\alpha }{C}_{f}+2{{\Vert}f{\Vert}}_{1}\right),\end{equation*}$$

and finally arrive at

$$\begin{equation}{{\Vert}{u}^{{\ast}}{\Vert}}_{p}{\leqslant}\frac{1}{c}\left(\frac{2\alpha +C{\Vert}K{\Vert}+cC}{\alpha }{C}_{f}+2{{\Vert}f{\Vert}}_{1}\right).\end{equation} \tag{ 18 }$$

This constitutes an a priori estimate similar to (17) for the Kullback–Leibler discrepancy, however, with the difference that also a suitable constant C_f has to determined.

Remark 3.32. In order to show the effect of TV² regularisation in contrast to TV regularisation, we performed a numerical denoising experiment for f shown in figure 2(a), i.e., solved ${\mathrm{min}}_{u\in {L}^{2}\left({\Omega}\right)}\frac{1}{2}{{\Vert}u-f{\Vert}}_{2}^{2}+{\mathcal{R}}_{\alpha }\left(u\right)$ where ${\mathcal{R}}_{\alpha }=\alpha \mathrm{T}\mathrm{V}$ or ${\mathcal{R}}_{\alpha }=\alpha {\mathrm{T}\mathrm{V}}^{2}$. One clearly sees that TV² regularisation (figure 2(c)) reduces the staircasing effect of TV regularisation (figure 2(b)) and piecewise linear structures are well recovered. However, TV² regularisation also blurs the object boundaries which appear less sharp in contrast to TV regularisation.

This is due to the fact that TV^k regularisation for k ⩾ 2 is not able to produce solutions with jump discontinuities. Indeed, TV^k regularisation implies that a solution u has be to contained in BV^k (Ω) which embeds into the Sobolev space H^k−1,1(Ω) ↪ H^1,1(Ω). As we have seen, for instance, in example 2.1, this means that characteristic functions cannot be solutions. More generally, for u ∈ H^1,1(Ω) ⊂ BV(Ω), the derivative ∇u interpreted as a measure is absolutely continuous with respect to the Lebesgue measure such that the singular part satisfies ∇^s u = 0. Theorem 2.21 then implies that the jump set J_u is a ${\mathcal{H}}^{d-1}$-negligible set, i.e., u cannot jump on (d − 1)-dimensional hypersurfaces.

Zoom In Zoom Out Reset image size

Remark 3.33. Instead of taking higher-order TV which bases on the full gradient, one could also try to regularise with other differential operators, for instance with the (weak) Laplacian:

$$\begin{equation*}{\mathcal{R}}_{\alpha }\left(u\right)=\alpha {{\Vert}{\Delta}u{\Vert}}_{\mathcal{M}}.\end{equation*}$$

However, the kernel of this seminorm is the space of p-integrable harmonic functions on Ω, the Bergman spaces, which are infinite-dimensional. Therefore, in view of theorem 3.26, to use ${\mathcal{R}}_{\alpha }$ for the regularisation of ill-posed linear inverse problems, the forward operator K must be continuously invertible on a complement of $\mathrm{ker}\left({\mathcal{R}}_{\alpha }\right)\cap \mathrm{ker}\left(K\right)$, i.e., well-posed. This limits the applicability of this regulariser. Nevertheless, denoising problems can, for instance, still be solved, see figure 2(d), leading to 'speckle' artefacts in the solutions. Another possibility would be to add more regularising functionals, which is discussed in the next section.

Higher order TV for multichannel images. In analogy to TV, also higher order TV can be extended to colour and multichannel images represented by functions mapping into a vector space, say R^m . This is achieved by testing with ${\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m}$-valued tensor fields, where

$$\begin{equation*}{\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m}=\left\{\xi =\left({\xi }_{1},\dots ,{\xi }_{m}\right)\enspace \vert \enspace {\xi }_{i}\in {\mathrm{S}\mathrm{y}\mathrm{m}}^{k}\left({\mathbf{R}}^{d}\right),\enspace i=1,\dots ,m\right\}\end{equation*}$$

and requires to choose a norm for this space. While, also in view of the Frobenius norm used in Sym^k (R^d ), the most natural choice seems to pick the norm that is induced by the inner product

$$\begin{equation*}\xi \cdot \eta =\sum _{i=1}^{m}{\xi }_{i}\cdot {\eta }_{i}\quad \text{for}\quad \xi ,\eta \in {\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m},\end{equation*}$$

as with TV, this is not the only possible choice and different norms imply different types of coupling of the multiple channels. Generally, we can take |⋅|_◦ to be any norm on ${\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m}$, set |⋅|_∗ to be the corresponding dual norm and extend TV^k to functions $u\in {L}_{\mathrm{l}\mathrm{o}\mathrm{c}}^{1}\left({\Omega},{\mathbf{R}}^{m}\right)$ as

$$\begin{equation}{\mathrm{T}\mathrm{V}}^{k}\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\cdot {\mathrm{d}\mathrm{i}\mathrm{v}}^{k}\varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m}\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty ,{\ast}}{\leqslant}1\right\}\end{equation} \tag{ 19 }$$

where ||φ||_∞,∗ is the pointwise supremum of the scalar function x ↦ |φ(x)|_∗. By equivalence of norms in finite dimensions, the functional-analytic properties of TV^k and the results on regularisation for inverse problems transfer one-to-one to its multichannel extension. Further, TV^k is invariant under rotations whenever the tensor norm |⋅|_∗ is unitarily invariant in the sense that for any orthonormal matrix O ∈ R^d×d and $\left({\xi }_{1},\dots ,{\xi }_{m}\right)\in {\mathrm{S}\mathrm{y}\mathrm{m}}^{k}{\left({\mathbf{R}}^{d}\right)}^{m}$ it holds that

$$\begin{equation*}\vert \left({\xi }_{1}O,\dots ,{\xi }_{m}O\right){\vert }_{{\ast}}=\vert \left({\xi }_{1},\dots ,{\xi }_{m}\right){\vert }_{{\ast}},\end{equation*}$$

where we define (ξ_i O)(a₁, ..., a_k ) = ξ_i (Oa₁, ..., Oa_k ) for i = 1, ..., m.

Fractional-order TV. Recently, ideas from fractional calculus started to be transferred to construct new classes of higher-order TV, namely fractional-order total variation. The latter bases on fractional partial differentiation with respect to the coordinate axes. The partial fractional derivative of a non-integral order α > 0 of a function u compactly supported on the interval $\left.\right]a,b\left[\right.\to \mathbf{R}$ can, for instance, be defined as

$$\begin{equation*}\frac{{\partial }_{\left[a,b\right]}^{\alpha }u}{\partial {x}^{\alpha }}\left(x\right)=\frac{1}{2}\left(\frac{{\partial }_{\left[a,x\right]}^{\alpha }u}{\partial {x}^{\alpha }}+{\left(-1\right)}^{k}\frac{{\partial }_{\left[x,b\right]}^{\alpha }u}{\partial {x}^{\alpha }}\right),\end{equation*}$$

where k ∈ N is such that k − 1 < α < k and, denoting by Γ the gamma-function, i.e., ${\Gamma}\left(t\right)={\int }_{0}^{\infty }{s}^{t-1}{\mathrm{e}}^{-t}\enspace \mathrm{d}s$,

$$\begin{equation*}\frac{{\partial }_{\left[a,x\right]}^{\alpha }u}{\partial {x}^{\alpha }}=\frac{1}{{\Gamma}\left(k-\alpha \right)}\frac{{\partial }^{k}}{\partial {x}^{k}}{\int }_{a}^{x}\frac{u\left(t\right)}{{\left(x-t\right)}^{\alpha -k+1}}\enspace \mathrm{d}t\end{equation*}$$

as well as

$$\begin{equation*}\frac{{\partial }_{\left[x,b\right]}^{\alpha }u}{\partial {x}^{\alpha }}=\frac{{\left(-1\right)}^{k}}{{\Gamma}\left(k-\alpha \right)}\frac{{\partial }^{k}}{\partial {x}^{k}}{\int }_{x}^{b}\frac{u\left(t\right)}{{\left(t-x\right)}^{\alpha -k+1}}\enspace \mathrm{d}t.\end{equation*}$$

This fractional-order derivative corresponds to a central version of the Riemann–Liouville definition [140, 199]. However, one has to mention that there are also other possibilities to define fractional-order derivatives [146]. On a rectangular domain ${\Omega}=\left.\right]{a}_{1},{b}_{1}\left[\right.{\times}\cdots {\times} \left.\right]{a}_{d},{b}_{d}\left[\right.\subset {\mathbf{R}}^{d}$ and for test vector fields $\varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathbf{R}}^{d}\right)$, the fractional divergence of order α can then be defined as ${\mathrm{d}\mathrm{i}\mathrm{v}}^{\alpha }\varphi ={\sum }_{i=1}^{d}\frac{{\partial }_{\left[{a}_{i},{b}_{i}\right]}^{\alpha }{\varphi }_{i}}{\partial {x}_{i}^{\alpha }}$ which is still a bounded function. Consequently, the fractional total variation of order α for u ∈ L¹(Ω) is given as

$$\begin{equation*}{\mathrm{T}\mathrm{V}}^{\alpha }\left(u\right)=\mathrm{sup}\enspace \left\{{\int }_{{\Omega}}u\cdot {\mathrm{d}\mathrm{i}\mathrm{v}}^{\alpha }\varphi \enspace \mathrm{d}x\enspace \vert \enspace \varphi \in {\mathcal{C}}_{\mathrm{c}}^{k}\left({\Omega},{\mathbf{R}}^{d}\right),\enspace {{\Vert}\varphi {\Vert}}_{\infty }{\leqslant}1\right\}.\end{equation*}$$

It is easy to see that this defines a proper, convex and lower semi-continuous functional on each L^p (Ω) which makes the functional suitable as a regulariser for denoising [194, 199], typically for 1 < α < 2. The use of TV^α for the regularisation of linear inverse problems, however, seems to be unexplored so far, and not many properties of the solutions appear to be known.

In this section, we consider the additive combination of total variation functionals with different orders. That is, we are interested in the following Tikhonov approach:

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}{S}_{f}\left(Ku\right)+{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)\end{equation} \tag{ 20 }$$

with α_i > 0 for i = 1, 2 and 1 ⩽ k₁ < k₂. With k₁ = 1, k₂ = 2, such an approach has for instance been considered in [142] for the regularisation of linear inverse problems.

The following proposition summarises, in the general setting of seminorms, basic properties of the function spaces arising from the additive combination of two different regularisers. Its proof is straightforward.

Proposition 4.1. Let |⋅|₁ and |⋅|₂ be two lower semi-continuous seminorms on the Banach space X. Then,

• (a)  
  The functional $\left\vert \cdot \right\vert =\vert \cdot {\vert }_{1}+\vert \cdot {\vert }_{2}$ is a seminorm on X.
• (b)  
  We have

  $$\begin{equation*}\mathrm{ker}\left(\left\vert \cdot \right\vert \right)=\mathrm{ker}\left(\vert \cdot {\vert }_{1}\right)\cap \mathrm{ker}\left(\vert \cdot {\vert }_{2}\right).\end{equation*}$$
• (c)  
  The seminorm $\left\vert \cdot \right\vert$ is lower semi-continuous and

  $$\begin{equation*}Y=\left\{x\in X\enspace \vert \enspace \left\vert \cdot \right\vert {< }\infty \right\},\quad {{\Vert}x{\Vert}}_{Y}={{\Vert}x{\Vert}}_{X}+\left\vert x\right\vert \end{equation*}$$

  constitutes a Banach space.
• (d)  
  With Y_i the Banach spaces arising from the norms ||⋅||_X + |⋅|_i , i = 1, 2 (see lemma 3.15),

  $$\begin{equation*}Y\hookrightarrow {Y}_{i}\quad \text{for}\enspace i=1,2.\end{equation*}$$

Setting $\vert \cdot {\vert }_{i}={\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}$ for i = 1, 2 shows in particular that the function space associated with the additive combination of the ${\mathrm{T}\mathrm{V}}^{{k}_{i}}$ is embedded in ${\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$, i.e., the BV space corresponding to the highest order. Hence non-trivial combinations of different ${\mathrm{T}\mathrm{V}}^{{k}_{i}}$ again do not allow to recover jumps and, as the following proposition shows, in fact even yield the same space as the single TV term with the highest order.

Theorem 4.2. Let 1 ⩽ k₁ < k₂, α₁ > 0, α₂ > 0 and Ω be a bounded Lipschitz domain. For X = L¹(Ω) and the seminorm $\left\vert \cdot \right\vert ={\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$, let Y be the associated Banach space according to lemma 3.15. Then,

$$\begin{equation*}Y={\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)\end{equation*}$$

in the sense of Banach space equivalence, and for p ∈ [1, ∞], p ⩽ d/(d − k₂) if k₂ < d, $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$ a continuous, linear projection, there is a C > 0 independent of u such that

$$\begin{equation*}{{\Vert}u-Ru{\Vert}}_{p}{\leqslant}C\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)\end{equation*}$$

for all u ∈ L^p (Ω).

Proof. For the claimed norm equivalence, one estimate is immediate, while the other one follows from theorem 3.19. Denoting by ${R}_{2}:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ a continuous, linear projection, the estimate on ||u − Ru||_p follows from corollary 3.23 and norm equivalence in finite-dimensional spaces as

$$\begin{align*}\hfill {{\Vert}u-Ru{\Vert}}_{p}& {\leqslant}{{\Vert}u-{R}_{2}u{\Vert}}_{p}+{{\Vert}Ru-{R}_{2}u{\Vert}}_{p}\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)+{{\Vert}Ru-{R}_{2}u{\Vert}}_{1}\right)\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)+{{\Vert}u-Ru{\Vert}}_{1}+{{\Vert}u-{R}_{2}u{\Vert}}_{1}\right)\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right)+{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)\right)\hfill \\ \hfill & {\leqslant}C\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right)\right),\hfill \end{align*}$$

with C > 0 a generic constant. □

Tikhonov regularisation. For employing ${\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$ as regularisation in a Tikhonov setting, the coercivity estimate in theorem 4.2 is crucial since it allows to transfer the well-posedness result of theorem 3.26. Observe in particular that $\mathrm{ker}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ is finite-dimensional, such that assumption (a) in theorem 3.26 is satisfied.

Proposition 4.3. With X = L^p (Ω), $p\in \left.\right] 1,\infty \left[\right.$, Ω a bounded Lipschitz domain, Y a Banach space, K : X → Y linear and continuous, S_f : Y → [0, ∞] proper, convex, lower semi-continuous and coercive, 1 ⩽ k₁ < k₂, α₁ > 0, α₂ > 0 the Tikhonov minimisation problem

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(u\right).\end{equation} \tag{ 21 }$$

is well-posed in the sense of theorem 3.26 whenever p ⩽ d/(d − k₂) if k₂ < d.

It is interesting to note that the necessary coercivity estimate on ${\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$ uses a projection to the smaller kernel of ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and an L^p norm with a larger exponent corresponding to ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$. Hence, in view of the assumptions in theorem 3.26, the additive combination of ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ inherits the best properties of the two summands, i.e., the ones that are the least restrictive for applications in an inverse problems context.

Regarding the convergence result of theorem 3.28 and the rates of proposition 3.29, a direct extension to regularisation with ${\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$ can be obtained by regarding the weights α₁, α₂ to be fixed and introducing an additional factor α > 0 for both terms, which then acts as the regularisation parameter. A more tailored approach, however, would be to regard both α₁, α₂ as regularisation parameters and study the limiting behaviour of the method as α₁, α₂ converge to zero in some sense. This is covered by the following theorem.

Theorem 4.4. In the situation of proposition 4.3, let for each δ > 0 the data f^δ be given such that ${S}_{{f}^{\delta }}\left({f}^{{\dagger}}\right){\leqslant}\delta$, let $\left\{{S}_{{f}^{\delta }}\right\}$ be equi-coercive and converge to ${S}_{{f}^{{\dagger}}}$ for some data f^† ∈ Y in the sense of (4) with ${S}_{{f}^{{\dagger}}}\left(v\right)=0$ if and only if v = f^†.

Choose the positive parameters α = (α₁, α₂) in dependence of δ such that

$$\begin{equation*}\mathrm{max}\left\{{\alpha }_{1},{\alpha }_{2}\right\}\to 0,\quad \frac{\delta }{\mathrm{max}\left\{{\alpha }_{1},{\alpha }_{2}\right\}}\to 0,\quad \text{as}\enspace \delta \to 0,\end{equation*}$$

and $\left({\tilde {\alpha }}_{1},{\tilde {\alpha }}_{2}\right)=\left({\alpha }_{1},{\alpha }_{2}\right)/\mathrm{max}\left\{{\alpha }_{1},{\alpha }_{2}\right\}\to \left({\alpha }_{1}^{{\dagger}},{\alpha }_{2}^{{\dagger}}\right)$ as δ → 0. Set

$$\begin{equation*}k=\begin{cases}_{1}\hfill & \text{if}\enspace {\alpha }_{2}^{{\dagger}}=0,\hfill \\ {k}_{2}\hfill & \text{else},\hfill \end{cases}\end{equation*}$$

and assume p ⩽ d/(d − k) in case of k < d, and that there exists u₀ ∈ BV^k (Ω) such that Ku₀ = f^†. Then, up to shifts in $\mathrm{ker}\left(K\right)\cap {\mathbf{P}}^{{k}_{1}-1}$, any sequence {u^α,δ }, with each u^α,δ being a solution to (20) for parameters (α₁, α₂) and data f^δ , has at least one L^p -weak accumulation point. Each L^p -weak accumulation point is a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution of Ku = f^† and ${\mathrm{lim}}_{\delta \to 0}\left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)=\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)$.

Proof. First note that, as consequence of theorem 3.26 and the fact that u₀ ∈ BV^k (Ω) with Ku₀ = f^†, there exists a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution to Ku = f^†, that we denote by u^†, such that TV^k (u^†) < ∞. Using optimality of u^α,δ compared to u^† gives

$$\begin{equation*}{S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right){\leqslant}\delta +\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right).\end{equation*}$$

Since max{α₁, α₂} → 0 as δ → 0, we have that ${S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)\to 0$ as δ → 0. Moreover, as also δ/max{α₁, α₂} → 0, it follows that

$$\begin{align*}\hfill \underset{\delta \to 0}{\text{lim sup}}\enspace \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)& {\leqslant}\underset{\delta \to 0}{\text{lim sup}}\enspace \left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)\hfill \\ \hfill & {\leqslant}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right).\hfill \end{align*}$$

The choice of k allows to conclude that {TV^k (u^α,δ )} is bounded, which, in case k = k₁, means that $\left\{{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }\right)\right\}$ is bounded. Now we show that also in the other case when k = k₂, $\left\{{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }\right)\right\}$ is bounded. To this aim, denote by $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ and ${P}_{Z}:\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\to Z$ linear, continuous projections, where Z is a complement of $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ in $\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$, i.e., id − P_Z projects $\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ to $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$. Then, by optimality and invariance of K and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ on $\mathrm{ker}\left(K\right)\cap \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ we estimate

$$\begin{align*}\hfill {S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)& {\leqslant}{S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)\hfill \\ \hfill & \quad +{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }-\left(\mathrm{i}\mathrm{d}-{P}_{Z}\right)R{u}^{\alpha ,\delta }\right),\hfill \end{align*}$$

which, together with lemma 3.25, norm equivalence on finite-dimensional spaces and injectivity of K on the finite-dimensional space Z, yields

$$\begin{equation*}\begin{aligned}\hfill {\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }\right)& {\leqslant}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }-\left(\mathrm{i}\mathrm{d}-{P}_{Z}\right)R{u}^{\alpha ,\delta }\right){\leqslant}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }-R{u}^{\alpha ,\delta }\right)+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({P}_{Z}R{u}^{\alpha ,\delta }\right)\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)+{\Vert}{P}_{Z}R{u}^{\alpha ,\delta }{{\Vert}}_{p}\right){\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)+{\Vert}K{P}_{Z}R{u}^{\alpha ,\delta }{{\Vert}}_{Y}\right)\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)+{\Vert}K\left({u}^{\alpha ,\delta }-R{u}^{\alpha ,\delta }+{P}_{Z}R{u}^{\alpha ,\delta }\right){{\Vert}}_{Y}+{\Vert}K{\Vert}{\Vert}\left({u}^{\alpha ,\delta }-R{u}^{\alpha ,\delta }\right){{\Vert}}_{p}\right)\hfill \\ \hfill & {\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)+{\Vert}K{u}^{\alpha ,\delta }{{\Vert}}_{Y}\right).\hfill \end{aligned}\end{equation*}$$

Now, the last expression is bounded due to boundedness of ${\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{\alpha ,\delta }\right)$ and equi-coercivity of $\left\{{S}_{{f}^{\delta }}\right\}$. Hence, $\left\{{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{\alpha ,\delta }\right)\right\}$ is always bounded and, again using the equi-coercivity of $\left\{{S}_{{f}^{\delta }}\right\}$ and the techniques in the proof of theorem 3.26, one sees that with possible shifts in $\mathrm{ker}\left(K\right)\cap {\mathbf{P}}^{{k}_{1}-1}$, one can achieve that {u^α,δ } is bounded in BV^k (Ω). Therefore, by continuous embedding and reflexivity, it admits a weak accumulation point in L^p (Ω).

Next, let u* be a L^p -weak accumulation point associated with {δ_n }, δ_n → 0 as well as the corresponding parameters {α_n } = {(α_1,n , α_2,n )}. Then, ${S}_{{f}^{{\dagger}}}\left(K{u}^{{\ast}}\right){\leqslant}{\text{lim inf}}_{n\to \infty }{S}_{{f}^{{\delta }_{n}}}\left(K{u}^{{\alpha }_{n},{\delta }_{n}}\right)=0$ by convergence of ${S}_{{f}^{\delta }}$ to ${S}_{{f}^{{\dagger}}}$, so Ku* = f^†. Moreover,

$$\begin{align*}\hfill \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)& {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace {\tilde {\alpha }}_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)+\underset{n\to \infty }{\text{lim inf}}\enspace {\tilde {\alpha }}_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)\hfill \\ \hfill & {\leqslant}\underset{n\to \infty }{\text{lim inf}}\left({\tilde {\alpha }}_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)\hfill \\ \hfill & {\leqslant}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right),\hfill \end{align*}$$

hence, u* is a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution. In particular,

$$\begin{equation*}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)=\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right),\end{equation*}$$

so

$$\begin{equation*}\underset{n\to \infty }{\mathrm{lim}}\left({\tilde {\alpha }}_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)=\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right).\end{equation*}$$

Finally, each sequence of {δ_n }, δ_n → 0 contains another subsequence (not relabelled) for which $\left({\tilde {\alpha }}_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)\to \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)$ as n → ∞, so we have $\left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)\to \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)$ as δ → 0. □

Remark 4.5. 

• Theorem 4.4 shows that, with the additive combination higher-order TV functionals, the maximum of the parameters plays the role of the regularisation parameter. The regularity assumption on u₀ such that Ku₀ = f^† on the other hand depends on whether some parameters converge to zero faster than the maximum or not, i.e., it depends on the ratio of the parameters. Assuming for instance that α₂/max{α₁, α₂} → 0 leads to the weaker ${\mathrm{B}\mathrm{V}}^{{k}_{1}}$-regularity requirement for u₀.In view of a practical parameter choice strategy, this motivates a parametrisation via (α₁, α₂) = α(λ₁, λ₂), where α > 0 is interpreted as the regularisation parameter and chosen in dependence of the noise level, and (λ₁, λ₂) with max{λ₁, λ₂} = 1 is interpreted as model parameter, which is chosen (or learned) once for a particular class of image data and then fixed independent of the concrete forward model or noise level.
• Although (20) incorporates multiple orders, a solution is always contained in ${\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$. Since k₂ ⩾ 2, this space is always contained in H^1,1(Ω), so jump discontinuities cannot appear. One can observe that for numerical solutions, this is reflected in blurry reconstructions of edges while higher-order smoothness is usually captured quite well, see figure 2(c).
• Naturally, it is also possible to consider the weighted sum of more than two TV-type functionals for regularisation, i.e.,
  $$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}{S}_{f}\left(Ku\right)+\left(\sum _{i=1}^{m}{\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\right)\left(u\right).\end{equation} \tag{ 22 }$$
  with orders k₁, ..., k_m ⩾ 1 and weights α₁, ..., α_m > 0. Solutions then exist, for appropriate p, in the space BV^k (Ω) for k = max{k₁, ..., k_m }.

Optimality conditions. As for TV^k , one can also consider optimality conditions for variational problems with ${\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$ as regularisation. Again, in the case that Y is a Hilbert space, q = 2 and ${S}_{f}\left(v\right)=\frac{1}{2}{{\Vert}v-f{\Vert}}_{Y}^{2}$, one can argue according to proposition 3.30 and obtain that u* is optimal for (20) if and only if

$$\begin{equation*}{K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)\in \partial \left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)\end{equation*}$$

or, equivalently,

$$\begin{equation*}{u}^{{\ast}}\in \partial \left({\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)}^{{\ast}}\right)\left({K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)\right).\end{equation*}$$

A difficulty with a further specification of these statements, however, is that it is not immediate that either the subdifferential is additive in this situation or that the dual of the sum of the ${\mathrm{T}\mathrm{V}}^{{k}_{i}}$ equals the infimal convolution of the duals (see definition 4.6 in the next subsection for a definition of the infimal convolution). A possible remedy is to consider the original minimisation problem in the space ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ instead, such that ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ becomes continuous. This, however, yields subgradients in the dual of ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ instead of L^p*(Ω) making the optimality conditions again difficult to interpret.

A priori estimates. In order to obtain a bound on a solution u* for a quadratic Hilbert-norm discrepancy, i.e., ${S}_{f}\left(v\right)=\frac{1}{2}{{\Vert}v-f{\Vert}}_{Y}^{2}$, Y Hilbert space, on can proceed analogously to remark 3.31, provided that K is injective on the space ${\mathbf{P}}^{{k}_{1}-1}$. We then also arrive at (17), with α replaced by α₁, C being the coercivity constant for ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and c the inverse bound for K on ${\mathbf{P}}^{{k}_{1}-1}$. Of course, in case K is still injective on the larger space ${\mathbf{P}}^{{k}_{2}-1}$, the analogous bound can be obtained with α₂ instead of α₁ and respective constants C, c. In case of the Kullback–Leibler discrepancy, i.e., S_f (v) = KL(v, f), the analogous statements apply to the estimate (18).

Denoising performance. Figure 3 shows the effect of α₁TV + α₂TV² regularisation compared to pure TV-regularisation. While staircase artefacts are slightly reduced, the overall image is more blurry than the one obtained with TV, see figures 3(b) and (c). This is expected as additive regularisation inherits the analytical properties of the stronger regularisation term, hence α₁TV + α₂TV² is not able to recover jumps. The result is not much different when ${{\Vert}{\Delta}\cdot {\Vert}}_{\mathcal{M}}$ is used instead of TV², see figure 3(d). Nevertheless, although not discussed in this paper, the issue of limited applicability of ${{\Vert}{\Delta}\cdot {\Vert}}_{\mathcal{M}}$ for regularisation of general inverse problems, as mentioned in remark 3.33, is overcome in an additive combination with TV since the properties of TV are sufficient to guarantee well-posedness results.

Zoom In Zoom Out Reset image size

In order to overcome the smoothing effect of total variation of order two and higher, and additive combinations thereof, another idea would be to model an image u as the sum of a first-order part and a second order part, i.e.,

$$\begin{equation*}u={u}_{1}+{u}_{2}\quad \text{with}\;{u}_{1}\in \mathrm{B}\mathrm{V}\left({\Omega}\right),{u}_{2}\in {\mathrm{B}\mathrm{V}}^{2}\left({\Omega}\right).\end{equation*}$$

This has originally been proposed in [59], and different variants have subsequently been analysed in [16] and considered in [174, 175] in a discrete setting.

Obviously, such a decomposition exists for each u ∈ BV(Ω) but is, of course, not unique. The parts u₁ and u₂ are now regularised with first and second-order total variation associated with some weights α₁ > 0, α₂ > 0. The associated Tikhonov minimisation problem reads as

$$\begin{equation*}\underset{\genfrac{}{}{0pt}{}{{u}_{1}\in \mathrm{B}\mathrm{V}\left({\Omega}\right),}{{u}_{2}\in {\mathrm{B}\mathrm{V}}^{2}\left({\Omega}\right)}}{\mathrm{min}}\enspace {S}_{f}\left(K\left({u}_{1}+{u}_{2}\right)\right)+{\alpha }_{1}\mathrm{T}\mathrm{V}\left({u}_{1}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{2}\left({u}_{2}\right).\end{equation*}$$

As we are only interested in u, we rewrite this problem as

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+\underset{\genfrac{}{}{0pt}{}{{u}_{1}\in \mathrm{B}\mathrm{V}\left({\Omega}\right),}{\genfrac{}{}{0pt}{}{{u}_{2}\in {\mathrm{B}\mathrm{V}}^{2}\left({\Omega}\right),}{u={u}_{1}+{u}_{2}}}}{\mathrm{inf}}\enspace {\alpha }_{1}\mathrm{T}\mathrm{V}\left({u}_{1}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{2}\left({u}_{2}\right).\end{equation} \tag{ 23 }$$

This regularisation functional is called infimal convolution of α₁TV and α₂TV².

Definition 4.6. Let ${F}_{1},{F}_{2}:X\to \left.\right] - \infty ,\infty \left.\right]$. Then,

$$\begin{equation*}\left({F}_{1}{\triangle}{F}_{2}\right)\left(u\right)=\underset{{u}_{1}+{u}_{2}=u}{\mathrm{inf}}\enspace {F}_{1}\left({u}_{1}\right)+{F}_{2}\left({u}_{2}\right)\end{equation*}$$

is the infimal convolution of F₁ and F₂.

An infimal convolution is called exact, if for each u ∈ X there is a pair u₁, u₂ ∈ X with

$$\begin{equation*}{u}_{1}+{u}_{2}=u\quad \text{and}\quad {F}_{1}\left({u}_{1}\right)+{F}_{2}\left({u}_{2}\right)=\left({F}_{1}{\triangle}{F}_{2}\right)\left(u\right).\end{equation*}$$

The infimal convolution may or may not be exact and may or may not be lower semi-continuous, even if both F₁, F₂ are lower semi-continuous. The next proposition, which should be compared to proposition 4.1 above, provides basic properties and the function spaces associated with infimal convolutions.

Proposition 4.7. Let |⋅|₁ and |⋅|₂ be two lower semi-continuous seminorms on the Banach space X. Then,

• (a)  
  The functional $\left\vert \cdot \right\vert =\vert \cdot {\vert }_{1}{\triangle}\vert \cdot {\vert }_{2}$ is a seminorm on X.
• (b)  
  We have

  $$\begin{equation*}\mathrm{ker}\left(\vert \cdot {\vert }_{1}\right)+\mathrm{ker}\left(\vert \cdot {\vert }_{2}\right)\subset \mathrm{ker}\left(\left\vert \cdot \right\vert \right)\end{equation*}$$

  with equality if |⋅|₁△|⋅|₂ is exact.
• (c)  
  If |⋅|₁△|⋅|₂ is lower semi-continuous, then

  $$\begin{equation*}Y=\left\{u\in X\enspace \vert \enspace \left\vert u\right\vert {< }\infty \right\},\quad {{\Vert}u{\Vert}}_{Y}={{\Vert}u{\Vert}}_{X}+\left\vert u\right\vert \end{equation*}$$

  constitutes a Banach space.
• (d)  
  With Y_i the Banach spaces arising from the norms ||⋅||_X + |⋅|_i , i = 1, 2 (see lemma 3.15),

  $$\begin{equation*}{Y}_{i}\hookrightarrow Y\quad \text{for}\enspace i=1,2.\end{equation*}$$
• (e)  
  It holds that ${\left(\vert \cdot {\vert }_{1}{\triangle}\vert \cdot {\vert }_{2}\right)}^{{\ast}}={\vert \cdot {\vert }_{1}}^{{\ast}}+{\vert \cdot {\vert }_{2}}^{{\ast}}$ and if |⋅|₁△|⋅|₂ is exact, then

  $$\begin{equation*}\partial \left({\vert \cdot {\vert }_{1}}^{{\ast}}+{\vert \cdot {\vert }_{2}}^{{\ast}}\right)=\partial {\vert \cdot {\vert }_{1}}^{{\ast}}+\partial {\vert \cdot {\vert }_{2}}^{{\ast}}\end{equation*}$$

Proof. The seminorm axioms can easily be verified for $\left\vert \cdot \right\vert$. If u = u₁ + u₂ for u_i ∈ ker(|⋅|_i ), i = 1, 2, then

$$\begin{equation*}\left\vert u\right\vert {\leqslant}\vert {u}_{1}{\vert }_{1}+\vert {u}_{2}{\vert }_{2}=0.\end{equation*}$$

The converse inclusion follows directly from the exactness.

The third statement is a direct consequence of lemma 3.15 while the forth immediately follows from ${{\Vert}u{\Vert}}_{X}+\left\vert u\right\vert {\leqslant}{{\Vert}u{\Vert}}_{X}+\vert u{\vert }_{i}$ for i = 1, 2.

For the fifth statement, the assertion on the Fenchel dual follows by direct computation. Regarding equality of the subdifferentials, let w ∈ X* and u ∈ X such that $u\in \partial \left({\vert \cdot {\vert }_{1}}^{{\ast}}+{\vert \cdot {\vert }_{2}}^{{\ast}}\right)\left(w\right)$. Then, the Fenchel identity yields

$$\begin{equation}\langle w,\enspace u\rangle =\left(\vert \cdot {\vert }_{1}{\triangle}\vert \cdot {\vert }_{2}\right)\left(u\right)+{\vert \cdot {\vert }_{1}}^{{\ast}}\left(w\right)+{\vert \cdot {\vert }_{2}}^{{\ast}}\left(w\right)+=\vert {u}_{1}{\vert }_{1}+{\vert \cdot {\vert }_{1}}^{{\ast}}\left(w\right)+\vert {u}_{2}{\vert }_{1}+{\vert \cdot {\vert }_{2}}^{{\ast}}\left(w\right)\end{equation} \tag{ 24 }$$

for the minimising u₁, u₂ ∈ X with u₁ + u₂ = u. As by the Fenchel inequality

$$\begin{equation}\langle w,\enspace {u}_{1}\rangle {\leqslant}\vert {u}_{1}{\vert }_{1}+{\vert \cdot {\vert }_{1}}^{{\ast}}\left(w\right)\quad \text{and}\quad \langle w,\enspace {u}_{2}\rangle {\leqslant}\vert {u}_{2}{\vert }_{2}+{\vert \cdot {\vert }_{2}}^{{\ast}}\left(w\right),\end{equation} \tag{ 25 }$$

the equation (24) can only be true when there is equality in (25). But this means, in turn, that ${u}_{1}\in \partial {\vert \cdot {\vert }_{1}}^{{\ast}}\left(w\right)$ and ${u}_{2}\in \partial {\vert \cdot {\vert }_{2}}^{{\ast}}\left(w\right)$. Hence, $\partial \left({\vert \cdot {\vert }_{1}}^{{\ast}}+{\vert \cdot {\vert }_{2}}^{{\ast}}\right)\subset \partial {\vert \cdot {\vert }_{1}}^{{\ast}}+\partial {\vert \cdot {\vert }_{2}}^{{\ast}}$. The other inclusion holds trivially. □

The statement (e) will be relevant for obtaining optimality conditions and we note that, as can be seen from the proof, it holds true for arbitrary convex functionals, not necessarily seminorms.

The previous proposition shows in particular that lower semi-continuity and exactness of the infimal convolution are important for obtaining an appropriate function space setting. Regarding the infimal convolution of TV functionals, this holds true on L^p -spaces as follows.

Proposition 4.8. Let Ω be a bounded Lipschitz domain, 1 ⩽ k₁ < k₂ and p ∈ [1, ∞] with p ⩽ d/(d − k₁) if k₁ < d. Then, for α = (α₁, α₂), α₁ > 0, α₂ > 0, the infimal convolution

$$\begin{equation}{\mathcal{R}}_{\alpha }={\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}},\end{equation} \tag{ 26 }$$

is exact and lower semi-continuous in L^p (Ω).

Proof. By continuous embedding, we may assume without loss of generality that p < ∞. Take a sequence {uⁿ } converging to some u in L^p (Ω) for which ${\text{lim inf}}_{n\to \infty }\enspace {\mathcal{R}}_{\alpha }\left({u}^{n}\right){< }\infty$. For each n, we can select ${u}_{1}^{n},{u}_{2}^{n}\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ such that ${u}^{n}={u}_{1}^{n}+{u}_{2}^{n}$,

$$\begin{equation*}{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right){\leqslant}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right)+\frac{1}{n},\end{equation*}$$

and ${u}_{1}^{n}$ is in the complement of $\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$ in the sense that $R{u}_{1}^{n}=0$ for $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$ a linear and continuous projection. The latter condition can always be satisfied since both ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ are invariant on $\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$. Now, by coercivity of ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ as in corollary 3.23, we get that $\left\{{u}_{1}^{n}\right\}$ is bounded in ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$. Hence, by the embedding of ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ in either L^∞(Ω) or ${L}^{d/\left(d-{k}_{1}\right)}\left({\Omega}\right)$ in case of k₁ < d as in theorem 3.20, the choice of p and convergence of {uⁿ } in L^p (Ω), we can extract (non-relabelled) subsequences of $\left\{{u}_{1}^{n}\right\}$ and $\left\{{u}_{2}^{n}\right\}$ converging weakly to some u₁ and u₂ in L^p (Ω), respectively, such that u = u₁ + u₂. Thus, lower semi-continuity of both ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ implies

$$\begin{align*}\hfill \left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)& {\leqslant}{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}\right)\hfill \\ \hfill & {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace \left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right)\right)\hfill \\ \hfill & =\underset{n\to \infty }{\text{lim inf}}\enspace \left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right)\hfill \end{align*}$$

such that lower semi-continuity holds. Finally, exactness for u ∈ L^p (Ω) with ${\mathcal{R}}_{\alpha }\left(u\right){< }\infty$ follows from choosing {uⁿ } as the sequence that is constant u. □

Given this, the special case $\vert \cdot {\vert }_{i}={\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}$ and X = L¹(Ω) of proposition 4.7 shows that both ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ and ${\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$ are embedded in the Banach space Y. Hence, in contrast to the sum of different TV terms, their infimal convolution allows to recover jumps whenever k₁ = 1, independent of k₂. In fact, as the following proposition shows, the space Y is even equivalent to the BV space corresponding to the lowest order, in particular to BV(Ω) for k₁ = 1. Again, the result should be compared to theorem 4.2 above.

Theorem 4.9. Let 1 ⩽ k₁ < k₂, α₁ > 0, α₂ > 0, Ω be a bounded Lipschitz domain, and Y be the Banach space associated with X = L¹(Ω) and total-variation infimal convolution according to (26). Then,

$$\begin{equation*}Y={\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)\end{equation*}$$

in the sense of Banach space equivalence, and for p ∈ [1, ∞], p ⩽ d/(d − k₁) if k₁ < d, and for $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ a linear, continuous projection there exists a C > 0 such that

$$\begin{equation}{{\Vert}u-Ru{\Vert}}_{p}{\leqslant}C\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)\end{equation} \tag{ 27 }$$

for all u ∈ L^p (Ω).

Proof. We first show the claimed norm equivalence. For this purpose, note that one estimate corresponds to the fourth statement in proposition 4.7.

For the converse estimate, let $u\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ and $R:{L}^{1}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ be a projection. Then,

$$\begin{equation}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right){\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u-Rw\right)\right)\end{equation} \tag{ 28 }$$

for C independent of $u,w\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$. Indeed, if for {uⁿ } and {wⁿ } we have ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{n}\right)=1$ and ${{\Vert}{u}^{n}{\Vert}}_{1}\to 0$ as well as ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{n}-R{w}^{n}\right)\to 0$, meaning uⁿ → 0 in L¹(Ω) and ${\nabla }^{{k}_{1}}\left({u}^{n}-R{w}^{n}\right)\to 0$ in $\mathcal{M}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$. The latter implies that $\left\{{\nabla }^{{k}_{1}}R{w}^{n}\right\}$ is bounded in a finite-dimensional space, hence there is a convergent subsequence (not relabelled) with limit $v\in \mathcal{M}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$. Then, ${\nabla }^{{k}_{1}}{u}^{n}\to v$ in $\mathcal{M}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{{k}_{1}}\left({\mathbf{R}}^{d}\right)\right)$ and the closedness of ${\nabla }^{{k}_{1}}$ yields v = 0 which is a contradiction to ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{n}\right)=1$ for all n.

Using this, together with the estimate

$$\begin{equation}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(w-Rw\right){\leqslant}C{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(w\right)\end{equation} \tag{ 29 }$$

from lemma 3.25, it holds for $u\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ and $w\in {\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$ that

$$\begin{align*}\hfill {\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right)& {\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u-Rw\right)\right)\hfill \\ \hfill & {\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u-w\right)+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(w-Rw\right)\right)\hfill \\ \hfill & {\leqslant}C\left({{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u-w\right)+{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left(w\right)\right).\hfill \end{align*}$$

Taking the infimum over all $w\in {\mathrm{B}\mathrm{V}}^{{k}_{2}}\left({\Omega}\right)$, adding ||u||₁ on both sides as well as observing that ${\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\mathrm{T}\mathrm{V}}^{{k}_{2}}{\leqslant}\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ yields

$$\begin{equation*}{{\Vert}u{\Vert}}_{1}+{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right){\leqslant}C\left({{\Vert}u{\Vert}}_{1}+\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)\right),\end{equation*}$$

and, consequently, the desired norm estimate. Likewise, the estimate ${\Vert}u-Ru{{\Vert}}_{p}{\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)$ follows in analogy to proposition 3.22 and corollary 3.23, which immediately gives the claimed estimate for arbitrary α₁ > 0, α₂ > 0. □

Tikhonov regularisation. Again, the second estimate in theorem 4.9 is crucial as it allows to apply the well-posedness result of theorem 3.26.

Proposition 4.10. With X = L^p (Ω), $p\in \left.\right] 1,\infty \left[\right.$, Ω being a bounded Lipschitz domain, Y a Banach space, K : X → Y linear and continuous, S_f : Y → [0, ∞] proper, convex lower semi-continuous and coercive, 1 ⩽ k₁ < k₂, α₁ > 0, α₂ > 0, the Tikhonov minimisation problem

$$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}\enspace {S}_{f}\left(Ku\right)+\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)\end{equation} \tag{ 30 }$$

is well-posed in the sense of theorem 3.26 whenever p ⩽ d/(d − k₁) if k₁ < d.

Compared to the sum of different TV terms, we see that now the necessary coercivity estimate incorporates a projection to the larger kernel of ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ and an L^p norm with a smaller exponent corresponding to ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$. Hence, in view of the assumptions of theorem 3.26, the infimal convolution of ${\mathrm{T}\mathrm{V}}^{{k}_{1}}$ and ${\mathrm{T}\mathrm{V}}^{{k}_{2}}$ inherits the worst properties of the two summands, i.e., the ones that are more restrictive for applications in an inverse problems context. Nevertheless, such a slightly more restrictive assumption on the continuity of the forward operator is compensated by the fact that the infimal convolution with k₁ = 1 allows to reconstruct jumps. In addition, each solution u* of a Tikhonov functional admits an optimal decomposition ${u}^{{\ast}}={u}_{1}^{{\ast}}+{u}_{2}^{{\ast}}$ with ${u}_{i}^{{\ast}}\in {\mathrm{B}\mathrm{V}}^{{k}_{i}}\left({\Omega}\right)$, i = 1, 2, which follows from the exactness of the infimal convolution.

Regarding the convergence result of theorem 3.28 and the rates of proposition 3.29, again a direct extension to regularisation with ${\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}$ can be obtained by regarding the weights α₁, α₂ to be fixed and introducing an additional factor α > 0 for both terms, which then acts as the regularisation parameter. Considering the limiting behaviour for both weights converging to zero, a counterpart of theorem 4.4 can be obtained as follows. There, we allow also for infinite weights α_i , i.e., for α_i = ∞, we set ${\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left(u\right)=0$ if $u\in \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{i}}\right)$ and ${\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left(u\right)=\infty$ else. We first need a lower semi-continuity result.

Lemma 4.11. Let Ω be a bounded Lipschitz domain, $p\in \left[\right.1,\infty \left[\right.$  with 1 ⩽ p ⩽ d/(d − k₁) if k₁ < d, {(α_1,n , α_2,n )} be a sequence of positive parameters converging to some $\left({\alpha }_{1}^{{\dagger}},{\alpha }_{2}^{{\dagger}}\right)\in { \left.\right]0,\infty \left.\right]}^{2}$ and {uⁿ } be a sequence in L^p (Ω) weakly converging to u* ∈ L^p (Ω). Then,

$$\begin{equation*}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right){\leqslant}\underset{n\to \infty }{\mathrm{lim inf}}\enspace \left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right).\end{equation*}$$

Proof. By moving to a subsequence, we can assume that $\left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right)$ converges to the limes inferior on the right-hand side of the claimed assertion and that the latter is finite. Choose $\left\{{u}_{1}^{n}\right\}$, $\left\{{u}_{2}^{n}\right\}$ sequences such that for each n, we have ${u}_{1}^{n}+{u}_{2}^{n}={u}^{n}$, $\left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right)={\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right)$ and ${u}_{1}^{n}$ being in a complement of $\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$ in the sense that ${u}_{1}^{n}\in \mathrm{ker}\left(R\right)$ for a linear, continuous projection $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$. Setting ${\hat{\alpha }}_{i}=\mathrm{inf}\enspace \left\{{\alpha }_{i,n}\right\}{ >}0$, we obtain

$$\begin{equation*}{\hat{\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\hat{\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right){\leqslant}{\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right).\end{equation*}$$

In particular, for a constant C > 0 it holds that

$$\begin{equation*}{{\Vert}{u}_{1}^{n}{\Vert}}_{p}{\leqslant}C{\hat{\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right){\leqslant}C\left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right)\right),\end{equation*}$$

which implies that $\left\{{u}_{1}^{n}\right\}$ is bounded in ${\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$. By the embedding of theorem 3.20 and since {uⁿ } is convergent, both $\left\{{u}_{1}^{n}\right\}$ and $\left\{{u}_{2}^{n}\right\}$ admit subsequences (not relabelled) weakly converging to some ${u}_{1}^{{\ast}}$ and ${u}_{2}^{{\ast}}$ in L^p (Ω), respectively. Now, in case ${\alpha }_{i}^{{\dagger}}{< }\infty$, we can conclude

$$\begin{equation*}{\alpha }_{i}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}_{i}^{{\ast}}\right){\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace {\alpha }_{i,n}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}_{i}^{n}\right).\end{equation*}$$

Otherwise, we get by boundedness of ${\alpha }_{i,n}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}_{i}^{n}\right)$ that ${\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}_{i}^{n}\right)\to 0$ and by lower semi-continuity that ${\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}_{i}^{{\ast}}\right)=0$. Together, this implies

$$\begin{align*}\hfill \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)& {\leqslant}{\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{{\ast}}\right)+{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{{\ast}}\right)\hfill \\ \hfill & {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace \left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}_{1}^{n}\right)+{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({u}_{2}^{n}\right)\right)\hfill \\ \hfill & =\underset{n\to \infty }{\text{lim inf}}\enspace \left({\alpha }_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{n}\right).\hfill \end{align*}$$

This implies the desired statement. □

Theorem 4.12. In the situation of proposition 4.10 and for $p\in \left.\right] 1,\infty \left[\right.$  with p ⩽ d/(d − k₁) in case of k₁ < d, let for each δ > 0 the data f^δ be given such that ${S}_{{f}^{\delta }}\left({f}^{{\dagger}}\right){\leqslant}\delta$, let $\left\{{S}_{{f}^{\delta }}\right\}$ be equi-coercive and converge to ${S}_{{f}^{{\dagger}}}$ for some data f^† in Y in the sense of (4) and ${S}_{{f}^{{\dagger}}}\left(v\right)=0$ if and only if v = f^†.

Choose the parameters α = (α₁, α₂) in dependence of δ such that

$$\begin{equation*}\mathrm{min}\left\{{\alpha }_{1},{\alpha }_{2}\right\}\to 0,\quad \frac{\delta }{\mathrm{min}\left\{{\alpha }_{1},{\alpha }_{2}\right\}}\to 0,\quad \text{as}\enspace \delta \to 0,\end{equation*}$$

and assume that $\left({\tilde {\alpha }}_{1},{\tilde {\alpha }}_{2}\right)=\left({\alpha }_{1},{\alpha }_{2}\right)/\mathrm{min}\left\{{\alpha }_{1},{\alpha }_{2}\right\}\to \left({\alpha }_{1}^{{\dagger}},{\alpha }_{2}^{{\dagger}}\right)\in {\left.\right]0,\infty \left.\right]}^{2}$ as δ → 0. Set

$$\begin{equation*}k=\begin{cases}_{1}\hfill & \text{if}\enspace {\alpha }_{1}^{{\dagger}}{< }\infty ,\hfill \\ {k}_{2}\hfill & \text{else},\hfill \end{cases}\end{equation*}$$

and assume that there exists u₀ ∈ BV^k (Ω) such that Ku₀ = f^†.

Then, up to shifts in $\mathrm{ker}\left(K\right)\cap {\mathbf{P}}^{{k}_{2}-1}$, any sequence {u^α,δ }, with each u^α,δ being a solution to (30) for parameters (α₁, α₂) and data f^δ , has at least one L^p -weak accumulation point. Each L^p -weak accumulation point is a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution of Ku = f^† and ${\mathrm{lim}}_{\delta \to 0}\left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)=\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)$.

Proof. First note that, with $R:{L}^{p}\left({\Omega}\right)\to \mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$ a linear, continuous projection, for any u ∈ L^p (Ω), we have

$$\begin{equation*}{\Vert}u-Ru{{\Vert}}_{p}{\leqslant}C\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right){\leqslant}C\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right),\end{equation*}$$

and by the choice of k as well as u₀ ∈ BV^k (Ω), that $\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}_{0}\right){< }\infty$. Hence, as a consequence of theorem 3.26, there exists a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution u^† ∈ BV^k (Ω) to Ku = f^†. Using optimality of u^α,δ compared to u^† gives

$$\begin{equation*}{S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)+\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right){\leqslant}\delta +\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right).\end{equation*}$$

Now since $\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right){\leqslant}{\mathrm{min}}_{i=1,2}\left\{{\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\left({u}^{{\dagger}}\right)\right\}$ and min{α₁, α₂} → 0 as δ → 0, we have that ${S}_{{f}^{\delta }}\left(K{u}^{\alpha ,\delta }\right)\to 0$ as δ → 0. Moreover, as also δ/min{α₁, α₂} → 0, it follows that

$$\begin{align*}\hfill \underset{\delta \to 0}{\text{lim sup}}\enspace \left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right)& {\leqslant}\underset{\delta \to 0}{\text{lim sup}}\enspace \left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)\hfill \\ \hfill & {\leqslant}\left(1+\varepsilon \right)\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right)\hfill \end{align*}$$

for ɛ > 0 independent of u^α,δ and letting ɛ → 0, we obtain

$$\begin{equation*}\underset{\delta \to 0}{\text{lim sup}}\left({\tilde {\alpha }}_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\tilde {\alpha }}_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{\alpha ,\delta }\right){\leqslant}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right).\end{equation*}$$

In particular, using (27), we can conclude that {u^α,δ − Ru^α,δ } is bounded in L^p (Ω). By introducing appropriate shifts in $\mathrm{ker}\left(K\right)\cap {\mathbf{P}}^{{k}_{2}-1}$ as done in theorem 3.26 and using the equi-coercivity of $\left\{{S}_{{f}^{\delta }}\right\}$, one can then achieve that {u^α,δ } is bounded in L^p (Ω) such that by reflexivity, it admits a L^p -weak accumulation point.

Next, let u* be a L^p -weak accumulation point associated with {δ_n }, δ_n → 0 as well as the corresponding parameters {α_n } = {(α_1,n , α_2,n )}. Then, ${S}_{{f}^{{\dagger}}}\left(K{u}^{{\ast}}\right){\leqslant}{\text{lim inf}}_{n\to \infty }\enspace {S}_{{f}^{{\delta }_{n}}}\left(K{u}^{{\alpha }_{n},{\delta }_{n}}\right)=0$ by convergence of ${S}_{{f}^{\delta }}$ to ${S}_{{f}^{{\dagger}}}$, so Ku* = f^†. Moreover, employing lemma 4.11, we get

$$\begin{align*}\hfill \left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)& {\leqslant}\underset{n\to \infty }{\text{lim inf}}\enspace \left({\tilde {\alpha }}_{1,n}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\tilde {\alpha }}_{2,n}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\alpha }_{n},{\delta }_{n}}\right)\hfill \\ \hfill & {\leqslant}\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\dagger}}\right),\hfill \end{align*}$$

hence, u* is a minimum-$\left({\alpha }_{1}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}^{{\dagger}}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)$-solution. The remaining assertions follow as in the proof of theorem 4.4 by replacing the sum with the infimal convolution. □

Remark 4.13. 

• Compared to the convergence result for the sum of higher-order TV functionals as in theorem 4.4, we see that now the minimum of the parameters plays the role of the regularisation parameter, but again the ratio of the parameters defines the required regularity assumption on u₀ such that Ku₀ = f^†. For a parameter choice in practice, this again motivates the choice (α₁, α₂) = α(λ₁, λ₂) as in remark 4.5, with α > 0 being interpreted as regularisation parameter and (λ₁, λ₂) with min{λ₁, λ₂} = 1 being interpreted as model parameter.
• It is also possible to construct infimal convolutions of more than two TV-type functionals and, of course, other functionals than TV^k .
• Introducing orders k₁, ..., k_m ⩾ 1 and weights α₁, ..., α_m > 0, one can consider
  $$\begin{equation}\underset{u\in {L}^{p}\left({\Omega}\right)}{\mathrm{min}}{S}_{f}\left(Ku\right)+\left({{\triangle}}_{i=1}^{m}{\alpha }_{i}{\mathrm{T}\mathrm{V}}^{{k}_{i}}\right)\left(u\right).\end{equation} \tag{ 31 }$$
  Solutions then exist, for appropriate p, in the space BV^k (Ω) for k = min{k₁, ..., k_m }.
• The latter is in contrast to the multi-order TV regularisation (20) where the solution space is determined by the highest effective order of differentiation. Letting k_i = 1 for some i, the solution space is then BV(Ω) which allows for discontinuities; a desirable property for image restoration.

Optimality conditions. Again, in the situation that Y is a Hilbert space, q = 2 and ${S}_{f}\left(v\right)=\frac{1}{2}{{\Vert}v-f{\Vert}}_{Y}^{2}$, we obtain some first-order optimality conditions. Noting that the dual of the infimal convolution of two functions is the sum of the respective duals, and arguing according to proposition 3.30, an u* is optimal for (30) if and only if

$$\begin{equation*}{u}^{{\ast}}\in \partial \left({\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)}^{{\ast}}+{\left({\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)}^{{\ast}}\right)\left({K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)\right).\end{equation*}$$

By proposition 4.7, the subgradients are additive, so in terms of the normal cones introduced in proposition 3.30, the optimality condition reads as

$$\begin{equation}{u}^{{\ast}}\in {\mathcal{N}}_{{\mathrm{T}\mathrm{V}}^{{k}_{1}}}\left(\frac{{K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)}{{\alpha }_{1}}\right)+{\mathcal{N}}_{{\mathrm{T}\mathrm{V}}^{{k}_{2}}}\left(\frac{{K}^{{\ast}}\left(f-K{u}^{{\ast}}\right)}{{\alpha }_{2}}\right).\end{equation} \tag{ 32 }$$

A priori estimates. Also here, in the above Hilbert space situation, i.e., ${S}_{f}\left(v\right)=\frac{1}{2}{{\Vert}v-f{\Vert}}_{Y}^{2}$ and Y Hilbert space, an a-priori bound of solutions u* can be derived thanks to the coercivity estimate (27). One indeed has ${{\Vert}u-Ru{\Vert}}_{p}{\leqslant}\frac{1}{2\mathrm{min}\left\{{\alpha }_{1},{\alpha }_{2}\right\}}C{{\Vert}f{\Vert}}_{Y}^{2}$ with R and C coming from (27). Hence, assuming that K is injective on ${\mathbf{P}}^{{k}_{2}-1}$, which leads to c||Ru||_p ⩽ ||KRu||_Y for all u and some c > 0, one proceeds analogously to remark 3.31 to obtain the bound (17) with α replaced by min{α₁, α₂}. By analogy, for S_f (v) = KL(v, f) being the Kullback–Leibler discrepancy, an a priori estimate of the type (18) follows.

Moreover, it is possible to control w* up to ${\mathbf{P}}^{{k}_{1}-1}$ whenever $\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{{\ast}}\right)={\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\ast}}-{w}^{{\ast}}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({w}^{{\ast}}\right)$. Let, in the following, C_f ⩾ 0 be an a priori estimate for the optimal functional value, for instance, ${C}_{f}=\frac{1}{2}{{\Vert}f{\Vert}}_{Y}^{2}$ in case of ${S}_{f}\left(v\right)=\frac{1}{2}{{\Vert}v-f{\Vert}}_{Y}^{2}$, and ${C}_{f}=\mathrm{K}\mathrm{L}\left(K{u}^{0},f\right)+\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left({u}^{0}\right)$ for a ${u}^{0}\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left({\Omega}\right)$ with KL(Ku⁰, f) < ∞ in case of S_f (v) = KL(v, f). Further, denoting by $\tilde {C}{ >}0$ a constant such that ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left(u\right){\leqslant}\tilde {C}\left({{\Vert}u{\Vert}}_{1}+\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}\left({\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}{\triangle}{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\right)\left(u\right)\right)$ for all $u\in {\mathrm{B}\mathrm{V}}^{{k}_{1}}\left(u\right)$ (which exists by virtue of the norm equivalence in theorem 4.9), we see that ${\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\ast}}\right){\leqslant}\tilde {C}\left({\left\vert {\Omega}\right\vert }^{1/p}{{\Vert}{u}^{{\ast}}{\Vert}}_{p}+\mathrm{min}{\left\{{\alpha }_{1},{\alpha }_{2}\right\}}^{-1}{C}_{f}\right)$, hence

$$\begin{equation*}{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({w}^{{\ast}}\right){\leqslant}{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\ast}}-{w}^{{\ast}}\right)+{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\ast}}\right)+{\alpha }_{2}{\mathrm{T}\mathrm{V}}^{{k}_{2}}\left({w}^{{\ast}}\right){\leqslant}{\alpha }_{1}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({u}^{{\ast}}\right)+{C}_{f}.\end{equation*}$$

Consequently, we obtain the bound

$$\begin{equation}{\mathrm{T}\mathrm{V}}^{{k}_{1}}\left({w}^{{\ast}}\right){\leqslant}\tilde {C}{\left\vert {\Omega}\right\vert }^{1/p}{{\Vert}{u}^{{\ast}}{\Vert}}_{p}+\frac{\left(\tilde {C}+1\right){C}_{f}}{\mathrm{min}\left\{{\alpha }_{1},{\alpha }_{2}\right\}},\end{equation} \tag{ 33 }$$

which gives an a priori estimate when plugging in the already-obtained bound on ${{\Vert}{u}^{{\ast}}{\Vert}}_{p}$. Moreover, this estimate implies a bound on ${{\Vert}{w}^{{\ast}}-R{w}^{{\ast}}{\Vert}}_{p}$ by the Poincaré–Wirtinger inequality. However, the norm of w* can not fully be controlled since adding an element in ${\mathbf{P}}^{{k}_{1}-1}=\mathrm{ker}\left({\mathrm{T}\mathrm{V}}^{{k}_{1}}\right)$ to w* would still realise the infimum in the infimal convolution. Thus, an estimate of the type (33) is the best one could except in the considered setting.

Denoising performance. Figure 4 shows that it is indeed beneficial for denoising to regularise with α₁TV△α₂TV² compared to pure TV-regularisation: higher-order features as well as edges are recognised by this image model. Nevertheless, staircase artefacts are still present, see figure 4(c). Essentially, this does not change when the second-order component of the infimal convolution is replaced, for instance by ${{\Vert}{\Delta}\cdot {\Vert}}_{\mathcal{M}}$ as in remark 3.33, see figure 4(d). (For the latter penalty functional, basically the same problems as the ones mentioned in remark 3.33 appear.)

Zoom In Zoom Out Reset image size

As a motivation for TGV, consider the formal predual ball associated with the infimal convolution ${\mathcal{R}}_{\alpha }={\alpha }_{1}\mathrm{T}\mathrm{V}{\triangle}{\alpha }_{0}{\mathrm{T}\mathrm{V}}^{2}$ for α = (α₀, α₁), α₀, α₁ > 0. Then

$$\begin{equation*}{\mathcal{R}}_{\alpha }^{{\ast}}={\left({\alpha }_{1}\mathrm{T}\mathrm{V}\right)}^{{\ast}}+{\left({\alpha }_{0}{\mathrm{T}\mathrm{V}}^{2}\right)}^{{\ast}}={\mathcal{I}}_{\overline{\mathcal{B}}}\quad \text{with}\quad \overline{\mathcal{B}}={\alpha }_{1}\overline{{\mathcal{B}}_{\mathrm{T}\mathrm{V}}}\cap {\alpha }_{0}\overline{{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{2}}}\end{equation*}$$

and

$$\begin{align*}\hfill {\alpha }_{1}{\mathcal{B}}_{\mathrm{T}\mathrm{V}}& =\left\{\mathrm{d}\mathrm{i}\mathrm{v}\enspace {\varphi }_{1}\enspace \vert \enspace {\varphi }_{1}\in {\mathcal{C}}_{\mathrm{c}}^{1}\left({\Omega},{\mathbf{R}}^{d}\right),\enspace {{\Vert}{\varphi }_{1}{\Vert}}_{\infty }{\leqslant}{\alpha }_{1}\right\},\hfill \\ \hfill {\alpha }_{0}{\mathcal{B}}_{{\mathrm{T}\mathrm{V}}^{2}}& =\left\{{\mathrm{d}\mathrm{i}\mathrm{v}}^{2}{\varphi }_{2}\enspace \vert \enspace {\varphi }_{2}\in {\mathcal{C}}_{\mathrm{c}}^{2}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{2}\left({\mathbf{R}}^{d}\right)\right),\enspace {{\Vert}{\varphi }_{2}{\Vert}}_{\infty }{\leqslant}{\alpha }_{0}\right\}.\hfill \end{align*}$$

Neglecting the closure for a moment, this leads to the predual ball according to

$$\begin{equation}\mathcal{B}=\left\{\varphi \enspace \vert \enspace \varphi =\mathrm{d}\mathrm{i}\mathrm{v}\enspace {\varphi }_{1}={\mathrm{d}\mathrm{i}\mathrm{v}}^{2}{\varphi }_{2},\enspace {\varphi }_{i}\in {\mathcal{C}}_{\mathrm{c}}^{i}\left({\Omega},{\mathrm{S}\mathrm{y}\mathrm{m}}^{i}\left({\mathbf{R}}^{d}\right)\right),\enspace {{\Vert}{\varphi }_{i}{\Vert}}_{\infty }{\leqslant}{\alpha }_{2-i},\enspace i=1,2\right\}.\end{equation} \tag{ 34 }$$

Each $\varphi \in \mathcal{B}$ possesses a representation as an ∞-bounded first and second-order divergence of some φ₁ and φ₂. However, as the kernel of the divergence is non-trivial (and even infinite-dimensional for d ⩾ 2), we can only conclude that φ₁ = div φ₂ + η for some η with div η = 0. Enforcing η = 0 thus gives the set