Beating noise with abstention in state estimation

To enable the possibility of abstaining, the POVM representing the measurement must include the abstention operator, which we denote by Π₀, in addition to the operators {Π_χ}, each of which are associated with a specific estimate  $|\vec n_\chi \rangle$. Thus, the completeness relation reads

$$\begin{equation} \sum_\chi \Pi_\chi +\Pi_0=\mathbbm{1}. \end{equation} \tag{ 19 }$$

The probability of abstention (abstention rate) and that of producing an estimate (acceptance rate) are then given, respectively, by

$$\begin{equation} Q=\int {\rm d}n\,{\rm tr}\left[\Pi_0\tau(\vec n)\right]\quad \mbox{and}\quad \skew3\bar Q=1-Q, \end{equation} \tag{ 20 }$$

and the mean fidelity defined in (4) now becomes

$$\begin{equation} F(Q)=\frac{1}{\skew3\bar Q}\sum_\chi\int {\rm d}n\, f(\vec n,\vec n_\chi){\rm tr}\left[\Pi_\chi \tau(\vec n)\right] , \end{equation} \tag{ 21 }$$

where note that the sum does not include the Π₀ operator and $\skew3\bar Q$ takes into account the abstentions excluded from the average.

We next note that for any unitary transformation U of the type defined after equation (6), the operators {UΠ_χU^†,UΠ₀U^†} give the same value of Q and F(Q) as the original set {Π_χ,Π₀}, provided we change the guessing rule as $\vec n_\chi \to {\mathcal R}_U\vec n_\chi$, where  ${\mathcal R}_U$ is the SO(3) rotation whose unitary representation is U. Therefore, one can easily prove that Π₀ (the set {Π_χ}) can always be chosen to be SU(2) invariant (covariant) by simply averaging over U. In other words, with no loss of generality, the POVM elements that provide a guess  $|\vec s\rangle$ can be chosen as

$$\begin{equation} \tilde\Pi(\vec s)=U(\vec{s})\,\Pi\, U^\dagger(\vec{s}) , \end{equation} \tag{ 22 }$$

where Π ⩾ 0 is the so-called seed of the POVM (in particular, note that  $\tilde \Pi (\hat z)=\Pi$). The abstention operator then reads

$$\begin{equation} \Pi_0=\mathbbm{1}- \int {\rm d}s\, \tilde{\Pi}(\vec{s}) , \end{equation} \tag{ 23 }$$

which is manifestly rotationally invariant (as claimed above). It is thus proportional to the identity on each invariant subspace

$$\begin{equation} \Pi_0= \bigoplus_{j=j_{\rm min}}^J a_j \mathbbm{1}_j, \end{equation} \tag{ 24 }$$

where a_j are coefficients that satisfy the condition 0 ⩽ a_j ⩽ 1 and $\mathbbm {1}_j$ is the projector onto the corresponding eigenspace j previously defined. We can also choose  $\tilde {\Pi }(\vec s)$ to have the block-diagonal form of the input state  $\tau (\vec n)$, namely,

$$\begin{equation} \tilde{\Pi}(\vec{s})=\bigoplus_{j=j_{\rm min}}^J \tilde{\Pi}_{j}(\vec{s})\otimes\mathbbm{1}^{(j)}_{\rm rep}. \end{equation} \tag{ 25 }$$

For a given {a_j}, the optimality of $\Pi _{j}(\vec {s})$, defined in equation (14), clearly ensures that

$$\begin{equation} \tilde{\Pi}_{j}(\vec{s})=(1-a_j) \Pi_{j}(\vec{s}) \end{equation} \tag{ 26 }$$

are also optimal for estimation with abstention. We have from (20) that the abstention probability is simply

$$\begin{equation} Q=\sum_{j=j_{\rm min}}^J p_j a_j , \end{equation} \tag{ 27 }$$

where p_j is given in equation (12). The coefficients a_j can be understood as the probabilities of abstention conditional on the input state having total angular momentum j, i.e. a_j = p(abstention|j). Similarly, for a given j, the probability of producing an estimate, or accepting, is $\bar {a}_j=1-a_j=p ~(\mbox {acceptance}| j)$.

From equation (21) we obtain

$$\begin{equation} F(Q)=\frac{1}{2} \left( 1 + \sum_{j=j_{\rm min}}^J p_j\tilde{\Delta}_j\right), \end{equation} \tag{ 28 }$$

where

$$\begin{equation} \tilde{\Delta}_j=\frac{1-a_j}{1-Q}\Delta_j=\frac{\bar{a}_j}{\bar{Q}}\Delta_j, \end{equation} \tag{ 29 }$$

and the quantity Δ_j is given in equations (16) and (17). Thus, we are only left with the free parameters a_j, which have to be optimized in order to maximize F(Q), subject to the constraints 0 ⩽ a_j ⩽ 1 and (27). Somehow expected, one can show that Δ_j is a monotonically increasing function of j, i.e. Δ_j−1 < Δ_j; therefore the largest contribution to the fidelity is given by Δ_J. This corresponds to $\bar a_{J}=1$ and  $\bar a_j=0$, j < J. Hence, for an unrestricted probability of abstention, the optimal protocol discards any contribution with j < J. This protocol, however, would provide an estimate with a probability that decreases exponentially with N, for r < 1, as p_J ≃ (1/r)[(1 + r)/2]^N+1. Note that in a noiseless scenario, r = 1, there is only the contribution j = J, which is already the optimal one and therefore abstention is of no use in such a case.

Clearly, for finite Q there can be contributions from other total angular momentum eigenspaces (j < J) compatible with equation (27). Recalling the monotonicity of Δ_j, and by convexity, it is obvious from equations (28) and (29) that there must exist an angular momentum threshold j* such that  $\bar a_j=0$ ($\bar a_j=1$), if j < j* (j > j*). The value j* is determined through equation (27) to be

$$\begin{equation} j^*=\max\left\{j \ \mbox{such that} \ Q-\sum \limits_{j'=j_{{\rm min}}}^{j-1} p_{j'} \geqslant 0 \right\}. \end{equation} \tag{ 30 }$$

Thus, we have

$$\begin{equation} a_j=\left\{\begin{array}{ll} 1, & j<j^*, \\ \displaystyle p_j^{-1} \left(Q-\sum\limits_{j'=j_{{\rm min}}}^{j^*-1}p_{j'}\right), & j=j^*, \\ 0 , & j>j^* . \end{array} \right. \end{equation} \tag{ 31 }$$

In a more physical language, the optimal strategy consists actually of two successive measurements. The experimentalist first makes a weak measurement to find the total angular momentum j of the input state $\tau (\vec n)$ and decides to abstain (provide a guess) if j < j* (j > j*). If j = j*, she simply decides randomly, by tossing a Bernoulli coin with probability a_j* of coming up heads, and if heads (tails) show up, abstain (provide a guess). In order to provide the actual guess, if she decides to do so, she makes the optimal POVM measurement  $\{\Pi (\vec s)\}$ (or just $\{\Pi _{j}(\vec s)\}$) in equation (13) on the state $\tau _{j}(\vec n)$ that resulted from the first measurement.

In figure 1 we plot the fidelity gain due to abstention [F(Q) − F(0)]/F(0) = ΔF/F(0) versus Q for N = 6, 8, and purities of r = 0.3 and 0.7. The structure of equation (31) is apparent from these plots: at Q = 0 (a_j = 0 for all j) there is, naturally, no gain; kinks sequentially appear at the precise values of Q where a new coefficient a_j in (31) becomes positive (and j* increases by one); the curves are convex between successive kinks, where the one a_j that has become positive, a_j*, keeps increasing. This pattern repeats until the abstention rate Q reaches a critical value Q_crit at which j* = J,

$$\begin{equation} Q_{{\rm crit}}=1-p_{J}=1-{1\over r}\left(\frac{1+r}{2}\right)^{2J+1}\kern-1em+{1\over r}\left(\frac{1-r}{2}\right)^{2J+1} \end{equation} \tag{ 32 }$$

(see equation (12)). Increasing Q further will not provide any additional gain, as the flat plateaus of figure 1 illustrate. This is so, since one can view the optimal abstention protocol as a filtering process where the low angular momentum components of the input state are filtered out. Hence, keeping the maximum value of j = J is the optimal filtering beyond which no further improvement is possible. Figure 1(a) shows that in noisy scenarios, e.g. r = 0.3, abstention can increase the fidelity quite notably, up to 15%. For higher purities the gain is more moderate, as shown in figure 1(b). The enhancement in this case is about 4–5% but with an abstention rate slightly above 50%. Further results are shown in figure 2, where we plot the fidelity gain as a function of the number of copies N for abstention rates larger than Q_crit, and for various values of the purity r. All the curves have a maximum at a value of N that varies with the purity. The lower the purity, the higher the value of N at which the maximum occurs (e.g. for r = 0.3 the maximum gain occurs at N = 12; for r = 0.1 the maximum is off scale at the right of the figure).

Zoom In Zoom Out Reset image size

Zoom In Zoom Out Reset image size

As we have seen, the possibility of abstaining enables us to reach values of the fidelity that otherwise we could only attain with lower levels of noise. To quantify this effective reduction of noise, let us define an effective purity r_eff by the implicit equation F(r_eff,N,0) = F(r,N,Q). That is, for an estimation setting, given by r, N and Q, r_eff is the purity of the input states that would provide the same fidelity if the standard strategy without abstention (Q = 0) were used instead. Since r is related to the probability of error η in our model of noisy measurements in (1), an increase of the effective purity corresponds to an effective reduction of the amount of noise in the measurement through the relation η_eff = (3/4)(1 − r_eff). Figure 3 shows a plot of the effective purity r_eff as a function of Q for various values of r and N. As can be seen, r_eff increases faster at low values of N, but it saturates earlier (lower Q_crit), reaching a lower value. For low N and for a wide range of purities, 0.1 ≲ r ≲ 0.9, we observe a constant effective increase of the purity, r_eff ≈ r + 0.2, for reasonable values of the abstention rate Q. As N increases one has to go to higher values of the abstention rate, Q ∼ Q_crit, to have a significant gain. Hence, a moderate abstention rate is most effective in noisy scenarios when a small, but fair, number of copies are available.

Zoom In Zoom Out Reset image size

Finally, let us point out that the protocol we have presented requires a projection on the total angular momentum eigenspaces. This is a non-local measurement that nonetheless can be implemented efficiently [31]. In a more extreme scenario where there are no restrictions on the abstention rate, one can attain the maximum fidelity with an even simpler strategy: make a local Stern–Gerlach measurement on every qubit (say, of the z-component of the spin) and abstain unless all outcomes agree. This strategy renders an abstention probability of Q = 1 − [(1 + r)/2]^N, which might be comparable to Q_crit in equation (32).

We next compute the analytical expressions of the fidelity in the large N limit. Here it is useful to define the variable x as

$$\begin{equation} x={j\over J},\quad 0\leqslant x\leqslant 1, \end{equation} \tag{ 33 }$$

which becomes continuous in the limit N → ∞ (J → ∞). In this case, we can replace p_j by the continuous probability distribution in [0,1] defined by

$$\begin{equation} p(x)=J p_{j=x J}, \end{equation} \tag{ 34 }$$

so that $\int _0^1 \mathrm {d}x\,p(x)=1$ as N goes to infinity. Equation (28) can then be approximated by its continuous version, which reads

$$\begin{equation} F={1\over2}\left[1+\int_0^1 {\rm d}x\, p(x)\tilde\Delta(x)\right], \end{equation} \tag{ 35 }$$

where

$$\begin{equation} \tilde\Delta(x)=\tilde\Delta_{j=x J}, \end{equation} \tag{ 36 }$$

where recall that $\tilde \Delta _j$ is given in equation (29). From equation (31) we see that asymptotically $\bar a_j$ becomes the step function θ(x − x*), where x* = j*/J, and we have used the standard definition

$$\begin{equation} \theta(x)=\left\{ \begin{array}{@{}ll@{}} 1, & x\geqslant 0,\\ 0, & x<0 . \end{array} \right. \end{equation} \tag{ 37 }$$

With this, equation (29) becomes

$$\begin{equation} \tilde\Delta(x)={\theta(x-x^*)\over \skew3\bar Q}\Delta(x), \end{equation} \tag{ 38 }$$

and, in turn,

$$\begin{equation} F={1\over2}\left[1+{1\over\skew3\bar Q}\int_{x^*}^1{\rm d}x\,p(x)\Delta(x)\right]. \end{equation} \tag{ 39 }$$

It also follows from (27) that

$$\begin{equation} \skew3\bar Q=\int_0^1 {\rm d}x\, p(x)\theta(x-x^*)=\int_{x^*}^1 {\rm d}x\, p(x) . \end{equation} \tag{ 40 }$$

At this point, we need to find a good approximation to p(x) that would enable us to obtain the explicit form for the asymptotic fidelity. From equation (12), and using the Stirling formula, we obtain

$$\begin{equation} p(x)\simeq \sqrt{\frac{N}{2 \pi}}\frac{1}{\sqrt{1-x^2}} {x(1+r)\over r(1+x)}\;{\rm e}^{- N H(\frac{1+x}{2}\parallel\frac{1+r}{2})}, \end{equation} \tag{ 41 }$$

where H(s∥t) is the (binary) relative entropy

$$\begin{equation} H(s\parallel t)=s \log\frac{s}{t}+ (1-s)\log\frac{1-s}{1-t} , \end{equation} \tag{ 42 }$$

and the approximation is valid for both x and r in the open unit interval (0,1). The appearance of a relative entropy³ in equation (41) can be understood as follows. Our N-copy input state (diagonal in the canonical J_n basis) can be thought of as a classical coin tossing distribution of N identical coins with a bias of (1 + r)/2. From the theory of types [30], it is well known that the probability to obtain k heads is given by the Kulback–Leibler distance (or relative entropy) between the empirical distribution {f = k/N,1 − f} and the distribution {(1 + r)/2,(1 − r)/2}. That is, $p(k)\sim \exp \{-N H[ f \parallel (1+r)/2]\}$ to first order in the exponent. The number of heads k is in one-to-one correspondence with the magnetic quantum number, m = k − J, and the conditioned probability p(j|m) is strongly peaked at m = j, as one can easily check. It follows that the probability that the input state has total angular momentum j, given by  $p(j)=\sum _{m} p(j|m)p(m)$, will be asymptotically determined by the probability distribution p(m), which has a convenient expression in terms of the typical and the empirical distribution of up/down outcomes.

From equation (41) it follows that p(x) is peaked at the value x = r, i.e. at j = rJ, as shown in figure 4 and stated without a proof in section 2. Actually, around the peak, x ∼ r, the exponent becomes quadratic and p(x) approaches the Gaussian distribution

$$\begin{equation} p(x)\simeq\sqrt{N\over2\pi(1-r^2)} \,{\rm e}^{ -N \frac{(x-r)^2}{2(1-r^2)}}, \end{equation} \tag{ 43 }$$

as also follows from the central limit theorem, whereas it falls off exponentially elsewhere.

Zoom In Zoom Out Reset image size

It is now apparent that, asymptotically, abstention has negligible impact if components with j below rJ are filtered out (x* < r), since the main contribution to the fidelity, which comes from the peak around x ≃ r, is not excluded from the integral in equation (39) (only the left exponentially decaying tail is). For the same reason (see equation (40)), $\skew3\bar Q\simeq 1$ (the abstention rate Q is exponentially small), and equation (39) yields

$$\begin{equation} F = 1- \frac{1}{2N}\frac{r+1}{r^{2}}+\cdots \quad \mbox{ for }\ \ x^*< r, \end{equation} \tag{ 44 }$$

which is the same expression as the asymptotic fidelity of the protocol without abstention, equation (18).

It is then clear that, in order to have a discernible improvement in the fidelity, the abstention threshold x* must lie on the right of the peak of the probability distribution. The fidelity in (39) then can be written as

$$\begin{equation} F \simeq \frac{1}{2} \left[1+ {p(x^{*})\over\skew3\bar Q}\Delta(x^{*})\right] \simeq \frac{1}{2}\left[1+\Delta(x^{*})\ \right],\quad x^*> r, \end{equation} \tag{ 45 }$$

where we have used that for x ⩾ x* > r and for large enough N, p(x) falls off exponentially and the integral can be approximated by the value of the integrand at its lower limit. By the very same argument equation (40) gives

$$\begin{equation} \skew3\bar Q\simeq p(x^*), \end{equation} \tag{ 46 }$$

which has also been used in (45). Using now (41) we obtain that in the asymptotic limit of many copies, the rate at which our protocol provides a guess is

$$\begin{equation} \bar{Q}\sim \exp\left[- N H\left(\frac{1+x^*}{2}\left\| \frac{1+r}{2}\right)\right.\right] . \end{equation} \tag{ 47 }$$

Recalling equations (16) and (17) we obtain the optimal fidelity

$$\begin{equation} F=1- \frac{1}{2Nx^{*}}\frac{r+1}{r}+\cdots ,\quad \mbox{for $r\leqslant x^{*}\leqslant 1$}, \end{equation} \tag{ 48 }$$

for a value of Q given by (47). For x* = r the results (18) and (44) are recovered, whereas for x* → 1 (Q ⩾ Q_crit) the maximum average fidelity is attained:

$$\begin{equation} F_{{\rm max}}=1- \frac{1}{2N }\frac{r+1}{r}+\cdots . \end{equation} \tag{ 49 }$$

The advantage provided by our estimation with the abstention protocol can be quantified by the effective number of copies that the standard protocol without abstention would require to achieve the same fidelity: $N_{\mathrm {eff}}=(x^*\kern -.2em/r) N$, where x*∈[r,1) is determined by the abstention rate Q through (47). For high noise levels (low purity, r ≪ 1) our protocol provides an important saving of resources/copies, as N_eff/N = 1/r ≫ 1, whereas for nearly ideal detectors the saving in this asymptotic regime is more modest.

Alternatively, the advantage discussed above can also be quantified by the effective measurement-noise reduction, or equivalently, the effective purity r_eff (see section 3.2). Using (48) one can easily find a simple expression for the effective purity in the asymptotic limit and for large abstention rate: $r_{\mathrm {eff}} = (r+\sqrt {4r+5 r^{2}})/[2 (1+r)]$. In the limit of very low noise levels the errors probability η (recall equation (1)) is effectively reduced by a factor of three, i.e. η_eff = η/3, while in the opposite limit of very noisy measurements, one finds that $r_{\mathrm {eff}}=\sqrt {r}$.

In the previous section we have seen how a gain in fidelity can be obtained provided the 'acceptance' rate $\skew3\bar Q$ falls off exponentially as N becomes very large. Here we give an example where this gain takes place even at finite  $\skew3\bar Q$.

At a fixed noise level (purity r), the fidelity is an increasing function of N. However, one could imagine an experimental setup where the noise (purity) also increases (decreases) with N. If this is so, the asymptotic fidelity could be strictly less than one, or in other words, perfect estimation could be unattainable even with unbounded resources. This is the case in our example, were we assume that $r={a}/\sqrt N$, a being a positive constant. Note that the threshold x* must also scale as $1/\sqrt {N}$ in order to have a reasonably low abstention rate. Therefore, it is convenient to use a new variable $\xi =\sqrt {N} x=\sqrt N\, j/J=2j/\sqrt N$ instead. Then, the probability distribution in this new variable is

$$\begin{equation} p(\xi)={\sqrt N\over2} p_{j=\xi\sqrt N/2},\quad {\rm with}\ r={a\over\sqrt{N}}. \end{equation} \tag{ 50 }$$

Recalling equation (12) and using the Stirling formula this equation gives

$$\begin{equation} p(\xi)={{\rm e}^{-\left({\xi-{a}\over\sqrt2}\right)^2}-{\rm e}^{-\left({\xi+{a}\over\sqrt2}\right)^2}\over\sqrt{2\pi}{a}}\xi \end{equation} \noindent \tag{ 51 }$$

to leading order in inverse powers of N. The subleading terms are of order N^−1/2 and will be neglected here. For a given threshold value $\xi ^* =2 j^*/\sqrt {N}$ the abstention rate is

$$\begin{equation} Q=\int_0^{\xi^*}p(\xi)\,{\rm d}\xi={1\over2}\left({\rm erf}\,{\xi^*_+}+{\rm erf}\,{\xi_{-}^*}\right)-{{\rm e}^{-{\xi^*_-}^2}-{\rm e}^{-{\xi^*_+}^2}\over\sqrt{2\pi}{a}} , \end{equation} \noindent \tag{ 52 }$$

where ${\xi ^*_{\pm }}=(\xi ^*\pm {a})/\sqrt 2$ and erf x is the error function.

From equations (16) and (17) we have in this same regime and at leading order

$$\begin{equation} \Delta(\xi)=\Delta_{j=\xi\sqrt N/2}=1-{2\over1-{\rm e}^{2{a}\xi}}-{1\over{a}\xi} . \end{equation} \noindent \tag{ 53 }$$

With the above, the fidelity (28) (or rather, the counterpart of (35)) is

$$\begin{equation} F={1\over2}\left[1+\int_0^\infty {\rm d}\xi\, p(\xi)\tilde\Delta(\xi)\right]= {1\over2}\left[1+{1\over\skew3\bar Q}\int_{\xi^*}^\infty {\rm d}\xi\, p(\xi)\Delta(\xi)\right], \end{equation} \noindent \tag{ 54 }$$

where the last integral can be computed to be

$$\begin{eqnarray} \Delta^*&\equiv&\int_{\xi^*}^\infty\Delta(\xi)p(\xi)\,{\rm d}\xi\\ &=&{1-{a}^2\over2{a}^2}\left({\rm erf}\,{\xi^*_-}-{\rm erf}\,{\xi^*_+}\right) +{{\rm e}^{-{\xi^*_-}^2}+{\rm e}^{-{\xi^*_+}^2}\over\sqrt{2\pi}{a}} .\nonumber \end{eqnarray} \noindent \tag{ 55 }$$

We can finally write the fidelity as

$$\begin{equation} F={1\over2}\left(1+\frac{\Delta^*}{1- Q}\right)+{\mathcal O}(N^{-1/2}). \end{equation} \noindent \tag{ 56 }$$

As shown in (52) and (55), both Q and Δ* are functions of the filtering threshold ξ*, which is just a properly scaled version of the original threshold j*. Finding the maximum fidelity for a given rate of abstention Q requires inverting equation (52) to obtain ξ*(Q), but this cannot be done analytically and one has to resort to numerical methods.

In figure 5 we plot F as a function of Q for a = 1. The increase of the fidelity in the asymptotic regime of large N is clearly seen: e.g., an abstention rate of 50% yields a rise of about 10%, and it goes up to about 30% for higher (but still reasonable) values of Q. The figure also shows the agreement between the approximate form of the fidelity given by equations (52)–(56) and the numerical evaluation of its exact expression in (28).

Zoom In Zoom Out Reset image size

It should be noted that in the regime described here a rise of the input size N fails to replicate the fidelity improvement that results from increasing the rate of abstention (no N_eff can be defined in this regime); thus abstention appears to be the only means by which one can improve estimation.