Diversity of sharp restart

Sharp restart is an algorithm that is described as follows [18–20]. There is a general task with completion time T, a positive-valued random variable. To this task a three-steps algorithm, with a positive deterministic timer τ, is applied. Step I: initiate simultaneously the task and the timer. Step II: if the task is accomplished up to the timer's expiration—i.e. if $T\leqslant \tau$—then stop upon completion. Step III: if the task is not accomplished up to the timer's expiration—i.e. if $T\gt\tau$—then, as the timer expires, go back to Step I.

The sharp-restart algorithm generates an iterative process of independent and statistically identical task-completion trials. This process halts during its first successful trial, and we denote by T_R its halting time. Namely, T_R is the overall time it takes—when the sharp-restart algorithm is applied—to complete the task. The sharp-restart algorithm is a non-linear mapping whose input is the random variable T, whose output is the random variable T_R , and whose parameter is the deterministic timer τ.

Henceforth, we set the task-completion process to start at time t = 0; thus, the process takes place over the non-negative time axis $t\geqslant 0$. Along this paper we use the following notation regarding the input's statistics: distribution function, $F\left( t\right) = \Pr \left( T\leqslant t\right)$ ($t\geqslant 0$); survival function, $\bar{F}\left( t\right) = \Pr \left( T\gt t\right)$ ($t\geqslant 0$); and density function, $f\left( t\right) = F^{^{\prime} }\left(t\right) = -\bar{F}^{^{\prime} }\left( t\right)$ (t > 0). The input's density function is considered to be positive-valued over the positive half-line: $f\left( t\right) \gt0$ for all t > 0.

In terms of the input's survival and density functions, the output's density function $f_{R}\left( t\right)$ admits the following representation [20]:

$$\begin{align} f_{R}\left( \tau n+u\right) =\bar{F}\left( \tau \right) ^{n}f\left( u\right), \end{align} \tag{ 1 }$$

where $n = 0,1,2,\ldots$, and where $0\leqslant u\lt\tau$. Indeed, in order that the output T_R be realized at time $t = \tau n+u$ we need that: (A) the first n task-completion trials be unsuccessful; and (B) the task be accomplished right at the time epoch u of the $(n+1){\text{th}}$ task-completion trial. Event A occurs with probability $\bar{F}\left( \tau\right) ^{n}$, event B occurs with likelihood $f\left( u\right)$, and hence: the likelihood that the output be realized right at time $t = \tau n+u$ is given by the right-hand side of equation (1).

Due to the structure of the sharp-restart algorithm, its output admits the stochastic representation

$$\begin{align} T_{R}=\tau N+U\text{,} \end{align} \tag{ 2 }$$

where: $N = \lfloor T_{R}/\tau \rfloor$ is the number of unsuccessful task-completion trials; and $U = T_{R}-\tau N$ is the time epoch, within the successful trial, at which the task is accomplished. The random variables N and U are mutually independent, and their statistics are as follows [20]. The random variable N is Geometrically-distributed with success probability $p = F\left( \tau \right)$, i.e.: $\Pr \left(N = n\right) = \left( 1-p\right) ^{n}p$ ($n = 0,1,2,\ldots$). The random variable U is equal, in law, to the input T—given the information that the input is no-larger than the timer, $\left\{T\leqslant \tau \right\}$; consequently, the statistical distribution of the random variable U is governed by the density function $f\left( u\right) /F\left( \tau \right)$ ($0\lt u\lt\tau$).

Perhaps the most common quantitative gauge of randomness is the standard deviation. Specifically, the standard deviation $\sigma \left[ X\right]$ of a real-valued random variable X is the square root of the random variable's variance. A principal property of the standard deviation is its response to affine transformations: $\sigma \left[ aX+b\right] = \left\vert a\right\vert \cdot \sigma \left[ X\right]$, where a and b are real parameters.

The aforementioned principal property captures four different features that one would reasonably expect from any non-negative gauge of randomness (of real-valued random variables). (I) The randomness of a deterministic, i.e. constant, random variable is zero. (II) Translating a random variable does not change its randomness: $\sigma[X+b] = \sigma[X]$, where b is a real translation parameter. (III) Mirroring a random variable does not change its randomness: $\sigma[-X] = \sigma[X]$. (IV) Changing the scale of a random variable changes its randomness by the very same scale: $\sigma[s \cdot X] = s \cdot \sigma[X]$, where s is a positive scale parameter.

The standard deviation is constructed via a Euclidean-geometry perspective. Shifting the perspective from geometric to probabilistic—while maintaining the aforementioned principal property in mind—we shall now show how to construct a gauge of randomness that quantifies the inherent 'determinism'/'stochasticity' of random variables.

Consider a real-valued random variable X, whose statistical distribution is governed by the density function $\varphi \left( x\right)$ ($-\infty\lt x\lt\infty$). Further consider a sample $\left\{X_{1},\ldots ,X_{\epsilon}\right\}$ comprising ε IID copies of the random variable X, where ε is an integer that is greater than one ($\epsilon = 2,3,\ldots$). The likelihood of the coincidence event $X_{1} = \cdots = X_{\epsilon }$ is:

$$\begin{align} \mathscr{C}_{\epsilon }\left[ X\right] =\int_{-\infty }^{\infty}\varphi \left( x\right) ^{\epsilon }dx. \end{align} \tag{ 3 }$$

The term $\mathscr{C}_{\epsilon }\left[ X\right]$ appearing in equation (3) is a coincidence likelihood that can be interpreted as follows: it is the likelihood that the sample $\left\{X_{1},\ldots ,X_{\epsilon }\right\}$ will appear as if it was generated deterministically. Hence, the coincidence likelihood $\mathscr{C}_{\epsilon }\left[ X\right]$ is a measure of the inherent 'determinism' of the random variable X. In turn, the reciprocal of the coincidence likelihood, $1/\mathscr{C}_{\epsilon }\left[ X\right]$, is a measure of the inherent 'stochasticity' of the random variable X.

Now, consider the aforementioned affine transformation, $X\mapsto aX+b$, where a and b are real parameters. It is straightforward to check that the response of the randomness measure $1/\mathscr{C}_{\epsilon }\left[ X\right]$ to the affine transformation is: $1/\mathscr{C}_{\epsilon }\left[aX+b\right] = \left\vert a\right\vert ^{\epsilon -1}/\mathscr{C}_{\epsilon }\left[ X\right]$. Consequently, if we want to obtain a response (to the affine transformation) that is identical to that of the standard deviation—we need to raise the randomness measure $1/\mathscr{C}_{\epsilon }\left[ X\right]$ by the power $1/(\epsilon -1)$. Doing so we arrive at the diversity [28–32]:

$$\begin{align} \mathscr{D}_{\epsilon }\left[ X\right] =\left\{\frac{1}{\mathscr{C}_{\epsilon }\left[ X\right] }\right\} ^{\frac{1}{\epsilon -1}}=\left[\int_{-\infty }^{\infty }\varphi \left( x\right) ^{\epsilon }dx\right]^{1/(1-\epsilon )}. \end{align} \tag{ 4 }$$

The diversity $\mathscr{D}_{\epsilon }\left[ X\right]$ is a measure of the inherent 'stochasticity' of the random variable X. The diversity $\mathscr{D}_{\epsilon }\left[ X\right]$ exhibits the principal property that the standard deviation exhibits—the following response to affine transformations: $\mathscr{D}_{\epsilon }\left[ aX+b\right] = \left\vert a\right\vert \cdot \mathscr{D}_{\epsilon }\left[ X\right]$, where a and b are real parameters.

In the special case ε = 2—which manifests a sample comprising two IID copies of the random variable X—the diversity is the reciprocal of the coincidence likelihood: $\mathscr{D}_{2}\left[ X\right] = 1/\mathscr{C}_{2}\left[ X\right]$. This diversity is known as IPR [21, 22] in physics—where it is used as a measure of localization. Also, this diversity has a profound relation to the overall correlation of moving-average random processes [48]. The coincidence likelihood $\mathscr{C}_{2}\left[ X\right]$ is known as: Simpson's index [25] in biology and ecology—where it is used as a measure of biodiversity; Hirschman's index [26] in economics and competition law—where it is applied as a measure of concentration ⁴ .

Observing equation (4) it is evident that the inverse relationship between the diversity $\mathscr{D}_{\epsilon }\left[ X\right]$ and the coincidence likelihood $\mathscr{C}_{\epsilon }\left[ X\right]$ holds for all ε > 1. Thus, mathematically, we can extend equation (4) from the integer range $\epsilon = 2,3,\ldots$ to the continuous range ε > 1. Doing so, the logarithm of the diversity $\ln ( \mathscr{D}_{\epsilon} \left[ X\right] )$ is the Renyi entropy of the random variable X [33–35], and ε is the corresponding Renyi exponent. In the special case ε = 2—which manifests a sample of size two—the logarithm of the diversity, $\ln ( \mathscr{D}_{2} \left[ X\right] ) = - \ln ( \mathscr{C}_{2} \left[ X\right] )$, is the collision entropy of the random variable X [23, 24].

Yet another special case of the Renyi entropy is obtained in the Renyi-exponent limit $\epsilon \rightarrow 1$, which manifests a sample of size one. Indeed, in this limit the diversity $\mathscr{D}_{\epsilon }\left[ X\right]$ converges to the perplexity of the random variable X [50]. And, the Renyi entropy $\ln ( \mathscr{D}_{\epsilon }\left[ X\right] )$ converges to the Shannon entropy of the random variable X [36].

In the context of the Shannon and Renyi entropies, the following fact should be underscored: while the Shannon entropy is a special case of the Renyi entropy, there are marked differences between the two [24, 51]. One key difference emerges via entropy maximization [52–54]. Specifically, the problem of maximizing entropy (under certain constraints) leads to profoundly different probability distributions—when maximizing with respect to the Shannon and Renyi entropies [48, 55, 56]. So, the Shannon and Renyi entropies gauge randomness differently, and hence both the Shannon and Renyi approaches should be considered when exploring randomness.

The first part of this duo [1] explored the effect of the sharp-restart algorithm on randomness via the perspective of the Shannon entropy. Elevating from this particular entropy to the general Renyi entropy, this part shall explore the effect of the sharp-restart algorithm on randomness via the perspective of diversity.

With equation (1) at hand, the statistical analysis of sharp restart is—in theory—complete. Indeed, equation (1) presents an explicit closed-form formula for the output's density function $f_{R}(t)$ in terms of: the input's density and survival functions, f(t) and $\bar{F}(t)$; and the sharp-restart timer parameter τ. Thus, given the input's statistical distribution, and setting the timer parameter, one can calculate the output's density function. In turn, various statistics of the output can be further calculated, e.g.: the output's mean (which was the focus of the studies [18, 19]), as well as higher-order moments of the output; the output's Shannon entropy (which was the focus of the first part of this duo [1]); and the output's diversity (which is the focus of this paper). So, why bother with the mean-performance analyses of [18, 19], and with the randomness analyses of this duo?

Well, as stated in the opening sentence of this subsection, the aforementioned 'calculational approach' is fine in theory, yet in practice this approach is rather impractical. Firstly, consider the ideal situation in which the input's statistical distribution is known in full detail. In this situation the calculation of equation (1)—which is performed case-by-case, per a given input and per a given timer—provides no general insights regarding the effect of sharp restart on mean-performance and on randomness. Secondly, consider situations in which full information regarding the input's statistical distribution is lacking. In 'real life' the latter situations are prevalent, and in such situations equation (1) is outright unusable. Thus, equation (1) is a foundational theoretical result, but it is not a practical 'working tool'.

As established in the mean-performance analyses of [18, 19] and in the entropy analysis of [1], and as shall be established here: 'standing on the shoulders' of equation (1), it is possible to construct highly insightful and highly implementable results that altogether circumvent the need to use equation (1) directly. Indeed, with remarkably little information about the input's statistical distribution—rather than full knowledge—explicit criteria can be drawn regarding the effect of sharp restart on mean-performance, on randomness, and jointly on mean-performance and randomness. Moreover, these explicit criteria are universal: they are applicable to any input, with any statistical distribution.

Equation (5) conceals two statistical distributions that are induced by the input's statistical distribution, and that are defined over the positive half-line. We shall now describe these statistical distributions, and thereafter re-formulate equation (5) in their terms. This re-formulation will yield general 'survival criteria' that shall determine—for any given input T, and for any given timer τ—if sharp restart decreases or increases diversity.

The first statistical distribution is governed by the following density function:

$$\begin{align} \psi \left( t\right) =\frac{1}{\gamma }f\left( t\right) ^{\epsilon } \end{align} \tag{ 7 }$$

(t > 0). Evidently, as the input's density function $f\left( t\right)$ is positive-valued over the positive half-line, so is the function $\psi \left( t\right)$. Moreover, the very definition of the coincidence likelihood γ implies that the function $\psi \left( t\right)$ has a unit mass over the positive half-line: $\int_{0}^{\infty }\psi \left( t\right) dt = 1$. Hence, over the positive half-line, $\psi \left( t\right)$ is indeed a density function. We denote by $\Psi \left( t\right) = \int_{0}^{t}\psi \left( s\right) ds$ ($t\geqslant 0$) the corresponding distribution function, and by $\bar{\Psi} \left( t\right) = \int_{t}^{\infty }\psi \left( s\right) ds$ ($t\geqslant 0$) the corresponding survival function.

For an integer Renyi exponent ($\epsilon = 2,3,\ldots$) the density function $\psi \left( t\right)$ has the following probabilistic meaning. Consider a sample $\left\{T_{1},\ldots ,T_{\epsilon }\right\}$ comprising ε IID copies of the random variable T. Then—provided that the coincidence event $T_{1} = \cdots = T_{\epsilon }$ occurred—$\psi \left( t\right)$ is the likelihood that the common value attained by the IID copies is t.

The second statistical distribution is governed by the following survival function:

$$\begin{align} \bar{\Phi}\left( t\right) =\bar{F}\left( t\right) ^{\epsilon } \end{align} \tag{ 8 }$$

($t\geqslant 0$). Evidently, as the input's survival function $\bar{F}\left(t\right)$ is monotone decreasing from the unit-level $\lim_{t\rightarrow 0} \bar{F}\left( t\right) = 1$ to the zero-level $\lim_{t\rightarrow \infty } \bar{F}\left( t\right) = 0$, so does the function $\bar{\Phi}\left( t\right)$. Hence, over the positive half-line, $\bar{\Phi}\left( t\right)$ is indeed a survival function. We denote by $\Phi \left( t\right) = 1-\bar{F} \left( t\right) ^{\epsilon }$ ($t\geqslant 0$) the corresponding distribution function, and by $\phi \left( t\right) = \epsilon \bar{F}\left( t\right)^{\epsilon -1}f\left( t\right)$ (t > 0) the corresponding density function.

For an integer Renyi exponent ($\epsilon = 2,3,\ldots$) the survival function $\bar{\Phi}\left( t\right)$ has the following probabilistic meaning. As above, consider a sample $\left\{T_{1},\ldots ,T_{\epsilon }\right\}$ comprising ε IID copies of the random variable T. Then, $\bar{\Phi}\left( t\right)$ is the survival function of the sample's minimum, $\min \left\{T_{1},\ldots ,T_{\epsilon }\right\}$. Namely: $\bar{\Phi} \left( t\right) = \Pr \left( \min \left\{T_{1},\ldots ,T_{\epsilon }\right\} \gt t\right)$.

With the two statistical distributions that were introduced, we can now re-formulate equation (5). To that end we divide both sides of equation (5) by the input's coincidence likelihood γ, and then use the above distribution and survival functions. This yields the following formula for the ratio of the output's coincidence likelihood to the input's coincidence likelihood:

$$\begin{align} \frac{C\left( \tau \right) }{\gamma }=\frac{\Psi \left( \tau \right) }{\Phi\left( \tau \right) }=\frac{1-\bar{\Psi}\left( \tau \right) }{1-\bar{\Phi}\left( \tau \right) }. \end{align} \tag{ 9 }$$

Recalling the inverse relation between the diversity and the coincidence likelihood, equation (9) straightforwardly implies the following pair of survival criteria.

• $\bullet $  
  Sharp restart with timer τ decreases diversity if and only if $\bar{\Psi}\left( \tau \right) \lt\bar{\Phi}\left(\tau \right)$.
• $\bullet $  
  Sharp restart with timer τ increases diversity if and only if $\bar{\Psi}\left( \tau \right) \gt\bar{\Phi}\left(\tau \right)$.

The survival criteria determine if sharp restart decreases/increases diversity by comparing the survival functions of the two statistical distributions that were introduced above. We emphasize that, in general, the survival criteria may provide different answers for different timers τ. Namely, it may be that while sharp restart decreases diversity for some timers τ, it increases diversity for other timers τ (and vice versa).

As an illustrative demonstration of the survival criteria, we re-visit the above example of a Pareto type II input. Recall that in this example the input's survival and density functions are, respectively, $\bar{F}(t) = [1+(t/s)]^{-\alpha}$ and $f(t) = \frac{\alpha}{s}[1+(t/s)]^{-\alpha -1}$ (where s is a positive scale parameter, and where α is a positive Pareto power). Consequently, straightforward calculations imply that $\bar{\Psi}\left( \tau \right) = [1+(\tau/s)]^{-(\epsilon \alpha + \epsilon -1)}$ and $\bar{\Phi}\left(\tau \right) = [1+(\tau/s)]^{-\epsilon \alpha}$. Note that these survival functions are type II Pareto with scale parameter s, and with Pareto powers $\epsilon \alpha + \epsilon -1$ and εα (respectively). The fact that ε > 1 implies that $\bar{\Psi}\left( \tau \right) \lt\bar{\Phi}\left(\tau \right)$ (see figure 1). So, the aforementioned conclusion regarding inputs that are Pareto type II is re-attained: sharp restart, with any timer τ, decreases diversity.

To conclude this section, we address the following question: are there inputs that sharp restart neither increases nor decreases their diversity? Specifically, inputs that yield the 'flat' output diversity $D(\tau) = \delta$ for all timers τ—which is tantamount to the 'flat' output coincidence-likelihood $C\left( \tau \right) = \gamma$ for all timers τ.

On the one hand, equation (9) implies that $C\left( \tau \right) = \gamma$ for all timers τ if and only if the two statistical distributions—governed, respectively, by the density functions $\psi \left( t\right)$ and $\phi \left( t\right)$—are identical. On the other hand, it is straightforward to check that the density functions $\psi \left( t\right)$ and $\phi \left( t\right)$ are identical if and only if the input's density function is that of the Exponential distribution: $f\left( t\right) = \frac{1}{s} \exp(-t/s)$ (t > 0), where s is a positive scale parameter. Thus, combining these two facts together, we arrive at the following diversity conclusions.

• $\bullet $  
  The output's diversity is identical to the input's diversity, $D\left( \tau \right) = \delta$ for all timers τ, if and only if the input T is Exponentially-distributed.
• $\bullet $  
  If the input T is not Exponentially-distributed then there exist timers τ with which the output's diversity differs from the input's diversity, $D\left( \tau \right) \neq \delta$.

Note that the diversity of an Exponentially-distributed input is $s \cdot \delta_{\exp}$, where s is the aforementioned scale parameter, and where $\delta_{\exp} = \epsilon ^{1/(\epsilon -1)}$ is the diversity of the unit-scale Exponential distribution ⁶ . The diversity value $\delta_{\exp}$ will play a key role in sections 5 and 6 below.

The mean of a statistical distribution that is defined over the positive half-line is given by the integral of its survival function. Hence, the mean of the statistical distribution that is governed by the density function $\psi \left( t\right)$ is $\mu _{\psi } = \int_{0}^{\infty }\bar{\Psi}\left(t\right) dt$. Also, the mean of the statistical distribution that is governed by the density function $\phi \left( t\right)$ is $\mu _{\phi} = \int_{0}^{\infty }\bar{\Phi}\left( t\right) dt$. Consequently, we have

$$\begin{align} \int_{0}^{\infty }\left[ \bar{\Psi}\left( t\right) -\bar{\Phi}\left( t\right) \right] dt=\mu _{\psi }-\mu _{\phi }. \end{align} \tag{ 10 }$$

Note that if the integral appearing in equation (10) is negative then its integrand—the difference $\left[ \bar{\Psi}\left( \tau\right) -\bar{\Phi}\left(\tau\right) \right]$—must be negative for some τ. Similarly, if the integral is positive then its integrand must be positive for some τ. Thus, combining equation (10) together with the survival criteria of section 3 yields the following pair of existence criteria.

• $\bullet $  
  If $\mu _{\psi }\lt\mu _{\phi }$ then there exist timers τ with which sharp restart decreases diversity.
• $\bullet $  
  If $\mu _{\psi }\gt\mu _{\phi }$ then there exist timers τ with which sharp restart increases diversity.

The existence criteria determine if sharp restart decreases/increases diversity by comparing the means of the two statistical distributions that were introduced in the previous section: the one governed by the density function of equation (7), and the one governed by the survival function of equation (8). While the survival criteria of the previous section required full knowledge of the two statistical distributions, the existence criteria of this section require knowing only the means of the two statistical distributions.

We note that an alternative way of obtaining these existence criteria is via the following formula:

$$\begin{align} \int_{0}^{\infty }\left[ \frac{C\left( \tau \right) -\gamma }{\gamma}\right] \Phi \left( \tau \right) d\tau =\mu _{\phi }-\mu _{\psi }. \end{align} \tag{ 11 }$$

Equation (11) stems from equation (9) via the following steps: subtract 1 from both sides of equation (9), multiply by the quantity $\Phi \left( \tau \right)$, integrate over positive half-line, and then use equation (10).

As an illustrative demonstration of these existence criteria, we re-visit the type II Pareto example of the previous section. Recall that in this example the survival function is $\bar{F}(t) = [1+(t/s)]^{-\alpha}$, where s is a positive scale parameter, and where α is a positive Pareto power. Consequently, as noted above, $\bar{\Psi}\left( \tau \right) = [1+(\tau/s)]^{- (\epsilon \alpha + \epsilon -1)}$ and $\bar{\Phi}\left(\tau \right) = [1+(\tau/s)]^{- \epsilon \alpha}$. Considering the Pareto power α to be greater than one, the corresponding means are $\mu _{\psi } = s/(\epsilon \alpha + \epsilon -2)$ and $\mu _{\phi } = s/(\epsilon \alpha-1)$. Evidently, the fact that ε > 1 implies that $\mu _{\psi }\lt\mu _{\phi }$, and hence: there exist timers τ with which sharp restart decreases diversity.

Regarding the type II Pareto example, the existence conclusion of this section is in full accord with the survival conclusion of the previous section (which asserted that sharp restart, with any timer τ, decreases diversity). As pointed out above: in order to deduce the existence conclusion one needs to know only the means µ_ψ and µ_φ , whereas in order to deduce the survival conclusion one needs to have full information of the survival functions $\bar{\Psi}\left( t\right)$ and $\bar{\Phi}\left(t\right)$.

Set T_ψ and T_φ to be positive-valued random variables whose statistical distributions are governed, respectively, by the density functions $\psi \left( t\right)$ and $\phi \left( t\right)$. The existence criteria of the previous subsection contested the means of the random variables T_ψ and T_φ against each other. In this subsection we shall present existence criteria that contest the random variables T_ψ and T_φ directly against each other.

To that end we introduce the density function $\phi _{\max }\left( t\right) = 2\Phi \left(t\right) \phi \left( t\right)$ (t > 0). Specifically, $\phi _{\max }\left(t\right)$ is the density function of the maximum of two IID copies of the random variable T_φ . The following formula emanates from equation (9):

$$\begin{align} \int_{0}^{\infty }\left[ \frac{C\left( \tau \right) -\gamma }{\gamma }\right] \phi _{\max }\left( \tau \right) d\tau =2\Pr \left( T_{\psi }\leqslant T_{\phi}\right) -1, \end{align} \tag{ 12 }$$

where the random variables that appear on the right-hand side of equation (12), T_ψ and T_φ , are independent of each other. The derivation of equation (12) is detailed in section 8.

Using argumentation which is similar to that following equation (10), equation (12) yields the following pair of existence criteria.

• $\bullet $  
  If $\Pr \left( T_{\psi }\leqslant T_{\phi }\right) \gt\frac{1}{2}$ then there exist timers τ with which sharp restart decreases diversity.
• $\bullet $  
  If $\Pr \left( T_{\psi }\leqslant T_{\phi }\right) \lt\frac{1}{2}$ then there exist timers τ with which sharp restart increases diversity.

The existence criteria determine if sharp restart decreases/increases diversity by contesting the random variable T_ψ against the random variable T_φ , and then comparing the probability of the event $T_{\psi }\leqslant T_{\phi }$ to the 'benchmark' probability $\frac{1}{2}$.

Equation (12) can be re-formulated in terms of the minimum $\min \left\{T_{1},T_{2}\right\}$ of two IID copies of the input T. Specifically, denoting by $\delta _{\min } = \mathscr{D}_{\epsilon } \left[ \min \left\{T_{1},T_{2}\right\} \right]$ the minimum's diversity, the re-formulation of equation (12) is:

$$\begin{align} \int_{0}^{\infty }\left[ \frac{C\left( \tau \right) -\gamma }{\gamma }\right] \phi _{\max }\left( \tau \right) d\tau =\left( \frac{\delta }{2\delta _{\min}}\right) ^{\epsilon -1}-1. \end{align} \tag{ 13 }$$

The derivation of equation (13) is detailed in section 8.

Equation (13) yields the following alternative formulation of this subsection's pair of existence criteria.

• $\bullet $  
  If $\delta \gt 2\delta _{\min }$ then there exist timers τ with which sharp restart decreases diversity.
• $\bullet $  
  If $\delta \lt 2\delta _{\min }$ then there exist timers τ with which sharp restart increases diversity.

In this formulation, the existence criteria determine if sharp restart decreases/increases diversity by comparing two diversities: that of the input T to that of the minimum $\min \left\{T_{1},T_{2}\right\}$ of two IID copies of the input T.

As an illustrative demonstration of these existence criteria, consider an input whose distribution is Weibull. The Weibull distribution [62] is one of the three universal laws emanating from the Fisher–Tippett–Gnedenko theorem [63, 64] of Extreme Value Theory [65–67], and it has numerous uses in science and engineering [68–70]. The survival function of a Weibull input is $\bar{F}\left( t\right) = \exp [-(t/s)^{\alpha }]$, where s is a positive scale parameter, and where α is a positive Weibull exponent.

The Weibull input displays the following scaling property: $\min \left\{T_{1},T_{2}\right\} = 2^{-1/\alpha }T$, where the equality is in law. In turn, this scaling property implies that $\delta _{\min } = 2^{-1/\alpha }\delta$. So, when the Weibull exponent is in the 'sub-Exponential' range α < 1 then: $\delta \gt2\delta _{\min }$, and hence there exist timers τ with which sharp restart decreases diversity. And, when the Weibull exponent is in the 'super-Exponential' range α > 1 then: $\delta \lt2\delta _{\min }$, and hence there exist timers τ with which sharp restart increases diversity.

In the 'sub-Exponential' range α < 1 the Weibull distribution manifests the Stretched Exponential distribution—which plays a key role in statistical physics [71–73]. So, in particular, the aforementioned Weibull example yields the following Stretched-Exponential conclusion: there exist timers τ with which sharp restart can always decrease the diversity of Stretched-Exponential tasks. We shall re-visit the Weibull example, and bolster the above Weibull conclusions, in section 6 below.

The cases of sharp restart with fast and slow timers correspond, respectively, to the timer limits $\tau \rightarrow 0$ and $\tau \rightarrow \infty$. In order to analyze the asymptotic behavior of sharp restart in these limits we shall use the notion of hazard function (described below). Also, we shall use the shorthand notation $\varphi \left( 0\right) = \lim_{t\rightarrow 0}\varphi \left( t\right)$ and $\varphi \left( \infty \right) = \lim_{t\rightarrow \infty }\varphi \left( t\right)$ for the limit values of a general positive-valued function $\varphi \left( t\right)$ that is defined over the positive half-line (t > 0); these limit values are assumed to exist in the wide sense, $0\leqslant \varphi \left( 0\right) ,\varphi \left( \infty \right) \leqslant \infty$.

The input's hazard function played a focal role in the entropy analysis [1], in the mean-performance analysis [18], and in the tail-behavior analysis [20] of sharp restart. The input's hazard function is the negative logarithmic derivative of the input's survival function:

$$\begin{align} H\left( t\right) =-\left\{\ln \left[ \bar{F}\left( t\right) \right] \right\} ^{^{\prime} }=\frac{f\left( t\right) }{\bar{F}\left( t\right) }. \end{align} \tag{ 14 }$$

The hazard function of equation (14) has the following probabilistic meaning: $H\left(t\right)$ is the likelihood that the input be realized right at time t, given the information that the input was not realized up to time t. The hazard function—a.k.a. 'hazard rate' and 'failure rate'—is a widely applied tool in survival analysis [74–76] and in reliability engineering [77–79].

Now, with the input's hazard function $H\left( t\right)$ at hand, we are all set to analyze the asymptotic behavior of sharp restart. Equations (16) and (17) below will incorporate the number

$$\begin{align} \delta_{\exp} =\epsilon ^{1/(\epsilon -1)}. \end{align} \tag{ 15 }$$

As noted at the end of section 3, the number $\delta_{\exp}$ is the diversity of the unit-scale Exponential distribution.

Firstly, we address the case of fast timers ($\tau\ll1$). Taking the limit $\tau \rightarrow 0$ in the middle part of equation (9), while using L'Hospital's rule, yields the following limit-value of the output's diversity:

$$\begin{align} D\left( 0\right) =\frac{\delta_{\exp} }{H\left( 0\right) }. \end{align} \tag{ 16 }$$

The derivation of equation (16) is detailed in section 8. In turn, equation (16) implies the following pair of fast criteria.

• $\bullet $  
  If $H\left( 0\right) \lt\delta_{\exp} /\delta$ then sharp restart with fast timers increases diversity; in particular, this criterion applies whenever the hazard function vanishes at the origin, $H\left(0\right) = 0$.
• $\bullet $  
  If $H\left( 0\right) \gt\delta_{\exp} /\delta$ then sharp restart with fast timers decreases diversity; in particular, this criterion applies whenever the hazard function explodes at the origin, $H\left(0\right) = \infty$.

Secondly, we address the case of slow timers ($\tau \gg1$). Taking the limit $\tau \rightarrow \infty$ in the middle part of equation (9) yields the following limit-value of the output's coincidence likelihood: $C\left( \infty \right) = \gamma$. In turn, a calculation using equation (9) and L'Hospital's rule asserts that the asymptotic behavior of the output's coincidence likelihood $C\left( \tau \right)$ about its limit value $C\left( \infty \right) = \gamma$ is given by the following limit:

$$\begin{align} \lim_{\tau \rightarrow \infty }\frac{C\left( \tau \right) -\gamma }{\bar{\Phi}\left( \tau \right) }=\left( \frac{1}{\delta }\right) ^{\epsilon -1}-\left[\frac{1}{\delta_{\exp} }H\left( \infty \right) \right] ^{\epsilon -1}. \end{align} \tag{ 17 }$$

The derivation of equation (17) is detailed in section 8. In turn, equation (17) implies the following pair of slow criteria.

• $\bullet $  
  If $H\left( \infty \right) \lt\delta_{\exp} /\delta$ then sharp restart with slow timers decreases diversity; in particular, this criterion applies whenever the hazard function vanishes at infinity, $H\left( \infty \right) = 0$.
• $\bullet $  
  If $H\left( \infty \right) \gt\delta_{\exp} /\delta$ then sharp restart with slow timers increases diversity; in particular, this criterion applies whenever the hazard function explodes at infinity, $H\left( \infty \right) = \infty$.

The fast and slow criteria display a common pattern: comparing the limit-values of the hazard function H(t) to the ratio of diversities $\delta_{\exp} /\delta$. This ratio manifests the height of a flat hazard function that characterizes an Exponential distribution whose diversity is equal to the input's diversity δ. Hence, in effect, the ratio of diversities $\delta_{\exp} /\delta$ serves—in the fast and slow criteria—as an 'Exponential hazard-function benchmark' to which the limit-values of the input's hazard function are compared.

In the previous subsection we addressed the case of fast timers ($\tau\ll1$), and the case of slow timers ($\tau \gg1$). However, we did not provide the precise meanings of these timers. Namely: how small should the timer τ be in order to qualify as 'fast'? and how large should the timer τ be in order to qualify as 'slow'? Considering the input's hazard function $H\left( t\right)$ to be continuous over the positive half-line (t > 0), in this subsection we answer these two questions.

The interplay between the following terms assumed a key role in the previous subsection: the input's hazard function $H\left( t\right)$ on the one hand, and the ratio of diversities $\delta_{\exp} /\delta$ on the other hand. This interplay will assume a key role also in this subsection, via the two following thresholds:

$$\begin{align} \tau _{\ast }=\inf \left\{t>0 \vert H\left( t\right) =\frac{\delta_{\exp} }{\delta }\right\}, \end{align} \tag{ 18 }$$

and

$$\begin{align} \tau ^{\ast }=\sup \left\{t>0 \vert H\left( t\right) =\frac{\delta_{\exp} }{\delta }\right\}. \end{align} \tag{ 19 }$$

Namely, the thresholds $\tau _{\ast }$ and $\tau ^{\ast }$ are, respectively, the smallest and largest times at which the hazard function $H\left( t\right)$ intersects the level $\delta_{\exp} /\delta$. In particular, if the hazard function $H\left( t\right)$ does not intersect the level $\delta_{\exp} /\delta$ then: $\tau _{\ast } = \infty$ and $\tau ^{\ast } = 0$.

A calculation based on equation (9) yields the following formulae:

$$\begin{align} C\left( \tau \right) -\gamma =\frac{1}{\epsilon \Phi \left( \tau \right) }\int_{0}^{\tau }\left[ H\left( t\right) ^{\epsilon -1}-\left( \frac{\delta_{\exp} }{\delta }\right) ^{\epsilon -1}\right] \phi \left( t\right) dt . \end{align} \tag{ 20 }$$

and

$$\begin{align} C\left( \tau \right) -\gamma =\frac{1}{\epsilon \Phi \left( \tau \right) }\int_{\tau }^{\infty }\left[ \left( \frac{\delta_{\exp} }{\delta }\right)^{\epsilon -1}-H\left( t\right) ^{\epsilon -1}\right] \phi \left( t\right) dt. \end{align} \tag{ 21 }$$

The derivations of equations (20) and (21) are detailed in section 8. In equations (20) and (21), the input's hazard function H(t) is compared to the level $\delta_{\exp} /\delta$. As noted at the close of the previous subsection, the level $\delta_{\exp} /\delta$ manifests an 'Exponential hazard-function benchmark'.

Consider the case of fast timers: $\tau \ll1$. The fast-timer criteria of the previous subsection, combined together with equation (20), straightforwardly yield the two following conclusions. (I) If $H\left(0\right) \lt\delta_{\exp} /\delta$ then sharp restart increases diversity for all timers τ in the range $\tau \lt\tau _{\ast }$. (II) If $H\left( 0\right)\gt\delta_{\exp} /\delta$ then sharp restart decreases diversity for all timers τ in the range $\tau \lt\tau _{\ast }$. Hence, the threshold $\tau _{\ast}$ of equation (18) defines the range of fast timers.

Consider the case of slow timers: $\tau \gg1$. The slow-timer criteria of the previous subsection, combined together with equation (21), straightforwardly yield the two following conclusions. (I) If $H\left(\infty \right) \lt\delta_{\exp} /\delta$ then sharp restart decreases diversity for all timers τ in the range $\tau \gt\tau ^{\ast }$. (II) If $H\left(\infty \right) \gt\delta_{\exp} /\delta$ then sharp restart increases diversity for all timers τ in the range $\tau \gt\tau ^{\ast }$. Hence, the threshold $\tau ^{\ast }$ of equation (19) defines the range of slow timers.

As an illustrative demonstration of the ranges of fast and slow timers, let us re-visit the Weibull example of section 4. Recall that in this example the survival function is $\bar{F}\left( t\right) = \exp [-(t/s)^{\alpha }]$, where s is a positive scale parameter, and where α is a positive Weibull exponent. Consequently, the hazard function is $H(t) = ct^{\alpha-1}$, with coefficient $c = \alpha s^{-\alpha}$.

When the Weibull exponent is in the 'sub-Exponential' range α < 1 then the hazard function is monotone decreasing from $H(0) = \infty$ to $H(\infty) = 0$. Hence, the results of the previous subsection assert that sharp restart with either fast or slow timers decreases diversity. The shape of the hazard function implies that it intersects the level $\delta_{\exp} /\delta$ at the unique threshold $\tau _{\ast} = \tau ^{\ast} = [c \delta / \delta_{\exp}]^{1/(1-\alpha)}$. Consequently, equations (20) and (21) assert that: sharp restart with any timer decreases diversity (see figure 2).

Zoom In Zoom Out Reset image size

When the Weibull exponent is in the 'super-Exponential' range α > 1 then the hazard function is monotone increasing from $H(0) = 0$ to $H(\infty) = \infty$. Hence, the results of the previous subsection assert that sharp restart with either fast or slow timers increases diversity. The shape of the hazard function implies that it intersects the level $\delta_{\exp} /\delta$ at the aforementioned unique threshold ($\tau _{\ast} = \tau ^{\ast} = [c \delta / \delta_{\exp}]^{1/(1-\alpha)}$). Consequently, equations (20) and (21) assert that: sharp restart with any timer increases diversity (see figure 2).

In fact, the above conclusion regarding the Weibull example in the 'sub-Exponential' range holds for any input whose hazard function is continuous and monotone decreasing from $H(0) = \infty$ to $H(\infty) = 0$. And, the above conclusion regarding the Weibull example in the 'super-Exponential' range holds for any input whose hazard function is continuous and monotone increasing from $H(0) = 0$ to $H(\infty) = \infty$.

Set $p = F\left( \tau \right)$ and $q = \bar{F}\left( \tau \right)$, and note that $p+q = 1$. Using the periodic parameterization $t = \tau n+u$ ($n = 0,1,2,\ldots$ and $0\leqslant u\lt\tau$) of the time axis, and using equation (1), we have:

$$\begin{align} \begin{aligned} \mathscr{C}_{\epsilon }\left[ T_{R}\right] & =\int_{0}^{\infty}f_{R}\left(t\right) ^{\epsilon }dt \\ & =\sum_{n=0}^{\infty }\int_{\tau n}^{\tau n+\tau }f_{R}\left( t\right)^{\epsilon}dt=\sum_{n=0}^{\infty }\int_{0}^{\tau }f_{R}\left( \tau n+u\right) ^{\epsilon }du \\ & =\sum_{n=0}^{\infty }\int_{0}^{\tau }\left[ q^{n}f\left( u\right) \right]^{\epsilon }du=\sum_{n=0}^{\infty }\left[ \left( q^{n}\right) ^{\epsilon}\int_{0}^{\tau }f\left( u\right) ^{\epsilon }du\right] \\ & =\left[ \sum_{n=0}^{\infty }\left( q^{\epsilon }\right) ^{n}\right] \cdot \int_{0}^{\tau }f\left( u\right) ^{\epsilon}du. \end{aligned} \end{align} \tag{ 22 }$$

Note that

$$\begin{align} \sum_{n=0}^{\infty }\left( q^{\epsilon }\right)^{n}=\frac{1}{1-q^{\epsilon }}=\frac{1}{1-\bar{F}\left( \tau \right) ^{\epsilon }}. \end{align} \tag{ 23 }$$

Substituting equation (23) into the bottom line of equation (22) yields equation (5).

To derive equation (6), we re-write equation (22) as follows:

$$\begin{align} \begin{aligned} \mathscr{C}_{\epsilon }\left[ T_{R}\right] & =\left[ \sum_{n=0}^{\infty}\left( q^{\epsilon }\right) ^{n}\left( p^{\epsilon }\right) \right] \cdot \int_{0}^{\tau}\frac{1}{p^{\epsilon }}f\left( u\right) ^{\epsilon }du \\ & =\left[ \sum_{n=0}^{\infty }\left( q^{n}p\right) ^{\epsilon }\right] \cdot \int_{0}^{\tau }\left[ \frac{1}{p}f\left(u\right) \right] ^{\epsilon }du. \end{aligned} \end{align} \tag{ 24 }$$

Consider the random variable N of equation (2), whose probability distribution is $\Pr \left( N = n\right) = q^{n}p$ ($n = 0,1,2,\ldots$); for this random variable note that

$$\begin{align} \mathscr{C}_{\epsilon }\left[ N\right] =\sum_{n=0}^{\infty }\Pr \left(N=n\right) ^{\epsilon }=\sum_{n=0}^{\infty }\left( q^{n}p\right) ^{\epsilon }. \end{align} \tag{ 25 }$$

Consider the random variable U of equation (2)—whose density function is $\frac{1}{p}\,f\left( u\right)$ ($0\leqslant u\lt\tau$); for this random variable note that

$$\begin{align} \mathscr{C}_{\epsilon }\left[ U\right] =\int_{0}^{\tau }\left[ \frac{1}{p} f\left( u\right) \right] ^{\epsilon }du. \end{align} \tag{ 26 }$$

Substituting equations (25) and (26) into the bottom line of equation (24) yields

$$\begin{align} \mathscr{C}_{\epsilon }\left[ T_{R}\right] =\mathscr{C}_{\epsilon }\left[ N\right] \cdot \mathscr{C}_{\epsilon }\left[ U\right]. \end{align} \tag{ 27 }$$

In turn, equation (27) yields equation (6).

Equation (9) implies that

$$\begin{align} \frac{C\left( \tau \right) -\gamma }{\gamma }=\frac{\Psi \left( \tau \right) -\Phi \left( \tau \right) }{\Phi \left( \tau \right) }. \end{align} \tag{ 28 }$$

Multiplying both sides of equation (28) by $\phi _{\max }\left( \tau \right) = 2\Phi \left( \tau \right) \phi \left( \tau \right)$ further implies that

$$\begin{align} \frac{C\left( \tau \right) -\gamma }{\gamma }\phi _{\max }\left( \tau \right) =2\Psi \left( \tau \right) \phi \left( \tau \right) -\phi _{\max}\left( \tau \right) . \end{align} \tag{ 29 }$$

In turn, integrating equation (29) over the positive half-line yields

$$\begin{align} \begin{aligned} & \int_{0}^{\infty }\left[ \frac{C\left( \tau \right) -\gamma }{\gamma }\right] \phi _{\max }\left( \tau \right) d\tau \\ & \quad =\int_{0}^{\infty }2\Psi \left( \tau \right) \phi \left( \tau \right) d\tau -\int_{0}^{\infty }\phi _{\max }\left( \tau \right) d\tau \\ & \quad =2\int_{0}^{\infty }\Psi \left( \tau \right) \phi \left( \tau \right) d\tau-1. \end{aligned} \end{align} \tag{ 30 }$$

For independent random variables T_ψ and T_φ , using conditioning, note that

$$\begin{align} \begin{aligned} \Pr \left( T_{\psi }\leqslant T_{\phi}\right) & =\int_{0}^{\infty }\Pr \left(T_{\psi }\leqslant \tau \vert T_{\phi }=\tau \right) \phi \left( \tau \right) d\tau \\ & =\int_{0}^{\infty }\Psi \left( \tau \right) \phi \left( \tau \right) d\tau. \end{aligned} \end{align} \tag{ 31 }$$

Substituting equation (31) into the bottom line of equation (30) yields equation (12).

Similarly to equation (31), note that

$$\begin{align} \begin{aligned} \Pr \left( T_{\psi }\leqslant T_{\phi }\right) & =\Pr \left( T_{\phi }>T_{\psi}\right) \\ & =\int_{0}^{\infty }\Pr \left( T_{\phi }>t \vert T_{\psi}=t \right) \psi \left( t\right) dt \\ & =\int_{0}^{\infty }\bar{\Phi}\left( t \right) \psi \left( t\right) dt=\int_{0}^{\infty }\left[ \bar{F}\left( t\right) ^{\epsilon }\right] \left[\frac{1}{\gamma }f\left( t\right) ^{\epsilon }\right] dt \\ & =\frac{1}{2^{\epsilon }\gamma }\int_{0}^{\infty }\left[ 2\bar{F}\left(t\right) f\left( t\right) \right] ^{\epsilon }dt; \end{aligned} \end{align} \tag{ 32 }$$

in the third line of equation (32) we used equations (7) and (8). The term $2\bar{F}\left( t\right) f\left( t\right)$—which appears in the bottom line of equation (32)—is the density function of the random variable $\min \left\{T_{1},T_{2}\right\}$, the minimum of two IID copies of the input T. Hence, equation (32) implies that

$$\begin{align} \Pr \left( T_{\psi }\leqslant T_{\phi }\right) =\frac{1}{2^{\epsilon }}\frac{\mathscr{C}_{\epsilon }\left[ \min \left\{T_{1},T_{2}\right\} \right] }{\mathscr{C}_{\epsilon }\left[ T\right] }. \end{align} \tag{ 33 }$$

In turn, equation (33) implies that

$$\begin{align} 2\Pr \left( T_{\psi }\leqslant T_{\phi }\right) =\left( \frac{\mathscr{D}_{\epsilon }\left[ T\right] }{2\mathscr{D}_{\epsilon }\left[ \min \left\{T_{1},T_{2}\right\} \right] }\right) ^{\epsilon -1}. \end{align} \tag{ 34 }$$

Substituting equation (34) into equation (12)—while using the shorthand notations $\delta = \mathscr{D}_{\epsilon }\left[ T\right]$ and $\delta _{\min } = \mathscr{D}_{\epsilon }\left[ \min \left\{T_{1},T_{2}\right\} \right]$—yields equation (13).

Using the density function $\psi \left( t\right) = \frac{1}{\gamma }f\left( t\right) ^{\epsilon }$ of equation (7), the density function $\phi \left( t\right) = \epsilon \bar{F}\left( t\right) ^{\epsilon -1}f\left( t\right)$ induced by the survival function of equation (8), the input's hazard function of equation (14), equation (15), and the connection between the coincidence likelihood and the diversity (recall equation (4)), note that:

$$\begin{align} \begin{aligned} \frac{\psi \left( t\right) }{\phi \left( t\right) } & =\frac{\frac{1}{\gamma } f\left( t\right) ^{\epsilon }}{\epsilon \bar{F}\left( t\right) ^{\epsilon -1}f\left( t\right) } \\ & =\frac{1}{\gamma \epsilon }\left[ \frac{f\left( t\right) }{\bar{F}\left( t\right) }\right] ^{\epsilon -1}=\frac{1}{\gamma \epsilon }H\left( t\right) ^{\epsilon -1} \\ & =\frac{1}{\gamma }\left[ \frac{1}{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1}=\left[ \frac{\delta }{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1}. \end{aligned} \end{align} \tag{ 35 }$$

Taking the limit $\tau \rightarrow 0$ in the middle part of equation (9), while using L'Hospital's rule and equation (35), we have:

$$\begin{align} \begin{aligned} \lim_{\tau \rightarrow 0}\frac{C\left( \tau \right) }{\gamma }& =\lim_{\tau \rightarrow 0}\frac{\Psi \left( \tau \right) }{\Phi \left( \tau \right) } \\ & =\lim_{t\rightarrow 0}\frac{\psi \left( t\right) }{\phi \left( t\right) } =\lim_{t\rightarrow 0}\frac{1}{\gamma }\left[ \frac{1}{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1} \\ & =\frac{1}{\gamma }\left[ \frac{1}{\delta_{\exp} }\lim_{t\rightarrow 0}H\left( t\right) \right] ^{\epsilon -1}. \end{aligned} \end{align} \tag{ 36 }$$

Equation (36) implies that

$$\begin{align} C\left( 0\right) =\left[ \frac{1}{\delta_{\exp} }H\left( 0\right) \right] ^{\epsilon -1}. \end{align} \tag{ 37 }$$

In turn, using the connection between the coincidence likelihood and the diversity (recall equation (4)), equation (37) implies equation (16).

Taking the limit $\tau \rightarrow \infty$ in the middle part of equation (9) yields

$$\begin{align} \lim_{\tau \rightarrow \infty }\frac{C\left( \tau \right) }{\gamma } =\lim_{\tau \rightarrow \infty }\frac{\Psi \left( \tau \right) }{\Phi \left( \tau \right) }=1, \end{align} \tag{ 38 }$$

and hence $C\left( \infty \right) = \gamma$. Equation (9) implies that

$$\begin{align} \frac{C\left( \tau \right) -\gamma }{\gamma }=\frac{\bar{\Phi}\left( \tau \right) -\bar{\Psi}\left( \tau \right) }{\Phi \left( \tau \right) }. \end{align} \tag{ 39 }$$

In turn, equation (39) implies that

$$\begin{align} \frac{C\left( \tau \right) -\gamma }{\gamma \bar{\Phi}\left( \tau \right) }= \frac{1}{\Phi \left( \tau \right) }\left[ 1-\frac{\bar{\Psi}\left( \tau \right) }{\bar{\Phi}\left( \tau \right) }\right] . \end{align} \tag{ 40 }$$

Taking the limit $\tau \rightarrow \infty$ in equation (40), while using L'Hospital's rule and equation (35), we have:

$$\begin{align} \begin{aligned} \lim_{\tau \rightarrow \infty }\frac{C\left( \tau \right) -\gamma }{\gamma \bar{\Phi}\left( \tau \right) }& =1-\lim_{\tau \rightarrow \infty }\frac{\bar{ \Psi}\left( \tau \right) }{\bar{\Phi}\left( \tau \right) } \\ & =1-\lim_{t\rightarrow \infty }\frac{\psi \left( t\right) }{\phi \left( t\right) }=1-\lim_{t\rightarrow \infty }\frac{1}{\gamma }\left[ \frac{1}{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1} \\ & =1-\frac{1}{\gamma }\left[ \frac{1}{\delta_{\exp} }\lim_{t\rightarrow \infty }H\left( t\right) \right] ^{\epsilon -1}. \end{aligned} \end{align} \tag{ 41 }$$

Equation (41) implies that

$$\begin{align} \lim_{\tau \rightarrow \infty }\frac{C\left( \tau \right) -\gamma }{\bar{\Phi }\left( \tau \right) }=\gamma -\left[ \frac{1}{\delta_{\exp} }H\left( \infty \right) \right] ^{\epsilon -1}. \end{align} \tag{ 42 }$$

In turn, using the connection between the coincidence likelihood and the diversity (recall equation (4)), equation (42) implies equation (17).

Equation (35), equation (15), and the connection between the coincidence likelihood and the diversity (recall equation (4)) imply that:

$$\begin{align} \begin{aligned} \Psi \left( \tau \right) -\Phi \left( \tau \right) & =\int_{0}^{\tau }\psi \left( t\right) dt-\int_{0}^{\tau }\phi \left( t\right) dt \\ & =\int_{0}^{\tau }\left[ \psi \left( t\right) -\phi \left( t\right) \right] dt=\int_{0}^{\tau }\left[ \frac{\psi \left( t\right) }{\phi \left( t\right) } -1\right] \phi \left( t\right) dt \\ & =\int_{0}^{\tau }\left\{\left[ \frac{\delta }{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1}-1\right\} \phi \left( t\right) dt \\ & =\left( \frac{\delta }{\delta_{\exp} }\right) ^{\epsilon -1}\int_{0}^{\tau }\left[ H\left( t\right) ^{\epsilon -1}-\left( \frac{\delta_{\exp} }{\delta }\right) ^{\epsilon -1}\right] \phi \left( t\right) dt \\ & =\frac{1}{\gamma \epsilon }\int_{0}^{\tau }\left[ H\left( t\right) ^{\epsilon -1}-\left( \frac{\delta_{\exp} }{\delta }\right) ^{\epsilon -1}\right] \phi \left( t\right) dt. \end{aligned} \end{align} \tag{ 43 }$$

Substituting equation (43) into equation (28) yields equation (20).

Equations (15) and (35), and the connection between the coincidence likelihood and the diversity (recall equation (4)) imply that:

$$\begin{align} \begin{aligned} \bar{\Phi}\left( \tau \right) -\bar{\Psi}\left( \tau \right) & =\int_{\tau }^{\infty }\phi \left( t\right) dt-\int_{\tau }^{\infty }\psi \left( t\right) dt \\ & =\int_{\tau }^{\infty }\left[ \phi \left( t\right) -\psi \left( t\right) \right] dt=\int_{\tau }^{\infty }\left[ 1-\frac{\psi \left( t\right) }{\phi \left( t\right) }\right] \phi \left( t\right) dt \\ & =\int_{\tau }^{\infty }\left\{1-\left[ \frac{\delta }{\delta_{\exp} }H\left( t\right) \right] ^{\epsilon -1}\right\} \phi \left( t\right) dt \\ & =\left( \frac{\delta }{\delta_{\exp} }\right) ^{\epsilon -1}\int_{\tau }^{\infty } \left[ \left( \frac{\delta_{\exp} }{\delta }\right) ^{\epsilon -1}-H\left( t\right) ^{\epsilon -1}\right] \phi \left( t\right) dt \\ & =\frac{1}{\gamma \epsilon }\int_{\tau }^{\infty }\left[ \left( \frac{\delta_{\exp} }{ \delta }\right) ^{\epsilon -1}-H\left( t\right) ^{\epsilon -1}\right] \phi \left( t\right) dt. \end{aligned} \end{align} \tag{ 44 }$$

Substituting equation (44) into equation (39) yields equation (21).

8.5.1. Row I.

Note that the condition $\bar{\Psi}\left( \tau \right) \lessgtr \bar{\Phi} \left( \tau \right)$ is equivalent to the condition

$$\begin{align} \gamma \frac{\bar{\Phi}\left( \tau \right) -\bar{\Psi}\left( \tau \right) }{\epsilon -1}\gtrless 0, \end{align} \tag{ 45 }$$

since γ > 0 and ε > 1. Using equations (7) and (8), as well as the input's coincidence likelihood γ, further note that

$$\begin{align} \begin{aligned} \gamma \left[ \bar{\Phi}\left( \tau \right) -\bar{\Psi}\left( \tau \right) \right] & =\gamma \left[ \bar{F}\left( \tau \right) ^{\epsilon }-\int_{\tau }^{\infty }\frac{1}{\gamma }f\left( t\right) ^{\epsilon }dt\right] \\ & =\left[ \int_{0}^{\infty }f\left( t\right) ^{\epsilon }dt\right] \bar{F} \left( \tau \right) ^{\epsilon }-\int_{\tau }^{\infty }f\left( t\right) ^{\epsilon }dt. \end{aligned} \end{align} \tag{ 46 }$$

Applying the limit $\epsilon \rightarrow 1$ to the term on the right-hand side of equation (45), while using equation (46) and L'Hospital's rule, we have

$$\begin{align} & \lim_{\epsilon \rightarrow 1}\gamma \frac{\bar{\Phi}\left( \tau \right) -\bar{\Psi}\left( \tau \right) }{\epsilon -1} \nonumber\\ & \quad =\lim_{\epsilon \rightarrow 1}\left\{\left[ \int_{0}^{\infty }f\left(t\right) ^{\epsilon }\ln \left[\, f\left( t\right) \right] dt\right] \bar{F} \left( \tau \right) ^{\epsilon }\right. \nonumber\\ & \qquad \left. +\left[ \int_{0}^{\infty }f\left( t\right) ^{\epsilon}dt\right] \left[ \bar{F}\left( \tau \right) ^{\epsilon }\ln \left[ \bar{F} \left( \tau \right) \right] \right] -\int_{\tau }^{\infty }f\left( t\right)^{\epsilon }\ln \left[\, f\left( t\right) \right] dt\right\} \nonumber\\ & \quad =\left[ \int_{0}^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt\right] \bar{F}\left( \tau \right) \nonumber\\ & \qquad +\left[ \int_{0}^{\infty }f\left( t\right) dt\right] \left[ \bar{F}\left( \tau \right) \ln \left[ \bar{F}\left( \tau \right) \right] \right] -\int_{\tau }^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt \nonumber\\ & \quad =-\ln \left( \delta \right) \bar{F}\left( \tau \right) +\bar{F}\left( \tau\right) \ln \left[ \bar{F}\left( \tau \right) \right] -\int_{\tau }^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt,\end{align} \tag{ 47 }$$

where δ in the bottom line of equation (47) denotes the input's perplexity. Equation (47) implies that

$$\begin{align} \lim_{\epsilon \rightarrow 1}\gamma \frac{\bar{\Phi}\left( \tau \right) - \bar{\Psi}\left( \tau \right) }{\epsilon -1}=\bar{F}\left( \tau \right) \left\{\ln \left[ \frac{\bar{F}\left( \tau \right) }{\delta }\right] -\frac{ 1}{\bar{F}\left( \tau \right) }\int_{\tau }^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt\right\} . \end{align} \tag{ 48 }$$

Combined together, equations (45) and (48) lead from row I of table 3 to row I of table 4.

8.5.2. Row II.

Note that the condition $\mu _{\psi }\lessgtr \mu _{\phi }$ is equivalent to the condition

$$\begin{align} \gamma \frac{\mu _{\phi }-\mu _{\psi }}{\epsilon -1}\gtrless 0. \end{align} \tag{ 49 }$$

Using equations (7) and (8), as well as the input's coincidence likelihood γ, further note that

$$\begin{align} \begin{aligned} \gamma \left[ \mu _{\phi }-\mu _{\psi }\right] & =\gamma \left[ \int_{0}^{\infty }\bar{\Phi}\left( t\right) dt-\int_{0}^{\infty }t\psi \left( t\right) dt\right] \\ & =\gamma \left[ \int_{0}^{\infty }\bar{F}\left( t\right) ^{\epsilon }dt-\int_{0}^{\infty }t\frac{1}{\gamma }f\left( t\right) ^{\epsilon }dt \right] \\ & =\left[ \int_{0}^{\infty }f\left( t\right) ^{\epsilon }dt\right] \left[ \int_{0}^{\infty }\bar{F}\left( t\right) ^{\epsilon }dt\right] -\int_{0}^{\infty }t\,f\left( t\right) ^{\epsilon }dt. \end{aligned} \end{align} \tag{ 50 }$$

Applying the limit $\epsilon \rightarrow 1$ to the term on the left-hand side of equation (49), while using equation (50) and L'Hospital's rule, we have

$$\begin{align} & \lim_{\epsilon \rightarrow 1}\gamma \frac{\mu _{\phi }-\mu _{\psi }}{\epsilon -1} \nonumber\\ & \quad =\lim_{\epsilon \rightarrow 1}\left\{\left[ \int_{0}^{\infty }f\left(t\right) ^{\epsilon }\ln \left[\, f\left( t\right) \right] dt\right] \left[ \int_{0}^{\infty }\bar{F}\left( t\right) ^{\epsilon }dt\right] \right. \nonumber\\ & \qquad \left. +\left[ \int_{0}^{\infty }f\left( t\right) ^{\epsilon}dt\right] \left[ \int_{0}^{\infty }\bar{F}\left( t\right) ^{\epsilon }\ln \left[ \bar{F}\left( t\right) \right] dt\right] -\int_{0}^{\infty }t\,f\left(t\right) ^{\epsilon }\ln \left[\, f\left( t\right) \right] dt\right\} \nonumber\\ & \quad =\left[ \int_{0}^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt\right] \left[ \int_{0}^{\infty }\bar{F}\left( t\right) dt\right] \nonumber\\ & \qquad +\left[ \int_{0}^{\infty }f\left( t\right) dt\right] \left[\int_{0}^{\infty }\bar{F}\left( t\right) \ln \left[ \bar{F}\left( t\right) \right] dt\right] -\int_{0}^{\infty }t\,f\left( t\right) \ln \left[\, f\left(t\right) \right] dt \nonumber\\ & \quad =-\ln \left( \delta \right) \mu +\int_{0}^{\infty }\bar{F}\left( t\right)\ln \left[ \bar{F}\left( t\right) \right] dt-\int_{0}^{\infty }t\,f\left( t\right) \ln \left[\, f\left( t\right) \right] dt, \end{align} \tag{ 51 }$$

where: δ in the bottom line of equation (51) denotes the input's perplexity; and µ denotes the input's mean. Equation (51) implies that

$$\begin{align} \lim_{\epsilon \rightarrow 1}\gamma \frac{\mu _{\phi }-\mu _{\psi }}{ \epsilon -1}=\int_{0}^{\infty }\bar{F}\left( t\right) \ln \left[ \bar{F} \left( t\right) \right] dt-\int_{0}^{\infty }t\,f\left( t\right) \ln \left[\, f\left( t\right) \right] dt-\ln \left( \delta \right) \mu . \end{align} \tag{ 52 }$$

Combined together, equations (49) and (52) lead from row II of table 3 to row II of table 4.

8.5.3. Row III'.

Note that the condition $\Pr \left( T_{\psi }\leqslant T_{\phi }\right) \gtrless \frac{1}{2}$ is equivalent to the condition

$$\begin{align} \gamma \frac{2\Pr \left( T_{\phi }>T_{\psi }\right) -1}{\epsilon -1}\gtrless 0. \end{align} \tag{ 53 }$$

Using equations (7) and (8), equation (32), and the input's coincidence likelihood γ, further note that

$$\begin{align} \gamma \left[ 2\Pr \left( T_{\phi }>T_{\psi }\right) -1\right] & =\gamma \left[2\int_{0}^{\infty }\bar{\Phi}\left( \tau \right) \psi \left( t\right) dt-1\right] \nonumber\\ & =\gamma \left[ 2\int_{0}^{\infty }\bar{F}\left( t\right) ^{\epsilon }\frac{1}{\gamma }f\left( t\right) ^{\epsilon }dt-1\right] \nonumber\\ & =2\int_{0}^{\infty }\left[ \bar{F}\left( t\right) f\left( t\right) \right]^{\epsilon }dt-\int_{0}^{\infty }f\left( t\right) ^{\epsilon }dt. \end{align} \tag{ 54 }$$

Applying the limit $\epsilon \rightarrow 1$ to the term on the left-hand side of equation (53), while using equation (54) and L'Hospital's rule, we have

$$\begin{align} & \lim_{\epsilon \rightarrow 1}\gamma \frac{2\Pr \left( T_{\phi }>T_{\psi}\right) -1}{\epsilon -1} \nonumber\\ & \quad =\lim_{\epsilon \rightarrow 1}\left\{2\int_{0}^{\infty }\left[ \bar{F}\left( t\right) f\left( t\right) \right] ^{\epsilon }\ln \left[ \bar{F} \left( t\right)\, f\left( t\right) \right] dt-\int_{0}^{\infty }f\left(t\right) ^{\epsilon }\ln \left[\, f\left( t\right) \right] dt\right\} \nonumber\\ & \quad =2\int_{0}^{\infty }\left[ \bar{F}\left( t\right) f\left( t\right) \right]\ln \left[ \bar{F}\left( t\right) f\left( t\right) \right] dt-\int_{0}^{\infty }f\left( t\right) \ln \left[\, f\left( t\right) \right] dt\nonumber\\ & \quad =\int_{0}^{\infty }f_{\min }\left( t\right) \ln \left[ \frac{1}{2}f_{\min}\left( t\right) \right] dt+\ln \left( \delta \right), \end{align} \tag{ 55 }$$

where: $f_{\min }\left( t\right) = 2\bar{F} \left( t\right) f\left( t\right)$ in the bottom line of equation (55) denotes the density function of the minimum of two IID copies of the input T; and δ denotes the input's perplexity. Denoting by $\delta _{\min }$ the perplexity of the minimum of two IID copies of the input T, note that

$$\begin{align} & \int_{0}^{\infty }f_{\min }\left( t\right) \ln \left[ \frac{1}{2}f_{\min}\left( t\right) \right] dt \nonumber\\ & \quad =\int_{0}^{\infty }f_{\min }\left( t\right) \ln \left[\, f_{\min }\left(t\right) \right] dt-\ln \left( 2\right) \int_{0}^{\infty }f_{\min }\left( t\right) dt \nonumber\\ & \quad =-\ln \left( \delta _{\min }\right) -\ln \left( 2\right). \end{align} \tag{ 56 }$$

Equations (55) and (56) imply that

$$\begin{align} \lim_{\epsilon \rightarrow 1}\gamma \frac{2\Pr \left( T_{\phi }>T_{\psi}\right) -1}{\epsilon -1} =-\ln \left( \delta _{\min }\right) -\ln \left( 2\right) +\ln \left( \delta\right) =\ln \left( \frac{\delta }{2\delta _{\min }}\right) . \end{align} \tag{ 57 }$$

Combined together, equations (53) and (57) lead from row III of table 3 to row III' of table 4—which is the 'perplexity counterpart' of row III' of table 3.

8.5.4. Rows IV and V.

Applying the limit $\epsilon \rightarrow 1$ to the level of equation (15), while using the change-of-variables $\epsilon \mapsto n = 1/(\epsilon -1)$, note that:

$$\begin{align} \lim_{\epsilon \rightarrow 1}\delta_{\exp} =\lim_{\epsilon \rightarrow 1}\epsilon ^{1/(\epsilon -1)}=\lim_{n\rightarrow \infty }\left( 1+\frac{1}{n}\right) ^{n}=e, \end{align} \tag{ 58 }$$

where e is the base of the natural logarithm (Euler's number). Consequently, applying the limit $\epsilon \rightarrow 1$ to equations (18) and (19), we obtain the thresholds

$$\begin{align} \tau _{\ast }=\inf \left\{t>0 \vert H\left( t\right) = \frac{e}{\delta }\right\}, \end{align} \tag{ 59 }$$

and

$$\begin{align} \tau ^{\ast }=\sup \left\{t>0 \vert H\left( t\right) = \frac{e}{\delta }\right\}. \end{align} \tag{ 60 }$$

Equations (58)–(60) lead from rows IV and V of table 3 to rows IV and V of table 4, respectively.