Dissipative timescales from coarse-graining irreversibility

We propose and investigate a method for identifying timescales of dissipation in nonequilibrium steady states modeled as discrete-state Markov jump processes. The method is based on how the irreversibility-measured by the statistical breaking of time-reversal symmetry-varies under temporal coarse-graining. We observe a sigmoidal-like shape of the irreversibility as a function of the coarse-graining time whose functional form we derive for systems with a fast driven transition. This theoretical prediction allows us to develop a method for estimating the dissipative time scale from time-series data by fitting estimates of the irreversibility to our predicted functional form. We further analyze the accuracy and statistical fluctuations of this estimate.


Introduction
Dissipation, the irrecoverable loss of energy by a system to its surroundings, is a defining feature of nonequilibrium steady states. Typically, it is quantified by the entropy production rate Σ [1]. This single number, however, elides over the details of the microscopic dynamics: different systems may have the same entropy production rate although their microscopic details may differ. This observation suggests that the development of more refined characterizations of dissipation could aid in our understanding of the nature of nonequilibrium steady states.
One way to characterize a physical process is to identify the characteristic time and length scales on which it operates [2,3]. For dissipation, we are then aiming to probe the characteristics of the microscopic dynamics that result in energy loss to the surroundings. However, nonequilibrium steady states continually exchange energy with their environment, making it somewhat unclear how to tweeze apart, without a model, the various processes that contribute to dissipation. In this article, we introduce a method to identify and measure these timescales by exploiting the deep connection between irreversibility and dissipation.
One of the central results of stochastic thermodynamics has been the formulation of a quantitative connection between the entropy production rate Σ and the statistical irreversibility (time-reversal asymmetry) of the dynamics as measured by the relative entropy between the probability P (γ t ) of a microscopic trajectory γ t of length t and the probability P (γ t ) of the time-reversed trajectoryγ t [4,5,6,7,8,9], where k B is the Boltzmann constant, which we will set to one for the remainder of this paper for notational simplicity. The relative entropy between two probability distributions p and q, D(p||q) = p(x) ln[p(x)/q(x)]dx, is an information-theoretic measure of distinguishability. For this reason, we can interpret the relative entropy in Eq. (1) as a measure of the distinguishability between the forward and reverse trajectories, namely, the irreversibility. Furthermore, the entropy production rate itself can be linked to energy dissipation, once we have identified the thermodynamic reservoirs in the environment. For example, when a system is coupled to two heat baths at temperatures T 1 > T 2 , the entropy production rate Σ = (1/T 2 − 1/T 1 )Q is proportional to the heat flow from reservoir 1 to 2. A useful implication of (1) is that the detection of a nonzero irreversibility from experimental time-series data allows one to deduce that a system is out of equilibrium, just from observations of dynamic fluctuations [10,11,12,13,14,15,16]. However, it has been observed in theoretical and experimental studies that the amount of irreversibility detected is sensitive to the types and numbers of observed variables as well as to the data acquisition rate [17,18,19].
Although the data-dependence of the measured irreversibility has been considered a nuisance, here we are going to attempt to exploit it to identify a dissipative timescale. To this end, imagine coarse-graining the dynamics in time by removing all information on a timescale shorter than τ . Examples include making stroboscopic observations at intervals of length τ or obtaining data as local time averages over windows of width τ . From an experimental point of view, such coarse graining is quite natural as one can never make measurements with infinite precision. No matter the procedure, we assume the result is a collection of N sequential observations that we can collect into a coarsegrained trajectory γ τ N , from which we can determine the observed irreversibility above the timescale τ Under the mild assumption that coarse-graining commutes with time reversal, the observed irreversibility bounds the entropy production from below, σ(τ ) ≤ Σ [20,21,22]. At each level of coarse-graining, we remove information about the dynamics at timescales shorter than τ , implying that the relative entropy σ(τ ) remains sensitive to any irreversible processes operating on timescales longer than τ . These arguments suggest that the variation of σ(τ ) with coarse-graining scale τ is potentially sensitive to a characteristic timescale relevant to dissipation. Indeed, this was observed in an experimental study of the actomyosin network of a starfish oocyte [15]. The fluctuations of endogenous probe particles embedded in the cortex were measured and used to compute σ(τ ). The observed irreversibility as a function of τ was found to peak at a value consistent with the rate of turnover of adenosine triphosphate (ATP) by the myosin motors responsible for the nonequilibrium fluctuations in the network. In this article, we use simple models of nonequilibrium steady states with well-defined characteristic timescales to analyze this dependence of the observed irreversibility on coarse-graining timescale τ in an effort to understand what can be inferred from measurements of σ(τ ). It is worth noting that coarse-graining irreversibility as a means to characterize dissipation is not unique to this study and represents a potentially powerful approach to assessing the properties of dissipation in nonequilibrium steady states. Indeed, similar observations were exploited in Ref. [23,24] where the authors coarse-grained progressively larger interdependencies among a set of interacting degrees of freedom. This allowed them to identify how the complexity of interactions among the system's degrees of freedom contributes to dissipation.
Our approach to identify dissipative timescales complements other proposals in the literature that decompose the dissipation at distinct locations and length scales. Like in our present work, the authors of Ref. [25] begin their analysis with the connection between dissipation and irreversibility embodied in Eq. (1). They then identify a local irreversibility by computing the observed irreversibility within small grids of a spatially extended system. Applications of this methodology to active fluids identified regions where irreversibility was prominent, either as a consequence of mobility-induced phase separation or the inclusion of fixed asymmetric obstacles, such as chevrons. Alternative approaches aim to decompose the irreversibility into distinct frequency and wave number components. For instance, the authors of Ref. [26], suggest evaluating the relative entropy rate by replacing the true microscopic dynamics by a Gaussian diffusion process with the same power spectrum. The resulting approximate relative entropy rate depends solely on the power spectrum and can be decomposed into frequency and wavenumber contributions. This allows one to identify time and length scales from the shape of this decomposition. A similar though distinct approach has been proposed based on the Harada-Sasa equality [27] that decomposes the dissipation into frequency and wavenumber dependent violations of the fluctuation-dissipation theorem [28,29]. While offering a principled identification of spatiotemporal scales for diffusive dynamics, the Harada-Sasa equality is known to fail for discrete-state Markov models [30,31].
We begin in Sec. 2 by introducing our models of interest and collecting their relevant properties. Then in Sec. 3, we numerically and theoretically analyze a model with a single dissipative timescale. We observe that for highly dissipative systems the observed irreversibility σ(τ ) has a simple functional form that we are able to predict using a perturbation theory. This functional form then serves as an ansatz whose fit to experimental or numerical data can be used to estimate the dissipative timescale. The performance of this estimation procedure is analyzed numerically in Sec. 4, before concluding in Sec. 5.

Coarse-graining Markov jump processes
Our focus will be systems that can be modeled as discrete-state Markov jump processes. Their dynamics are captured by how the vector of probabilities p(t) = {p 1 (t), . . . , p n (t)} over the states i = 1, . . . , n evolves with time t [32], where the off-diagonal elements of the transition rate matrix W ij give the transition probabilities per unit time to jump from j → i and the diagonal elements W ii = − j̸ =i W ji are fixed by probability conservation. We will further assume that W ij ̸ = 0 only if W ji ̸ = 0, which as we will see guarantees a well-defined entropy production rate [1]. The formal solution of (3) with initial condition p(0) is given by the matrix exponential We will assume that our dynamics are ergodic such that (4) tends to a unique steadystate distribution π = {π 1 , . . . , π n } given as the solution ofŴ π = 0. Our interest lies in the thermodynamics of the steady state. For such Markov processes, the entropy production rate Σ can be readily obtained (say by direct evaluation of (1)) [33], The link to energy dissipation can be made once we have identified the reservoirs coupled to the system that supply the energy (and particles) required for transitions between the states. Formally, this is implemented by assuming that the transition rates satisfy a local detailed balance condition [33]: for example, if the transition from k → l is mediated by a single thermal reservoir at inverse temperature β, then we require ln(W kl /W lk ) = βq kl , where q kl is the heat flow into the reservoir accompanying the transition. Now, a natural and analytically tractable method for coarse-graining continuoustime dynamics are stroboscopic observations every τ . The output is a collection of N observations γ τ N = {i 1 , . . . , i N }. Such stroboscopic measurements are attractive because the resulting coarse-grained dynamics is a discrete-time Markov chain with the same steady-state distribution π. The transition probabilities M ij (τ ) are given as the solution of (3) with Kronecker delta initial condition p q (0) = δ qj : For such observed dynamics, we can readily evaluate the relative entropy rate per unit time in Eq. (2) to obtain the observed irreversibility. Note that the probability to observe the trajectory γ τ N = {i 1 , . . . , i N } beginning in the steady-state is determined by the transition probabilities via Thus, comparing the forward and reverse trajectoryγ τ Upon taking the limit, and noting that only transitions between different states contribute, we arrive at Before proceeding to our analysis of the observed irreversibility, it is worth noting a few algebraic properties of the transition rate matrix that will be helpful in our theoretical calculations [32]. First,Ŵ need not be symmetric, which implies that each eigenvalue λ α can have associated with it a distinct right u α and left v α eigenvector given as the solutions of We have included a minus sign for later convenience in order that all eigenvalues have a non-negative real part [32]. While the eigenvectors need not exist, for simplicity we assume that they do and that they form a complete basis: that is, the eigenvectors can be chosen to satisfy u α · v β = δ αβ such that they verify the completeness relation α u α v α,T =Î, withÎ being the identity matrix. Furthermore, we choose to label the eigenvalue ofŴ corresponding to the steady state distribution as λ 0 = 0. The remaining n − 1 eigenvalues can be ordered using their real part from smallest to largest λ 0 ≤ λ 1 ≤ · · · ≤ λ n−1 . A useful consequence of these assumptions is that the transition matrixM (τ ) can be decomposed aŝ 3. Observed irreversibility with a dissipative timescale

Model with single dissipative transition
To test our hypothesis that the observed irreversibility contains information about the dissipative timescales, we consider models containing a single dissipative transition with rate ν.
With this in mind, our transition rate matrix can be divided into two parts, an undriven or equilibrium partŴ eq and a driven or nonequilibrium partŴ neq , with ν setting the relative scale. We choose the equilibrium transition rates to have an Arrhenius form, with symmetric 'energy barriers' B ij = B ji and 'state energies' E j . IfŴ =Ŵ eq (ν = 0), then the system would relax to a steady state characterized by an equilibrium Gibbs distribution π eq j ∝ e −E j with weights determined by the dimensionless energies E j . It is for this reason that we sayŴ eq describes undriven or equilibrium-like dynamics. Every transition will be an equilibrium transition except for transitions between 1 ↔ 2 (W eq 12 = W eq 21 = 0), which we will drive out of equilibrium. Thus, we choose all the off-diagonal elements ofŴ neq to be zero, apart from those entries corresponding to the transition 1 ↔ 2. Here, driving is implemented by adding an asymmetric force F to the Arrhenius form, with a barrier B neq . By construction, the sole source of nonequilibrium driving comes from the asymmetry of the driving force inŴ neq . This effect becomes explicit when calculating the entropy production rate (5) for this model where we have introduced the probability flux between states 1 and 2, J 12 = ν(W neq 21 π 1 − W neq 12 π 2 ). Thus, nonzero F is required to have nonequilibrium systems in steady state.

Numerical analysis
For our system with a single driven transition, we can determine closed-form expressions for the transition matrixM (τ ), and from (10) the observed irreversibility σ(τ ). To gain insight, we focus on a system with n = 4 states and make the simplifying choice of setting all E j = B ij = B neq = 0. This assumption sets the timescale W eq ij = 1 against which the rates W neq 12 = (W neq 21 ) −1 = e F/2 are compared. The resulting expressions for σ(τ ) are plotted in figure 1 for a representative value of the driving force (F = 5) as a function of timescale ν and coarse-graining scale τ . In figure 1(a), we see that for every value of the timescale ν, as we vary τ the observed irreversibility displays a cross over from the steady-state entropy production rate Σ (the left limit) to zero. Moreover, the shape of this crossover is sensitive to the value of ν. For ν ≤ 1 (blue tones), the dissipative transition rate limits Σ (inset). As ν increases, not only does the value of Σ increase, but the knee of σ(τ ) shifts in location and steepness. However, when ν ≥ 1, we see that Σ is no longer limited by ν, rather by the driving force (inset). As we continue to increase ν in this regime (red tones), the dominant effect on σ(τ ) is to simply shift the knee leftward, while maintaining the same relative shape. These observations become more pronounced in figures 1(b)&(c), where we normalize the observed irreversibility by the entropy production rate, σ(τ )/Σ. Such normalization highlights the qualitative change in the ν dependence above and below ν = 1. This analysis reveals that even for this relatively simple model, the shape of σ(τ ) is sensitive to the interplay between the timescale ν and the strength of thermodynamic driving F . The exception is when the driven transitions occur on a fast enough timescale (here ν > 1), such that the driving force is the limiting factor determining the entropy production rate. In such a limit, the shape of σ(τ ) does not vary with timescale ν, only the location of the knee.

Functional form of the observed irreversibility
When the dissipative transition rate is large, we have observed a rather stable shape of the observed irreversibility. To understand this simple dependence of σ(τ ) on ν we will now analyze the observed irreversibility perturbatively for ν ≫ 1. An additional goal of this section will be finding a sufficiently generic functional form for σ(τ ) that can be leveraged in Sec. 4 as a data-fitting method to estimate the dissipative timescale.
To this end, we will need to approximateM (τ ) for fast driven transitions, which is facilitated by looking at the four eigenvalues of the transition rate matrixŴ in our 4-state model with ν and F the only nonzero parameters: Clearly, for ν ≫ 1, one eigenvalue is much larger than the others and scales with the driven transition rate, λ 3 ∼ ν. Thus, at least for times up to the dissipative timescale τ ∼ 1/ν the dynamics will be dominated by the largest eigenvalue. We can now approximateM (τ ) and the observed irreversibility on this timescale assuming a single large dominant eigenvalue. This approximation can be done in some generality, so that our theoretical analysis will apply beyond the simple model analyzed in the previous subsection. We begin the derivation by considering a transition rate matrixŴ whose largest eigenvalue is much larger than all other eigenvalues, λ ≡ λ n−1 ≫ λ α . For notational convenience, we also introduce the projectorP = u n−1 v n−1,T onto the eigenspace of λ. Then for times τ ≲ 1/λ, we can approximate (13) aŝ Substituting in the completeness relation and the eigendecomposition ofŴ , allows us to arrive at the suggestive form Using this approximation for the transition matrix, we can now approximate the observed irreversibility as, While this is an accurate approximation in our region of interest, it remains a complicated function of the elements of the transition rate matrix, making (24) unsuitable for inference. We can simplify this expression further by noting that (e −λτ − 1 + λτ ) ≪ λτ when τ ≲ 1/λ. This allows us to expand the logarithm to second order, giving rise to The resulting expression (25) has the advantage of making the dependence of σ(τ ) on λ and τ explicit, and hides all the details of the elements of the transition rate matrix in two multiplicative constants, a and b.
To test the accuracy of our approximation, we plot the normalized observed irreversibility σ(τ )/Σ in figure 2 for ν ≥ 1 (solid) alongside our prediction in Eq. (25) (dashed). As anticipated, our approximation is fairly accurate for times τ ≲ 1/λ and improves as λ ∼ ν increases. We see that when there is a strong timescale separation between the equilibrium and driven transitions a simple story emerges for the observed irreversibility: as we coarse grain past the timescale of the dissipative transition, the shape of the observed irreversibility takes on a simple form whose scale is controlled by the largest eigenvalue of the transition rate matrix. We can thus identify in these scenarios the (inverse) eigenvalue 1/λ as the dissipative timescale: observations faster than 1/λ appear irreversible, while observations slower appear reversible.

Measuring the dissipative timescale
As mentioned in the introduction, part of our motivation for this study was the experiments conducted in Ref. [15] where the observed irreversibility was measured. Naturally, this raises the question of whether the analysis in the previous section can be used as a means to extract the dissipative timescale, that is the dominant eigenvalue λ from experimental data.

Estimates using a fit ansatz
The rather simple functional form for the observed irreversibility derived in Eq. (25) suggests a method to infer λ from measured values of the normalized observed irreversibility σ(τ )/Σ. Namely, we use (25) to inform a fitting ansatz with fitting parametersã,b andλ. We can then generate an estimate for the dominant eigenvalueλ by fitting (28) to measured values of the normalized observed irreversibility.
To test the accuracy of this approach, we consider the case where measured values of the observed irreversibility are unbiased and have infinite precision. To this end, we generate 10 3 random transition rate matrices according to the model in Eqs. (14)- (17), where the E j , B ij and B neq are sampled uniformly in the range [0, 1], the driving force F is sampled uniformly on [0, 5], and the timescale ν is sampled from a log-base 10 uniform distribution from [1, 10 2 ]. For each transition rate matrix we evaluate σ(τ ) numerically at values uniformly-spaced on a log-scale over the range τ ∈ [τ m , τ M ] = [10 −5 , 10 1 ]. This forms the data that we use to fit our ansatz f (τ ) (28) using a nonlinear least-squares fit function. We plot the estimatedλ versus the true eigenvalue λ for the 10 3 random transition rate matrices in figure 3.
Clearly our estimates correlate well with the true value as the cluster of points lie near the diagonal. However, there is a clear bias in our estimate that is uniform on a log scale, as the estimates tend to fall just below the diagonal. Indeed, we can estimate this bias by determining the (normalized) average deviation over our random samples: ⟨(λ −λ)/λ⟩ ≈ 8%. This bias is likely due to the fact that we are fitting a highly nonlinear function of λ, and such nonlinearities are known to lead to biased estimates [34]. Nevertheless, we can conclude in this highly-dissipative regime that fitting the observed irreversibility to our ansatz f (τ ) does allow one to reasonably well estimate the order of magnitude of the dissipative timescale, if not the true value.  (28) to the normalized observed irreversibility σ(τ )/Σ for 10 3 randomly generated transition rate matrices as described in the text. The black line is a guide to the eye and represents perfect estimation.

Implications of finite data
Numerically or experimentally generated time-series data is finite, which can result in bias and noise in the estimates of the observed irreversibility. In this section, we assess how these errors due to a finite sample size propagate to our estimate ofλ.
To perform this assessment, we numerically generate continuous-time trajectories γ T of a finite duration T for random transition rate matrices using the Gillespie algorithm [35]. We then coarse grained these trajectories by observing the system state every τ , forming the coarse-grained trajectory γ τ N = {i 1 , i 2 , . . . i N }. From each such coarse-grained trajectory, we estimate the observed irreversibility using a plug-in estimator [10,11]. Namely, we directly estimate the steady-state distribution as well as the transition probabilities by counting the number of visits to each state and the number of each type of transition, These estimates are then plugged into (10) to generate our estimate of the observed irreversibilityσ While more sophisticated methods are available to estimate the observed irreversibility [10, 16, 19, 24, 25], we chose this method for its computational simplicity. Moreover, our concern here is not with how to accurately estimate the observed irreversibility, but how errors, which are inevitable in any method, propagate to estimates of the dissipative timescale. The estimatedσ(τ ) is normalized usingσ(τ m ), our estimate at the smallest observation time τ considered (τ m = 10 −5 ), and then fit by f (τ ) (28) to generate an estimate of the timescaleλ. The results for 10 2 random transition rate matrices are presented in figure 4 using the same ensemble described above in Sec. 4.1 except with fixed ν = 10 2 . For each rate matrix a trajectory of length T = 10 7 was generated from whichλ was estimated. We then repeated the procedure on the same trajectory, but instead taking the first T = 10 6 of the trajectory, then T = {10 5 , 10 4 }, allowing us to visualize how reducing the amount of data affects our estimates. Figure 4(a) displays the normalized relative entropy rates for a representative trajectory, displaying almost no discernible disagreement on this scale. Indeed, the expected mean deviation of the observed irreversibilities remain below 6.3% (Table 1). In figure 4(b) estimates ofλ are compared for various observation times T = {10 4 , 10 5 , 10 6 , 10 7 }. In addition, we include estimates from the noiseless data studied in the previous subsection Sec. 4.1, which in the present language corresponds to a trajectory of infinite length T = ∞. For observation times T ≥ 10 5 , the finite sample estimates are nearly as accurate as having perfectly accurate estimates of the observed irreversibility. Noise appears detrimental only for the shortest studied observation time T = 10 4 , reaching 35%, about 3 times larger than our unavoidable error floor of ≈ 12% due to our nonlinear fitting method. The potential origin of this increased error can be seen in the inset of figure 4(a) where we plot the absolute error, as a function of τ for various values of T . This error is concentrated near the crossover where τ ∼ 1/λ with additional deviations for long coarse-graining times where the number of samples N = T /τ is least. This suggests that accurate estimation of the knee region of the observed irreversibility is required when implementing our proposed estimation method, and even small deviations can adversely affect the estimate of the dissipative timescale.  (28) to the normalized observed irreversibility estimatesσ(τ )/σ(τ m ) for 10 2 randomly generated transition rate matrices. The diagonal black line is a guide to the eye and represents perfect estimation.

Conclusion
Under coarse-graining the observed irreversibility σ(τ ) displays a characteristic crossover from its steady state value Σ to zero. When there is strong timescale separation between the driven transition and the undriven transitions, this profile takes on a simple shape with a single characteristic timescale. Using a perturbative approach we derived an approximate functional form and identified this dissipative scale with the largest eigenvalue λ of the transition rate matrix. Above this scale irreversibility is observable, whereas below it irreversibility is hidden. General theoretical analysis that goes beyond the fast driven limit remains an open question, complicated by our observation that the functional form of the observed irreversibility can vary as we change the transition speed and driving. Building on our theoretical analysis, we proposed a fitting ansatz that allows us to estimate the dissipative timescale λ from time-series data. We observed that this method reasonably well reproduces the magnitude of λ, but remains biased due to the nonlinear fitting method. The error of our estimates is further sensitive to any error in the estimates of the observed irreversibility when data is finite. Accurate estimates in the cross-over region are necessary to most effectively use our proposed estimator. More refined fitting forms that account for a wider range of scales may be less affected by errors in the relative entropy determination, but this remains for future research.