Thermodynamically consistent model of an active Ornstein–Uhlenbeck particle

Identifying the full entropy production of active particles is a challenging task. We introduce a microscopic, thermodynamically consistent model, which leads to active Ornstein–Uhlenbeck statistics in the continuum limit. Our minimal model consists of a particle with a fluctuating number of active reaction sites that contribute to its active self-propulsion on a lattice. The model also takes ordinary thermal noise into account. This approach allows us to identify the full entropy production stemming from both thermal diffusion and active driving. Extant methods based on the comparison of forward and time-reversed trajectory underestimate the physical entropy production when applied to the Langevin equations obtained from our model. Constructing microscopic Markovian models can thus provide a benchmark for determining the entropy production in non-Markovian active systems.


Introduction
The research of active matter [1][2][3] has explored many facets for which thermodynamic properties play an important role.Examples include heat engines driven by active matter and their design principles [4][5][6][7][8][9][10] and the onset of motility induced phase separation [11][12][13].In most of this theoretical work, the underlying active matter models for particles can be grouped into three distinct categories with different characteristic motion.Run and tumble particles (RTP) [14,15] perform straight motion at a constant speed in conjunction with Poisson-distributed tumble events in which the particles change their orientation.Active Brownian particles (ABP) [16,17] move at a constant velocity in the direction of a director, which changes its orientation continuously.Finally, active Ornstein-Uhlenbeck particles (AOUPs) [18][19][20] additionally feature a stochastically changing velocity, described by an Ornstein-Uhlenbeck process.
Fundamentally, all of these active matter models are driven out of equilibrium by an active reservoir such that they break time-reversal symmetry.In addition, the state of the particle is not only given by its position, but also by its orientation, which makes the dynamics of the former non-Markovian if the orientation is not accessible.
This property has made it difficult to straightforwardly apply the methods of stochastic thermodynamics [21][22][23][24] to investigate thermodynamic quantities such as the physical entropy production of these systems [25][26][27][28].In turn, attempts have been made to find new measures for the departure from equilibrium on the level of Langevin trajectories.These, however, do not have a straight-forward interpretation as physical entropy production, but rather measure the irreversibility by comparing the probability of forward and backward trajectory [29][30][31][32][33][34] or by only identifying dissipated heat [35].Constructing the backward trajectory is ambiguous however, since the timereversal for the active noise history can be chosen to be even or odd.For an in-depth discussion about the role and intricacies of time-reversal, and different interpretations of the resulting pseudo-entropy productions, we point to the reviews in [36,37].Another way to approach this issue is given in [38,39], where the colored noise history is integrated out of the path weight and encoded in a memory kernel.
In addition, exact expressions for the entropy production of active particles have been derived on the level of microscopic Markovian models, to which the principles from stochastic thermodynamics are easily applied.So far, this has been done for ABPs [40] and for RTPs [41].For active Ornstein-Uhlenbeck particles, however, such a model still does not exist to the best of our knowledge.The aim of this paper is to fill this gap and to provide a minimal, thermodynamically consistent model which leads to the equations of motion of an AOUP assumed a priori in other work.The paper is structured as follows: In section 2 we introduce the model in one dimension and calculate the exact expression for the mean entropy production rate.In section 3, we then compare this expression to results found previously.Finally, in section 4, we show that our model also exhibits AOUP statistics in higher dimensions and conclude in section 5.In the active state, the reaction site catalyzes some fuel-consuming chemical reaction with rate k + , which releases chemical free energy ∆µ.Thermodynamic consistency requires that the reverse reaction with rate k − is also possible.The blue circles symbolize the reactants.Switching between the active and inactive state occurs with rates ω on,off .(b) The particle with internal degree of freedom (n l , n r ) at position x i on the one dimensional lattice.On the left side of the particle, n l = 2 sites are active, and on the right one n r = 1 site is active.Each active site on the left pushes the particle to the right if the fuel-consuming reaction with rate k + l (x i ) takes place and pulls it to the left if the fuel-generating reaction with rate k − l (x i−1 ) occurs.Reactions on the right side of the particle have the opposite effect, pushing it to the left with rate k + r (x i−1 ) and pulling it to the right with rate k − r (x i ).In addition, the particle can diffuse thermally, indicated by the dashed arrows between the states.

Definition
We start by considering the model in one dimension, where the particle is on a discrete lattice with lattice constant a and sites x i .The motion of the particle can be driven in either direction by some process, like a chemical reaction, which draws its energy from the environment, e.g., from a chemical reservoir.This process has a forward rate k + , which extracts free energy from the reservoir, and a backward rate k − , which puts it back into the reservoir.These rates are parametrized by their asymmetric part where ∆µ is the free energy drawn from the reservoir with each forward reaction.We set the inverse temperature β = 1 from here on and measure entropy in units of k B .
As shown in figure 1, the reaction takes place at phosphorylation sites at the surface of the particle, either on the right side of the particle, where the forward reaction drives it to the left and the reverse reaction to the right, or, vice versa, on the left side of the particle.Whatever the exact underlying mechanism is, thermodynamic consistency requires that the reverse process is also possible.
On each side of the particle, there are N reaction sites in total, each of which is in either of two states.They can be active with probability p A , or inactive with a probability p I = 1 − p A .Only active sites can catalyze reactions, inactive sites do not contribute to the driving.The number of active reaction sites on the left and the right are n l and n r respectively.The activation of an inactive site occurs with rate ω on and the deactivation of an active one with rate ω off , which are given by where κ sets the timescale at which these transitions take place.Thus, the dynamics of the internal state of the particle, characterized by (n l , n r ), follows the master equation Each active site contributes to the mean velocity and mean diffusion of the particle.We call this characteristic mean velocity per reaction site u 0 and the characteristic mean diffusion coefficient per site D 0 , which are defined as For fixed (n l , n r ), the particle will then move with the average velocity ⟨ ẋ|n l , n r ⟩ = n l − n r u 0 and diffuse as ⟨(x(t) − x(0)) 2 |n l , n r ⟩ = 2 n l + n r D 0 t.With these definitions we can write If there is an additional external spatial potential V (x), thermodynamic consistency requires for the rates of reaction sites on the left side and for the reaction sites on the right side.This distinction takes into account that the forward and backward direction drive the particle in different directions of the potential depending on the side of the particle at which they take place.

Dynamics
Turning to the dynamics of the particle, we introduce the active current for a fixed number of active reaction sites n l and n r as The first term, j l ac (x i , n l , n r ), accounts for the current generated by the n l active reaction sites on the left side of the particle and has two contributions: The particle can be pushed from x i to x i+1 by a forward reaction with rate k + l (x i ) and can be pulled from x i+1 to x i by the fuel synthesizing backward reaction with rate k − l (x i ).The second term, j r ac (x i , n l , n r ), accounts for the n r active reaction sites on the right side of the particle, where the reaction with rate k + r (x i ) displaces it to the left, and the reverse reaction with rate k − r (x i ) displaces it to the right, as is illustrated in figure 1. Preparing for the continuum limit, where we treat x as a continuous variable and let a → 0, we expand the rates in a as and as well as the stationary distribution and obtain in leading order 1/a such that the total current becomes We have also introduced as a shorthand.Describing the internal state through n − and n + as independent degrees of freedom is equivalent to describing it via n l and n r .Since the physical expressions below will depend on the former description, these variables will prove more convenient.

Entropy production
The entropy production rate arising from the active motion is given as usual in stochastic thermodynamics as [23,24] with the currents given in leading order 1/a by (12).It is crucial to distinguish whether a jump occurred due to a reaction on the left or right side of the particle, as each has different contributions to the entropy production.By expanding the rates as in ( 9) and (10), we obtain Putting these together and replacing the sum with an integral, we obtain the entropy production rate from the active motion as where the averages ⟨. ..⟩ are performed with regard to p(x, n − , n + ).Additionally, we allow for ordinary thermal translational diffusion with the current [40] where D tr is the thermal diffusion constant.For this active system, the entropy production stemming from thermal translation does not vanish and is given by where the second equality follows after a partial integration.

Activation statistics
Since the activation and deactivation of reaction sites is a stochastic process, we now turn to the statistics of n − and n + , which we have so far treated as fixed values.The probability of finding a single site either activated or deactivated is independent of the state of the other sites and identically distributed for all sites.Thus, the probability of finding n l,r active sites is given by a binomial distribution B(n l,r |N, p A ).For large N , this is well approximated by a Gaussian N (µ, σ 2 ) with mean µ and variance σ 2 , such that The probability distributions for n − and n + , which are the difference and sum of n l and n r , respectively, are then also Gaussian.For n − (n + ) the mean is subtracted (added) and the variance is the sum of the variance of n l and n r , i.e., In the limit N → ∞, these approximations are exact.From ( 21), we see that n + scales linearly with the total number of sites, whereas n − does not.However, in order to recover AOUP-statistics, we require that, for large N , the contribution to the dynamics from the terms proportional to n ± in (8) should be of the same order and independent of N .This implies that N p A does not depend on N and is O(1), which we enforce by scaling p A ∝ 1/N .The constant N p A is the mean number of activated sites nA .Thus, we choose p A = nA /N , where nA will turn out to be a crucial parameter in the model.Physically, this rescaling of the activation probability ensures that the fluctuations around 0 in n − , which lead to the active propulsion, are not suppressed as the number of sites increases.At the same time, it ensures that the diffusive contribution to the active current stays finite.The rates ω on,off from (2) can then also be expressed through nA as Since the dynamics of n ± is an equilibrium process, the distribution p(n ± ) can also be interpreted as a Boltzmann distribution p(n ± ) ∼ exp [−U n ± (n ± )] with some effective potential U n ± .We have already established that p(n ± ) are Gaussian distributions, which implies that the effective potentials for n − and n + read respectively.The centers of these harmonic potentials correspond to the mean value of the distribution p(n ± ) and their widths correspond to the variance.Since the Boltzmann distribution is the steady-state solution of the Fokker-Planck equation for an overdamped particle in a harmonic potential, we get the condition on the current Here, µ n ± and D n ± are an effective mobility and an effective diffusion coefficient, respectively, for this internal degree of freedom.Since these are effective coefficients, they are not related by the Einstein relation and can be determined independently.Both are related to the timescale at which n ± change and the appropriate choice of µ n ± and D n ± lets us express the Langevin equation, which we will derive later, in such a way that we can identify n − as the colored noise source canonically used in describing AOUPs.The rate at which n ± change is based on the rates ω on/off from our microscopic model.The exit rate from a state with 2n A active sites is the sum of the rate at which an inactive site activates and the rate at which an active site deactivates.The timescale τ , at which changes to n ± take place is then the inverse of this exit rate, i.e.
where we have inserted (2) and dropped terms of order N −2 .Choosing will yield the correct Langevin equations, as we will show later.

Total entropy production
Putting together (24) and the currents ( 13) and ( 18) we obtain the stationary Fokker-Planck equation for the probability p(x, n − , n + ) as Since the activation and deactivation of reaction sites is an equilibrium process, uncoupled from the dynamics in x, it does not produce entropy.The total entropy production is then the sum of ( 17) and ( 19), which reads By multiplying (27) with V (x) and integration by parts, we obtain which we plug into (28) to obtain This mean total entropy production rate in the steady state for this active particle constitutes our first main result.

Comparing different measures for the departure from equilibrium
To show that our model is equivalent to the AOUP model, we start by briefly introducing the canonical description of AOUPs [30,31,[36][37][38], where we adopt the conventions from [38,39] to our notation, which then reads where ζ(t) and ξ(t) are independent unit white noises, by which we mean that they are correlated as While early papers [30,31] have not considered thermal noise ξ(t), it has been included in more recent publications [36][37][38].
The diffusion coefficient D is equivalent to the thermal diffusion coefficient D tr in our model.The active diffusion coefficient D a measures the amplitude of the Ornstein-Uhlenbeck process η(t), which has correlations with a correlation time τ a .The Langevin equation corresponding to our model follows with (26) from the Fokker-Planck equation (27) as with The stochastic processes ζ n + , ζ n − , ζ ac and ζ tr are all independent unit white noises.The variable n − follows an Ornstein-Uhlenbeck process and plays the role of the active noise source, while u 0 measures its amplitude and is equivalent to √ 2D a .There is, however, a difference compared to the conventional description in (31), which is the additional degree of freedom n + .It enters the dynamics of x(t) in two places.First, there is an additional noise source from the active process, which has an amplitude proportional to n + .It arises because each active site contributes to diffusion and thus increases the active noise amplitude.This new noise source leads to an effective diffusion coefficient, which can be seen as active mobility.Both of these additional phenomena also appear in the microscopic model for ABPs [40].Importantly, the fluctuations of n − , which lead to the self-propulsion, and the amplitude of n + are of the same order, that is, as the self-propulsion speed increases, so does diffusion.
With these Langevin equations, we can now compare our result for the total entropy production, (30), to results already established in the literature, based on (31).Calculating the entropy production of AOUPs on the level of Langevin dynamics relies on the time reversal of the trajectory, in which the reversal of the noise history can be chosen either even or odd.By choosing an odd time reversal, one implicitly also reverses the director of the particle, which is in conflict with the physical entropy production [40].In our model, the path weight for a trajectory x(t), given a trajectory of n − (t) and n + (t), reads where we omit the explicit time-dependence on the right-hand side for a more compact notation.The entropy production obtained from comparing the forward and backward probability under the time reversal x(t) = x(T − t), ñ± (t) = n ± (T − t), denoted by S + in [37], reads For a comparison between σ tot and S + , we use due to the fact that D tr > 0 from (34) and n l,r > 0. Hence, it follows that S + < σ tot .As discussed in [40] for ABPs, part of the underestimation comes from the fact that the Langevin description of the AOUP does not distinguish whether a jump occurs due to thermal or active noise, which both contribute differently to entropy production.In addition, the Langevin description does not distinguish whether the active noise causes a fuel-synthesizing or fuel-consuming reaction, which further increases the difference between these two measures of irreversibility.
In regimes where the energy exchanged with the active bath is much larger than the energy scale of the thermal bath, i.e., for n + D 0 ≫ D tr , setting ζ tr = 0 and discarding thermal noise is a tempting approximation.While this simplification is still thermodynamically consistent in our model, even in this case the entropy production S + underestimates the total entropy production σ tot .In contrast, the active noise ζ ac is crucial for thermodynamic consistency in our model, as it ensures micro-reversibility.However, it has not been included in the canonical description (31).
In our model, a putative time reversal that involves an odd reversal of the noise history would read ñ− (t) = −n − (T − t), such that The fact that this measure vanishes for V = 0 highlights that this quantity cannot measure a physical entropy production [40].Furthermore, it is clear that in the present model n ± are necessarily even variables for a sensible time-reversal which can be interpreted as playing a recorded movie of the motion of the particle backwards.The interpretation given in [38] that S + measures the departure from equilibrium for a passive particle in an active environment and S − for a self-propelled active particle can thus not be sustained in the present analysis.

Generalization to d dimensions
So far, we have dealt with a particle moving in one dimension.However, our approach is generalizable to any number of dimensions, as we will show in this section.To this end, each reaction site k is now associated with a director n k , which points in the direction of forward propulsion if a reaction with rate k + takes place while activated.The quantity n − is then replaced by the quantity which is the sum of all directors in the set of active sites, A. This generalizes the definition from (14), where the director for particles on the left is n k = +1 and for particles on the right n k = −1.The diffusion amplitude n + is replaced by a tensor The quantity u 0 n − still has the interpretation of a mean propulsion velocity for a given internal state.The internal state of the particle is now given by {α k }, where α k is the state of a reaction site k, which is either active or inactive.The probability distributions p(n ± ) are then replaced by multivariate distributions p(n − ) and p(M + ), which depend on the shape of the particle and the distribution of reaction sites, which are so far arbitrary.
To obtain AOUP statistics in higher dimensions, we can consider a spherical particle with a dense, uniform distribution of N total reaction sites in the continuum limit.In this case, n − is a multivariate Gaussian distribution with mean ⟨n − ⟩ = µ n − = 0 and covariance matrix where i and j are Cartesian coordinates.Since n − is the sum of identically distributed and independent random variables n k , it follows from the central limit theorem that the probability distribution for n − is Gaussian, if we sum over sufficiently many active reaction sites.Spherical symmetry implies that its mean is 0 and that there are no cross correlations.The same argument applies to p(M + ), which is now a matrix normal distribution with the same among-row and among-column covariance matrix as in (41).However, since the product n k ⊗ n k has strictly positive values, the mean of p(M + ) is not 0, but rather π −1 nA 1.All the relations we have derived for the one-dimensional case remain valid upon replacing n − with n − and n + with M + .The total entropy production, (30), thus reads In particular, the corresponding Langevin equations read ẋ(t) = − D eff (t)∇V (x(t)) + u 0 n − (t) + 2D tr ζ tr (t) + ξ ac (t) , (43a)

Conclusion
We have presented a microscopic Markovian model for a particle which leads to AOUP statistics in the continuum limit.For our model, the well-established tools of stochastic thermodynamics can easily be applied, which let us calculate the mean total entropy production in the steady state.From a Fokker-Planck equation, we have then constructed the corresponding Langevin equation, which additionally features a fluctuating diffusivity.We have then applied concepts developed for measuring the departure from equilibrium for AOUP dynamics and found that the physical entropy production in our microscopic model is always larger than pseudo-entropy productions inferred from Langevin trajectories alone.In addition, we do not have to deal with the intricacies of an ambiguous time-reversal.Even though we have treated only one active particle, it is straightforward to extend our approach to many AOUPs and to AOUPs interacting with passive particles following the lines of [40], where this has been done for ABPs.The resulting thermodynamically consistent model for an active bath of AOUPs can then be used to study efficiency and alike in models of such generalized active heat engines.Furthermore, our analysis has revealed one possible mechanism leading to AOUP-like motion, which is a uniform distribution of many propulsion sites and a low activation probability of each site.The construction of thermodynamically consistent models underlying the effective dynamics of active particles can thus yield insight and serve as a benchmark for effective descriptions.These models bridge the gap between understanding the underlying, thermodynamically relevant, but inaccessible processes on the one hand and the observable degrees of freedom and effective measures for the departure from equilibrium on the other.

Figure 1 .
Figure1.Illustration of the active driving mechanism.(a) A single reaction site is either inactive (dotted and gray) or active (red).In the active state, the reaction site catalyzes some fuel-consuming chemical reaction with rate k + , which releases chemical free energy ∆µ.Thermodynamic consistency requires that the reverse reaction with rate k − is also possible.The blue circles symbolize the reactants.Switching between the active and inactive state occurs with rates ω on,off .(b) The particle with internal degree of freedom (n l , n r ) at position x i on the one dimensional lattice.On the left side of the particle, n l = 2 sites are active, and on the right one n r = 1 site is active.Each active site on the left pushes the particle to the right if the fuel-consuming reaction with rate k + l (x i ) takes place and pulls it to the left if the fuel-generating reaction with rate k − l (x i−1 ) occurs.Reactions on the right side of the particle have the opposite effect, pushing it to the left with rate k + r (x i−1 ) and pulling it to the right with rate k − r (x i ).In addition, the particle can diffuse thermally, indicated by the dashed arrows between the states.