Role of mutual information in entropy production under information exchanges

We relate the information exchange between two stochastic systems to the nonequilibrium entropy production in the whole system. By deriving a general formula that decomposes the total entropy production into the thermodynamic and informational parts, we obtain nonequilibrium equalities such as the fluctuation theorem in the presence of information processing. Our results apply not only to situations under measurement and feedback control but also to those under multiple information exchanges between two systems, giving the fundamental energy cost for information processing and elucidating the thermodynamic and informational roles of a memory in information processing. We point out a dual relationship between measurement and feedback.


Introduction
Thermodynamics of information processing has seen a resurgence of interest recently.From a theoretical point of view, the advances in nonequilibrium statistical mechanics over the last two decades have opened up a new avenue of research to generally and quantitatively investigate the relationship between nonequilibrium thermodynamics and information theory , shedding new light on the longstanding problem concerning Maxwell's demon [56][57][58][59][60][61].From an experimental point of view, developments in experimental techniques have led to the realization of Maxwell's demon with small thermodynamic systems [62][63][64].
Furthermore, the nonequilibrium equalities such as the fluctuation theorem (FT) [65][66][67][68][69][70][71][72][73][74][75][76][77] have been generalized to the case under information processing.For example, we have derived a generalized FT in the presence of an information exchange [44].However, a fundamental question remains elusive: What is the relationship between the exchanged information inside the universe and the total entropy production in the universe?Here, the "universe" means the relevant entire system including heat baths.
In the present paper, we address this question by focusing on the role of the mutual information in the total entropy production in the whole system.By deriving a decomposition formula of the total entropy production into the thermodynamic and informational parts, we investigate FT and the second law of thermodynamics (SL) in the presence of information processing.In particular, we examine SL under multiple information exchange.We also point out that there exists a certain duality between measurement and feedback, which relates the entropic cost for measurement to that for feedback.Moreover, we study the detailed structure of a memory that stores information, and obtain a general formula that determines the fundamental energy cost needed for measurement and feedback control.
This paper is organized as follows.In Sec. 2, we consider the case of a single information exchange, and derive a general formula of the decomposition of the entropy production.In Sec. 3, we consider the case of multiple information exchanges, and apply the obtained general result to the composite process of measurement and feedback control; this process includes a typical setup of Maxwell's demon.In Sec. 4, we analyze the entropic and informational roles played by the memory, which enables us to derive the minimal energy cost needed for measurement.In Sec. 5, we conclude this paper.In Appendix A, we discuss the entropy production in the heat bath, and clarify the physical meaning of the total entropy production along the line with the standard nonequilibrium statistical mechanics.

Single information exchange
In this section, we consider the case of a single information exchange.In Sec.2.1, we briefly review as much of information theory as is needed for later discussions.In Sec.2.2, we derive a general formula of the decomposition of the entropy production under information processing.In Sec.2.3 and 2.4, we apply the general formula to situations under feedback and measurement, respectively.In Sec.2.5, we discuss a duality between measurement and feedback.

Information contents
We first review the Shannon entropy (or information) and the mutual information [79,80], which play key roles in following discussions.
Let x be a probability variable with probability distribution P [x].The stochastic Shannon entropy is defined by which characterizes how rare the occurrence of an outcome x is; the rarer it is, the greater s[x] becomes.The average of s[x] over the probability distribution P [x] gives the Shannon entropy If x is a continuous variable, the sum in Eq. ( 2) is replaced by the integral.Let x and y be two probability variables with joint probability distribution P [x, y].The marginal distributions are given by P [x] := y P [x, y] and P [y] := x P [x, y].The stochastic mutual information is defined by The ensemble average of I[x, y] gives the mutual information: The mutual information characterizes the correlation between the two probability variables.We also note the relation where The Shannon entropy of x and the mutual information between x and y satisfy the following inequalities: where the left equality is achieved if and only if the two variables are not correlated, or equivalently statistically independent (i.e., P [x, y] = P [x]P [y]); the right equality is achieved if and only if, for any y, there exists a unique x such that P [x, y] = 0.A parallel argument holds true if we replace s x by s y in Eq. ( 7).

Decomposition formula
We consider stochastic dynamics of two systems X and Y in the presence of information exchange between them.We assume that X is attached to heat baths with inverse temperatures We denote the baths collectively as B. System X then evolves under the influence of system Y , where we assume that the phase-space point of Y at a particular time, denoted as y, only affects the dynamics of X (see also Fig. 1).We note that the present situation is the same as the one in our previous paper [44], but we here adopt a different approach to deriving FT and SL. .Time evolution of X under the influence of Y .System X evolves from x to x ′ along trajectory X F , where the phase-space point of Y at a particular time, denoted as y, only affects the dynamics of X .There may be initial and final correlations between X and Y which are characterized by mutual information contents I i XY and I f XY .
Let x and x ′ be the initial and final phase-space points of X, and y be the phasespace point of Y .Let P i F [x, y] and P f F [x ′ , y] be the initial and final joint probability distributions of the composite system XY .Here the subscript "F " indicates the "forward process."We define We note that the marginal distribution of y does not change in time.We assume that there may, in general, be the initial and final correlations between X and Y , i.e., . We consider the difference between the Shannon entropy of (x, y) and that of (x ′ , y), which is given by ∆s XY := (− ln It can be rewritten as where ∆s X := (− ln ∆I XY := I f XY − I i XY := ln Since ∆s Y = 0, we obtain ∆s XY = ∆s X − ∆I XY .
In the following, we denote the initial and final Shannon entropies of X as Let Q X,k be the heat absorbed by system X from the kth bath.Following to the standard nonequilibrium thermodynamics [68,69,73], the entropy production in the total system (X, Y , and B) during the present dynamics is given by where is the entropy production in B (see Appendix A for details).We then obtain the decomposition of the total entropy production as follows: where is the entropy increase in XB.We examine the above result in terms of DFT.Let X F be the trajectory of X in the forward process.The joint probability distribution of X F and y is given by where P F [X F |x, y] is the conditional probability of X F under the initial condition (x, y), where the dependence on y reflects the effect of information exchange.We write the ensemble average of an arbitrary quantity A[X F , y] as To formulate DFT, we need to introduce the concept of backward processes, where the time dependence of external parameters such as the magnetic field is time-reversed.The backward probability distribution is given by where P B [X B |x, ỹ] is the conditional probability of X B under the initial condition (x, ỹ).Let x * and y * be the time-reversal of the phase-space points x and y, respectively.For example, if x = (r, p) with position r and momentum p, then x * = (r, −p).For X F = {x(t)} 0≤t≤τ , we define its time-reversal as X † F := {x † (t)} 0≤t≤τ := {x * (τ − t)} 0≤t≤τ .In a broad class of nonequilibrium dynamics, the entropy production in B satisfies [67][68][69] ∆s B = ln where the left-hand side (lhs) is the entropy production in B in the forward process, and the right-hand side (rhs) is the ratio of the probability distributions of the forward and backward trajectories.We then assume that the initial distribution of the backward processes is given by the time-reversal of the final distribution of the forward process: which leads to DFT for the total system: We then have By noting that we reproduce Eq. ( 17).
In the present setup, the Kawai-Parrondo-van den Broeck (KPB) equality [74] is given by where the rhs is the relative entropy between the forward and backward trajectories.
From the positivity of the relative entropy [80], we obtain SL for the total process: which is equivalent to Inequality (29) implies that the lower bound of the entropy increase in XB is given by the change in the mutual information between X and Y .Let S be the set of (x, y) such that P i F [x, y] = 0. We then have where we used dX F = dX † F and dy = dy * .If S is the whole phase space, we obtain the integral fluctuation theorem (IFT) or the Jarzynski equality: which is equivalent to The crucial assumption here is that the dynamics of X is affected only by the phase-space point y at a particular time.Therefore, Y does not necessarily stay at y as X evolves, as long as the evolution of Y does not affect the dynamics of X.Therefore, the probability distribution of X F is characterized by P F [X F |x, y] that is not affected by the time evolution of Y .
Although we have obtained the same results as ( 27), ( 29), and IFT (32) in a previous paper [44], we stress that in this paper we have adopted a new approach to deriving them on the basis of the decomposition formula (17).The present approach gives a new insight compared with the previous one, in that it enables us to understand the generalized FT and SL as a result of the decomposition of the total entropy production.We note that a decomposition formula similar to Eq. ( 17) has been discussed in Ref. [53] for special cases.
In the absence of information exchange, P F [X F |x, y] is independent of y so that P F [X F |x].In this case, ∆s XB satisfies the conventional DFT and therefore its expectation value is nonnegative: We also have which is a special case of the data processing inequality [80].Therefore, in the absence of information processing, we obtain In other words, inequality (33) is stronger than inequality (29) in this case; ∆s XB cannot be negative due to inequality (33), even when inequality (29) gives a negative lower bound.Therefore, in the absence of information exchange, it is consistent to regard XB as the whole "universe" even when there are initial and final correlations with Y ; we can ignore what's happening outside XB if there is no interaction between inside and outside of the universe.

Feedback control
We apply the foregoing general framework to feedback control, where X is the system to be controlled and Y is the memory that initially has the information about the initial condition of the system and controls it depending on that information (see also Fig. 2 (a)).The mutual information that is initially shared between the system and the memory is given by I := I i XY , and the final remaining correlation is given by I rem := I f XY .The decomposition (17) of the total entropy production is then given by which, together with inequality (28), leads to Inequality (37) implies that the entropy in XB can be decreased by the amount up to I −I rem that characterizes the upper bound of the utilized information during feedback control.(a) Dynamics of feedback control, where X is the system to be controlled and Y is the memory.(b) Dynamics of measurement, where X is the memory and Y is the measured system.These schematics illustrate the dual relationship between measurement and feedback control; they have a one-to-one correspondence under timereversal and exchange of the roles of the system and the memory.
We next consider the energetics of feedback control.Let E i X [x] and E f X,y [x ′ ] be the initial and final Hamiltonians of system X.Here, we assume that the initial Hamiltonian is independent of y, and that the final one can depend on y through feedback control.The intermediate Hamiltonians during the feedback process can also depend on y.For simplicity, we neglect the interaction Hamiltonian between X and Y in the initial and final states.The energy change in this process is given by The first law of thermodynamics is given by where W X is the work performed on X through the time dependence of external parameters.
We now assume that there is a single heat bath at inverse temperature β.Inequality (37) then reduces to where ∆F eff is the change in the effective (nonequilibrium) free energy defined by We next define the initial and final equilibrium free energies as follows: We further assume that the initial distribution of X is the thermal equilibrium: We then obtain On the other hand, the final distribution can be different from the canonical distribution in general.Let sf X,y [x ′ ] := − ln P f F [x ′ |y] be the conditional Shannon entropy of the final distribution.We then have an inequality: where the equality is achieved if and only if P f X [x ′ |y] is the conditional canonical distribution for a given y: We note that We finally obtain where is the average change in the conditional free energy.Inequality (48) sets the fundamental lower bound of the energy cost for feedback control, which is smaller by the amount of β −1 I than the usual thermodynamic bound.We note that the same bound as (48) has been obtained in Refs.[6,10] for a different setup.

Measurement
We next apply our general framework to measurement processes, where X is the memory and Y is the measured system (see also Fig. 2 (b)).In other words, X performs a measurement on Y in this setup.We first assume that the initial correlation is zero (i.e., I i XY = 0) before the measurement, and the final correlation is characterized by the information (I := I f XY ) obtained by the measurement.The total entropy production is given by which, together with inequality (28), leads to Inequality (51) implies that the entropy in XB inevitably increases due to the obtained information by the measurement.
If the memory has prior knowledge about the system before the measurement, there is the corresponding initial correlation I ini := I i XY .We then obtain ∆s XY B = ∆s XB − (I − I ini ), (52) which, together with inequality (28), leads to Inequality (53) implies that the entropy increase in XB is bounded from below by the obtained information I − I ini .
To discuss the energetics of the memory, we need to examine the more detailed structure of the memory, which will be discussed in Sec. 4.

Duality between measurement and feedback control
We now discuss a fundamental relationship between measurement and feedback control.Let us consider the time-reversal transformation of the dynamics and exchange the roles of the system and the memory at the same time (see also Fig. 2).We then find that the measurement becomes feedback and vice versa, where I in measurement corresponds to I in feedback, and I ini in measurement corresponds to I rem in feedback.This implies a kind of dual structure between the measurement and feedback, as summarized in Table 1.
We consider a special case of I ini = I rem = 0.In this case, the lower bound of ∆s XB is given by I for measurement and by − I for feedback, where the opposite signs are due to the fact that the final correlation in measurement corresponds to

Multiple information exchanges
We generally consider the case of multiple information exchanges in Sec.3.1, and then focus on the case of Maxwell's demon in Sec.3.2.

General framework
We consider multiple information exchanges between two systems X and Y , which are attached to different heat baths with each other.For simplicity, we use notation B to indicate all baths.If the correlation time in the baths is sufficiently small compared with the time scale of the systems, we may apply this assumption to the situation in which the systems are attached to the same baths.We consider a composite process consisting of the following two processes (see also Fig. 3 (a)).In the first process (i), Y evolves under the influence of the initial phase-space point of X, denoted as x.Let P 0 F [x, y] be the initial distribution of the first process.System Y evolves along trajectory Y F with probability P F [Y F |x, y] under the initial condition of (x, y).The final distribution of Y is given by P , where y ′ is the final phase-space point of Y .Let ∆s In the second process (ii), X evolves under the influence of the final phase-space point of Y , denoted as y ′ (see Fig. 3 (a)).Let P 1 F [x, y ′ ] be the initial distribution of the second process.System X evolves along trajectory X F with probability P F [X F |x, y ′ ] under the condition of (x, y ′ ).The final distribution of X is given by P 2 F [x ′ , y ′ ], where x ′ is the final phase-space point of X.Let ∆s  Y evolves under the influence of the initial phase-space point of X, denoted by x.In the second process (ii), X evolves under the influence of the final phase-space point of Y , denoted by y ′ .(b) Typical situation of Maxwell's demon.X is the system to be controlled and Y is the memory of the demon, where the first process describes the measurement with outcome y ′ and the second process describes the feedback control.
The total entropy production in the composite process, denoted by ∆s tot XY B , is given by the sum of the entropy productions of the two processes: The change in the mutual information in the total process is given by ∆I tot XY = ln which can also be expressed as the sum of the changes in the two processes: In terms of DFT, the entropy productions are given by ∆s (i) Y B = ln XY B = ln , ∆s and ∆s tot XY B = ln Here, we have assumed that the initial distributions of the two backward processes are given by P 1 F [x * , y ′ * ] and We note that the initial distribution of the backward process of (i) is not necessarily equal to the final distribution of the backward process of (ii).In other words, the first backward process is not necessarily followed by the second backward process; one cannot start the backward process of (i) immediately after the backward process of (ii), but one should change the probability distribution to start the backward process of (ii).On the other hand, the initial distribution of the forward process (i) is equal to the final distribution of the forward process (ii).Therefore, the forward process (i) is actually followed by the forward process (ii), and one can start the forward process (ii) immediately after the forward process (i).
Since the total entropy production is nonnegative, we obtain and therefore ∆s Inequality (63) implies that the sum of the entropy increases is bounded by the total change in the mutual information.We note that the foregoing argument can straightforwardly be generalized to the case of information exchanges which take place more than once.

Maxwell's demon
We next consider the composite process of measurement and feedback, which is a typical situation of Maxwell's demon (see also Fig. 3 (b)).In this case, X is the system to be controlled and Y is the memory of the demon.We assume that there is no initial correlation: I 0 XY = 0.After the measurement, the memory obtains the mutual information I XY := I 1 XY and then uses it for feedback control.The remaining correlation after feedback control is given by I rem XY := I 2 XY .By applying Eq. ( 56) to this case, the total entropy production of the composite process is given by Therefore, we obtain Since I rem XY is non-negative, we obtain ∆s feed XB + ∆s meas This inequality implies that the entropy decrease in XB by feedback control is compensated for by the entropy increase in Y B by measurement.We note that, the total entropy productions ∆s meas XY B and ∆s feed XY B are both nonnegative during measurement and feedback, which confirms that the role of the demon does not contradict SL.The crucial observation here is that the mutual information I XY which is stored during the measurement is used as a resource of the entropy decrease during the feedback process.

Memory structure
We next discuss the detailed structure of the memory, and its roles in measurement and feedback control.

Setup and decomposition of entropy
We consider a situation in which the phase space of the memory, which we refer to as Y, is divided into several subspaces (see also Fig. 4).Each subspace is written as Y m labeled by m (= 1, 2, • • •), where M := {m} may be regarded as the set of measurement outcomes.We assume that Y m 's do not overlap with each other, and m Y m = Y.For any y ∈ Y, there is a single m such that y ∈ Y m , which we write as m y .
We consider probability distribution P [y] over Y. Let p[m] be the probability of y ∈ Y m , and P [y|m] be the conditional probability of y under the condition of y ∈ Y m .We note that P [y|m] = 0 if m = m y , because Y m 's do not overlap with each other.The joint probability distribution is given by where δ(•, •) is the Kronecker delta.The unconditional probability distribution is then given by We define the stochastic Shannon entropies as which satisfy Therefore, we obtain where Equality (73) implies that the total Shannon entropy is decomposed into the Shannon entropy over m and the average Shannon entropy of the phase-space points in Y m , where the former characterizes the randomness of the measurement outcomes, while the latter characterizes the average of the fluctuations within individual subspaces.

Measurement
We now consider measurement processes with the memory structure in the presence of heat baths B. Let us choose a subspace Y 0 which may be one of Y m 's, but not necessarily be so.In fact, Y 0 may be equal to the whole phase space Y.We assume that the initial phase-space point y is in Y 0 with unit probability; in this case, we say that the memory is in the standard state.Let P i F [y] be the initial distribution of y; by assumption, P i F [y] = 0 if y does not belong to Y 0 .We also assume that there is no initial correlation between X and Y .
The memory then evolves along trajectory Y F under the influence of X with phase-space point x, and stores outcome m with probability p F [m].This measurement establishes the correlation between x and m.After the measurement, the final phasespace point is y ′ .We note that the probability that y ′ is in subspace Y m is given by p be the final probability distribution of y ′ under the condition of m.The total entropy production during the measurement is then given by where In the following, we write s i Y := − ln P i F [y] and s f Y,m := − ln P f F [y ′ |m].We next assume that there is a single heat bath at inverse temperature β.Let E i Y,0 [y] be the initial Hamiltonian defined on subspace Y 0 .We assume that the initial distribution is given by the canonical distribution in Y 0 : where the conditional free energy is given by In this case, Let E f Y,m [y ′ ] be the final Hamiltonian defined only on Y m .We define the conditional free energy as We refer to the memory as symmetric if F f Y,m takes on the same value for all m (see also Fig. 4).We then have where the equality is achieved if and only if which vanishes outside of Y m .We then have where ∆F meas Therefore, we have where W meas Y is the work performed on the memory during the measurement.Since ∆s meas XY B ≥ 0, we finally obtain which determines the minimal energy cost for measurement.The lower bound is characterized by the average free-energy difference, the Shannon information of measurement outcomes, and the mutual information between X and Y .On the rhs of inequality (92), −β −1 h M arises from the increase in the Shannon entropy of the memory by the measurement, and β −1 I XY arises from the increase of the mutual information between the system and the memory by the measurement.The reason why the signs of −β −1 h M and β −1 I XY are different from each other is that the Shannon information and the mutual information contribute to the total entropy with opposite signs as shown in Eq. ( 5).We note that the actually utilizable information obtained by the memory is characterized by the mutual information between X and outcome M: where P f F [x, m] is the joint probability distribution of x and m after the measurement.We then have where P f F [x|y] and P f F [x|m] are the conditional probabilities of x under the condition of y and m, respectively.The ensemble average ĨXY is the conditional mutual information between X and Y under the condition of m, which is by construction nonnegative [see Eq. (94)]: Therefore, we obtain an inequality which is weaker than (92): Inequality ( 96) is physically more transparent than inequality (92), because the lower bound in (96) is characterized by the physically utilizable information I XM rather than the total correlation I XY .We note that the same bound as (96) has been derived in Ref. [38] for a different setup.
As an illustration, we consider a simple model of measurement.Figure 5 (a) shows a model of error-free measurement.The memory is a single particle in a box with a single heat bath at inverse temperature β −1 , where Y 0 is the whole phase space.We assume that the measured state is x = L or R with equal probability 1/2.After the quasi-static and isothermal measurement described in Fig. 5  Figure 5 (b) shows a model of measurement with error rate ε (0 ≤ ε ≤ 1), where Y 0 , Y L , and Y R are the same as in the previous example.We assume that the measured state is x = L or R with the equal probability of 1/2.In this case, ∆F meas Therefore, the equality in (96) is again achieved in this model.We now briefly discuss the information erasure from the memory.During the erasure, memory Y is detached from the measured system X, and Y returns to the standard state; after the erasure, the phase-space point of Y is in Y 0 with unit probability.The Shannon entropy in M after the erasure is 0 by definition; it changes by − h M during the erasure, whose sign is opposite to that in the measurement.Since Y is detached from X during the erasure, DFT and SL can apply to Y B (see also arguments in the last paragraph of Sec.2.2).Therefore, the entropy change in Y B during the erasure satisfies where the equality can be achieved in the quasi-static erasure.We assume that there is a single heat bath at inverse temperature β, and that the probability distribution of Y in each Y m before the erasure is the canonical distribution under the condition of m.By applying a similar argument used in deriving (92) to ∆s eras Y B , we obtain the lower bound of the work performed on the memory during the erasure: which is the generalized Landauer principle [36,38].We note that the free-energy change ∆F eras Y during the erasure satisfies ∆F eras In the special case of ∆F erase Y = 0, inequality (100) reduces to the conventional Landauer principle [31,60,64], which is satisfied in the case of a symmetric memory as shown in Fig. 4 (a).
By summing up inequalities (92) and (100), the total work for measurement erasure is given by where the lower bound is only determined by the mutual information; ∆F meas Y and −β −1 h M on the rhs of inequality (92) are canceled by the corresponding terms in inequality (100).In fact, the measurement and erasure are time-reversal with each other if we only focus on Y B and ignore the interaction with X.However, they are not completely time-reversal if we take into consideration their interaction; Y interacts with X and establishes the correlation only in the measurement process.Therefore, the mutual information obtained by the measurement process plays an essential role in determining the work for the entire process of measurement and erasure.
We note that the assumption of the conditional canonical distribution before the erasure is not necessary to derive only inequality (101); we only need to assume that the probability distribution before the erasure is the same as that after the measurement.In fact, by summing up the entropy changes in measurement and erasure, we obtain ∆s meas

Feedback control
We next consider feedback control on X by Y after the measurement.More precisely, we assume that the dynamics of X is determined only by the outcome m.Therefore, we can consider a composite system XM instead of XY .We assume that system X is attached to heat baths that are different from those in contact with the memory.We denote the baths attached to X again by B.
The probability distribution of the forward trajectory of X and m is given by where P f F [x, m] is the pre-feedback (post-measurement) distribution of (x, m), and P F [X F |x, m] is the conditional probability of X F under the initial condition (x, m) of the feedback process.
The argument is then completely parallel to that in Sec.2.3 if we replace Y with M. The total entropy production in XMB is given by ∆s feed XM B = ∆s feed XB + (I XM − I rem XM ), where I rem XM describes the remaining correlation after the feedback control.SL is then expressed as If there is a single heat bath at inverse temperature β and the initial state of system X is in the canonical distribution, we obtain On the other hand, by considering XY B, we can also obtain We note that inequality (105) is stronger than inequality (106) in the present setup.

Conclusion
We have established the general relationship between the total entropy production of the whole system and the mutual information that is exchanged between two stochastic systems.
In Sec. 2, we have derived the general decomposition formula (17) for a single information exchange.Correspondingly, we have obtained the KPB equality (27), SL (29), and IFT (32), such that they explicitly include the mutual information.We have applied the general formula to the cases of feedback control (36) and measurement (53).In Sec. 3, we have discussed the case of multiple information exchanges, and obtained a general decomposition formula (56) and the corresponding SL (63).In Sec. 4, we have considered the structure of the memory; its phase space is divided into several subspaces corresponding to the measurement outcomes.This formulation has clarified the role of the Shannon information of measurement outcomes as well as the mutual information, as shown for the cases of measurement (96) and feedback control (105).
Our theory has clarified the role of mutual information in nonequilibrium thermodynamics with information processing, which is not restricted to the conventional case of Maxwell's demon.As a consequence, we have revealed the fundamental relationship between the entropy production in the whole universe (system and bath) and the exchanged information inside the universe.Our results would serve as the theoretical foundation of nonequilibrium thermodynamics of complex systems in the presence of information processing.

Figure 1
Figure1.Time evolution of X under the influence of Y .System X evolves from x to x ′ along trajectory X F , where the phase-space point of Y at a particular time, denoted as y, only affects the dynamics of X .There may be initial and final correlations between X and Y which are characterized by mutual information contents I i XY and I f XY .
Figure2.(a) Dynamics of feedback control, where X is the system to be controlled and Y is the memory.(b) Dynamics of measurement, where X is the memory and Y is the measured system.These schematics illustrate the dual relationship between measurement and feedback control; they have a one-to-one correspondence under timereversal and exchange of the roles of the system and the memory.
be the entropy productions in XY B and Y B in this process.The change in the mutual information is given by ∆I (i) (ii) XY B and ∆s(ii)XB be the entropy productions in XY B and XB in this process.The change in the mutual information is given by ∆I (ii)

Figure 3 .
Figure 3. (a) Dynamics of the two-step composite process.In the first process (i), Y evolves under the influence of the initial phase-space point of X, denoted by x.In the second process (ii), X evolves under the influence of the final phase-space point of Y , denoted by y ′ .(b) Typical situation of Maxwell's demon.X is the system to be controlled and Y is the memory of the demon, where the first process describes the measurement with outcome y ′ and the second process describes the feedback control.

Figure 4 .
Figure 4. Schematic of the double-well memory structure with m = L, R. (a) Symmetric memory withF f Y,L = F f Y,R .(b) Asymmetric memory F f Y,L = F f Y,R .
(a), the particle is in the left box or the right box corresponding to m = L or R, where Y L and Y R correspond to the left and right box, respectively.We note that x = m in this model.In this case, ∆F meas Y = β −1 ln 2, W meas Y = β −1 ln 2, h M = ln 2, and I XM = ln 2. Therefore, the equality in inequality (96) is achieved in this model.

Figure 5 .
Figure 5. Simple models of measurement.(a) Error-free measurement.The memory is initially in the standard state, which is the global equilibrium in the box.If the measured state is x = L (x = R), the box is compressed from the right (left) quasi-statically and isothermally with the particle confined in the left (right) box corresponding to m = L (m = R).In the final state, x and m are perfectly correlated.(b) Measurement with error rate ε.The standard state is the same as in (a).If the measured state is x = L (x = R), a barrier is inserted and the box is divided into two compartments with volume ratio 1 − ε : ε (ε : 1 − ε).The barrier is moved to the center of the box.The particle is finally in the left (right) box corresponding to m = L (m = R), where x and m are not perfectly correlated if 0 < ε < 1.If ε = 0, this model is equivalent to the error-free model of (a).
eras Y B ≥ I XY .By applying a similar argument used in deriving (92) to the entire entropy change ∆s meas Y B + ∆s eras Y B in measurement and erasure, we again obtain inequality (101).

Table 1 .
Duality between measurement and feedback.I − I ini ∆s XB ≥ I rem − I the initial correlation in feedback because of the time-reversal transformation.This explains the reason why the entropy in XB is increased by measurement but decreased by feedback control.