Probing the extensive nature of entropy

We have devised a general numerical scheme applied to a system of independent, distinguishable, non-interacting particles, to demonstrate in a direct manner the extensive nature of statistical entropy. Working within the microcanonical ensemble, our methods enable one to directly monitor the approach to the thermodynamic limit (N → ∞) in a manner that has not been known before. We show that (sN − s∞) → N−α where sN is the entropy per particle for N particles and S∞ is the entropy per particle in the thermodynamic limit. We demonstrate universal behaviour by considering a number of different systems each defined by its unique single-particle spectrum. Various thermodynamic quantities as a function of N may be computed using our methods; in this paper, we focus on the entropy, the chemical potential and the temperature. Our results are applicable to systems of finite size, e.g. nano-particle systems. Furthermore, we demonstrate a new phenomenon, referred to as entropic interference, which manifests as a cancellation of terms in the thermodynamic limit and which results in the additive nature of entropy.


Introduction
In 1871, Ludwig Boltzmann wrote down his celebrated equation (which is inscribed on his tombstone) relating statistical entropy S of an isolated atomistic system to the number of accessible system microstates W subject to macroscopic constraints, where k B is the Boltzmann constant. This is one of the most fundamental equations in physics. Given the importance of the second law of thermodynamics [1] in governing the flow of entropy in physical and chemical processes [2], including in biological systems and ultimately in processes that shape life and the evolution of the universe, there is little question about the significance of Boltzmann's work in endeavours that go well beyond physics. Equation (1) connects the microscopic details of a system with its macroscopic properties in a simple and elegant way. This equation, which forms the basis of the statistical interpretation of thermodynamics [3], has been the subject of rigourous verification over the past more than 100 years, but was initially met with strong resistance by so-called energeticists including some of the most famous physicists of that time who felt that energy conservation alone was necessary to describe spontaneous processes and the arrow of time. This fierce opposition ultimately led to the depressive Boltzmann taking his own life in 1906 [4].
In our paper we make observations concerning (1) that have important implications for physics.
We demonstrate in a direct manner the extensive nature of statistical entropy. A property of any system is extensive if, in the thermodynamic limit (infinite system size limit), the property depends on the size of the system. We are able to do this now in a manner that was not possible 100 years ago because of our access to computation. Boltzmann only concluded that statistical entropy is extensive by first equating statistical temperature with thermodynamic temperature, and by then inferring the equivalence of statistical entropy with thermodynamic entropy. The extensive nature of thermodynamic entropy was already well established in the early part of the 19 th century.
An important step for us was the formulation of an efficient computational algorithm that enables us to calculate statistical entropy in the microcanonical ensemble as a function of the number of particles N in a recursive manner. Our algorithm is quick and, despite the exponential increase of W as a function of N , we are able to derive converged results in real time on a modern desktop computer with no overflow errors that have dogged previous attempts.
We also demonstrate the manner in which entropy approaches the thermodynamic limit. To the best of our knowledge, this has not been done before. We demonstrate universality by considering a number of different systems each defined by its unique single-particle spectrum. Despite the vast differences in the microscopic details of the single particle energy states for differing systems, we show that (s N − s ∞ ) ∼ N −α , where s N is the entropy per particle for N particles, s ∞ is the entropy per particle in the thermodynamic limit (i.e. in the infinite system size limit where N → ∞), and α is the exponent. This slow N −α powerlaw behaviour of statistical entropy, as it converges in the thermodynamic limit, is an important subject of this paper.
Our results have important implications for finite system sizes. Using our results, we are able to derive in a consistent manner, various thermodynamic properties in the microcanonical ensemble for finite systems of size N , such as temperature, pressure, chemical potential, magnetism, the heat capacities, the various thermodynamic potentials, etc. The corollary is that our work enables finite size corrections for systems that are large but not so large that the thermodynamic limit has been reached. Finite systems are more the rule than the exception for typical systems that are measured in the laboratory, and so finite size corrections would usually be very necessary. Our work is of particular relevance to nano-particle physical, chemical and biological systems, and is thus applicable in the quantitative study of such systems at finite temperature.
In addition, we demonstrate entropic interference, which manifests as a cancellation of terms in the thermodynamic limit and which results in the additive nature of entropy.

Extensive behaviour
We consider an isolated system comprising N independent, distinguishable, non-interacting particles with total energy E. The number of microstates for such a system within the microcanonical ensemble can be calculated recursively by where w(E ′ ) is the degeneracy of the single-particle state with energy E ′ . The entropy for such a system may then be determined recursively by A derivation of (3) from (2) is given in [5]. These equations may be used to calculate statistical entropy recursively at energy E systematically for N = 2, 3, 4, . . . , N max , where N max can be made sufficiently large. In so-doing we are able to analyse the approach to the thermodynamic limit in a manner that has not been possible before. Equation (3) is an extremely efficient means to calculate entropy on a modern desktop computer. We have applied and successfully tested and verified (3) for a range of analytically solvable models including a system of harmonic oscillators; free particles in one, two and three dimensions; spin 1/2 and spin 1 particles, etc., and for a range of analytically intractable problems including a host of hypothetical models, each defined by its unique single-particle spectrum (for example: ε = p 2 ; ε = p 3 ; ε = p 4 ; ε = p + q; ε = p + q 2 , etc., where p, q = 0, 1, 2, 3, . . .). For the purposes of this paper and to demonstrate our main ideas, we shall primarily focus only on two models, namely the well-known analytically solvable system of harmonic oscillators with single-particle spectrum (in appropriate units) given by ε(p) = p where p = 0, 1, 2, 3, . . . (Model 1); and the analytically insolvable model system with singleparticle spectrum (in appropriate units) given by ε(p, q) = p 2 + q 4 where p, q = 0, 1, 2, 3, . . . (Model 2). The second model is sufficiently complex (two-parameter quantum model with unequally-spaced energy levels and with degenerate states) that it may be considered to be representative of the general class of discrete-level problems, and therefore a resolution of this problem is a measure of the power of our methods.
In figure 1 we plot for Model 1 the entropy per particle s ∞ (in units of k B ) versus the energy per particle e in the thermodynamic limit. We actually present three plots which, to within numerical error, coincide with each other, namely (i) s ∞ (e) = (1 + e) log(1 + e) − e log e, (ii) s ∞ (e) calculated in the canonical ensemble, and (iii) s 3000 (e) calculated using (3). From (iii) we conclude that N = 3000 is a sufficiently large number of particles to estimate, for our purposes, the thermodynamic limit. The plot involving (i) is possible only because this model is solvable in the thermodynamic limit using the Stirling approximation. This is not the case for the second model so the equivalence of plot (i) for Model 2 does not exist. For Model 2 we plot in figure 1 (iv) s ∞ (e) calculated in the canonical ensemble, and (v) s 3000 (e) calculated using (3) where once again, to within numerical error, these two plots coincide.
This demonstrates the accuracy and the efficiency with which (3) may be used to calculate the entropy of any system within the microcanonical ensemble. We have shown here that independent of the single particle spectrum of Models 1 and 2, s N (e) → constant real number, designated as s ∞ (e), as N → ∞. Similarly (3) may be used to show that this is true for any single particle spectrum. This establishes in a general manner the extensive nature of statistical  Figure 2. The deviation of the entropy per particle from its value in the thermodynamic limit (s N − s ∞ ) versus the number of particles N for Model 1 for energy per particle e = 10.

Powerlaw behaviour
In the course of our numerical work we made the fortuitous discovery that the approach of (3) to the thermodynamic limit is slow. For example, a ten-fold increase in the number of particles from N = 100 to 1000 for Model 1 at e = 10 results in a reduction in the deviation of the entropy per particle from its value in the thermodynamic limit by less than 1.5%. What proved to be frustrating for us initially, on closer inspection turned out to be revealing of the powerlaw behaviour of entropy as we approached the thermodynamic limit. Using (3), we are able to monitor the approach of the entropy per particle to the thermodynamic limit. In figure 2, we plot (s N − s ∞ ) versus N for Model 1 for e = 10. The curve is negative which indicates that the thermodynamic limit is approached from below. The asymptotic behaviour is observed to be slow (powerlaw rather than exponential) and may be analysed more carefully by plotting log |s N − s ∞ | versus log N which we do in figure 3. The behaviour is linear which confirms powerlaw behaviour, with a negative slope of α = 0.92623 ± 0.00003.
We observe similar behaviour for log |s N − s ∞ | versus log N for Model 1 separately for each value of e. We are therefore able to trace the curve α(e), which we present in α(e) exhibits only a very weak dependence on e and a very weak dependence on the microscopic details of the system single particle levels. We discovered the same behaviour for Model 2 and for other models studied in the general class of similar problems each defined by its unique single-particle spectrum. It is significant also that α is only weakly dependent on e, and only weakly dependent on the details of the single particle spectrum as shown in figure 4.

Thermodynamic properties to order N −α
We conclude that the first order correction to s N (e) may be written as where η(e) is the slope of each curve indicated in figure 5, which we have calculated and plotted in figure 6. We find smooth behaviour for η(e) and we expect η(e) → 0 as e → 0 by the third law of thermodynamics. The function η(e) is unique for each system under consideration, and    N (up to 2000) for Model 1 at energy per particle e = 10. The results in this graph are calculated using (5) and the derivative of (3) with respect to E. The range on the vertical axis is from 0 to 1%. must be evaluated separately for each system to enable one to proceed to calculate the various thermodynamic properties of the system to order N −α . For example, the temperature of a finite system of size N is given by where the prime indicates the first derivative with respect to e, and T ∞ represents the temperature in the thermodynamic limit. This establishes the expression for temperature for a finite system of size N . For Model 1 at e = 10 we plot in figure 7 the percentage difference between T N and T ∞ as a function of N . We note only about ∼ 1% elevation in the temperature for N = 100 compared with the temperature in the infinite system size limit. This is a surprising result: even through the entropy per particle converges rather slowly to its value in the thermodynamic limit, the derivative of entropy with respect to the energy converges much more rapidly. This is a general feature of our conclusions -the physically measurable quantities that are expressible as a gradient of the entropy, such as temperature, converge much more rapidly to the thermodynamic limit than does the entropy itself.  Similarly, the ratio of chemical potential to temperature µ T for a finite system of size N is given by where µ T ∞ defines the ratio of the chemical potential to temperature in the thermodynamic limit.
All thermodynamic quantities such as pressure, magnetism, the heat capacities, the various thermodynamic potentials, etc. may now be calculated for any finite system of size N .

Entropic interference
To demonstrate our ideas of entropic interference, we focus on Model 2 for which ε = p 2 + q 4 . To start with, we note that the partition function is separable and therefore entropy is additive in the thermodynamic limit However we don't expect Ω p 2 +q 4 (N, E) to be related to Ω p 2 (N, E) and Ω q 4 (N, E) in a simple way since Ω corresponds to the counting of states! For finite N , we may therefore write where J p 2 +q 4 N (e) may be referred to as an interference term. This can only happen if In figure 8 we plot J p 2 +q 4 N (e) as a function of N which verifies this result. We therefore conclude that entropy is only additive in the thermodynamic limit.

Further theoretical notions
Boltzmann is sometimes credited with being a founding father of quantum mechanics because he was first to assume a discrete set of states (which, unbeknownst to him at that time, was the levels of the quantum harmonic oscillator; this is the same Model 1 that we presented above) for an atomistic system, which he used to derive (1). However, it should be kept in mind that Boltzmann did this for mathematical convenience, and in the end he tried to extract the classical limit and he argued that his results still applied in this continuum limit. There is no evidence to suggest that Boltzmann genuinely believed that the discrete set of levels was anything more than an intermediate step in his arguments for essentially classical systems.
Using (3), we can now see why Boltzmann's argument was valid. Equation (3) is valid independent of the overall scaling factor for energy (which we stated earlier was measured in appropriate units). This energy scale factor may be finite, which is appropriate for quantum systems and is a measure of the quantum unit of energy for the system, or the scale factor may approach zero which is appropriate for classical systems, whereupon (3) will then need to be re-written in terms of an integral rather than a summation over E. The overall conclusions that we have drawn in this paper stand independent of the actual value of this energy scale factor.
A further point worth noting is that Boltzmann used physical arguments to conclude that statistical entropy is an extensive quantity. We have derived and used what may be referred to as a direct numerical method embodied in (3), to show that statistical entropy is an extensive quantity for any arbitrary system defined by its own unique single particle spectrum. In this process, we discovered universal scaling behaviour for statistical entropy.

Conclusions
We conclude that (3) is a computationally efficient means to calculate the entropy of any system comprising N independent, indistinguishable, non-interacting particles in the microcanonical ensemble. We have shown that the entropy per particle converges to a constant real number in the thermodynamic limit independent of the details of the single-particle spectrum, thus verifying for the first time in a direct manner, the extensive nature of statistical entropy. We investigated the manner in which the entropy per particle approaches the thermodynamic limit and discovered universal scaling behaviour for (s N − s ∞ ) ∼ N −α , where α is the exponent. For the range of systems that we have considered, α(e) exhibits only a very weak dependence on e and a very weak dependence on the microscopic details of the system single particle levels. Our work is applicable in the quantitative study of nano-particle physical, chemical and biological systems at finite temperature. We showed that entropy is only additive in the thermodynamic limit.