Entanglement area law for shallow and deep quantum neural network states

The Schrödinger equation of condensed matter system usually involves a large number of degrees of freedom which makes it extremely difficult to be solved exactly. However, the eigenstates of the Hamiltonians of these natural systems often have an internal simplified structure, which makes many approximating or even exact methods possible. Neural network states are introduced as ansatz states of many-body quantum systems recently, and because of their good performance in solving some problems which can not be solved using the state-of-the-art method, many attentions are attracted [27, 28, 30, 40]. Here, to explore the area-law entanglement of the neural network states, we first introduce the concept of quasi-product states. As we will show later, the locality constraint imposed on the neural network architecture results in states of quasi-product form.

Let $\mathcal{S}=\left\{{s}_{1},\dots ,\;{s}_{N}\right\}$ be a system with N particles, by a local K-cluster cover we mean a class of local subsets of $\mathcal{S}$, viz., ${\mathcal{C}}_{1},\dots ,\;{\mathcal{C}}_{M}$, called local cluster, for which each ${\mathcal{C}}_{i}$ only contain at most K particles in a local region and ${\cup }_{i=1}^{M}{\mathcal{C}}_{i}=\mathcal{S}$. A local K-cluster quasi-product state can then be defined as ${\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={{\Phi}}_{1}\left({\mathcal{C}}_{1}\right){\times}\cdots {\times}{{\Phi}}_{M}\left({\mathcal{C}}_{M}\right)$, where each cluster term ${{\Phi}}_{i}\left({\mathcal{C}}_{i}\right)$ is a function of degrees of freedom of particles contained in ${\mathcal{C}}_{i}$, and the size of clusters $K{:=}\mathrm{max}\left\{\vert {\mathcal{C}}_{i}\vert \right\}$ does not depend on the system size N. It is obvious that product state is just one-local quasi-product state, i.e., each ${{\Phi}}_{i}\left({\mathcal{C}}_{i}\right)$ is just the function Φ_i(s_i), since each local cluster ${\mathcal{C}}_{i}$ only contains one particle s_i, we will also refer this kind of states as local one-cluster quasi-product state.

It turns out that many crucial classes of quantum states can be expressed as local quasi-product states, such as cluster state, ${\mathbb{Z}}_{2}$-toric code states, graph states, ${\mathbb{Z}}_{2}$-stabilizer code states, Kitaev's $D\left({\mathbb{Z}}_{d}\right)$ quantum double ground states. They are all explicitly constructed in local RBM form [29–32], but we will show later in this section that all local RBM states are the local quasi-product state.

Many of examples of local gapped systems come from local commutative Hamiltonian H = ∑_kH_k for which [H_k, H_l] = 0, ∀k, l and each local term H_k only acts on a local region ${\mathcal{S}}_{k}$ of the system⁷. It is very natural to use the quasi-product state as an ansatz state to solve the eigenvalue equation HΨ(s₁, ..., s_N) = E₀Ψ(s₁, ..., s_N). We can assign a cluster ${\mathcal{C}}_{k}$ to each local term H_k and usually we also make constraint that ${\mathcal{S}}_{k}\subseteq {\mathcal{C}}_{k}$, i.e., ${\mathcal{C}}_{k}$ contain all particles which H_k acts on nontrivially. In this way, the eigenvalue equation can be simplified as

$$\begin{equation*}{H}_{k}\prod _{j:{\mathcal{C}}_{j}\cap {\mathcal{S}}_{k}\ne \varnothing }{{\Phi}}_{j}\left({\mathcal{C}}_{j}\right)={E}_{0}^{\left(k\right)}\prod _{j:{\mathcal{C}}_{j}\cap {\mathcal{S}}_{k}\ne \varnothing }{{\Phi}}_{j}\left({\mathcal{C}}_{j}\right),\end{equation*}$$

where ${E}_{0}^{\left(k\right)}$ is the ground state energy of H_k. Then using some other properties, like symmetry, of the system, we can alternatively solve these equations with less variables to give the solution of the original eigenvalue problem. Here, for illustration, we choose cluster stabilizer code, toric code and graph state as examples.

Example 1. Cluster stabilizer code state, or equivalently, the ground state of (1 + 1)D symmetry protected topological (SPT) phase Hamiltonian

$$\begin{equation*}{H}_{\text{cluster}}=\sum _{k=1}^{N}\left(I-{\sigma }_{k-1}^{z}{\sigma }_{k}^{x}{\sigma }_{k+1}^{z}\right)\end{equation*}$$

defined on a 1d lattice with periodic boundary condition can be represented by a three-local quasi-product state. Each term ${\sigma }_{k-1}^{z}{\sigma }_{k}^{x}{\sigma }_{k+1}^{z}$ is called a stabilizer. The cluster state is a ${\mathbb{Z}}_{2}{\times}{\mathbb{Z}}_{2}$ protected topological state [41], which can used for measurement-based quantum computation [42–44]. Here, we validate the efficiency of the local quasi-product state representation by explicit constructions, the local cluster is chosen as three-local cluster corresponding to each stabilizer, i.e., ${{\Phi}}_{k}\left({\mathcal{C}}_{k}\right)={{\Phi}}_{k}\left({s}_{k-1},{s}_{k},{s}_{k+1}\right)$. The ground state satisfies ${\sigma }_{k-1}^{z}{\sigma }_{k}^{x}{\sigma }_{k+1}^{z}{\sum }_{{s}_{1},\dots ,\;{s}_{N}={\pm}1}{\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)\vert {s}_{1},\dots ,\;{s}_{N}\rangle ={\sum }_{{s}_{1},\dots ,\;{s}_{N}={\pm}1}{\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)\vert {s}_{1},\dots ,\;{s}_{N}\rangle$ for all k, which is equivalent to

$$\begin{equation}{s}_{k-1}{s}_{k+1}{\Psi}\left(\cdots \;,\;{s}_{k-1},\;{s}_{k},\;{s}_{k+1},\;\cdots \;\right)={\Psi}\left(\cdots \;,\;{s}_{k-1},\;-{s}_{k},\;{s}_{k+1},\;\cdots \;\right),\forall k.\end{equation} \tag{ 1 }$$

Using the three-local quasi-product state ansatz state ${\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={\prod }_{k=1}^{N}{{\Phi}}_{k}\left({s}_{k-1},\;{s}_{k},\;{s}_{k+1}\right)$ and canceling the same term from two sides of the equality, we obtain

$$\begin{align}\hfill & {s}_{k-1}{s}_{k+1}{{\Phi}}_{k-1}\left({s}_{k-2},{s}_{k-1},{s}_{k}\right){\times}{{\Phi}}_{k}\left({s}_{k-1},{s}_{k},{s}_{k+1}\right){\times}{{\Phi}}_{k+1}\left({s}_{k},{s}_{k+1},{s}_{k+2}\right)\hfill \\ \hfill & \quad ={{\Phi}}_{k-1}\left({s}_{k-2},{s}_{k-1},-{s}_{k}\right){\times}{{\Phi}}_{k}\left({s}_{k-1},-{s}_{k},{s}_{k+1}\right){\times}{{\Phi}}_{k+1}\left(-{s}_{k},{s}_{k+1},{s}_{k+2}\right),\forall k.\hfill \end{align}$$

These are highly nonlinear equations, thus are very difficult to solve directly and the solution is not unique in general. But noticing that the model is translationally invariant, we can assume that all local clusters ${{\Phi}}_{k}\left({\mathcal{C}}_{k}\right)$ are of the same form. Via this simplification, we can obtain a solution:

$$\begin{equation*}{{\Phi}}_{k}\left({s}_{k-1},{s}_{k},{s}_{k+1}\right)=2\mathrm{cos}\left(\frac{\pi +2\pi {s}_{k-1}+3\pi {s}_{k}+\pi {s}_{k+1}}{4}\right).\end{equation*}$$

It is easily verified that the local quasi-product state ${\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={\prod }_{k=1}^{N}{{\Phi}}_{k}\left({s}_{k-1},{s}_{k},{s}_{k+1}\right)$ satisfies the equation (1).

Example 2. Let us now consider the toric code model (${\mathbb{Z}}_{2}$-Kitaev quantum double model) [45], which is the simplest model of topologically ordered states and plays an important role in quantum error correcting codes and fault-tolerant quantum computation theories [46]. Given an L × L square lattice with periodic boundary condition (i.e., on a 2d torus ${\mathbb{T}}^{2}$), on each edge there is an associated spin space ${\mathbb{C}}^{2}$, thus there will be N = 2L × L qubits in total. To each vertex v and plaquette we assign a stabilizer operator ${A}_{v}={\prod }_{j\in \partial v}{\sigma }_{j}^{x}$ and ${B}_{p}={\prod }_{j\in \partial p}{\sigma }_{j}^{z}$ respectively, and the Hamiltonian is of the form

$$\begin{equation*}{H}_{\text{toric}}=-\sum _{v}{A}_{v}-\sum _{p}{B}_{p}.\end{equation*}$$

The ground state of the Hamiltonian is four-fold degenerate (which corresponds to the order of the first ${\mathbb{Z}}_{2}$ homology group of torus ${\mathbb{T}}^{2}$, viz., $\text{GSD}=\vert {H}_{1}\left({\mathbb{T}}^{2},{\mathbb{Z}}_{2}\right)\vert$). Let us briefly recall how to calculate the ground state of the toric code model. Consider the constraints imposed by plaquette operators, i.e., B_p|Ω⟩ = |Ω⟩. In the σ^z basis {|0⟩, |1⟩}, the set of spin configurations is then {|00⋯0⟩, |00⋯1⟩, ..., |11⋯1⟩}. Assume that

$$\begin{equation*}\vert {\Omega}\rangle =\sum _{\mathbf{s}}{c}_{\mathbf{s}}\vert \mathbf{s}\rangle ,\end{equation*}$$

then for ${B}_{p}={\sigma }_{{p}_{1}}^{z}{\sigma }_{{p}_{2}}^{z}{\sigma }_{{p}_{3}}^{z}{\sigma }_{{p}_{4}}^{z}$, we have ${B}_{p}\vert \mathbf{s}\rangle ={\left(-1\right)}^{{s}_{{p}_{1}}+{s}_{{p}_{2}}+{s}_{{p}_{3}}+{s}_{{p}_{4}}}\vert \mathbf{s}\rangle$, where $\vert \mathbf{s}\rangle =\vert {s}_{{p}_{1}}{s}_{{p}_{2}}{s}_{{p}_{3}}{s}_{{p}_{4}}\rangle \otimes \vert \cdots \rangle$ and the addition is modulo two. Therefore

$$\begin{equation*}{B}_{p}\vert {\Omega}\rangle =\sum _{\mathbf{s}}{c}_{\mathbf{s}}{\left(-1\right)}^{{s}_{{p}_{1}}+{s}_{{p}_{2}}+{s}_{{p}_{3}}+{s}_{{p}_{4}}}\vert \mathbf{s}\rangle =\sum _{\mathbf{s}}{c}_{\mathbf{s}}\vert \mathbf{s}\rangle .\end{equation*}$$

If ${s}_{{p}_{1}}+{s}_{{p}_{2}}+{s}_{{p}_{3}}+{s}_{{p}_{4}}=1$ the corresponding coefficient c_s must be zero, thus the spin configurations which do not vanish is the one only even number of |1⟩ is placed on the edges of each plaquette, which means that the |1⟩ spins form a close loop in the dual lattice (see figure 1):

$$\begin{equation*}\vert {\Omega}\rangle =\sum _{\text{loop}}{c}_{\text{loop}}\vert \mathrm{l}\mathrm{o}\mathrm{o}\mathrm{p}\rangle .\end{equation*}$$

Zoom In Zoom Out Reset image size

As shown in figure 1, the effect of vertex operators is just to deform one loop into another. We call two loops equivalent if they can be linked by some vertex operators, the GSD is just the number of equivalent classes of loops. There exist essentially four different kinds of loops like longitudinal loop and latitudinal loop shown in figure 1, they are actually the bases (logical states) of the ground state space of toric code model:

$$\begin{equation*}\begin{matrix}{c}\hfill \vert 0{0\rangle }_{L}=\sum _{\text{trivial}\;\text{loop}}\vert \text{loop}\rangle ,\hfill \\ \hfill \vert 0{1\rangle }_{L}=\sum _{\text{longitudinal}\;\text{loop}}\vert \text{loop}\rangle ,\hfill \\ \hfill \vert 1{0\rangle }_{L}=\sum _{\text{latitudinal}\;\text{loop}}\vert \text{loop}\rangle ,\hfill \\ \hfill \vert 1{1\rangle }_{L}=\sum _{\text{longitudinal}+\text{latitudinal}\;\text{loop}}\vert \text{loop}\rangle .\hfill \end{matrix}\end{equation*}$$

Here, by explicit construction, we show that the ground state of toric code model can be represented as four-local quasi-product state, the philosophy is very similar as what we have done above. Actually, we can assign a cluster to each vertex and plaquette, thus the state is of the form ${\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={\prod }_{v}{{\Phi}}_{v}\left({\mathcal{C}}_{v}\right){\prod }_{p}{{\Phi}}_{p}\left({\mathcal{C}}_{p}\right)$, where ${\mathcal{C}}_{v}$ (resp. ${\mathcal{C}}_{p}$) only contain spins which A_v (resp. B_p) acts nontrivially. Like in the cluster state construction, we have the constraints

$$\begin{align*}\hfill & {A}_{v}{\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={\Psi}\left(\cdots \;,-{s}_{{j}_{1}}^{\left(v\right)},-{s}_{{j}_{2}}^{\left(v\right)},-{s}_{{j}_{3}}^{\left(v\right)},-{s}_{{j}_{4}}^{\left(v\right)},\cdots \;\right)\hfill \\ \hfill & {B}_{p}{\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)=\prod _{i\in \partial p}{s}_{i}{\Psi}\left({s}_{1},\dots ,{s}_{N}\right).\hfill \end{align*}$$

This can be transformed into a set of equations only involves local clusters around vertex v and plaquette p, as we have show exlicitly in references [31, 32] for general stabilizer code. A solution in the RBM form in provided in [29], it is easily checked that the corresponding local clusters

$$\begin{align*}\hfill & {{\Phi}}_{v}\left({s}_{{i}_{1}}^{\left(v\right)},{s}_{{i}_{2}}^{\left(v\right)},{s}_{{i}_{3}}^{\left(v\right)},{s}_{{i}_{4}}^{\left(v\right)}\right)=\mathrm{cos}\left[\frac{\pi }{2}\left({s}_{{i}_{1}}^{\left(v\right)}+{s}_{{i}_{2}}^{\left(v\right)}+{s}_{{i}_{3}}^{\left(v\right)}+{s}_{{i}_{4}}^{\left(v\right)}\right)\right],\hfill \\ \hfill & {{\Phi}}_{p}\left({s}_{{j}_{1}}^{\left(p\right)},{s}_{{j}_{2}}^{\left(p\right)},{s}_{{j}_{3}}^{\left(p\right)},{s}_{{j}_{4}}^{\left(p\right)}\right)=\mathrm{cos}\left[\frac{\pi }{4}\left({s}_{{j}_{1}}^{\left(p\right)}+{s}_{{j}_{2}}^{\left(p\right)}+{s}_{{j}_{3}}^{\left(p\right)}+{s}_{{j}_{4}}^{\left(p\right)}\right)\right],\hfill \end{align*}$$

form a ground state of the toric code model. The excited state can also be represented in quasi-product form in a similar way. We must stress here that this is just one of the solutions, in fact there are a lot of other solutions depending on the choice of the local clusters.

Example 3. Another example we will consider here is the graph state, which is an important class of multipartite entangled quantum states and is useful for quantum error correcting codes, measurement-only quantum computation and so on [47]. For a given graph G with vertex set V(G) = {1, ..., N} and edge set E(G) ⊂ V(G) × V(G), the graph state is defined as

$$\begin{equation*}\vert {{\Psi}}_{G}\rangle =\prod _{\langle ij\rangle \in E\left(G\right)}{U}_{\langle ij\rangle }{\vert +\rangle }^{\otimes V\left(G\right)},\end{equation*}$$

where U_⟨ij⟩ is a two-qubit controlled-Z gate. The wave function thus takes the form

$$\begin{equation}{{\Psi}}_{G}\left({s}_{1},\dots ,\;{s}_{N}\right)=\prod _{\langle ij\rangle \in E\left(G\right)}\frac{{\left(-1\right)}^{{s}_{i}{s}_{j}}}{\sqrt{2}}.\end{equation} \tag{ 2 }$$

To represent the state using the local quasi-product state, we can assign a cluster to each edge e = ⟨i_ej_e⟩ ∈ E(G), i.e., ${{\Psi}}_{G}\left({s}_{1},\dots ,\;{s}_{N}\right)={\prod }_{e\in E\left(G\right)}{{\Phi}}_{e}\left({s}_{{i}_{e}},{s}_{{j}_{e}}\right)$. From equation (2) we see that ${{\Phi}}_{e}\left({s}_{{i}_{e}},{s}_{{j}_{e}}\right)=\frac{{\left(-1\right)}^{{s}_{{i}_{e}}{s}_{{j}_{e}}}}{\sqrt{2}}$, which is obviously a two-local quasi-product state.

Here we will construct two important classes of quasi-product states via feed-forward and stochastic recurrent neural networks, which will be the main focus of this work. To this end, we first need to introduce the notion of geometry for neural networks.

2.2.1. The geometry of neural network states

Inspired by the geometry of tensor network states [48], here we introduce the notion of the geometry of the neural network states, which turns out to be crucial for understanding entanglement features. Hereinafter, we will concentrate on the neural networks with a layered structure, which are also the most studied cases. The physical degrees of freedom are placed on some fixed layer of neural networks, e.g., the visible layer of restricted Boltzmann machine or input layer of the feed-forward neural network, the layer will be referred to as physical layer. Notice that the physical layer has its geometry given by the physical system. For example, if the physical degrees of freedom (like spins) are placed on the square lattice, we can impose the neurons (represent physical degrees of freedom) to have the same square lattice geometry. After the geometry of the physical layer is fixed by the geometry of the physical system, all other layers are imposed to have the same geometry duplicated from the physical layer as shown in figure 2.

Zoom In Zoom Out Reset image size

Recall that the geometry of tensor network states is characterized by the positions of local tensors and their contraction pattern, here for neural network states, similar results hold. We can compare the distance between neurons in different layers since these layers have the same geometry. Now we can define the notion of locality of a neural network. For a neuron ${h}_{i}^{\left(l\right)}$ in a given, say lth, layer, it is called local ɛ-connected with other neurons if it only connects to the neurons in (l − 1)th and (l + 1)th layers in the ɛ-neighborhood of h_i (see figure 2 for illustration). If all the neurons of a neural network are local ɛ-connected with each other, we say the neural network is a local ɛ-connected neural network. Similar construction has been used in references [29, 31, 32, 49] for exactly constructing neural network states of some physical systems. When the in a local ɛ-connected neural network, each neuron ${h}_{i}^{\left(l\right)}$ only connects with K neurons both in (l − 1)th and (l + 1)th layers, we call it a K-local neural network. For a K-local neural network, a corresponding quantum state can be given. Usually, there are two different ways to build quantum neural network states [40], the first approach, which is also the approach we choose to use in this work, is to introduce complex weights and biases into the neural network; the second approach is to represent the amplitude and phase of a wavefunction separately. We will prove that quantum states build from K-local neural networks obey the entanglement area-law, since there are all quasi-product states, and the entanglement area law of quasi-product states will be established later. To make this construction more clear, let us see two important examples.

2.2.2. Local restricted Boltzmann machine states

In this part, we will introduce the notion of restricted Boltzmann machine states, which were introduced in reference [27] for calculating ground state and unitary evolution of strongly correlated many-body systems. The RBM was invented by Smolensky [50], it is an energy-based neural network model [51, 52]. Since RBM only has two layers of neurons, one visible layer and one hidden layer, it can be regarded as a shallow neural network.

We now build quantum states from local RBM and show that they are local quasi-product states. The RBMs have a layered structure, which makes the locality defined above applicable. To construct a local RBM state, we first impose the locality constraints on the visible layer, which is nothing but the physical layer, each visible neuron corresponds to the physical degrees of freedom (e.g. spin), denoted as $\mathcal{S}=\left\{{v}_{1},\dots ,\;{v}_{n}\right\}$, and the geometry of the visible layer is inherited from the physical system. The hidden neurons are denoted as {h₁, ..., h_m}, which are placed on the hidden layer with geometry duplicated from visible layer, viz, the distance between two neurons can be defined the same as visiblelayer (the distance of visible layer is inherited from the physical system). The weight between h_j and v_i is denoted as W_ij, thebiases of v_i and h_j are a_i and b_j respectively. The RBM representation of quantum states is obtained by tracing out all hidden neurons, viz,

$$\begin{align}\hfill {{\Psi}}_{\text{RBM}}\left({v}_{1},\dots ,\;{v}_{n}\right)=& \sum _{{h}_{1},\dots ,\;{h}_{m}}\mathrm{exp}\left\{\sum _{i}{a}_{i}{v}_{i}+\sum _{j}{h}_{j}\left({b}_{j}+\sum _{i:\langle ij\rangle }{W}_{ij}{v}_{i}\right)\right\}\hfill \\ \hfill =& {\mathrm{e}}^{\sum _{i}{a}_{i}{v}_{i}}\prod _{j}{{\Gamma}}_{j}\left({v}_{i}:\langle {v}_{i}{h}_{j}\rangle \right),\hfill \end{align} \tag{ 3 }$$

where ${{\Gamma}}_{j}\left({v}_{i}:\langle {v}_{i}{h}_{j}\rangle \right)=2\mathrm{cosh}\left({b}_{j}+{\sum }_{i:\langle ij\rangle }{v}_{i}{W}_{ij}\right)$ if h_j = ±1 and ${{\Gamma}}_{j}\left({v}_{i}:\langle {v}_{i}{h}_{j}\rangle \right)=1+\mathrm{exp}\left({b}_{j}+{\sum }_{i:\langle ij\rangle }{v}_{i}{W}_{ij}\right)$ if h_j = 0, 1, and by notation ⟨ij⟩ we mean h_j and v_i are connected. The K-local RBM state can be defined as the one whose hidden neurons h_j only connect with at most K visible neurons ${v}_{{j}_{k}}$ in a local ɛ-neighborhood of h_j. From the construction it is easily checked that the Ψ_RBM(v₁, ..., v_n) = ∏_jΦ_j(v_i : ⟨v_ih_j⟩). This kind of construction has be used in references [29, 31, 32, 49] for investigating physical properties of complex systems.

Let us now see an one-dimensional example. As shown in figure 3(a), the quantum states built from three-local RBM neural network is a local quasi-product state. From equation (3), it is easily checked that Ψ(v₁, ..., v₉) = Φ₁(v₁, v₂) × Φ₂(v₁, v₂, v₃) × ⋯ × Φ₉(v₈, v₉), thus it is a three-local quasi-product state.

Zoom In Zoom Out Reset image size

2.2.3. Local feed-forward neural network states

Entanglement entropy is a crucial theoretical tool for investigating quantum many-body systems. We now establish the entanglement area law of local quasi-product states, since local RBM states and local feed-forward neural network states are special cases of local quasi-product states, they all obey the entanglement area law. We prove that in arbitrary spatial dimensions the local quasi-product states obey the entanglement area law for arbitrary connected bipartition of the system. More precisely, we have the following theorem:

Theorem 1. For an N-particle system $\mathcal{S}$, suppose that

$$\begin{equation*}\vert {\Psi}\rangle =\sum _{{s}_{1},\dots ,\;{s}_{N}}{\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)\vert {s}_{1}\rangle \otimes \cdots \otimes \vert {s}_{N}\rangle \end{equation*}$$

is a K-local quasi-product quantum states, then the Rényi entropies of the reduced density matrix ${\rho }_{\mathcal{A}}={\mathrm{Tr}}_{{\mathcal{A}}^{c}}\vert {\Psi}\rangle \langle {\Psi}\vert$ with respect to the bipartition $\mathcal{S}=\mathcal{A}\bigsqcup {\mathcal{A}}^{c}$ satisfies the following area law

$$\begin{equation}{S}_{\alpha }\left(\mathcal{A}\right){\leqslant}\zeta \left(K\right)\;\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\left(\mathcal{A}\right),\;\;\forall \alpha ,\end{equation} \tag{ 4 }$$

where $\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\left(\mathcal{A}\right)$ denotes the number of particles on the boundary of $\mathcal{A}$ and ζ(K) is a scaling factor only depends on the size of local cluster K.

Proof. We first define three kinds of local clusters: (i) the clusters which only contain particles in $\mathcal{A}$ (as ${\mathcal{C}}_{\mathrm{int}}$ in figure 4), called internal clusters; (ii) the clusters which only contain particles in ${\mathcal{A}}^{c}$ (as ${\mathcal{C}}_{\text{ext}}$ in figure 4), called external clusters; and (iii) the clusters which contain particles both in $\mathcal{A}$ and ${\mathcal{A}}^{c}$ (as ${\mathcal{C}}_{bd}$ in figure 4), called boundary clusters. We will denote the set of particles contained in boundary clusters as $\mathcal{B}=\partial \mathcal{A}\cup \partial {\mathcal{A}}^{c}$, where $\partial \mathcal{A}=\mathcal{A}\cap \left({\cup }_{i}{\mathcal{C}}_{bd}^{\left(i\right)}\right)$, ${\mathcal{C}}_{bd}^{\left(i\right)}$ are all boundary clusters, and similarly for $\partial {\mathcal{A}}^{c}$. The interior of $\mathcal{A}$, denoted as $\mathrm{I}\mathrm{n}\mathrm{t}\mathcal{A}$, is defined as $\mathrm{I}\mathrm{n}\mathrm{t}\mathcal{A}=\mathcal{A}{\backslash}\partial \mathcal{A}$; the exterior of $\mathcal{A}$, denoted as $\mathrm{E}\mathrm{x}\mathrm{t}\mathcal{A}$, is defined as $\mathrm{E}\mathrm{x}\mathrm{t}\mathcal{A}={\mathcal{A}}^{c}{\backslash}\partial {\mathcal{A}}^{c}$.

Since ${\Psi}\left({s}_{1},\dots ,\;{s}_{N}\right)={\prod }_{i=1}^{M}{{\Phi}}_{i}\left({\mathcal{C}}_{i}\right)$ are of quasi-product form, then using the locality feature of each local cluster, we have

$$\begin{align}\hfill \vert {\Psi}\rangle & =\sum _{{s}_{1},\dots ,\;{s}_{N}}\prod _{i=1}^{M}{{\Phi}}_{i}\left({\mathcal{C}}_{i}\right)\vert {s}_{1}\rangle \otimes \cdots \otimes \vert {s}_{N}\rangle \hfill \\ \hfill & =\sum _{{s}_{m}\in \mathcal{B}}\left[\prod _{{\mathcal{C}}_{bd}^{\left(i\right)}\subset \mathcal{B}}{{\Phi}}_{i}\left({\mathcal{C}}_{bd}^{\left(i\right)}\right)\right]\vert {{\Phi}}_{L}\left(\partial \mathcal{A}\right)\rangle \otimes \vert {{\Phi}}_{R}\left(\partial {\mathcal{A}}^{c}\right)\rangle \hfill \end{align} \tag{ 5 }$$

where $\vert {{\Phi}}_{L}\left(\partial \mathcal{A}\right)\rangle ={\sum }_{{s}_{n}\in \mathrm{I}\mathrm{n}\mathrm{t}\mathcal{A}}{\prod }_{{\mathcal{C}}_{\mathrm{int}}^{\left(j\right)}\subset \mathcal{A}}{\Phi}\left({\mathcal{C}}_{\mathrm{int}}^{\left(j\right)}\right)\vert \mathcal{A}\rangle$ and $\vert {{\Phi}}_{R}\left(\partial {\mathcal{A}}^{c}\right)\rangle ={\sum }_{{s}_{l}\in \mathrm{E}\mathrm{x}\mathrm{t}\mathcal{A}}{\prod }_{{\mathcal{C}}_{\text{ext}}^{\left(j\right)}\subset {\mathcal{A}}^{c}}{\Phi}\left({\mathcal{C}}_{\text{ext}}^{\left(k\right)}\right)\vert {\mathcal{A}}^{c}\rangle$ , states $\vert {{\Phi}}_{L}\left(\partial \mathcal{A}\right)\rangle$ are labeled by particles contained in $\partial \mathcal{A}$ and states $\vert {{\Phi}}_{R}\left(\partial {\mathcal{A}}^{c}\right)\rangle$ are labeled by the particles contained in $\partial {\mathcal{A}}^{c}$. If each local system s_i can take p values, there are at most ${p}^{\vert \mathcal{B}\vert }$ terms contained in summation of equation (5). We stress here that $\vert \mathcal{B}\vert$ only depends on the area $\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathcal{A}$ and cluster size K, more precisely, $\vert \mathcal{B}\vert {\leqslant}R{\times}\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathcal{A}$. Therefore after tracing out the ${\mathcal{A}}^{c}$ part, we get ${\rho }_{\mathcal{A}}$ with rank at most ${p}^{\vert \mathcal{B}\vert }$, thus the Rényi entropy ${S}_{\alpha }\left(\mathcal{A}\right)$ is upper bounded by $\zeta \left(K\right)\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathcal{A}$.□

Zoom In Zoom Out Reset image size

For a physical system with a fixed background space-time, particles only interact with their neighborhoods, the strength of the interaction usually decays to zero if particles are sufficiently far apart. Thus for any many-body physical system, there is a preexisted geometry which characterizes the distance of two particles. However, when we write down a many-body quantum state, this part of geometrical information is usually erased in some sense. For a multipartite quantum system $\mathcal{S}$, we have a state $\vert {{\Psi}}_{\mathcal{S}}\rangle$ which encodes the full information of the system. To understand entanglement feature of $\vert {{\Psi}}_{\mathcal{S}}\rangle$, we first divide $\mathcal{S}$ into two parts $\mathcal{A}$ and ${\mathcal{A}}^{c}$, note that Rényi entropy ${S}_{\alpha }\left(\mathcal{A}\right)$ of the reduced density matrix ${\rho }_{\mathcal{A}}$ quantifies the entanglement between $\mathcal{A}$ and ${\mathcal{A}}^{c}$, then we can define entanglement feature of $\vert {{\Psi}}_{\mathcal{S}}\rangle$ as the set of Rényi entropies ${S}_{\alpha }\left(\mathcal{A}\right)$ over all entanglement subsystem $\mathcal{A}$. The entanglement feature of the system usually contains the information of the geometry of the system, namely, using the entanglement-geometry correspondence (duality), we can recover the geometry information from entanglement features.

Since the paradigm of the neural network is to adjust the connection weights, when weights are zero we say there is no connection between two neurons. It is then natural that the geometry of the neural network reflects in the connectivity of the network. In the spirit of entanglement-geometry correspondence, the entanglement is then encoded in the connectivity of neural networks of the state. This is consistent with the intuition we get from tensor network states, for which we can easily read out entanglement properties from the geometry of the tensor network. Since locally connected neural network states are all local quasi-product states, thus they all obey the entanglement area law. The locality of the states is encoded in the connection pattern of the neural network, which agrees well with our intuition. This kind of construction will also be useful for understanding the entanglement-geometry correspondence [49], such as Ryu–Takayanagi formula in a discrete form [11, 25, 26].

Another issue worth mentioning is the topological entanglement entropy [54], which is a sub-leading term of $S\left(\mathcal{A}\right)$. More precisely, for a gapped system, the entanglement entropy is expected to take the form $S\left(\mathcal{A}\right)=\zeta \mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathcal{A}-\gamma +\mathcal{O}\left(\vert \mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\mathcal{A}{\vert }^{\beta }\right)$, here ζ, γ, β ⩾ 0, the first term is the area law contribution and γ is called topological entanglement entropy. Topological entanglement entropy S_top = γ is a universal quantum number which can be used to detect if a system is topologically ordered or equivalently, if the state is long-range entangled. The word universal means that for the states with the same topological order, the topological entanglement entropy is the same. The local quasi-product state is a good ansatz state for gapped many-body system, thus it can also be used to calculate topological entanglement entropy. The procedure is as follows, for a local gapped quantum system H = ∑_kH_k, take the ansatz state as ${\Psi}={{\Pi}}_{k}{{\Phi}}_{k}\left({\mathcal{C}}_{k},{{\Omega}}_{k}\right)$, here ${\mathcal{C}}_{k}$ is the local cluster corresponding to H_k which has the support (spins contained in ${\mathcal{C}}_{k}$) equal or lager than the support (spins which the operator acts nontivially) of H_k and Ω_k is the variational parameters of the local cluster state Φ_k which, for example, can be chosen as the weights and biases for the kth portion of local neural network of the state. Using the variational method, we can find the value of parameters which minimize the energy functional and thus obtain the state. Then taking a large disc region $\mathcal{D}$ with smooth boundary, we can divide it as three fan-shaped subregions $\mathcal{A},\mathcal{B},\mathcal{C}$, the topological entanglement entropy can be obtained from

$$\begin{equation*}{S}_{\text{top}}=S\left(\mathcal{A}\mathcal{B}\right)+S\left(\mathcal{C}\mathcal{B}\right)+S\left(\mathcal{A}\mathcal{C}\right)-S\left(\mathcal{A}\right)-S\left(\mathcal{B}\right)-S\left(\mathcal{C}\right)-S\left(\mathcal{D}\right).\end{equation*}$$

As what we have shown in example 2, the toric code state can be represented exactly as local quasi-product state in the most economic choice of local cluster, thus S_top of toric code model can be obtained exactly. In general, to improve the accuracy of the calculation, the lager local cluster ought to be chosen.

Here we show that the geometrical information, locality, of the deep neural network, also results in the area law of the entanglement entropy.

Theorem 2. For any K-local DBM state |Ψ⟩, Rényi entropy of the reduced density matrix ${\rho }_{\mathcal{A}}={\mathrm{Tr}}_{{\mathcal{A}}^{c}}\left(\vert {\Psi}\rangle \langle {\Psi}\vert \right)$ satisfy the following area law

$$\begin{equation}{S}_{\alpha }\left(\mathcal{A}\right){\leqslant}\zeta \left(K\right)\;\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\left(\mathcal{A}\right),\;\;\forall \alpha ,\end{equation} \tag{ 6 }$$

where $\mathrm{A}\mathrm{r}\mathrm{e}\mathrm{a}\left(\mathcal{A}\right)$ denotes the number of particles on the boundary of $\mathcal{A}$ and ζ(K) is a scaling factor only depends on local connection number K.

Proof. Inspired by the work of Deng et al [28], here we establish the area law by explicit construction. For K-local DBM states, we can group connections into several sets using the hidden neurons in the first hidden layer. Note that for DBM states, the coefficients are ${\Psi}\left(\mathbf{v}\right)={\sum }_{\mathbf{h}}{\sum }_{\mathbf{g}}\mathrm{exp}\left\{{\sum }_{i}{v}_{i}{a}_{i}+{\sum }_{k}{c}_{k}{g}_{k}+{\sum }_{j}{h}_{j}\left({b}_{j}+{\sum }_{i;\langle ij\rangle }{W}_{ij}{v}_{i}+{\sum }_{k;\langle kj\rangle }{W}_{kj}{g}_{k}\right)\right\}$ where sum is over hidden neurons h = (h₁, ..., h_m) of first hidden layer and g = (g₁, ..., g_l) of second hidden layer and all hidden neurons are assumed to take values 0, 1 here (of course, for the case of taking values ±1, the result also holds). Then, like what have usually been done for area-law tensor network states, we need to factorize the coefficients into a partial product form ${\Psi}\left(\mathbf{v}\right)={\prod }_{i=1}^{n}{\mathrm{e}}^{{v}_{i}{a}_{i}}{\sum }_{\mathbf{g}}\left({\prod }_{k=1}^{l}{\mathrm{e}}^{{g}_{k}{c}_{k}}{\prod }_{j=1}^{m}{{\Phi}}_{j}\right)$ in which ${{\Phi}}_{j}={{\Phi}}_{j}\left({v}_{{j}_{1}},\dots ,\;{v}_{{j}_{K}};{g}_{{j}_{1}},\dots ,\;{g}_{{j}_{K}}\right)={\sum }_{{h}_{j}=0,1}\mathrm{exp}\left\{{h}_{j}\left({b}_{j}+{\sum }_{i;\langle ij\rangle }{W}_{ij}{v}_{i}+{\sum }_{k;\langle kj\rangle }{W}_{kj}{g}_{k}\right)\right\}$, ${v}_{{j}_{1}},\dots ,\;{v}_{{j}_{K}}$ are (at most) K visible neurons connected to h_j, and ${g}_{{j}_{1}},\dots ,\;{g}_{{j}_{K}}$ (at most) K hidden neurons connected to h_j in deep hidden layer (where we have used the assumption of K-locality). Now using an important trick of reference [28], the visible layer neurons can be divided into six groups: ${\mathcal{A}}_{3}$ consists of visible neurons which connect ${\mathcal{A}}^{c}$ part by a hidden neuron, ${\mathcal{A}}_{2}$ consists of visible neurons connecting neurons of ${\mathcal{A}}_{3}$ via a hidden neuron, and ${\mathcal{A}}_{1}=\mathcal{A}{\backslash}\left({\mathcal{A}}_{3}\bigcup {\mathcal{A}}_{2}\right)$; similarly, we can define ${\mathcal{A}}_{3}^{c}$, ${\mathcal{A}}_{2}^{c}$ and ${\mathcal{A}}_{1}^{c}$. Obviously, $\mathcal{A}$ (${\mathcal{A}}^{c}$) is the disjoint union of ${\mathcal{A}}_{1}$, ${\mathcal{A}}_{2}$ and ${\mathcal{A}}_{1}$ (${\mathcal{A}}_{3}^{c}$, ${\mathcal{A}}_{2}^{c}$ and ${\mathcal{A}}_{1}^{c}$). Similar division can be applied to the deep hidden layer (since the layer has a fixed background geometry same as visible layer), we first assume that the corresponding bipartition of the layer is $\mathcal{B}\bigsqcup {\mathcal{B}}^{c}$, the deep hidden neurons are then grouped into six parts: ${\mathcal{B}}_{1}$, ${\mathcal{B}}_{2}$, ${\mathcal{B}}_{3}$, ${\mathcal{B}}_{3}^{c}$, ${\mathcal{B}}_{2}^{c}$ and ${\mathcal{B}}_{1}^{c}$.

Now, consider the state $\vert {\Psi}\rangle ={\sum }_{\mathbf{v}}{\Psi}\left(\mathbf{v}\right)\vert {\mathbf{v}}_{\mathcal{A}}\rangle \otimes \vert {\mathbf{v}}_{{\mathcal{A}}^{c}}\rangle ={\sum }_{\mathbf{v}}{\prod }_{i=1}^{n}{\mathrm{e}}^{{v}_{i}{a}_{i}}{\sum }_{\mathbf{g}}\left({\prod }_{k=1}^{l}{\mathrm{e}}^{{g}_{k}{c}_{k}}{\prod }_{j=1}^{m}{{\Phi}}_{j}\right)\vert {\mathbf{v}}_{\mathcal{A}}\rangle \otimes \vert {\mathbf{v}}_{{\mathcal{A}}^{c}}\rangle$, we denote the set of shallow hidden neurons which connect ${\mathcal{A}}_{3}$ and ${\mathcal{A}}_{3}^{c}$ (also ${\mathcal{B}}_{3}$ and ${\mathcal{B}}_{3}^{c}$) as ${\mathcal{C}}_{\text{Bd}}$, ones connect ${\mathcal{A}}_{1}$ and ${\mathcal{A}}_{2}$ (also ${\mathcal{B}}_{1}$ and ${\mathcal{B}}_{2}$) as ${\mathcal{C}}_{\text{Int}}$ and ones connect ${\mathcal{A}}_{1}^{c}$ and ${\mathcal{A}}_{2}^{c}$ (also ${\mathcal{B}}_{1}^{c}$ and ${\mathcal{B}}_{2}^{c}$) as ${\mathcal{C}}_{\text{Ext}}$. We can introduce the state $\vert {{\Psi}}_{\mathcal{A}}\rangle ={\sum }_{{\mathbf{v}}_{{\mathcal{A}}_{1}}}{\mathrm{e}}^{{\mathbf{v}}_{{\mathcal{A}}_{1}}\cdot {\mathbf{a}}_{{\mathcal{A}}_{1}}}{\sum }_{{\mathbf{g}}_{{\mathcal{B}}_{1}}}{\mathrm{e}}^{{\mathbf{g}}_{{\mathcal{B}}_{1}}\cdot {\mathbf{c}}_{{\mathcal{B}}_{1}}}{\prod }_{j\in {\mathcal{C}}_{\text{Int}}}{{\Phi}}_{j}\vert {\mathbf{v}}_{\mathcal{A}}\rangle$ and state $\vert {{\Psi}}_{{\mathcal{A}}^{c}}\rangle ={\sum }_{{\mathbf{v}}_{{\mathcal{A}}_{1}^{c}}}{\mathrm{e}}^{{\mathbf{v}}_{{\mathcal{A}}_{1}^{c}}\cdot {\mathbf{a}}_{{\mathcal{A}}_{1}^{c}}}{\sum }_{{\mathbf{g}}_{{\mathcal{B}}_{1}^{c}}}{\mathrm{e}}^{{\mathbf{g}}_{{\mathcal{B}}_{1}^{c}}\cdot {\mathbf{c}}_{{\mathcal{B}}_{1}^{c}}}{\prod }_{j\in {\mathcal{C}}_{\text{Ext}}}{{\Phi}}_{j}\vert {\mathbf{v}}_{{\mathcal{A}}^{c}}\rangle$, then the state Ψ can be decomposed as

$$\begin{equation*}\vert {\Psi}\rangle =\sum _{{\mathbf{v}}_{{\mathcal{A}}_{\text{Bd}}}}\sum _{{\mathbf{g}}_{{\mathcal{B}}_{\text{Bd}}}}{\mathrm{e}}^{{\mathbf{v}}_{{\mathcal{A}}_{\text{Bd}}}\cdot {\mathbf{a}}_{{\mathcal{A}}_{\text{Bd}}}}{\mathrm{e}}^{{\mathbf{g}}_{{\mathcal{B}}_{\text{Bd}}}\cdot {\mathbf{c}}_{{\mathcal{B}}_{\text{Bd}}}}\prod _{j\in {\mathcal{C}}_{\text{Bd}}}{{\Phi}}_{j}\vert {{\Psi}}_{\mathcal{A}}\rangle \otimes \vert {{\Psi}}_{{\mathcal{A}}^{c}}\rangle ,\end{equation*}$$

where ${\mathcal{A}}_{\text{Bd}}={\mathcal{A}}_{2}\cup {\mathcal{A}}_{3}\cup {\mathcal{A}}_{3}^{c}\cup {\mathcal{A}}_{2}^{c}$ and ${\mathcal{B}}_{\text{Bd}}={\mathcal{B}}_{2}\cup {\mathcal{B}}_{3}\cup {\mathcal{B}}_{3}^{c}\cup {\mathcal{B}}_{2}^{c}$. Tracing out the ${\mathcal{A}}^{c}$ part, we get ${\rho }_{\mathcal{A}}$ is the weighted sum of several one-dimensional projectors (which are not necessarily orthogonal), via Gram–Schmidt orthogonalization, it is clear that the rank of ${\rho }_{\mathcal{A}}$ is upper bounded by a function f(K) only depends on K. Since the Rényi entropy of a density matrix take the maximum value only if all eigenvalues p_i are equal, i.e., p₁ = ⋯ = p_r = 1/r ⩾ 1/f(K) with r the rank of matrix, we then complete the proof. □

Let us give some intuitive explanation about the construction. Since each visible neuron is correlated with neurons in ɛ-neighborhood, we can regard ɛ as the correlation length of the state, the correlation between $\mathcal{A}$ and ${\mathcal{A}}^{c}$ comes predominantly from visible neurons sit in 2ɛ-strip around the boundary $\partial \mathcal{A}$. Then the visible neurons deep in the region $\mathcal{A}$ can not be correlated with visible neurons deep in the region ${\mathcal{A}}^{c}$, only neurons near the boundary contribute to the Rényi entanglement entropy ${S}_{\alpha }\left(\mathcal{A}\right)$ and thus result in the area law.

As we have proved above, the locality of the neural network reflects in the area law of entanglement features. It is natural to ask what about the neural network with nonlocal connections. We can expect the fully connected DBMs exhibit entanglement volume law [58–60], actually this is the case. In contrast to tensor network for which the efficiency⁹ strongly depends on the validity of the entanglement area law of the state, the neural networks are still efficient in representing many-body states obeying volume law. As has been pointed out in references [29, 30], shallow neural networks are capable of some critical-system states obeying entanglement volume law. We can trivially add a deep hidden layer and give some trivial connections to make a deep neural network exhibit volume law. Despite the triviality of the construction, we want to stress some crucial point of the volume-law neural network: (i) there must be some nonlocal connections in the neural network architecture, which is the origin of the volume law entanglement; (ii) the representation is efficient, i.e., the number of hidden neurons and connections increases at most polynomially at the number of visible neurons.

The volume-law DBM states have a close relationship with the maximally multipartite entangled states [61]. The philosophy behind the construction is that we can make the particles in the smaller region $\mathcal{A}$ fully correlate with its complementary ${\mathcal{A}}^{c}$ such that all information of $\mathcal{A}$ is encoded in ${\mathcal{A}}^{c}$ in some way, then ${\rho }_{\mathcal{A}}$ is proportional to identity matrix ${\mathbb{1}}_{\text{Vol}\left(\mathcal{A}\right)}$ with order $\mathrm{V}\mathrm{o}\mathrm{l}\left(\mathcal{A}\right)$ (where $\mathrm{V}\mathrm{o}\mathrm{l}\left(\mathcal{A}\right)$ denotes the number of particles contained in region $\mathcal{A}$), which further implies that Rényi entropies satisfy the volume law. Another important issue is that the number of the neural network is closely related to the entanglement properties of the corresponding neural network states. A neural network state with more hidden layers will tend to exhibit the volume law entanglement. This can be seen from feed-forward neural network more easily, if the number of the hidden layers increases, eventually, the size of the local clusters will be comparable with the system size, this will break the entanglement area law.