Unsupervised learning spectral functions with neural networks

Reconstructing spectral functions from Euclidean Green’s functions is an ill-posed inverse problem that is crucial for understanding the properties of many-body systems. In this proceeding, we propose an automatic differentiation (AD) framework utilizing neural network representations for spectral reconstruction from propagator observables. We construct spectral functions using neural networks and optimize the network parameters unsupervisedly based on the reconstruction error of the propagator. Compared to the maximum entropy method, the AD framework demonstrates better performance in situations with high noise levels. It is noteworthy that neural network representations provide non-local regularization, which has the potential to significantly improve the solution of inverse problems.


Introduction
In lattice calculations for many-body systems, it's necessary to perform an analytic continuation of function from finite data points which however is ill-posed [1,2].In Euclidean Quantum Field Theory (QFT), one will encounter the dilemma when rebuilding spectral functions based on some discrete data points along the Euclidean axis.More specifically, the inverse problem occurs when we try to bridge the propagator data points collected from lattice QCD with physical spectra [3].The spectral function is strongly related to transport process and non-equilibrium phenomena in heavy ion collisions [3].Moreover, the problem of rebuilding spectral function is not unique to strong interaction many-body systems, but have similar counterparts in quantum liquid and superconductivity [1].
In past two decades, the most common approach in such reconstruction project is the Bayesian Inference(BI) which contains many statistical inference methods, e.g., maximum entropy method (MEM).It routinely regularizes the inversion task through incorporating the prior knowledge from the physical domain about the spectral function [3].One direction of improvement is for example combining two of axioms, the scale invariance and proper constant form prior which builds the Bayesian Approach with Shannon-Jaynes entropy [4,5].Besides, recent studies have attempted to rebuild spectral functions using deep learning [6,7,8,9,10].In a supervised learning manner, the prior knowledge is encoded in amounts of prepared data and the inverse mapping is approximating through a training process [6,7,8].To alleviate the dependence of redundant training data, there are also studies adopting the radial basis functions and Gaussian process [10,11].

Unsupervised rebuilding spectral functions
In an unsupervised learning approach, we discuss the feasibility of rebuilding spectral functions from the Källen-Lehmann(KL) representation [12], The test of our framework will be demonstrated on spectral functions which are prepared using a superposed collection of Breit-Wigner peaks [6,13].Each individual Breit-Wigner spectral function is given by ρ where M is mass, Γ denotes its width and A sets the amplitude as a positive normalization constant.We assembled different single peaks to prepare multi-peak structure data.In the following parts, we show the vectorized form of our method, which can be easily implemented through differential programming in Pytorch or other modern deep learning frameworks.3 ω i to be summation over N ω points at fixed frequency interval dω, then it is suitable to the vectorized framework.The width of the l-th layer is n l , in which the correlation among discrete outputs is contained in a concealed form.The second form is demonstrated in Figure 1a), which sets input node as a 0 = ω i and single output node as a L = ρ i .It is termed as point-to-point neural networks (NN-P2P) as Figure 1c shown, in which the continuity of function ρ(ω) is a regularization defined in domains of input and output.

Automatic differentiation The output of above representations is
, from which we can calculate the correlator as where '⊙' represents element-wise product.After the forward process, we can get both ⃗ ρ and , where D i is observed data at p i with N p points.To optimize the parameters of presentations {θ} with loss function, we implement the backward propagation (BP).The gradients for layer-l is ∂L ∂θ [l] = ∆ [l] and the input for the BP is, With iteration loops in backward direction the gradients, ) can be used to optimize parameters {θ}, where '⊤' represents the transpose, θ [l] is weights matrix at layer-l, Z [l] is output of layer-l and σ(•) is the corresponding activation function.The components of the framework are differentiable and therefore amenable to gradient descent.Due to the feasibility of regularizers in neural network representations, the optimization makes use of the Adam algorithm [15].Besides the existing regularization of neural network itself, the only physical prior we enforce into the framework is the positive-definiteness of hadron spectral functions, which is introduced through using Softplus activation function f (x) = log(1 + e x ).More details about the training process can be found in our work [14].It should be mentioned that the biases induced by using gradient descent-type optimizers are not avoided in our framework, but it could be improved by embedding ensemble learning strategies.

Numerical results
We set two profiles of spectral functions as ground truths.Noise is added to the mock data as Di = D(p i ) + nε, with detailed settings found in Ref. [14].Fig. 2 compares reconstructions at different levels of relative noises: ε = 10 −3 , 10 −4 , and 10 −5 , respectively.Neural network representations show remarkable performances for a single peak at each noise level.The baseline results of MEM are also shown as green lines, which exhibit oscillations around the zero-point under different noise backgrounds.In contrast, the reconstructed spectral function from NN-P2P does not oscillate.Although NN-P2P does not outperform MEM in all cases, it offers an alternative for building specialized representations to explore properties such as transport coefficients.

Summary
We propose an automatic differentiation framework, a general tool for unfolding spectral functions from observable data.Spectral representations are employed using two distinct neural network structures, enabling the application of modern optimization algorithms.Although the inverse problems cannot be entirely resolved, the impressive performance of the reconstructed spectral functions demonstrates that this framework and the ability to introduce regularization are inherent advantages of this method.In recent work, we discussed the uniqueness of solutions in neural network representations [16].In future work, we will explore neural network representations, with one potential direction being the design of specific neural networks that adhere to physical rules.

Figure 1 .
Figure 1.a) Automatic differential framework designed to rebuild spectral functions from observations.Different representations for spectral functions with neural networks: b) NN.L-layers neural networks and the output is a list point on spectral functions.c) NN-P2P.L-layers neural networks and the end-to-end nodes are (ω i , ρ i ) pairwise.

Figure 2 .
Figure 2. The predicted spectral functions from MEM, NN and NN-P2P.The upper is from a single peak spectrum with A = 1.0,Γ = 0.5, M = 2.0 and the below one is from double peak profile with A 1 = 0.8, A 2 = 1.0,Γ 1 = Γ 2 = 0.5, M 1 = 2.0, M 2 = 5.0.From left to right panels, different Gaussian noises are added to the propagator data in the case of N p = 25, N ω = 500 [14].