Focus on ML and the Physical Sciences

In this work we demonstrate that significant gains in performance and data efficiency can be achieved in High Energy Physics (HEP) by moving beyond the standard paradigm of sequential optimization or reconstruction and analysis components. We conceptually connect HEP reconstruction and analysis to modern machine learning workflows such as pretraining, finetuning, domain adaptation and high-dimensional embedding spaces and quantify the gains in the example usecase of searches of heavy resonances decaying via an intermediate di-Higgs system to four b-jets. To our knowledge this is the first example of a low-level feature extraction network finetuned for a downstream HEP analysis objective.

A real-time detection of the plasma confinement regime can enable new advanced plasma control capabilities for both the access to and sustainment of enhanced confinement regimes in fusion devices. For example, a real-time indication of the confinement regime can facilitate transition to the high-performing wide-pedestal (WP) quiescent H-mode, or avoid unwanted transitions to lower confinement regimes that may induce plasma termination. To demonstrate real-time confinement regime detection, we use the 2D beam emission spectroscopy (BES) diagnostic system to capture localized density fluctuations of long wavelength turbulent modes in the edge region at a 1 MHz sampling rate. BES data from 330 discharges in either L-mode, H-mode, quiescent H (QH)-mode, or WP QH-mode were collected from the DIII-D tokamak and curated to develop a high-quality database to train a deep-learning classification model for real-time confinement detection. We utilize the 6×8 spatial configuration with a time window of 1024 µs and recast the input to obtain spectral-like features via fast Fourier transform preprocessing. We employ a shallow 3D convolutional neural network for the multivariate time-series classification task and utilize a softmax in the final dense layer to retrieve a probability distribution over the different confinement regimes. Our model classifies the global confinement state on 44 unseen test discharges with an average F₁ score of 0.94, using only ∼1 ms snippets of BES data at a time. This activity demonstrates the feasibility for real-time data analysis of fluctuation diagnostics in future devices such as ITER, where the need for reliable and advanced plasma control is urgent.

Three machine learning models are used to perform jet origin classification. These models are optimized for deployment on a field-programmable gate array device. In this context, we demonstrate how latency and resource consumption scale with the input size and choice of algorithm. Moreover, the models proposed here are designed to work on the type of data and under the foreseen conditions at the CERN large hadron collider during its high-luminosity phase. Through quantization-aware training and efficient synthetization for a specific field programmable gate array, we show that $\mathcal{O}(100)$ ns inference of complex architectures such as Deep Sets and Interaction Networks is feasible at a relatively low computational resource cost.

We present a variational approach aimed at enhancing the training of physics-informed neural networks (PINNs) and more general surrogate models for learning partial differential equations (PDE). In particular, we extend our formerly introduced notion of Sobolev cubatures to negative orders, enabling the approximation of negative order Sobolev norms. We mathematically prove the effect of negative order Sobolev cubatures in improving the condition number of discrete PDE learning problems, providing balancing scalars that mitigate numerical stiffness issues caused by loss imbalances. Additionally, we consider polynomial surrogate models (PSMs), which maintain the flexibility of PINN formulations while preserving the convexity structure of the PDE operators. The combination of negative order Sobolev cubatures and PSMs delivers well-conditioned discrete optimization problems, solvable via an exponentially fast convergent gradient descent for λ-convex losses. Our theoretical contributions are supported by numerical experiments, addressing linear and non-linear, forward and inverse PDE problems. These experiments show that the Sobolev cubature-based PSMs emerge as the superior state-of-the-art PINN technique.

Foundation models are multi-dataset and multi-task machine learning methods that once pre-trained can be fine-tuned for a large variety of downstream applications. The successful development of such general-purpose models for physics data would be a major breakthrough as they could improve the achievable physics performance while at the same time drastically reduce the required amount of training time and data. We report significant progress on this challenge on several fronts. First, a comprehensive set of evaluation methods is introduced to judge the quality of an encoding from physics data into a representation suitable for the autoregressive generation of particle jets with transformer architectures (the common backbone of foundation models). These measures motivate the choice of a higher-fidelity tokenization compared to previous works. Finally, we demonstrate transfer learning between an unsupervised problem (jet generation) and a classic supervised task (jet tagging) with our new OmniJet-α model. This is the first successful transfer between two different and actively studied classes of tasks and constitutes a major step in the building of foundation models for particle physics.

Finding low-energy atomic ordering in compositionally complex materials is one of the hardest problems in materials discovery, the solution of which can lead to breakthroughs in functional materials—from alloys to ceramics. In this work, we present the Artificial Structure Arranging Net (ArtiSAN)—a reinforcement learning agent utilizing graph representation that is trained to find low-energy atomic configurations of multicomponent systems through a series of atomic switch operations. ArtiSAN is trained on small alloy supercells ranging from binary to septenary. Strikingly, ArtiSAN generalizes to much larger systems of more than a thousand atoms, which are inaccessible with state-of-the-art methods due to the combinatorially larger search space. The performance of the current ArtiSAN agent is tested and deployed on several compositions that can be correlated with known experimental and high-fidelity computational structures. ArtiSAN demonstrates transfer across size and composition and finds physically meaningful structures using no energy evaluation calls once fully trained. While ArtiSAN will require further modifications to capture all variability in structure search, it is a remarkable step towards solving the structural part of the problem of disordered materials discovery.

Physically informed neural networks (PINNs) are a promising emerging method for solving differential equations. As in many other deep learning approaches, the choice of PINN design and training protocol requires careful craftsmanship. Here, we suggest a comprehensive theoretical framework that sheds light on this important problem. Leveraging an equivalence between infinitely over-parameterized neural networks and Gaussian process regression, we derive an integro-differential equation that governs PINN prediction in the large data-set limit—the neurally-informed equation. This equation augments the original one by a kernel term reflecting architecture choices. It allows quantifying implicit bias induced by the network via a spectral decomposition of the source term in the original differential equation.

We present JefiAtten, a novel neural network model employing the attention mechanism to solve Maxwell's equations efficiently. JefiAtten uses self-attention and cross-attention modules to understand the interplay between charge density, current density, and electromagnetic fields. Our results indicate that JefiAtten can generalize well to a range of scenarios, maintaining accuracy across various spatial distribution and handling amplitude variations. The model showcases an improvement in computation speed after training, compared to traditional integral methods. The adaptability of the model suggests potential for broader applications in computational physics, with further refinements to enhance its predictive capabilities and computational efficiency. Our work is a testament to the efficacy of integrating attention mechanisms with numerical simulations, marking a step forward in the quest for data-driven solutions to physical phenomena.

The high computational demand of the Density Functional Theory (DFT) based method for screening new materials properties remains a strong limitation to the development of clean and renewable energy technologies essential to transition to a carbon-neutral environment in the coming decades. Machine Learning comes into play with its innate capacity to handle huge amounts of data and high-dimensional statistical analysis. In this paper, supervised Machine Learning models together with data analysis on existing datasets obtained from a high-throughput calculation using Density Functional Theory are used to predict the Seebeck coefficient, electrical conductivity, and power factor of inorganic compounds. The analysis revealed a strong dependence of the thermoelectric properties on the effective masses, we also proposed a machine learning model for the prediction of highly performing thermoelectric materials which reached an efficiency of 95 percent. The analyzed data and developed model can significantly contribute to innovation by providing a faster and more accurate prediction of thermoelectric properties, thereby, facilitating the discovery of highly efficient thermoelectric materials.

We propose a support vector machine (SVM) based approach for generating an entanglement witness that requires exponentially less training data than previously proposed methods. SVMs generate hyperplanes represented by a weighted sum of expectation values of local observables whose coefficients are optimized to sum to a positive number for all separable states and a negative number for as many entangled states as possible near a specific target state. Previous SVM-based approaches for entanglement witness generation used large amounts of randomly generated separable states to perform training, a task with considerable computational overhead. Here, we propose a method for orienting the witness hyperplane using only the significantly smaller set of states consisting of the eigenstates of the generalized Pauli matrices and a set of entangled states near the target entangled states. With the orientation of the witness hyperplane set by the SVM, we tune the plane's placement using a differential program that ensures perfect classification accuracy on a limited test set as well as maximal noise tolerance. For N qubits, the SVM portion of this approach requires only $O(6^N)$ training states, whereas an existing method needs $O(2^{4^N})$. We use this method to construct witnesses of 4 and 5 qubit GHZ states with coefficients agreeing with stabilizer formalism witnesses to within 3.7 percent and 1 percent, respectively. We also use the same training states to generate novel 4 and 5 qubit W state witnesses. Finally, we computationally verify these witnesses on small test sets and propose methods for further verification.

Currently, mainstream methods for multi-fidelity data fusion have achieved great success in many fields, but they generally suffer from poor scalability. Therefore, this paper proposes a $C_3^2$ combination modeling method for complex multi-fidelity data fusion, devoted to solving the modeling problems with three types of multi-fidelity data fusion, and explores a general solution for any n types of multi-fidelity data fusion. Different from the traditional direct modeling method—Multi-Fidelity Deep Neural Network (MFDNN)—the $C_3^2$ method is an indirect modeling method. The experimental results on three representative benchmark functions and the prediction tasks of SG6043 airfoil aerodynamic performance show that $C_3^2$ combination modeling has the following advantages: (1) It can quickly establish the mapping relationship between high, medium, and low fidelity data. (2) It can effectively solve the data imbalance problem in multi-fidelity modeling. (3) Compared with MFDNN, it has stronger noise resistance and higher prediction accuracy. Additionally, this paper discusses the scalability problem of the $C_n^2$ method when n = 4 and n = 5, providing a reference for further research on the combined modeling method.

We pursue the use of deep learning methods to improve state-of-the-art computations in theoretical high-energy physics. Planar $\mathcal{N} = 4$ Super Yang–Mills theory is a close cousin to the theory that describes Higgs boson production at the Large Hadron Collider; its scattering amplitudes are large mathematical expressions containing integer coefficients. In this paper, we apply transformers to predict these coefficients. The problem can be formulated in a language-like representation amenable to standard cross-entropy training objectives. We design two related experiments and show that the model achieves high accuracy (${\gt}{98\%})$ on both tasks. Our work shows that transformers can be applied successfully to problems in theoretical physics that require exact solutions.

Gravitational lensing data is frequently collected at low resolution due to instrumental limitations and observing conditions. Machine learning-based super-resolution techniques offer a method to enhance the resolution of these images, enabling more precise measurements of lensing effects and a better understanding of the matter distribution in the lensing system. This enhancement can significantly improve our knowledge of the distribution of mass within the lensing galaxy and its environment, as well as the properties of the background source being lensed. Traditional super-resolution techniques typically learn a mapping function from lower-resolution to higher-resolution samples. However, these methods are often constrained by their dependence on optimizing a fixed distance function, which can result in the loss of intricate details crucial for astrophysical analysis. In this work, we introduce DiffLense, a novel super-resolution pipeline based on a conditional diffusion model specifically designed to enhance the resolution of gravitational lensing images obtained from the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP). Our approach adopts a generative model, leveraging the detailed structural information present in Hubble space telescope (HST) counterparts. The diffusion model, trained to generate HST data, is conditioned on HSC data pre-processed with denoising techniques and thresholding to significantly reduce noise and background interference. This process leads to a more distinct and less overlapping conditional distribution during the model's training phase. We demonstrate that DiffLense outperforms existing state-of-the-art single-image super-resolution techniques, particularly in retaining the fine details necessary for astrophysical analyses.

The study of phonon dynamics is pivotal for understanding material properties, yet it faces challenges due to the irreversible information loss inherent in powder inelastic neutron scattering spectra and the limitations of traditional analysis methods. In this study, we present a machine learning framework designed to reveal obscured phonon dynamics from powder spectra. Using a variational autoencoder, we obtain a disentangled latent representation of spectra and successfully extract force constants for reconstructing phonon dispersions. Notably, our model demonstrates effective applicability to experimental data even when trained exclusively on physics-based simulations. The fine-tuning with experimental spectra further mitigates issues arising from domain shift. Analysis of latent space underscores the model's versatility and generalizability, affirming its suitability for complex system applications. Furthermore, our framework's two-stage design is promising for developing a universal pre-trained feature extractor. This approach has the potential to revolutionize neutron measurements of phonon dynamics, offering researchers a potent tool to decipher intricate spectra and gain valuable insights into the intrinsic physics of materials.

Anomaly, or out-of-distribution, detection is a promising tool for aiding discoveries of new particles or processes in particle physics. In this work, we identify and address two overlooked opportunities to improve anomaly detection (AD) for high-energy physics. First, rather than train a generative model on the single most dominant background process, we build detection algorithms using representation learning from multiple background types, thus taking advantage of more information to improve estimation of what is relevant for detection. Second, we generalize decorrelation to the multi-background setting, thus directly enforcing a more complete definition of robustness for AD. We demonstrate the benefit of the proposed robust multi-background AD algorithms on a high-dimensional dataset of particle decays at the Large Hadron Collider.

The alpha magnetic spectrometer (AMS) is a high-precision particle detector onboard the International Space Station containing six different subdetectors. The transition radiation detector and electromagnetic calorimeter (ECAL) are used to separate electrons/positrons from the abundant cosmic-ray proton background. The positron flux measured in space by AMS falls with a power law which unexpectedly softens above 25 GeV and then hardens above 280 GeV. Several theoretical models try to explain these phenomena, and a more accurate measurement of positrons at higher energies is needed to help test them. The currently used methods to reject the proton background at high energies involve extrapolating shower features from the ECAL to use as inputs for boosted decision tree and likelihood classifiers. We present a new approach for particle identification with the AMS ECAL using deep learning (DL). By taking the energy deposition within all the ECAL cells as an input and treating them as pixels in an image-like format, we train an MLP, a CNN, and multiple ResNets and convolutional vision transformers (CvTs) as shower classifiers. Proton rejection performance is evaluated using Monte Carlo (MC) events and ISS data separately. For MC, using events with a reconstructed energy between 0.2–2 TeV, at 90% electron accuracy, the proton rejection power of our CvT model is more than five times that of the other DL models. Similarly, for ISS data with a reconstructed energy between 50–70 GeV, the proton rejection power of our CvT model is more than 2.5 times that of the other DL models.

Causality is innate to the determination of the fundamental mechanism controlling any physical phenomena. However, combining causality within the standard practices of computational modelling to understand structure-functionality connections is extremely rare. This work proposes a fingerprint based on key structural modes for ABO₃-type perovskite oxides and its derivatives, combined with causal models, for predicting Kohn–Sham energies. Our study of causal models captures the inherent coupling between structural modes such as rotation, tilt and antiferroelectric displacements, responsible for phase transition, polarization, magnetization and metal–insulator transition, exhibited by these materials. Although developed for modelling specific functionality, this method is universally applicable to derive other functionalities and even different material classes while tracking hidden causal mechanisms via structural distortions.

One of the main challenges in surrogate modeling is the limited availability of data due to resource constraints associated with computationally expensive simulations. Multi-fidelity methods provide a solution by chaining models in a hierarchy with increasing fidelity, associated with lower error, but increasing cost. In this paper, we compare different multi-fidelity methods employed in constructing Gaussian process surrogates for regression. Non-linear autoregressive methods in the existing literature are primarily confined to two-fidelity models, and we extend these methods to handle more than two levels of fidelity. Additionally, we propose enhancements for an existing method incorporating delay terms by introducing a structured kernel. We demonstrate the performance of these methods across various academic and real-world scenarios. Our findings reveal that multi-fidelity methods generally have a smaller prediction error for the same computational cost as compared to the single-fidelity method, although their effectiveness varies across different scenarios.

We develop a general theory of flows in the space of Riemannian metrics induced by neural network (NN) gradient descent. This is motivated in part by recent advances in approximating Calabi–Yau metrics with NNs and is enabled by recent advances in understanding flows in the space of NNs. We derive the corresponding metric flow equations, which are governed by a metric neural tangent kernel (NTK), a complicated, non-local object that evolves in time. However, many architectures admit an infinite-width limit in which the kernel becomes fixed and the dynamics simplify. Additional assumptions can induce locality in the flow, which allows for the realization of Perelman's formulation of Ricci flow that was used to resolve the 3d Poincaré conjecture. We demonstrate that such fixed kernel regimes lead to poor learning of numerical Calabi–Yau metrics, as is expected since the associated NNs do not learn features. Conversely, we demonstrate that well-learned numerical metrics at finite-width exhibit an evolving metric-NTK, associated with feature learning. Our theory of NN metric flows therefore explains why NNs are better at learning Calabi–Yau metrics than fixed kernel methods, such as the Ricci flow.

We report a gravitational-wave parameter estimation algorithm, AMPLFI, based on likelihood-free inference using normalizing flows. The focus of AMPLFI is to perform real-time parameter estimation for candidates detected by machine-learning based compact binary coalescence search, Aframe. We present details of our algorithm and optimizations done related to data-loading and pre-processing on accelerated hardware. We train our model using binary black-hole (BBH) simulations on real LIGO-Virgo detector noise. Our model has ${\sim}6$ million trainable parameters with training times ${\lesssim}24$ h. Based on online deployment on a mock data stream of LIGO-Virgo data, Aframe + AMPLFI is able to pick up BBH candidates and infer parameters for real-time alerts from data acquisition with a net latency of ${\sim}6$ s.

The observation of gravitational waves is expected to allow new tests of general relativity to be performed. As the gravitational wave signal is hidden by detector noise in observed data, a method to reduce noise is required to analyze the ringdown phase of gravitational wave signals. Recently, some noise reduction methods based on a neural network have been proposed; however, the results of these methods must be considered with caution because the output can contain spurious components. To overcome this limitation, in this study, we developed a neural network–based method to design optimal digital filters for extracting ringdown gravitational wave signals. In this method, no spurious components appear in the output because the digital filters reduce the noise. We conducted simulations with waveforms of gravitational waves from binary black hole coalescence and confirmed that the proposed method designs appropriate filters that reduce detector noise.

The accurate prediction of phase diagrams is of central importance for both the fundamental understanding of materials as well as for technological applications in material sciences. However, the computational prediction of the relative stability between phases based on their free energy is a daunting task, as traditional free energy estimators require a large amount of simulation data to obtain uncorrelated equilibrium samples over a grid of thermodynamic states. In this work, we develop deep generative machine learning models based on the Boltzmann Generator approach for entire phase diagrams, employing normalizing flows conditioned on the thermodynamic states, e.g. temperature and pressure, that they map to. By training a single normalizing flow to transform the equilibrium distribution sampled at only one reference thermodynamic state to a wide range of target temperatures and pressures, we can efficiently generate equilibrium samples across the entire phase diagram. Using a permutation-equivariant architecture allows us, thereby, to treat solid and liquid phases on the same footing. We demonstrate our approach by predicting the solid–liquid coexistence line for a Lennard-Jones system in excellent agreement with state-of-the-art free energy methods while significantly reducing the number of energy evaluations needed.

We present the development of a new algorithm which combines state-of-the-art energy-dispersive x-ray (EDX) spectroscopy theory and a suitable machine learning formulation for the hyperspectral unmixing of scanning transmission electron microscope EDX spectrum images. The algorithm is based on non-negative matrix factorization (NMF) incorporating a physics-guided factorization model. It optimizes a Poisson likelihood, under additional simplex constraint together with user-chosen sparsity-inducing and smoothing regularizations, and is based on iterative multiplicative updates. The fluorescence of x-rays is fully modeled thanks to state-of-the-art theoretical work. It is shown that the output of the algorithm can be used for a direct chemical quantification. With this approach, it is straightforward to include a priori knowledge on the specimen such as the presence or absence of certain chemical elements in some of its phases. This work is implemented within two open-source Python packages, espm and emtables, which are used here for data simulation, data analysis and quantification. Using simulated data, we demonstrate that incorporating physical modeling in the decomposition helps retrieve meaningful components from spatially and spectrally mixed phases, even when the data are very noisy. For synthetic data with a higher signal, the regularizations yield a tenfold increase in the quality of the reconstructed abundance maps compared to standard NMF. Our approach is further validated on experimental data with a known ground truth, where state-of-the art results are achieved by using prior knowledge about the sample. Our model can be generalized to any other scanning spectroscopy techniques where underlying physical modeling can be linearized.

We automate the process of machine learning correlations between knot invariants. For nearly 200 000 distinct sets of input knot invariants together with an output invariant, we attempt to learn the output invariant by training a neural network on the input invariants. Correlation between invariants is measured by the accuracy of the neural network prediction, and bipartite or tripartite correlations are sequentially filtered from the input invariant sets so that experiments with larger input sets are checking for true multipartite correlation. We rediscover several known relationships between polynomial, homological, and hyperbolic knot invariants, while also finding novel correlations which are not explained by known results in knot theory. These unexplained correlations strengthen previous observations concerning links between Khovanov and knot Floer homology. Our results also point to a new connection between quantum algebraic and hyperbolic invariants, similar to the generalized volume conjecture.

Interpreting neutron reflectivity (NR) data using ad hoc multi-layer models and physics-based models provides information about spatially resolved neutron scattering length density (NSLD) profiles. Recent improvements in data acquisition systems have allowed acquiring thousands of NR curves in a couple of hours, which has led to a need for automated data analysis tools to interpret NR measurements in real-time. Here, we present a machine learning analysis workflow that uses a series of models, based on a convolutional neural network (CNN), to learn the relation between the NSLDs and the NRs, and subsequently produce continuous NSLD profiles directly from NRs. The usefulness of our CNN-based models is demonstrated by constructing NSLDs from NRs of several films containing homopolymer polyzwitterions and diblock copolymers mixed with different types of salts. Comparisons of the NSLDs with those constructed using ad hoc multi-layer models reveal a very good agreement, suggesting the potential of CNN-based models for real-time automated data analysis of NRs.

This paper introduces a new neural network architecture designed to forecast high-dimensional spatio-temporal data using only sparse measurements. The architecture uses a two-stage end-to-end framework that combines neural ordinary differential equations (NODEs) with vision transformers. Initially, our approach models the underlying dynamics of complex systems within a low-dimensional space; and then it reconstructs the corresponding high-dimensional spatial fields. Many traditional methods involve decoding high-dimensional spatial fields before modeling the dynamics, while some other methods use an encoder to transition from high-dimensional observations to a latent space for dynamic modeling. In contrast, our approach directly uses sparse measurements to model the dynamics, bypassing the need for an encoder. This direct approach simplifies the modeling process, reduces computational complexity, and enhances the efficiency and scalability of the method for large datasets. We demonstrate the effectiveness of our framework through applications to various spatio-temporal systems, including fluid flows and global weather patterns. Although sparse measurements have limitations, our experiments reveal that they are sufficient to forecast system dynamics accurately over long time horizons. Our results also indicate that the performance of our proposed method remains robust across different sensor placement strategies, with further improvements as the number of sensors increases. This robustness underscores the flexibility of our architecture, particularly in real-world scenarios where sensor data is often sparse and unevenly distributed.

A common setting in astronomy is the availability of a small number of high-quality observations, and larger amounts of either lower-quality observations or synthetic data from simplified models. Time-domain astrophysics is a canonical example of this imbalance, with the number of supernovae observed photometrically outpacing the number observed spectroscopically by multiple orders of magnitude. At the same time, no data-driven models exist to understand these photometric and spectroscopic observables in a common context. Contrastive learning objectives, which have grown in popularity for aligning distinct data modalities in a shared embedding space, provide a potential solution to extract information from these modalities. We present Maven, the first foundation model for supernova science. To construct Maven, we first pre-train our model to align photometry and spectroscopy from 0.5 M synthetic supernovae using a contrastive objective. We then fine-tune the model on 4702 observed supernovae from the Zwicky transient facility. Maven reaches state-of-the-art performance on both classification and redshift estimation, despite the embeddings not being explicitly optimized for these tasks. Through ablation studies, we show that pre-training with synthetic data improves overall performance. In the upcoming era of the Vera C. Rubin observatory, Maven will serve as a valuable tool for leveraging large, unlabeled and multimodal time-domain datasets.

The ability to model and sample from conditional densities is important in many physics applications. Implicit quantile networks (IQN) have been successfully applied to this task in domains outside physics. In this work, we illustrate the potential of IQNs as components of emulators using the simulation of jets as an example. Specifically, we use an IQN to map jets described by their 4-momenta at the generation level to jets at the event reconstruction level. The conditional densities emulated by our model closely match those generated by Delphes, while also enabling faster jet simulation.

Ultrafast pulse characterisation is crucial for studying processes that occur at femtosecond timescales and below. Because of this, various methods have been developed to recover a pulse's electric field profile at these durations, with the frequency-resolved optical gating (FROG) technique being the most common. However, this approach is computationally expensive and suffers from limitations in terms of robustness and reliability. In this regard, recent publications have demonstrated that applying machine learning towards ultrafast pulse recovery can alleviate these issues, providing more accurate retrievals. Inspired by these works, we propose an encoder–decoder scheme for a FROG system which exploits dual harmonic generation in low-index thin films. Specifically, we demonstrate enhanced reliability and accuracy of ultrafast pulse recovery when compared to machine learning approaches using second or third harmonic signals independently. As the amount of information used to train each neural network is kept constant, this study demonstrates and benchmarks the technological advantages of contextual information analysis involving multiple nonlinear processes.

Theoretical approaches to quantum many-body physics require developing compact representations of the complexity of generic quantum states. This paper explores an interpretable data-driven approach utilizing principal component analysis (PCA) and autoencoder neural networks to compress the two-particle vertex, a key element in Feynman diagram approaches. We show that the linear PCA offers more physical insight and better out-of-distribution generalization than the nominally more expressive autoencoders. Even with ∼10–20 principal components, we find excellent reconstruction across the phase diagram suggesting the existence of heretofore unrealized structures in the diagrammatic theory. We show that the principal components needed to describe the ferromagnetic state are not contained in the low rank description of the Fermi liquid (FL) state, unlike those for antiferromagnetic and superconducting states, suggesting that the latter two states emerge from pre-existing fluctuations in the FL while ferromagnetism is driven by a different process.

We present methods to estimate systematic uncertainties in unbinned large hadron collider (LHC) data analyses, focusing on constraining Wilson coefficients in the standard model effective field theory (SMEFT). Our approach also applies to broader parametric models of non-resonant phenomena beyond the standard model. By using machine-learned surrogates of the likelihood ratio, we extend well-established procedures from binned Poisson counting experiments to the unbinned case. This framework handles various theoretical, modeling, and experimental uncertainties, laying the foundation for future unbinned analyses at the LHC. We also introduce a tree-boosting algorithm that learns precise parametrizations of systematic effects, providing a robust, flexible alternative to neural networks for modeling systematics. We demonstrate this approach with an SMEFT analysis of highly energetic top quark pair production in proton–proton collisions.

The Koopman operator framework can be used to identify a data-driven model of a nonlinear system. Unfortunately, when the data is corrupted by noise, the identified model can be biased. Additionally, depending on the choice of lifting functions, the identified model can be unstable, even when the underlying system is asymptotically stable. This paper presents an approach to reduce the bias in an approximate Koopman model, and simultaneously ensure asymptotic stability, when using noisy data. Additionally, the proposed data-driven modeling approach is applicable to systems with inputs, such as a known forcing function or a control input. Specifically, bias is reduced by using a total least-squares, modified to accommodate inputs in addition to lifted inputs. To enforce asymptotic stability of the approximate Koopman model, linear matrix inequality constraints are augmented to the identification problem. The performance of the proposed method is then compared to the well-known extended dynamic mode decomposition (DMD) method and to the newly introduced forward–backward extended DMD method using a simulated Duffing oscillator dataset and experimental soft robot arm dataset.

Dynamics of coarse-grained particle systems derived via the Mori–Zwanzig projection formalism commonly take the form of a (generalized) Langevin equation with configuration-dependent friction tensor and diffusion coefficient matrix. In this article, we introduce a class of equivariant representations of tensor-valued functions based on the Atomic Cluster Expansion framework that allows for efficient learning of such configuration-dependent friction tensors from data. Besides satisfying the correct equivariance properties with respect to the Euclidean group E(3), the resulting heat bath models satisfy a fluctuation-dissipation relation. We demonstrate the capabilities of the model approach by fitting a model of configuration-dependent tensorial electronic friction calculated from first principles that arises during reactive molecular dynamics at metal surfaces.

Optimizing parameters of physics-based simulators is crucial in the design process of engineering and scientific systems. This becomes particularly challenging when the simulator is stochastic, computationally expensive, black-box and when a high-dimensional vector of parameters needs to be optimized, as e.g. is the case in complex climate models that involve numerous interdependent variables and uncertain parameters. Many traditional optimization methods rely on gradient information, which is frequently unavailable in legacy black-box codes. To address these challenges, we present SCOUT-Nd (Stochastic Constrained Optimization for N dimensions), a gradient-based algorithm that can be used on non-differentiable objectives. It can be combined with natural gradients in order to further enhance convergence properties. and it also incorporates multi-fidelity schemes and an adaptive selection of samples in order to minimize computational effort. We validate our approach using standard, benchmark problems, demonstrating its superior performance in parameter optimization compared to existing methods. Additionally, we showcase the algorithm's efficacy in a complex real-world application, i.e. the optimization of a wind farm layout.

This paper introduces an innovative method for fine-tuning a large multi-label model for abnormality detection, utilizing a smaller trainer and advanced knowledge distillation techniques. It studies the effects of fine-tuning on various abnormalities, noting different improvements based on the Original Model's performance in specific tasks. The experimental setup, optimized for on-device inference and fine-tuning with limited computational resources, demonstrates moderate yet promising enhancements in model performance post-fine-tuning. Key insights from the study include the significance of aligning the µ-Trainer's behavior with the Original Model and the influence of hyper-parameters like batch size on fine-tuning outcomes. The research acknowledges limitations such as the limited exploration of loss functions in multi-label models and constraints in architectural design, suggesting potential avenues for future investigation. While the proposed Naive Continual Fine-tuning Process is in its early stages, we highlight this paper's potential model personalization on long-term data. Moreover, weight transfer in our system is exclusively for fine-tuning; hence, it improves user privacy protection by failing data reconstruction attempts from weights, like an issue with Federated learning models. Our on-device fine-tuning prevents the transferring of data or gradients from the edge of the network to their server. Despite modest performance improvements after fine-tuning, these working layers represent a small fraction (0.7%) of the total weights in the Original Model and 1.6% in the µ-Trainer. This study establishes a foundational framework for advancing personalized model adaptation, on-device inference and fine-tuning while emphasizing the importance of safeguarding data privacy in model development.

Multi-dimensional parameter spaces are commonly encountered in physics theories that go beyond the standard model. However, they often possess complicated posterior geometries that are expensive to traverse using techniques traditional to astroparticle physics. Several recent innovations, which are only beginning to make their way into this field, have made navigating such complex posteriors possible. These include GPU acceleration, automatic differentiation, and neural-network-guided reparameterization. We apply these advancements to dark matter direct detection experiments in the context of non-standard neutrino interactions and benchmark their performances against traditional nested sampling techniques when conducting Bayesian inference. Compared to nested sampling alone, we find that these techniques increase performance for both nested sampling and Hamiltonian Monte Carlo, accelerating inference by factors of $\mathord{\sim} 100$ and $\mathord{\sim} 60$, respectively. As nested sampling also evaluates the Bayesian evidence, these advancements can be exploited to improve model comparison performance while retaining compatibility with existing implementations that are widely used in the natural sciences. Using these techniques, we perform the first scan in the neutrino non-standard interactions parameter space for direct detection experiments whereby all parameters are allowed to vary simultaneously. We expect that these advancements are broadly applicable to other areas of astroparticle physics featuring multi-dimensional parameter spaces.

We present an efficient tensor-network-based approach for simulating large-scale quantum circuits exemplified by quantum support vector machines (QSVMs). Experimentally, leveraging the cuTensorNet library on multiple GPUs, our method effectively reduces the exponential runtime growth to near-quadratic scaling with respect to the number of qubits in practical scenarios. Traditional state-vector simulations become computationally infeasible beyond approximately 50 qubits; in contrast, our simulator successfully handles QSVMs with up to 784 qubits, executing simulations within seconds on a single high-performance GPU. Furthermore, utilizing the message passing interface for multi-GPU environments, our method demonstrates strong linear scalability, effectively decreasing computation time as dataset sizes increase. We validate our framework using the MNIST and Fashion MNIST datasets, achieving successful multiclass classification and highlighting the potential of QSVMs for high-dimensional data analysis. By integrating tensor-network techniques with advanced high-performance computing resources, this work demonstrates both the feasibility and scalability of simulating large-qubit quantum machine learning models, providing a valuable validation tool within the emerging Quantum-HPC ecosystem.

We present new invariant machine learning models that approximate the Ricci-flat metric on Calabi–Yau (CY) manifolds with discrete symmetries. We accomplish this by combining state of the art models for predicting such metrics, based on the so-called φ-model of the cymetric package, with non-trainable, G-invariant, canonicalization layers that project the φ-model's input data (i.e. points sampled from the CY geometry) to the fundamental domain of a given symmetry group G. These G-invariant layers are easy to concatenate, provided one compatibility condition is fulfilled, and combine well with both standard and spectral versions of the φ-model. Through experiments on different CY geometries, we find that, for fixed point sample size and training time, canonicalized models give slightly more accurate metric approximations than the standard φ-model. On highly symmetric spaces, we also observe significantly faster convergence upon training. The method may also be used to compute the Ricci-flat metric on smooth CY quotients. We demonstrate this aspect by experiments on a smooth $\mathbb{Z}^2_5$ quotient of a 5-parameter quintic CY manifold.

Determining the universality class of reaction–diffusion processes with long-range interactions in non-equilibrium phase transitions is both challenging and intriguing. Identifying critical points is fundamental for studying the phase transition characteristics of these universality classes. Unlike Monte Carlo simulations of statistical system observables, machine learning methods can extract evolutionary information from clusters of such systems, enabling a faster approach to phase transition regions. We developed a method that uses the one-dimensional encoding output of a stacked autoencoder (SAE) to determine the critical point in systems undergoing (1+1)-dimensional directed percolation with spatial long-range interactions. We validate this method by examining the power-law behavior of particle density at the critical point, which strongly supports our approach. As the system adheres to the scaling relation $t_f{\sim}L^{z}$ at the critical point, we conducted extensive simulations at this critical probability to determine the dynamic exponent z. In addition, the SAE is also capable of identifying the characteristic time of critical states. Finally, we tested two other heavy-tailed distributions that generate random step lengths: the Lévy distribution and the Cauchy distribution, which introduce different global spreading mechanisms. This method remains effective for determining the critical points in these systems. Our findings highlight the promising applications of SAE techniques for processes involving long-range interactions.

We propose a novel tomographic method, nonlinear Gaussian process tomography (nonlinear GPT), that uses the Laplace approximation to impose constraints on non-negative physical quantities, such as the emissivity in plasma optical diagnostics. While positive-valued posteriors have previously been introduced through sampling-based approaches in the original GPT method, our alternative approach implements a logarithmic Gaussian process (log-GP) for faster computation and more natural enforcement of non-negativity. The effectiveness of the proposed log-GP tomography is demonstrated through a case study using the Ring Trap 1 device, where log-GPT outperforms existing methods, standard GPT, and the minimum Fisher information methods in terms of reconstruction accuracy. The results highlight the effectiveness of nonlinear GPT for imposing physical constraints in applications to an inverse problem.

Correctly capturing the transition to turbulence in a barotropic instability requires fine spatial resolution. To reduce computational cost, we propose a dynamic super-resolution approach where a transient simulation on a coarse mesh is frequently corrected using a U-net-type neural network. For the nonlinear shallow water equations, we demonstrate that a simulation with the Icosahedral Nonhydrostatic ocean model with a 20 km resolution plus dynamic super-resolution trained on a 2.5km resolution achieves discretization errors comparable to a simulation with 10 km resolution. The neural network, originally developed for image-based super-resolution in post-processing, is trained to compute the difference between solutions on both meshes and is used to correct the coarse mesh solution every 12 h. We show that the ML-corrected coarse solution correctly maintains a balanced flow and captures the transition to turbulence in line with the higher resolution simulation. After an 8 d simulation, the L₂-error of the corrected run is similar to a simulation run on a finer mesh. While mass is conserved in the corrected runs, we observe some spurious generation of kinetic energy.

Giving up and starting over may seem wasteful in many situations such as searching for a target or training deep neural networks (DNNs). Our study, though, demonstrates that resetting from a checkpoint can significantly improve generalization performance when training DNNs with noisy labels. In the presence of noisy labels, DNNs initially learn the general patterns of the data but then gradually memorize the corrupted data, leading to overfitting. By deconstructing the dynamics of stochastic gradient descent (SGD), we identify the behavior of a latent gradient bias induced by noisy labels, which harms generalization. To mitigate this negative effect, we apply the stochastic resetting method to SGD, inspired by recent developments in the field of statistical physics achieving efficient target searches. We first theoretically identify the conditions where resetting becomes beneficial, and then we empirically validate our theory, confirming the significant improvements achieved by resetting. We further demonstrate that our method is both easy to implement and compatible with other methods for handling noisy labels. Additionally, this work offers insights into the learning dynamics of DNNs from an interpretability perspective, expanding the potential to analyze training methods through the lens of statistical physics.

Neutrinoless double-beta decay ($0\nu\beta\beta$) is a rare nuclear process that, if observed, will provide insight into the nature of neutrinos and help explain the matter-antimatter asymmetry in the Universe. The large enriched germanium experiment for neutrinoless double-beta decay (LEGEND) will operate in two phases to search for $0\nu\beta\beta$. The first (second) stage will employ 200 (1000) kg of High-Purity Germanium (HPGe) enriched in ⁷⁶Ge to achieve a half-life sensitivity of 10²⁷ (10²⁸) years. In this study, we present a semi-supervised data-driven approach to remove non-physical events captured by HPGe detectors powered by a novel artificial intelligence model. We utilize affinity propagation to cluster waveform signals based on their shape and a support vector machine to classify them into different categories. We train, optimize, and test our model on data taken from a natural abundance HPGe detector installed in the Full Chain Test experimental stand at the University of North Carolina at Chapel Hill. We demonstrate that our model yields a maximum sacrifice of physics events of $0.024 ^{+0.004}_{-0.003} \%$ after data cleaning. Our model is being used to accelerate data cleaning development for LEGEND-200 and will serve to improve data cleaning procedures for LEGEND-1000.

Improving the prediction accuracy and stability of non-resonant high-order harmonics energetic particle modes (EPMs) is crucial for achieving tokamak plasmas steady state operation. This study adopts a stacking ensemble learning model for the EPMs prediction. This ensemble model has a 2-layer structure with the base learner and the meta-learner: the first layer, base learners include K-nearest neighbor regression, Extreme Gradient Boosting, gradient boosting regression (GBR), decision tree (DT) and support vector regression (SVR); the second layer, meta-learners output the final result via GBR. The developed stacking model is designed for multi-objective prediction, including the grow rate (γ) and mode frequency (ω) of the EPMs. The evaluation results indicate that the performance of the proposed model surpasses most supervised learning algorithms. Specifically, in comparison with the SVR and Bagging algorithms, the growth rate predictions of stacking model reduces Root mean squared error (RMSE) by 45% and 33%, mean absolute error (MAE) by 47% and 32%, and increases the R-squared coefficient (R²) by 5% and 3%, respectively. Similarly, for the mode frequency predictions, RMSE decreases by 34% and 65%, MAE by 23% and 50%, while R² improves by 2% and 7%, respectively. In terms of computational cost, the stacking model can conserve more time and expense than kinetic-hybrid simulations of EPMs. Therefore, the proposed model can be well applied to the prediction of the high-energy particle instability phenomenon in tokamak reactors.