Characterisation of Physical Processes from Anomalous Diffusion Data

Recently, a large number of research teams from around the world collaborated in the so-called 'anomalous diffusion challenge'. Its aim: to develop and compare new techniques for inferring stochastic models from given unknown time series, and estimate the anomalous diffusion exponent in data. We use various numerical methods to directly obtain this exponent using the path increments, and develop a questionnaire for model selection based on feature analysis of a set of known stochastic processes given as candidates. Here, we present the theoretical background of the automated algorithm which we put for these tasks in the diffusion challenge, as a counter to other pure data-driven approaches.

The results of the Anomalous Diffusion Challenge (AnDi Challenge) (Muñoz-Gil G et al 2021 Nat. Commun. 12 6253) have shown that machine learning methods can outperform classical statistical methodology at the characterization of anomalous diffusion in both the inference of the anomalous diffusion exponent α associated with each trajectory (Task 1), and the determination of the underlying diffusive regime which produced such trajectories (Task 2). Furthermore, of the five teams that finished in the top three across both tasks of the AnDi Challenge, three of those teams used recurrent neural networks (RNNs). While RNNs, like the long short-term memory network, are effective at learning long-term dependencies in sequential data, their key disadvantage is that they must be trained sequentially. In order to facilitate training with larger data sets, by training in parallel, we propose a new transformer based neural network architecture for the characterization of anomalous diffusion. Our new architecture, the Convolutional Transformer (ConvTransformer) uses a bi-layered convolutional neural network to extract features from our diffusive trajectories that can be thought of as being words in a sentence. These features are then fed to two transformer encoding blocks that perform either regression (Task 1 1D) or classification (Task 2 1D). To our knowledge, this is the first time transformers have been used for characterizing anomalous diffusion. Moreover, this may be the first time that a transformer encoding block has been used with a convolutional neural network and without the need for a transformer decoding block or positional encoding. Apart from being able to train in parallel, we show that the ConvTransformer is able to outperform the previous state of the art at determining the underlying diffusive regime (Task 2 1D) in short trajectories (length 10–50 steps), which are the most important for experimental researchers.

The inverse scattering transform allows explicit construction of solutions to many physically significant nonlinear wave equations. Notably, this method can be extended to fractional nonlinear evolution equations characterized by anomalous dispersion using completeness of suitable eigenfunctions of the associated linear scattering problem. In anomalous diffusion, the mean squared displacement is proportional to t^α, α > 0, while in anomalous dispersion, the speed of localized waves is proportional to A^α, where A is the amplitude of the wave. Fractional extensions of the modified Korteweg–deVries (mKdV), sine-Gordon (sineG) and sinh-Gordon (sinhG) and associated hierarchies are obtained. Using symmetries present in the linear scattering problem, these equations can be connected with a scalar family of nonlinear evolution equations of which fractional mKdV (fmKdV), fractional sineG (fsineG), and fractional sinhG (fsinhG) are special cases. Completeness of solutions to the scalar problem is obtained and, from this, the nonlinear evolution equation is characterized in terms of a spectral expansion. In particular, fmKdV, fsineG, and fsinhG are explicitly written. One-soliton solutions are derived for fmKdV and fsineG using the inverse scattering transform and these solitons are shown to exhibit anomalous dispersion.

Three-dimensional Monte Carlo simulations provide a striking confirmation to a recent theoretical prediction: the Brownian non-Gaussian diffusion of critical self-avoiding walks. Although the mean square displacement of the polymer center of mass grows linearly with time (Brownian behavior), the initial probability density function is strongly non-Gaussian and crosses over to Gaussianity only at large time. Full agreement between theory and simulations is achieved without the employment of fitting parameters. We discuss simulation techniques potentially capable of addressing the study of anomalous diffusion under complex conditions like adsorption- or Theta-transition.

The field of movement ecology has seen a rapid increase in high-resolution data in recent years, leading to the development of numerous statistical and numerical methods to analyse relocation trajectories. Data are often collected at the level of the individual and for long periods that may encompass a range of behaviours. Here, we use the power spectral density (PSD) to characterise the random movement patterns of a black-winged kite (Elanus caeruleus) and a white stork (Ciconia ciconia). The tracks are first segmented and clustered into different behaviours (movement modes), and for each mode we measure the PSD and the ageing properties of the process. For the foraging kite we find 1/f noise, previously reported in ecological systems mainly in the context of population dynamics, but not for movement data. We further suggest plausible models for each of the behavioural modes by comparing both the measured PSD exponents and the distribution of the single-trajectory PSD to known theoretical results and simulations.

Stochastic processes (SPs) appear in a wide field, such as ecology, biology, chemistry, and computer science. In transport dynamics, deviations from Brownian motion leading to anomalous diffusion (AnDi) are found, including transport mechanisms, cellular organization, signaling, and more. For various reasons, identifying AnDi is still challenging; for example, (i) a system can have different physical processes running simultaneously, (ii) the analysis of the mean-squared displacements (MSDs) of the diffusing particles is used to distinguish between normal diffusion and AnDi. However, MSD calculations are not very informative because different models can yield curves with the same scaling exponent. Recently, proposals have suggested several new approaches. The majority of these are based on the machine learning (ML) revolution. This paper is based on ML algorithms known as the convolutional neural network to classify SPs. To do this, we generated the dataset from published paper codes for 12 SPs. We use a pre-trained model, the ResNet-50, to automatically classify the dataset. Accuracy of 99% has been achieved by running the ResNet-50 model on the dataset. We also show the comparison of the Resnet18 and GoogleNet models with the ResNet-50 model. The ResNet-50 model outperforms these models in terms of classification accuracy.

Understanding and identifying different types of single molecules' diffusion that occur in a broad range of systems (including living matter) is extremely important, as it can provide information on the physical and chemical characteristics of particles' surroundings. In recent years, an ever-growing number of methods have been proposed to overcome some of the limitations of the mean-squared displacements approach to tracer diffusion. In March 2020, the anomalous diffusion (AnDi) challenge was launched by a community of international scientists to provide a framework for an objective comparison of the available methods for AnDi. In this paper, we introduce a feature-based machine learning method developed in response to task 2 of the challenge, i.e. the classification of different types of diffusion. We discuss two sets of attributes that may be used for the classification of single-particle tracking data. The first one was proposed as our contribution to the AnDi challenge. The latter is the result of our attempt to improve the performance of the classifier after the deadline of the competition. Extreme gradient boosting was used as the classification model. Although the deep-learning approach constitutes the state-of-the-art technology for data classification in many domains, we deliberately decided to pick this traditional machine learning algorithm due to its superior interpretability. After the extension of the feature set our classifier achieved the accuracy of 0.83, which is comparable with the top methods based on neural networks.

Quantum walks are known to propagate quadratically faster than their classical counterparts and are used to model dynamics in various quantum systems. The spread of the quantum walk in position space shows anomalous diffusion behavior. By controlling the action of quantum coin operation on the corresponding coin degree of freedom of the walker, one can demonstrate control over the diffusion behavior. In this work, we report different forms of coin operations on quantum walks exhibiting anomalous diffusion behavior. Homogeneous and accelerated quantum walks display superdiffusive behavior, whereas uncorrelated static and dynamic disorders in the evolution induce strong and weak localization of the particle indicating subdiffusive and normal diffusive behavior. The role played by the interference effects in the spreading of the walker has remained elusive and our aim in this work is to present the interplay between quantum coherence and mean squared displacement of the walker. We employ two reliable measures of coherence for conclusively establishing the role of quantum interference as the driving force behind the anomalous diffusive behavior in the dynamics of quantum walks.

We show through intensive simulations that the paradigmatic features of anomalous diffusion are indeed the features of a (continuous-time) random walk driven by two different Markovian hopping-trap mechanisms. If p ∈ (0, 1/2) and 1 − p are the probabilities of occurrence of each Markovian mechanism, then the anomalousness parameter β ∈ (0, 1) results to be β ≃ 1 − 1/{1 + log[(1 − p)/p]}. Ensemble and single-particle observables of this model have been studied and they match the main characteristics of anomalous diffusion as they are typically measured in living systems. In particular, the celebrated transition of the walker's distribution from exponential to stretched-exponential and finally to Gaussian distribution is displayed by including also the Brownian yet non-Gaussian interval.

We present a Bayesian inference scheme for scaled Brownian motion, and investigate its performance on synthetic data for parameter estimation and model selection in a combined inference with fractional Brownian motion. We include the possibility of measurement noise in both models. We find that for trajectories of a few hundred time points the procedure is able to resolve well the true model and parameters. Using the prior of the synthetic data generation process also for the inference, the approach is optimal based on decision theory. We include a comparison with inference using a prior different from the data generating one.

We present here the autoregressive tempered fractionally integrated moving average (ARTFIMA) process obtained by taking the tempered fractional difference operator of the non-Gaussian stable noise. The tempering parameter makes the ARTFIMA process stationary for a wider range of the memory parameter values than for the classical autoregressive fractionally integrated moving average, and leads to semi-long range dependence and transient anomalous behavior. We investigate ARTFIMA dependence structure with stable noise and construct Whittle estimators. We also introduce the stable Yaglom noise as a continuous version of the ARTFIMA model with stable noise. Finally, we illustrate the usefulness of the ARTFIMA process on a trajectory from the Golding and Cox experiment.

The characterization of diffusion processes is a keystone in our understanding of a variety of physical phenomena. Many of these deviate from Brownian motion, giving rise to anomalous diffusion. Various theoretical models exists nowadays to describe such processes, but their application to experimental setups is often challenging, due to the stochastic nature of the phenomena and the difficulty to harness reliable data. The latter often consists on short and noisy trajectories, which are hard to characterize with usual statistical approaches. In recent years, we have witnessed an impressive effort to bridge theory and experiments by means of supervised machine learning techniques, with astonishing results. In this work, we explore the use of unsupervised methods in anomalous diffusion data. We show that the main diffusion characteristics can be learnt without the need of any labelling of the data. We use such method to discriminate between anomalous diffusion models and extract their physical parameters. Moreover, we explore the feasibility of finding novel types of diffusion, in this case represented by compositions of existing diffusion models. At last, we showcase the use of the method in experimental data and demonstrate its advantages for cases where supervised learning is not applicable.

Anomalous diffusion occurs at very different scales in nature, from atomic systems to motions in cell organelles, biological tissues or ecology, and also in artificial materials, such as cement. Being able to accurately measure the anomalous exponent associated to a given particle trajectory, thus determining whether the particle subdiffuses, superdiffuses or performs normal diffusion, is of key importance to understand the diffusion process. Also it is often important to trustingly identify the model behind the trajectory, as it this gives a large amount of information on the system dynamics. Both aspects are particularly difficult when the input data are short and noisy trajectories. It is even more difficult if one cannot guarantee that the trajectories output in experiments are homogeneous, hindering the statistical methods based on ensembles of trajectories. We present a data-driven method able to infer the anomalous exponent and to identify the type of anomalous diffusion process behind single, noisy and short trajectories, with good accuracy. This model was used in our participation in the anomalous diffusion (AnDi) challenge. A combination of convolutional and recurrent neural networks was used to achieve state-of-the-art results when compared to methods participating in the AnDi challenge, ranking top 4 in both classification and diffusion exponent regression.

The Lévy walk (LW) is a non-Brownian random walk model that has been found to describe anomalous dynamic phenomena in diverse fields ranging from biology over quantum physics to ecology. Recurrently occurring problems are to examine whether observed data are successfully quantified by a model classified as LWs or not and extract the best model parameters in accordance with the data. Motivated by such needs, we propose a hidden Markov model for LWs and computationally realize and test the corresponding Bayesian inference method. We introduce a Markovian decomposition scheme to approximate a renewal process governed by a power-law waiting time distribution. Using this, we construct the likelihood function of LWs based on a hidden Markov model and the forward algorithm. With the LW trajectories simulated at various conditions, we perform the Bayesian inference for parameter estimation and model classification. We show that the power-law exponent of the flight-time distribution can be successfully extracted even at the condition that the mean-squared displacement does not display the expected scaling exponent due to the noise or insufficient trajectory length. It is also demonstrated that the Bayesian method performs remarkably inferring the LW trajectories from given unclassified trajectory data set if the noise level is moderate.

Anomalous diffusion, which shows a deviation of transport dynamics from the framework of standard Brownian motion, is involved in the evolution of various physical, chemical, biological, and economic systems. The study of such random processes is of fundamental importance in unveiling the physical properties of random walkers and complex systems. However, classical methods to characterize anomalous diffusion are often disqualified for individual short trajectories, leading to the launch of the anomalous diffusion (AnDi) challenge. This challenge aims at objectively assessing and comparing new approaches for single trajectory characterization, with respect to three different aspects: the inference of the anomalous diffusion exponent; the classification of the diffusion model; and the segmentation of trajectories. In this article, to address the inference and classification tasks in the challenge, we develop a WaveNet-based deep neural network (WADNet) by combining a modified WaveNet encoder with long short-term memory networks, without any prior knowledge of anomalous diffusion. As the performance of our model has surpassed the current 1st places in the challenge leaderboard on both two tasks for all dimensions (6 subtasks), WADNet could be part of state-of-the-art techniques to decode the AnDi database. Our method presents a benchmark for future research, and could accelerate the development of a versatile tool for the characterization of anomalous diffusion.

The study of the dynamics of natural and artificial systems has provided several examples of deviations from Brownian behavior, generally defined as anomalous diffusion. The investigation of these dynamics can provide a better understanding of diffusing objects and their surrounding media, but a quantitative characterization from individual trajectories is often challenging. Efforts devoted to improving anomalous diffusion detection using classical statistics and machine learning have produced several new methods. Recently, the anomalous diffusion challenge (AnDi, www.andi-challenge.org) was launched to objectively assess these approaches on a common dataset, focusing on three aspects of anomalous diffusion: the inference of the anomalous diffusion exponent; the classification of the diffusion model; and the segmentation of trajectories. In this article, I describe a simple approach to tackle the tasks of the AnDi challenge by combining extreme learning machine and feature engineering (AnDi-ELM). The method reaches satisfactory performance while offering a straightforward implementation and fast training time with limited computing resources, making it a suitable tool for fast preliminary screening of anomalous diffusion.

Diffusion processes are important in several physical, chemical, biological and human phenomena. Examples include molecular encounters in reactions, cellular signalling, the foraging of animals, the spread of diseases, as well as trends in financial markets and climate records. Deviations from Brownian diffusion, known as anomalous diffusion (AnDi), can often be observed in these processes, when the growth of the mean square displacement in time is not linear. An ever-increasing number of methods has thus appeared to characterize anomalous diffusion trajectories based on classical statistics or machine learning approaches. Yet, characterization of anomalous diffusion remains challenging to date as testified by the launch of the AnDi challenge in March 2020 to assess and compare new and pre-existing methods on three different aspects of the problem: the inference of the anomalous diffusion exponent, the classification of the diffusion model, and the segmentation of trajectories. Here, we introduce a novel method (CONDOR) which combines feature engineering based on classical statistics with supervised deep learning to efficiently identify the underlying anomalous diffusion model with high accuracy and infer its exponent with a small mean absolute error in single 1D, 2D and 3D trajectories corrupted by localization noise. Finally, we extend our method to the segmentation of trajectories where the diffusion model and/or its anomalous exponent vary in time.

Countless systems in biology, physics, and finance undergo diffusive dynamics. Many of these systems, including biomolecules inside cells, active matter systems and foraging animals, exhibit anomalous dynamics where the growth of the mean squared displacement with time follows a power law with an exponent that deviates from 1. When studying time series recording the evolution of these systems, it is crucial to precisely measure the anomalous exponent and confidently identify the mechanisms responsible for anomalous diffusion. These tasks can be overwhelmingly difficult when only few short trajectories are available, a situation that is common in the study of non-equilibrium and living systems. Here, we present a data-driven method to analyze single anomalous diffusion trajectories employing recurrent neural networks, which we name RANDI. We show that our method can successfully infer the anomalous exponent, identify the type of anomalous diffusion process, and segment the trajectories of systems switching between different behaviors. We benchmark our performance against the state-of-the art techniques for the study of single short trajectories that participated in the Anomalous Diffusion (AnDi) challenge. Our method proved to be the most versatile method, being the only one to consistently rank in the top 3 for all tasks proposed in the AnDi challenge.

Single particle tracking allows probing how biomolecules interact physically with their natural environments. A fundamental challenge when analysing recorded single particle trajectories is the inverse problem of inferring the physical model or class of models of the underlying random walks. Reliable inference is made difficult by the inherent stochastic nature of single particle motion, by experimental noise, and by the short duration of most experimental trajectories. Model identification is further complicated by the fact that main physical properties of random walk models are only defined asymptotically, and are thus degenerate for short trajectories. Here, we introduce a new, fast approach to inferring random walk properties based on graph neural networks (GNNs). Our approach consists in associating a vector of features with each observed position, and a sparse graph structure with each observed trajectory. By performing simulation-based supervised learning on this construct [1], we show that we can reliably learn models of random walks and their anomalous exponents. The method can naturally be applied to trajectories of any length. We show its efficiency in analysing various anomalous random walks of biological relevance that were proposed in the AnDi challenge [2]. We explore how information is encoded in the GNN, and we show that it learns relevant physical features of the random walks. We furthermore evaluate its ability to generalize to types of trajectories not seen during training, and we show that the GNN retains high accuracy even with few parameters. We finally discuss the possibility to leverage these networks to analyse experimental data.

Anomalous diffusion phenomena are observed in many areas of interest. They manifest themselves in deviations from the laws of Brownian motion (BM), e.g. in the non-linear growth (mostly power-law) in time of the ensemble average mean squared displacement (MSD). When we analyze the real-life data in the context of anomalous diffusion, the primary problem is the proper identification of the type of the anomaly. In this paper, we introduce a new statistic, called empirical anomaly measure (EAM), that can be useful for this purpose. This statistic is the sum of the off-diagonal elements of the sample autocovariance matrix for the increments process. On the other hand, it can be represented as the convolution of the empirical autocovariance function with time lags. The idea of the EAM is intuitive. It measures dependence between the ensemble-averaged MSD of a given process from the ensemble-averaged MSD of the classical BM. Thus, it can be used to measure the distance between the anomalous diffusion process and normal diffusion. In this article, we prove the main probabilistic characteristics of the EAM statistic and construct the formal test for the recognition of the anomaly type. The advantage of the EAM is the fact that it can be applied to any data trajectories without the model specification. The only assumption is the stationarity of the increments process. The complementary summary of the paper constitutes of Monte Carlo simulations illustrating the effectiveness of the proposed test and properties of EAM for selected processes.

In this paper we study properties of the diffusion limits of three different models of Lévy walks (LW). Exact asymptotic behavior of their trajectories is found using LePage series representation. We also prove an existing conjecture about total variation of LW sample paths. Based on this conjecture we verify martingale properties of the limit processes for LW. We also calculate their probability density functions and apply this result to determine the potential density of the associated non-symmetric α-stable processes. The obtained theoretical results for continuous LW can be used to recognize and verify this type of processes from anomalous diffusion experimental data. In particular they can be used to estimate parameters from experimental data using maximum likelihood methods.

Numerous examples for a priori unexpected non-Gaussian behaviour for normal and anomalous diffusion have recently been reported in single-particle tracking experiments. Here, we address the case of non-Gaussian anomalous diffusion in terms of a random-diffusivity mechanism in the presence of power-law correlated fractional Gaussian noise. We study the ergodic properties of this model via examining the ensemble- and time-averaged mean-squared displacements as well as the ergodicity breaking parameter EB quantifying the trajectory-to-trajectory fluctuations of the latter. For long measurement times, interesting crossover behaviour is found as function of the correlation time τ characterising the diffusivity dynamics. We unveil that at short lag times the EB parameter reaches a universal plateau. The corresponding residual value of EB is shown to depend only on τ and the trajectory length. The EB parameter at long lag times, however, follows the same power-law scaling as for fractional Brownian motion. We also determine a corresponding plateau at short lag times for the discrete representation of fractional Brownian motion, absent in the continuous-time formulation. These analytical predictions are in excellent agreement with results of computer simulations of the underlying stochastic processes. Our findings can help distinguishing and categorising certain nonergodic and non-Gaussian features of particle displacements, as observed in recent single-particle tracking experiments.