1. Complex Systems
In general, even though there is no precise definition of complex systems, a complex system can be thought of as a collection of nonlinearly interacting elements summing up as a whole which is characterized by novel large scale effects. These effects arise as emergent properties related with nonlinear-complex behavior, which is the most distinguishing feature of complex systems [1]. A nonexhaustive list of complex systems contains among others (in fact most systems can be considered as complex systems): geophysical processes (earthquakes), biophysics (brain dynamics, DNA), space plasmas (solar wind, solar flares, Earth’s magnetosphere), plastic deformation of materials, economy (stock indices), socio-technical systems and many others. The aforementioned complex systems, even though they are completely different in many aspects (e.g. different elements), share common characteristics such as:
- They consist of a large number of interacting elements. This fact allows the description of these systems in two different hierarchical levels, namely microscopic and macroscopic.
- Their subsystems and their interactions are nonlinear.
- They exhibit emergent behavior, namely a self-organizing collective behavior, which is not predictable from the knowledge of the element’s behavior.
- These systems are open, namely they interact with their environment. This allows these systems to be driven to metastable states far from equilibrium, where self-organization and collective behavior manifest.
These systems are usually spatially distributed systems [2]. They are of infinite dimension, since in order to be completely determined would require an infinite number of initial conditions. They can be thought of as a synthesis of many dynamical subsystems, each of which has its own dynamics. Their dynamical behavior varies in space and time leading to the emergence of spatio-temporal patterns [3]. In addition, since they are open systems, they can be considered as driven systems, exhibiting input-output (or equivalently load-unload) and threshold dynamics. In such systems, nothing happens since an external source (driver) exceeds a specific value (threshold) [4]. However, since the system is driven far from equilibrium, its’ behavior is characterized by fast unload processes in form of bursts, which produce avalanche events, giving rise to long range correlations in space and time. A representative paradigm of such a complex system is seismogenesis. In particular, earthquakes are caused by the deformation and sudden rupture of some parts of the Earth’s lithopshere. In this case the driving force is convection in the mantle which drives the plate tectonic motions. Under appropriate conditions, the energy stored in the Earth’s crust is released in a “bursting” mode, namely the earthquakes. Therefore, the interaction between the lithosphere and the mantle convection indicates the fact that plate tectonics and lithospheric internal dynamics, related with seismogenesis, constitute an input-output driven process [5].
This paper is summarized as follows: In chapter 2, the phenomenology and the basic characteristics of complex systems are presented, while in chapter 3 an overview of models used frequently in the study of complex systems is given. Chapter 4 is an introduction to extended nonlinear time series analysis presenting also some basic tools, while chapter 5 presents its application to some indicative complex systems. Finally, in chapter 6 a summary is given.
2. Phenomenology of Complex Systems
The phenomenology of the nonlinear behavior of complex systems includes emergent collective phenomena such as: far from equilibrium dynamics, power law scaling, multifractality, chaos, self organized criticality (SOC), intermittent turbulence, avalanches, non-Gaussian statistics, anomalous diffusion etc [7], to name a few. In the following we will briefly describe the most significant of them:
2.1 Input-Output, threshold dynamics
In general, the dynamics of an input-output complex system can be described by the differential equation [6]
The internal state of the system is described by the vector X and the input U by the vector, while λ is the control parameter. For spatially distributed systems, X and U are fields X= X(r,t), U= U(r,t) and their dynamics can be described by partial differential equations
In addition, in an input-output process the internal dynamics of the system is coupled with its environment and the output is a signal(s) z(t), which can be observed or measured. This signal includes valuable information for the internal dynamics as well as for the input dynamics according to the relation
2.2 Far from equilibrium thermodynamics
The rich and interesting behavior of complex systems usually takes place in conditions far from equilibrium. In this paragraph we briefly describe the notion of entropy S since it is the most significant notion concerning thermodynamics. We also introduce Tsallis entropy which is nowadays widely used in describing various complex systems’ statistics.
It is well known that in isolated systems, the energy is conserved and entropy is monotonically increases till it reaches its maximum, a state which is known as thermodynamic equilibrium
However, as we noted in the paragraph 2.1, most complex systems interact with their environment, namely they are open systems. In such systems, there exists a flow of entropy between the (sub)-systems of the system in study and its environment. In this case, the entropy change dS during a time interval dt, can be decomposed into two components [8],
where
These far from equilibrium, macroscopic, large-scale order structures, correspond dynamically to non-equilibrium (quasi)- stationary states (NESS's) [9], which have the topology of a percolating fractal set. The transition to a NESS is related to enhanced nonlinearities and developing of instabilities into a collective mode. In the NESS the system is in a turbulent state, exhibiting complex space distributions in the critical domain, while is fully determined by self organization processes which generate and maintain multi-scale correlations [9].
The most commonly used entropic function used for describing the thermodynamics of the complex systems is that based on Boltzmann- Gibbs (BG) statistics in which the entropy, for the discrete case, has the form of
where W is a set of discrete states, k is the Boltzmann constant pi is the probability of the (i) microscopic state of the system with
However, even though BG statistics-thermodynamics is used efficiently to study many interesting systems and their behavior, it fails to describe ubiquitous phenomena commonly observed in complex systems and their basic quantities, since these quantities exhibit long range interactions, non-Gaussian and non-exponential behavior, for example power law scaling and multi-fractality. In order to confront with this problem, Tsallis [10] introduced a new definition of entropy, which is generalization of BG definition of entropy. In particular, this non-extensive entropy Sq is given by
where q is the entropic index, which is related to the microscopic
dynamics and characterizes the degree of non-extensivity. In addition,
it can be proved that
For two independent systems A and B (e.g.
For all occasions Sq ≥ 0, q <1, q =1, q >1 and the cases of correspond to super-additivity, additivity, sub-additivity, respectively.
As we will show in Chapter 5, a vast amount of papers nowadays concerning different complex systems showed that indeed Tsallis non-extensive statistical mechanics is appropriate for dynamical regions such as between regular motions and standard chaos, regions called edge of chaos, suggesting that the entropic index q could be a convenient manner for quantifying some relevant aspects of complexity, such as (multi)fractal and similar, hierarchical, statistically scale-invariant, structures.
2.3 Power law Scaling
Power laws are considered to be a trademark of complex systems and appear in many cases of different complex systems, such as the ones mentioned in first paragraph. In general, a quantity x obeys a power law if it derives from a probability distribution (PDF)
where a is a parameter of the PDF known as scaling exponent. If we plot Eq. 4 in log-log axes we will get a straight line, the slope of which is the value of a. This law is also known as Zipf ’s law or Pareto distribution [11]. Sometimes a power law distribution is also said to be a scale invariant distribution, because it remains the same under any scale transformation.
2.4 Multi-fractality
Power laws are closely related to fractals [12]. Indeed, fractals are geometrical objects which cannot be described by Euclidean geometry since they possess a fractal dimension, they are characterized by selfsimilarity and they do not have a specific length. Paradigms are the trees, the brain, the clouds, the coast-shores and many others. For the characterization of fractal objects different fractal dimensions were developed such as the Hausdorff dimension DH, the capacity dimension D0, the information dimension D1, the correlation dimension D2 etc. In particular, an object is said to be fractal if parts or segments N of the object scale as a power law with an exponent D
N(L) = AL−D
where A is a constant and the exponent D fractal dimension.
However in many cases a simple fractal dimension cannot describe a complex inhomogeneous geometrical object. Then, this complexity can be described efficiently by the concept of multi-fractality [13]. In this case, the fractal distribution attains different degrees of clustering in different regions in the object. Therefore, multi-fractality quantifies the degree of clustering and the intermittency in a fractal object. For characterizing a multi-fractal object S the generalized fractal dimension is estimated, from which the other fractal dimension are derived also
where the function
where
2.5 Low Dimensional Chaos
As we mentioned before complex systems can exhibit rich behavior depending on the conditions and or/the control parameter. In particular, as, for example, a control parameter is increased and t→∞ the dynamical behavior of the system can be characterized in the phase space by limit cycles, low dimensional torus and low dimensional chaotic (strange) attractors. Low dimensional chaotic behavior was initially discovered on the behavior of a rather simple nonlinear system, described by three nonlinear differential equations, the well known Lorenz system [14]. In particular, low dimensional chaos describes the irregular-random behavior which emerges from a rigorous deterministic evolution of a low dimensional system in time, without the presence of any source of noise or external stochasticity. This aperiodic behavior manifests as sensitivity to initial conditions, namely nearby trajectories in the phase space diverge exponentially as time passes, excluding a long term prediction of the systems’ dynamics. This phenomenon is also called “butterfly effect”, is due to the nonlinear coupling between the variables of the system [15]. In the phase space, the strange attractor is a low dimensional confined set which is characterized by a fractal dimension, has self-similar properties and close trajectories diverge exponentially as time passes. Such a behavior can also be observed also in complex systems with many, practically infinite, degrees of freedom. Specific examples of such behavior are well studied in hydro-dynamics and refer to Rayleigh-Benard convection, Taylor-Couette flow and dripping faucet model [16-18].
2.6 Self Organized Criticality (SOC)
Another characteristic behavior of complex systems is the SOC behavior, which is strongly connected with power law scaling. This behavior is very different from the chaotic one we described in the previous paragraph, since in this case the system is high dimensional and is characterized by a zero maximum Lyapunov exponent. In particular, the theory of SOC by Bak, Tang και Wiesenfeld [19] was developed in order to explain the power law behavior of various systems such as earthquakes, solar storms, snow avalanches and many others. According this theory, dissipative spatially distributed systems far from equilibrium can physically evolve to critical point states without any characteristic temporal or spatial scale. In these critical points, the states of the systems are characterized by power laws of temporal correlations (e.g. flickering 1/f noise) and spatial correlations namely spatial fractal structures. In addition, these points are attractors for the system, are not being affected from various parameters of system and are solely due to the nonlinear interactions of the elements of the system. That is why it is called self organized. The fluctuations around the SOC state are called avalanches and characterized by long range correlations. Very small fluctuations can lead to avalanches of all sizes. Practically, the SOC mechanism involves slow accumulation of energy and fast unloading of this energy through avalanches. Finally, according to [19], in a SOC system the divergence of near-by trajectories follow a power law, instead of an exponential law which is characteristic of chaotic systems. In addition, the degrees of freedom are proportional to the size of the system, namely the system is high dimensional (practically infinite), in contrast with the low dimensional chaotic systems. The SOC attractor corresponds to a kind of weak chaos (zero maximum Lyapunov exponent) and the systems lives at the edge of chaos.
2.7 Strange Kinetics and Anomalous Diffusion (Transport) Processes
Complex systems are often connected to strange kinetics which involve anomalous diffusion (transport) processes, both sub-linear and super-linear [20], giving rise to long range correlations which manifest as non-Gaussian (‘strange’) behavior of the system’s kinetic behavior near non-equilibrium stationary states [9]. A common way to measure transport (diffusion) processes is to study the time evolution of the mean square displacement of the particle
where D is the normalization constant having the meaning of the generalized transport coefficient and the exponent classifies the type of transport: for a=1, the diffusion is normal, for a≠1, anomalous. In particular, for 0 < a < 1 there exists sub-diffusion and for a=1 superdiffusion [22,23]. Normal diffusion is usually related to systems near thermodynamical equilibrium, where the trajectories of the particles are irregular but homogeneous, namely they consist of small similar distances, whereas anomalous diffusion is connected with systems far from equilibrium, where the trajectories are very inhomogeneous. This inhomogeneity, is due to the fact that particles are “trapped” into eddies where they stay for unknown time and in the following they go through big “flights”, meaning they travel long distances. In addition, anomalous diffusion can also be connected with power law distributions, called Levy distributions. In this case, the “flights” in anomalous diffusion processes are called Levy flights, the waiting times are described by Poissonian distributions, whereas the distribution of flights λ(x)[23] are power law distributions according to the relation
2.8 Intermittency
An additional complex dynamical behavior in spatio-temporal extended driven systems is intermittency in turbulence. In particular, in classical fluid turbulence, scaling is often associated with a hierarchical structure of eddies extending over the inertial range [24,25]. In this picture (called cascade model), the shear friction between these two distinct formations in the flow creates, in a little bit downstream, big eddies with the diameter about lin . The eddies flow downstream with the mean current producing smaller eddies one after another. As a result, a turbulent state is formed comprised of a mixture of various sizes of eddies from large to small (fully developed turbulence). The diameter of the eddies is given by
Here, n is the number of steps in the cascade producing smaller eddies in a stream. Therefore, turbulence is fluctuating with different rhythms, intermittently, which allow one to observe the intermittent burst of fluctuations. The cascade model, described previously, was introduced to account for these intermittent phenomena. The inertial range is the region where the energy of eddies are delivered, consecutively, to smaller eddies and the diameter of eddies satisfy the relationship lin>>ln>>n. The highly intermittent character of turbulent phenomena manifested in complex systems, is also connected with inhomogeneity in fluctuations in different quantities of the turbulent system. This inhomogeneity is manifested as small-scale fluctuations of high intensity surrounded by extended areas of much lower fluctuations. Therefore, intermittency manifests itself via the burst-like behavior in temporal and spatial domains. In addition, intermittency is strongly related to non-Gaussian statistics and multifractality (since a Gaussian process is mono-fractal). Indeed, in this cascade picture of turbulence many multifractal models were developed such as the lognormal model, p-model, Arimitsu and Arimitsu model [25].
3. Modeling Complex Systems
A nonlinear complex system can be studied in various ways, e.g either macroscopically studying its behavior, either mesoscopically studying the operation of its subsystems, either microscopically. Even though the study of complex systems requires a vast amount of information, the development of computers provided and provides great help in understanding complex systems. In this direction various theoretical and computational models were developed which can be classified into two great categories [1]:
-
Equation-Based
- Partial Differential and Differential Equations
- Maps
- Agent Based, such as cellular automata and complex networks
Some representative examples of mean field type models are the Kuramoto – Sivashinsky equation [26], the complex Ginzburg- Landau equation [27], the reaction diffusion-equation [28], and others. In the following we describe briefly some partial differential equations commonly used to model complex systems’ behavior.
3.1 Kuramoto – Sivashinsky equations
This equation is widely studied in various phenomena such as plasma physics, flows, convection, chemical reactions and others, exhibiting a variety of nonlinear and turbulent states, such as low dimensional chaos, found in spatially extended systems [26]. Its form can be written as
where h(X+t) can be interpreted as the height of a d-dimensional interface embedded in d+1 dimensions.
3.2 Cubic complex Ginzburg-Landau equations
Another well studied nonlinear equation is the cubic complex Ginzburg-Landau equation which can efficiently describe a vast variety of nonequilibrium phenomena in spatially extended systems, from nonlinear waves to second-order phase transitions, from superconductivity, superfluidity, and Bose-Einstein condensation to liquid crystals and strings in field theory [27]. It has the form
where A is a complex function of (scaled) time t and space x and the real parameters b and c characterize linear and nonlinear dispersion.
3.3 Reaction-Diffusion equations
These equations are widely applicable in many areas, including chemistry, biology, physics and engineering [28]. The simplest reaction-diffusion equation can be written as
where
3.4 Fractional Calculus and the Fractal Integral
Fractals are closely related to complex systems. As we showed they are irregular over all length scales (non-integer dimensions) and this fact raises serious difficulties in studying them with usual calculus, which is unable in describing efficiently such structures and processes. Indeed, fractal functions do not possess first-derivative at any point. Therefore, new mathematical operators are needed and these can be found in fractional calculus [29]. In the last decade many fractional equations were developed, including diffusion equations, wave equations, relaxation equation (for a review see [30].
The most frequently used definition of a fractional integral of order p (p > 0) is the Reimann-Liouville’s definition,
with n-1 < q < n (n integer) as the nth integer of the (n-q)th fractional integral. The above formula is a direct generalization of Cauchy’s formula for repeated integration
When p is not an integer, the fractional derivative (20) is a nonlocal operator since it depends on the lower integration limit a. In addition, when a = 0, the following scaling property exists
providing a link between fractional calculus and fractals and p
being the fractal dimension [31], since the fractional operator shows
the same scaling laws as the a-dimensional Hausdorff measure of a
fractal set V, for e.g.
3.5 Cellular automata
One class of models used to describe complex systems and their behavior, without the use of differential equations is cellular automata (CA), which are examples of mathematical systems constructed from many identical components, each simple, but together capable of complex behavior [32,33]. A 1-D CA consists of a line of sites (“cells”), with each site carrying a value of (0, ..., k-1). The state of the CA is completely specified by the values of the variables ai at each site i. The evolution of the CA takes place in discrete time steps and the variables are updated simultaneously based on the values of the variables in the neighborhood according to a deterministic rule
The sites at microscopic level may represent points for example in a crystal lattice, while at macroscopic level a region of many molecules or others, depending on the scale and the system in study. In general, the overall behavior of CA can be extremely complex, since different rules yield different patterns that differ in detail but are similar in a statistical sense, exhibiting spatially homogeneous states, periodic structures, chaotic behavior, and complicated localized or propagating structures. The CA are used in modeling many physical complex systems such as earthquakes [19], brain dynamics [34] and many others.
3.6 Complex networks
Another promising tool for understanding complex systems and their behavior is complex network and network theory [35,36]. Examples of complex networks are systems composed by a large number of highly interconnected dynamical units, such as the Internet and the World Wide Web, neural networks, social interacting species, to name only a few. Complex networks can be studied either as graphs whose nodes represent the dynamical units, and whose links stand for the interactions between them, either as a dynamical system learning how a large ensemble of dynamical systems that interact through a complex wiring topology can behave collectively. Mathematically, a network is represented by a graph, consisting of a set of N nodes which are connected with a set of links. When a network is random, then the nodes are linked in a completely random manner. However, complex networks usually are far from random, often related to organizing principles. There are several measures to characterize network topology such as the clustering coefficient ci, the neighbor connectivity knn, but the simplest is the degree distribution p(k), which is the probability for finding a node i with degree k (number of links associated with it) in the network [37]. When the degree distribution is a power law, the network is called scale-free, which are of high interest nowadays, since, as we mentioned, many complex systems exhibit power law behavior, such as brain dynamics [38] and earthquakes [39].
3.7 Artificial Neural Networks (ANNs)
Very often complex systems are connected with a really vast amount of data, such as time series, images, various parameters, other spatiotemporal variables etc. For instance, for studying brain dynamics different variables are available such as interviews of patients and family members, physical exams, laboratory and cognitive tests and neurological exams, including magnetic resonance imaging (MRI) or computerized tomography (CT), positron emission tomography (PET) and functional magnetic resonance imaging (fMRI), as well as electroencephalography (EEG) time series. For the analysis and utilization of all this information advance techniques and algorithms are being developed summarized as machine learning. In particular, machine learning deals with algorithms that facilitate pattern recognition, classification and prediction, based on models derived from existing data [40]. Such computer algorithms included in machine learning arsenal are artificial neural networks (ANNs), which can recognize hidden patterns and relations of input data, model nonlinear complex functions, manage data and learn, improving their overall performance. Usually, ANNs consist of neurons or nodes which are grouped into layers: an input layer, one or several hidden layers and an output layer. The layers are connected with transfer functions and weights which are trained and initialized. A typical artificial neuron can be modeled as [41]: if we consider different inputs x1,...,xn in a neuron, then the output signal O is given by:
where wj is the weight vector, and the function f(net) is referred to as an activation (transfer) function. The variable net is defined as a scalar product of the weight and input vectors
where T is the transpose of a matrix, and, in the simplest case, the output value O is computed as
where θ is called the threshold level; and this type of node is called a linear threshold unit. Paradigms of neural networks are [41] the radial basis functions (RBF) which use radial basis functions as activation functions, the probabilistic neural network (PNN) which uses a kernel-based approximation to for classification problems, the self organized maps which are used for dimensionality reduction of the data input. Applications of ANNs can be found in many scientific fields for e.g. the wind power prediction [42], in cancer prediction and prognosis [43], geomagnetic activity [44], earthquakes [45], etc
4. Nonlinear Time series analysis and Statistics
In this section we present a summary of algorithms and methods commonly used in the study of complex systems, based on nonlinear time series analysis. In particular, the main purpose of experimental time series analysis is to extract significant information for the underlying dynamics of the observed signal. Modern nonlinear time series analysis includes [46]:
-
Computation of Phenomenological Characteristics
- Autocorrelation Coefficient and Power Spectrum (Linear correlations, periodicities, scaling laws
- Mutual Information (Linear and Nonlinear Correlations)
- Probability Distributions (Power Laws)
- Hurst exponent (Persistence, anti-persistence, white noise)
- Flatness Coefficient F (Intermittent turbulence)
- Structure Functions (Turbulence, intermittency exponent)
- Phase portrait (Low Dimensionality)
- Entropy, energy, multifractal structures
- Estimation of q-Tsallis Statistics
-
Computation of Geometrical and Dynamical characteristics in the reconstructed state space
- Correlation Dimension (Degrees of freedom)
- Generalized Dimension (Multifractals)
- False Neighbors (Degrees of Freedom)
- Singular values spectrum (SVD components, filtering)
- Maximum Lyapunov Exponent (Sensitivity in initial conditions)
- Power Spectrum of Lyapunov Exponents
-
Testing of Null Hypothesis in order to discriminate between low dimensional chaotic dynamics and linear high dimensional stochastic dynamics
- Surrogate data
- Discriminating statistics
-
Singular Value Analysis in order to
- Estimate Degrees of Freedom
- Filter signals from White or Colored Noise
- Search for input-output dynamics
In the following we will present more analytically some of the most important tools of modern nonlinear time series analysis:
4.1 The mutual information
The mutual information can be used to search for nonlinear correlations between the values of an experimental signal. In general, the mutual information between two observables, A and B is given by the relation
where H(A) is the amount of average information gained from a measurement of A and H(A/B) is the amount of information of A given that B is known. If this relation is applied to time series, it leads to [47]:
If the samples {B = x(i)} and {A = x(i-τ)} are statistically independent then the mutual information will vanish for this value of τ. Thus, knowledge of the second sample cannot be gained by knowing the first. On the other hand, if the first sample uniquely determines the second sample then I(τ)=Imax, which is true when τ = 0. For a statistically independent signal, the mutual information is zero for τ>0.
4.2 Flatness coefficient F
The intermittent nature of a complex system’s dynamics can be investigated through the Probability Density Functions (PDF) of a set of two-point differenced time series of an original time series δBτ(t) = B(t + τ) - B(t), which can be any physical quantity. The coefficient F corresponding to the flatness values of the two-point difference for the observed time series is defined as [48]:
The coefficient F for a Gaussian process is equal to 3, while deviation from this value imply non-Gaussian behavior and intermittency. The parameter τ represents the spatial size of the “eddies”, which contribute to the energy cascade process.
4.3 Structure functions
Another way to search for intermittency, is the generalized structure functions, which are the various order moments of the fluctuating quantity, namely a time series,
where <...> is the ensemble average of experimental time series. The structure functions should obey a power law as a function of lag time τ
The scaling index J(P)as function of the moment p, can be used to characterize the turbulent field [49], since deviation of J(p) from p/3 implies non-Gaussian behavior and intermittency.
4.4 The Tsallis q-triplet estimation
The most basic of Tsallis indices qsensitivity, qrelaxation, qstationary, known also as Tsallis q-triplet [50], constitute the best empirical quantifier of non-extensivity. In the following we describe briefly the underlying mathematical framework concerning Tsallis q-triplet:
4.4.1 Tsallis index qsen.
This index is related to q-exponential sensitivity to initial conditions and to the rate of entropy production of the system in study. The qsen index is given by
The amax and amin values correspond to zeros of multifractal spectrum
function , which is estimated by the Legendre transformation
4.4.1 Tsallis Index qstat.
The estimation of Tsallis q entropic index, referred to as stationary q = qstat, is related to the size of the distribution tail and is usually based on Tsallis q-Gaussian distributions which can describe metastable stationary states of the system. The Tsallis q-Gaussian distribution [51] is given by
where
For q < 1, the support of Gq(β; x) is compact since this density
vanishes for
4.4.3 Tsallis Index qrel.
This index is related to relaxation processes of the system. It is given by qrel = (s-1)/s , where s is the slope of the log-log plotting of mutual information I(τ) given in paragraph 4.1 or the autocorrelation coefficient.
4.5 The reconstructed state space
Modern analysis of complex dynamical systems is also based on the reconstructed dynamics for autonomous and purely deterministic systems with n dynamical variables, according to which a delay reconstruction map Φ is considered which maps the state X into the m – dimensional delay vectors
This map is an embedding when m ≥ 2n+1 , where ft describes the dynamical flow underlying the observed signal and n is the dimension of the manifold of the system phase space dynamics [52]. The embedding Φ is a diffeomorphism which maps the orbits of the original state space in the reconstructed space, preserving their orientation and dynamical and geometrical characteristics, such as Lyapunov exponents and dimension of attractors respectively. For the estimation of the best reconstruction time τ, the first local minimum of the autocorrelation coefficient or the mutual information, as well as higher values of τ for which the estimated geometrical and dynamical characteristics of the reconstructed attractor remain invariable, can be used.
In the reconstructed state space we can also use the SVD (Singular Value Decomposition) analysis in order to: (i) filter the time series and (ii) decompose the series in its SVD reconstructed components which can be used for the detection of the underlying dynamics. Singular value analysis is applied to the trajectory matrix X estimated by the reconstructed state space [53] The SVD analysis permits the reconstruction of the original flow, in terms of n eigenvectors Vi, known as SVD reconstructed components corresponding to the spectrum of the singular values {σi} . The d largest ones of them correspond to the Vi eigenvectors, i =1,2,…d, that are sufficient for an accurate description of the underlying dynamics.
4.5.1 Correlation dimension
The correlation dimension in the reconstructed phase space is given by
of the dynamical trajectories of the system in state space, where Cm(r) is the correlation integral of the trajectory and Dm is its slope. The correlation integral is given by
The low value saturation
4.5.2 Lyapunov Exponents
The Lyapunov spectrum is of high importance as it measures the rate of convergence or divergence of close trajectories in all d directions of the phase space, giving further evidence for the possible low dimensional and chaotic character of the attractor in the reconstructed phase space. In particular, positive exponents correspond to instability, negative exponents to convergence, while at least one zero exponent must exist for the expansion along the trajectory. The Lyapunov spectrum is found by the derivative matrix of the dynamics in the reconstructed state space. The spectrum of Lyapunov exponents ordered as the λ1 ≥ λ1 ≥... ≥λd by the relation [55].
where Ai is the local approximation of the derivative matrix DF
at the reconstructed trajectory points xr(ti), i = 1, 2,...,N of the d –
dimensional reconstructed state space and
4.6 Surrogate data analysis
The method of surrogate data is used to distinguish between linearity and nonlinearity as well as between chaoticity and pure stochasticity, since a linear stochastic signal can mimic a nonlinear chaotic process after a static nonlinear distortion, since they can mimic the geometrical or dynamical characteristics of the original data [56,57]. Therefore, they can be used for the rejection of every null hypothesis that identifies the observed low dimensional chaos as a purely non – chaotic stochastic linear process. In particular, surrogate data can be constructed according to [58], to mimic the original data, regarding their autocorrelation, power spectrum and their probability distribution.
In order to distinguish a nonlinear deterministic process from a linear stochastic one, we use as discriminating statistic a quantity L derived from a method sensitive to nonlinearity, for example the correlation dimension, the maximum Lyapunov exponent, the mutual information etc. The discriminating statistic L is then calculated for the original and the surrogate data and the null hypothesis is verified or rejected depending on the “number of sigmas”
where μsur and σsur are the mean value and standard deviation of L taken from the surrogate data and μobs is the mean value of L derived from the original data. For a single time series, μobs is the single L value [56]. The significance of the statistics is a dimensionless quantity and we report it in terms of units of Sigma “sigmas”. When Sigma takes values higher than 2 – 3 then the probability that the observed time series does not belong to the same family with its surrogate data is higher than 0.95 – 0.99, correspondingly.
5. Applications of Nonlinear Time Series Analysis in Various Complex Systems
In the following we present recent representative studies (a nonexhaustive list, mostly the ones the author of this paper was involved with), concerning the application of nonlinear time series algorithms, described in previous paragraphs, in various physical systems, such as seismogenesis, plastic deformation of materials, space plasmas, brain dynamics, economy and DNA (see also [59]).
5.1 Seismogenesis
In a series of papers [5,60-62] results were presented concerning seismogenesis in different Hellenic regions (land and sea of Greece), applying nonlinear analysis to various earthquake time series, such as magnitude, inter-event times, longitude and latitude time series. For the analysis, the model of the dripping faucet was used as a physical interpretation of the seismic process. In this model, the loading rate m(t) of mass in the mechanistic dripping faucet model of Shaw corresponds to the transfer of stress in the fault system by the mantle and plate tectonic dynamics (the external driver of the system), while mass unloading corresponds to earthquakes, as releases of the elastic strain energy stored along a fault. The dripping faucet similarly to the earthquake process can be understood as a local, driven, threshold process.
Geometrical and dynamical characteristics were then estimated in the reconstructed state space, such as the singular value spectrum, the correlation integrals and their slopes and the maximum Lyapunov exponent, for the original seismic time series and its SVD components. In addition, in order to exclude the case of linear stochastic dynamics that mimic low dimensional chaos, the authors compared their results with an efficient number of surrogate data. Tsallis statistics were also exploited to study the non-Gaussian character of the time series. Their results, showed that the spatio-temporal dynamics of earthquakes are of low dimensional, non-extensive, chaotic character, indicating that seismic events do not occur randomly in space and time and supported the hypothesis of seismogenesis, as an active chaotic walker, which corresponds to a point process realized in the continuously extended Hellenic lithospheric system. In addition, the nonlinear analysis of magnitude time series showed that an independent high dimensional SOC dynamics are connected with the energy release process. The above concepts showed that earthquakes can be understood via the general theory of statistical physics for dynamical processes of far from equilibrium phase transitions applied to distributed fault‘s systems.
5.2 Space plasmas
Space plasmas dynamics are also another example of complex distributed systems exhibiting complex behavior [63]. Studies concerning the nonlinear analysis of various time series derived from Earth’s magnetosphere, solar wind, solar flares and sunspots others revealed the complex behavior of those systems, which include among others chaotic, SOC and intermittent turbulent dynamics, non-equilibrium phase transitions, as well as Tsallis non-Gaussian statistics [53,64-71].
In particular, Earth’s magnetosphere is strongly coupled externally to the solar wind plasma flow revealing dissipative internal nonequilibrium and non-linear dynamics, related with the development of magnetospheric superstorms during which strong plasma flows can be developed along the magnetotail. For example, in [64,68], the analysis of AE index time series and bulk plasma velocity measurements (Vx component) of both calm and storm periods, revealed that during the development of a superstorm event, there is a phase transition of magnetopsheric dynamics from high dimensional (SOC state) to low dimensional and chaotic (chaos state), with a corresponding strengthening of the intermittent turbulent non-Gaussian character of the dynamics. The corresponding statistics can be well described within the theoretical framework of Tsallis statistics.
Similarly, the solar plasma dynamics is a prototype of non-linearity and non-integrability, exhibiting non-Gaussian turbulent and chaotic dynamics as well as spatial multifractal topology of the solar magnetic field. Solar activity is related to two different processes, corresponding to different regions of the solar system with different physical characteristics, namely sunspots in the photosphere and solar flares at the base of solar corona. The analysis of sunspot index and daily solar flares index time series [65-67,69] revealed the coexistence of two clearly discriminated physical processes underlying the solar activity, corresponding to SOC and Chaos processes, related to photospheric and sub-photospheric zones activity of the Solar system. Also, strong evidence for intermittent solar turbulence as well as for non-extensive statistical processes, according to Tsallis q–statistics, was found.
Another example of complex system is solar wind plasma. It consists of ionized and magnetized gas, composed mainly by protons, electrons, alpha particles and heavier ions, continuously flowing away from the solar corona in all directions pervading the interplanetary space. Its dynamics is also a nonlinear, far from thermodynamical equilibrium in which the development of hierarchical, self-organized and long-correlated dynamical states is possible. Indeed in recent studies [70-71] the non-extensive and non-Gaussian character of the solar wind plasma was verified, indicating the existence of multi-scale strong correlations from the microscopic to the macroscopic scales. These long range correlations are significantly enhanced during the phase transition from calm to “shock” events, as the analysis of Tsallis q-triplet and other nonlinear quantities revealed.
5.3 Unstable plastic flow in deformation of materials
Plasticity in material’s deformation is a highly complex spatiotemporal phenomenon. The complexity of the underlying dynamics is mainly connected to complex geometric objects, dislocations, which not only act as carriers of deformation, but they move, interact and evolve in a nonlinear way, resulting in a nonuniform and non-isotropic, non-random spatial distribution [72]. One representative paradigm of such an unstable plastic flow is the Portevin–Le Chatelier (PLC) effect, which is manifested when some dilute alloys undergo tensile tests in specific regimes of strain rate and/ or temperature and is connected with serrations in stress/strain graphs and shear bands in material’s surface. Recently, [73] conducted Tsallis statistical analysis for various stress serration time series of Cu-15%Al alloys corresponding to different deformation temperatures and PLC types (A, B, C), estimating Tsallis’ entropic index q=qstat (stat denotes stationary states). The results revealed the non-Gaussian (Tsallis q-Gaussian), nonextensive, sub-additive character of the underlying dynamics of the PLC type-A bands (q > 1) indicating dynamics at the edge of chaos, with global long range correlations and power law scaling. Similarly, for PLC type B bands, a Tsallis q-Gaussian, nonextensive, super-additive statistical profile was found with q < 1, while for the PLC type C bands, a stochastic, near Boltzmann-Gibbs (BG) statistical character was verified indicating Gaussian dynamics with short (weak) range correlations. These results verify and extend previous studies [e.g.74] which classified type B and type A PLC bands underlying dynamics to distinct dynamical behavior, namely chaotic behavior for the first and self organized critical (SOC) behavior for the latter.
Complex behavior is also manifested in ultra fine grained (UFG) materials, in which the deformation mechanisms, involve grain rotation and grain boundary sliding, dislocation nucleation, emission and absorption in GBs, twinning and faulting as well as diffusional creep and grain boundary migration [75]. In a recent study [76], statistical features of serrations, which are observed in low strain rates, as well as the spatial distribution of extensive shear bands which are formed in high strain rates of an UFG alloys, were studied. The analysis was based on Tsallis nonextensive statistics and fractal theory. The results demonstrated that plastic flow that the statistics of serrations and shear bands’ formation are connected with Tsallis q-Gaussians (q>1) and fractality [Iliopoulos et al., 2015b].
5.4 Brain dynamics, Economics, DNA
The human brain can also be modeled as a driven nonlinear threshold system, comprised of nonlinear units or cells, generating spatial complex networks, where each cell operates when the electrical potential or current reaches a threshold value. In [34] a phase transition process of brain activity was revealed from a high dimensional SOC state during the health period to a gradually developed low dimensional chaotic state corresponding to epileptic state. In addition, a cellular Automata (CA) model was developed which simulated faithfully the brain behavior during healthy and epileptic states.
Economics is also a very complex system. Indeed, various economic systems can exhibit ubiquitous complex dynamics evidenced by large amplitude and aperiodic fluctuations in economic and financial variables, including foreign exchange rates, gross domestic product, interest rates, production, stock market prices and unemployment. The presence of fluctuations in economic and financial systems are indications that these systems are driven far away from the equilibrium where non-linearity gives rise to complex system’s behavior [77,78]. For example, the statistical analysis of two time stock market time series, namely Standard & Poor’s 500 (S & P) 500 and TVIX [77], based on Tsallis non-extensive statistics and in particular the estimation of q – triplet, showed the non-Gaussian, non-extensive statistical character of the time series, indicating multi-fractal phase space dynamics, which can be described faithfully by Tsallis distribution functions. The non-extensive character of the underlying dynamics is related to the existence of multi-scale long range interactions in space and time. In addition, the detailed analysis of S & P index unraveled the existence of non-equilibrium phase transitions depicted clearly in the variations of Tsallis q-triplet values, which are connected with non-equilibrium stationary states of economical dynamics derived from processes of strong self organization.
DNA is another example of complex system, since it can be thought of as a growth or aggregation phenomenon, which results in a fractal cluster with power-law correlations over wide ranges of length scales, resulting in long-range correlations of nucleotides in DNA sequences. In a recent study [79] applied an extensive nonlinear algorithm in DNA sequences of Major Histocompatibility Complex (MHC) showing that the DNA complexity and self-organization can be related to fractional dynamical nonlinear processes with low dimensional deterministic chaotic and non-extensive statistical character.
6. Summary and Conclusions
Complex systems and their behavior are found in many fields, such as scientific, engineering and social. These systems are much more than a sum of their parts and they are connected to the emergence of collective behavior through self-organization, e.g. the spontaneous formation of temporal, spatial or functional structures. They are also nonlinear, inhomogeneous, and far from equilibrium as well as adaptive as they evolve and may contain self-driving feedback loops. The collective behavior of the whole system cannot be simply inferred from the understanding of the behavior of the individual components [80].
Interesting features commonly emerged in complex systems such as far from equilibrium thermodynamics and Tsallis statistics, power law scaling, multi-fractality, low dimensional chaos, SOC, strange kinetics and anomalous diffusion and turbulent intermittency. For describing the complex behavior of such systems, new modeling tools are generated. A non-exhaustive list includes models based either on equations, for example Kuramoto – Sivashinsky equation, cubic complex Ginzburg-Landau Equation, reaction-diffusion Equation, fractional equations etc, or agent based such as cellular automata, complex networks, artificial neural networks (for further reading see also [81]). In addition, for the analysis of experimental data (time series) of complex systems new concepts and sophisticated mathematical tools are developed, roundup as nonlinear time series analysis, in order to extract significant information for the underlying dynamics of the observed signals. Tools like mutual information, flatness coefficient, structure functions, Tsallis q-triplet, correlation dimension and Lyapunov exponents in the reconstructed phase space can provide valuable information for the system’s long range correlations, intermittent turbulence features, non-Gaussian statistics, degrees of freedoms and sensitivity to initial conditions.
As we presented in this paper, there is experimental evidence of intriguingly similar quantitative features traced from nonlinear time series analysis of data derived from different complex systems, such as earthquakes, Earth’s magnetosphere, solar plasma and solar wind, plastic deformation of materials, epilepsy, economical indices and DNA structure. These results strongly support the concept of universality, namely that systems arising in different disciplines can share common characteristics and behavior, irrespective of the underlying microscopic mechanisms responsible for the observed complex behavior [82]. However, much remains to be explored. Various theoretical concepts and mathematical tools, among others, are now in intensive active research regarding complex systems, based on non-extensive statistical mechanics, fractal topology, turbulence theory, strange dynamics, percolation theory, anomalous diffusion theory and anomalous transport theory, fractional dynamics and non-equilibrium phase transition theory [83 and refs therein]. The advances in these theories will provide new insights in the description, analysis and understanding in the complexity of physical, engineering and social systems, leading to uncertainty and unpredictability reduction, which are fundamental aspects of these systems.
Competing Interests
The author declare that there is no competing interests regarding the publication of this article.
Acknowledgments
The author gratefully acknowledges Prof. G.P. Pavlos, Dr. M. Athanasiou, Dr. L.P. Karakatsanis, E. Pavlos and M. Xenakis for various useful discussions on complexity theory and complex systems.