Time series Models

Time series analysis (One/more output, no input)

System Identification Toolbox™ enables you to create and estimate four general types of time series model.

Linear parametric models — Estimate parameters in structures such as autoregressive models and state-space models.
Frequency-response models — Estimate spectral models using spectral analysis.
''12''
Nonlinear ARX models — Estimate parameters in the nonlinear ARX structure.
Grey-box models — Estimate the coefficients of the ordinary differential or difference equations that represent your system dynamics.

ARMA typies

For the following functions,we all the notation that
$\left. \begin{array} { l } { A ( q ) = 1 + a _ { 1 } q ^ { - 1 } + \ldots + a _ { n _ { a } } q ^ { - n _ { a } } } \\ { B ( q ) = b _ { 1 } + b _ { 2 } q ^ { - 1 } + \ldots + b _ { n _ { b } } q ^ { - n _ { b } + 1 } } \\ { C ( q ) = 1 + c _ { 1 } q ^ { - 1 } + \ldots + c _ { n _ { c } } q ^ { - n _ { c } } }\end{array} \right.$

AR
$A ( q ) y ( t ) = e ( t )$
ARI
$A ( q ) y ( t ) = \frac { 1 } { 1 - q ^ { - 1 } } e ( t )$
MA
$y ( t ) = C ( q ) e ( t )$
ARMA
$A ( q )y ( t ) = C ( q ) e ( t )$
ARIMA
$A ( q )y ( t ) = C ( q )\frac { 1 } { 1 - q ^ { - 1 } } e ( t )$

where $( \frac { 1 } { 1 - q ^ { - 1 } } )$ is the integrator in the noise channel, $e(t)$ .

S-ARIMA
$A( q) \Phi ( q ) ( 1 - L ) ^ { D } ( 1 - L ^ { s } ) ^ { D _{s}}y ( t ) =c+ C ( q ) \Theta ( q ) e ( t )$
Over here
$\Phi ( q) = 1 - \phi _ { D _{s}} q ^ { D _{s} } ,\Theta ( q ) = ( 1 + \theta _ { D _{s} } q^ { D _{s} } )$
And the above coefficient $c$ , we can choose it or not and further more to specify the structure of S-ARIMA. Of cousre, for the above models the AR, ARI, MA, ARMA, and ARIMA, there are also the time series difference operator $( 1 - L ) ^ { D }$ that can be applied.

Time series analysis (with input)

idpoly:Polynomial model with identifiable parameters

$A ( q ) y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t ) + \frac { C ( q ) } { D ( q ) } e ( t )$

The variables $A$ , $B$ , $C$ , $D$ , and $F$ are polynomials expressed with the time-shift operator $q^{-1}$ .
$\left. \begin{array} { l } { A ( q ) = 1 + a _ { 1 } q ^ { - 1 } + \ldots + a _ { n _ { a } } q ^ { - n _ { a } } } \\ { B ( q ) = b _ { 1 } + b _ { 2 } q ^ { - 1 } + \ldots + b _ { n _ { b } } q ^ { - n _ { b } + 1 } } \\ { C ( q ) = 1 + c _ { 1 } q ^ { - 1 } + \ldots + c _ { n _ { c } } q ^ { - n _ { c } } \\ { F ( q ) = 1 + f _ { 1 } q ^ { - 1 } + \ldots + f _ { n _ { f } } q ^ { - n _ { f } }}\\ { D( q ) = 1 + d _ { 1 } q ^ { - 1 } + \ldots + d _ { n _ { d } } q ^ { - n _ { d } }}\\ }\end{array} \right.$
As we can see, the above all time series analysis models (ARMA typies) are all special case of a polynomial model without the input series, $u_{t}$ . So the input series, there are some polynomial models.

polyest:
$A ( q ) y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t - n _{k} ) + \frac { C ( q ) } { D ( q ) } e ( t )$
Linear ARX:

$A ( q ) y ( t ) = B ( q ) u ( t - n _ { k } ) + e ( t )$

For the multiple-Input, Single-Output Models, the ARX MISO structure is then given by the following equation:
$A ( q ) y ( t ) = B _ { 1 } ( q ) u _ { 1 } ( t - n k _ { 1 } ) + B _ { 2 } ( q ) u _ { 2 } ( t - n k _ { 2 } ) + \cdots + B _ { n u } ( q ) u _ { n u } ( t - n k _ { n u } ) +e(t)$

iv4: ARX model estimation using four-stage instrumental variable method.
It estimates an ARX polynomial model, using the four-stage instrumental variable method. The estimation algorithm is insensitive to the color of the noise term, $\nu( t )$ .
$A ( q ) y ( t ) = B ( q ) u ( t - n k ) +\nu( t )$
Estimation is performed in 4 stages. The first stage uses the arx function. The resulting model generates the instruments for a second-stage IV estimate. The residuals obtained from this model are modeled as a high-order AR model. At the fourth stage, the input-output data is filtered through this AR model and then subjected to the IV function with the same instrument filters as in the second stage.
ivar: AR model estimation using familiar method.
ivar estimates an AR polynomial model, sys, using the instrumental variable method and the time series data data.
$A ( q ) y ( t ) = e ( t )$
For this model, I am not familiar with its differnce with AR. How use the instrumental variable, it need more consideration.
Linear ARIX:

$A ( q ) y ( t ) = B ( q ) u ( t - n _ { k } ) + \frac { 1 } { 1 - q ^ { - 1 } } e ( t )$

armax:

$A ( q ) y ( t ) = B ( q ) u ( t - n _ { k } ) + C ( q ) e ( t )$

arimax:

$A ( q ) y ( t ) = B ( q ) u ( t - n _ { k } ) + C ( q ) \frac { 1 } { 1 - q ^ { - 1 } }e ( t )$

oe: Output-error models
$y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t - n _{k} ) + e ( t )$
bj: Box-Jenkins polynomial model.

$y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t - n _{k} ) + \frac { C ( q ) } { D ( q ) } e ( t )$

Also with the integrator in the noise channel,
$y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t - n _{k} ) + \frac { C ( q ) } { D ( q ) } \frac { e ( t ) } { 1 - q ^ { - 1 } }$
The general Box-Jenkins model (with more than one input series) structure is:
$y ( t ) = \sum _ { i = 1 } ^ { n u } \frac { B _ { i } ( q ) } { F _ { i } ( q ) } u _ { i } ( t - n k _ { i } ) + \frac { C ( q ) } { D ( q ) } e ( t )$

State-space model

idss: State-space model with identifiable parameters.
$\left. \begin{array} { l } { \frac { d x ( t ) } { d t } = A x ( t ) + B u ( t ) + K e ( t ) } \\ { y ( t ) = C x ( t ) + D u ( t ) + e ( t ) } \end{array} \right.$
$A$ , $B$ , $C$ , $D$ , and $K$ are state-space matrices. $u(t)$ is the input, $y(t)$ is the output, $e(t)$ is the disturbance, and $x(t)$ is the vector of $n_x$ states.

系统原理

变量定义

ssest: Estimate state-space model using time-domain or frequency-domain data. It estimates a continuous-time state-space model.
$\left. \begin{array} { l } { \dot { x } ( t ) = A x ( t ) + B u ( t ) + K e ( t ) } \\ { y ( t ) = C x ( t ) + D u ( t ) + e ( t ) } \end{array} \right.$
n4sid :estimates a discrete-time state-space model.

$\left. \begin{array} { l } { \dot { x } ( t ) = A x ( t ) + B u ( t ) + K e ( t ) } \\ { y ( t ) = C x ( t ) + D u ( t ) + e ( t ) } \end{array} \right.$
The the above two models have the same fomula, but apply to two different time format.

Grey-box model

greyest: Linear grey-box model estimation.
This model can be any linear structure. We choose the common DC motor model as a example. Choose the angular position (rad) and the angular velocity (rad/s) as the outputs and the driving voltage (V) as the input. Set up a linear state-space structure of the following form:
$\left. \begin{array} { l } { \dot { x } ( t ) = \left[ \begin{array} { c c } { 0 } & { 1 } \\ { 0 } & { - \frac { 1 } { \tau } } \end{array} \right] x ( t ) + \left[ \begin{array} { l } { 0 } \\ { \frac { G } { \tau } } \end{array} \right] u ( t ) } \\ { y ( t ) = \left[ \begin{array} { l l } { 1 } & { 0 } \\ { 0 } & { 1 } \end{array} \right] x ( t ) . } \end{array} \right.$
$\tau$ is the time constant of the motor in seconds, and $G$ is the static gain from the input to the angular velocity in rad/(V*s) .

Frequency response

idfrd: Frequency response data or model.
$y ( t ) = G ( q ) u ( t ) + H ( q ) e ( t )$

the transfer function estimate is $G ( e ^ { i \omega } )$ and the additive noise spectrum $\Phi _ { \nu} ( \omega )$ at the output is

$\Phi _ { \nu} ( \omega ) = \lambda T | H ( e ^ { i \omega T } ) | ^ { 2 }$

Here, $\lambda$ is the estimated variance of $e(t)$ and T is the sample time.

spa: Estimate frequency response with fixed frequency resolution using spectral analysis.

$y ( t ) = G ( q ) u ( t ) + \nu(t)$

The notation $G(q)u(t)$ represents the following operation:
$G ( q ) u ( t ) = \sum _ { k = 1 } ^ { \infty } g ( k ) u ( t - k )$
$q$ is the shift operator, defined by the following equation:
$G ( q ) = \sum _ { k = 1 } ^ { \infty } g ( k ) q ^ { - k } , q ^ { - 1 } u ( t ) = u ( t - 1 )$
$G(q)$ is the frequency-response function when it is evaluated on the unit circle, $G ( e ^ { i \omega } )$ .
The disturbance $\nu(t)$ can be described as filtered white noise:
$\nu(t)=H(q)e(t)$
Here, $e(t)$ is the white noise with variance $\lambda$ and the noise power spectrum is given by the following equation:
$\Phi _ {\nu } ( \omega ) = \lambda | H ( e ^ { i \omega } ) | ^ { 2 }$

spafdr:Estimate frequency response and spectrum using spectral analysis with frequency-dependent resolution.

$y ( t ) = G ( q ) u ( t ) + \nu ( t )$

etfe: Estimate empirical transfer functions and periodograms.

$y ( t ) = G ( q ) u ( t ) + \nu ( t )$

The fomula expressions of spa, spafdr and etfe are the same, but the solving algorithm are not the same.

Nonlinear models

nlarx(nonlinear ARX model): Estimate parameters of nonlinear ARX model.

A simulation scenario of nonlinear ARX

You can assign any of the regressors as inputs to the linear function block of the output function, the nonlinear function block, or both.
It maps the regressors to the model output using an output function block. We can spdcify any structures of Nonlinear models.
$F ( x ) = L ^ { T } ( x - r ) + g ( Q ( x - r ) ) + d$

$L ^ { T } ( x - r )$ and $g ( Q ( x - r ) )$ are respectively representing the linear and non-linear blocks of the model. And for non-linear block, $g ( Q ( x - r ) )$ , it is a abstract expression and can be any specific structure.

nlgreyest: Estimate nonlinear grey-box model parameters.

Grey-box Function

Hammerstein-Wiener Models

Hammerstein-Wiener models describe dynamic systems using one or two static nonlinear blocks in series with a linear block. The linear block is a discrete transfer function that represents the dynamic component of the model.

Structure of a Hammerstein-Wiener model

$f$ is a nonlinear function that transforms input data $u(t)$ as $\overline {u}(t) = f(u(t))$ . $w(t)$ , an internal variable, is the output of the Input Nonlinearity block and has the same dimension as $u(t)$ .
$B/F$ is a linear transfer function that transforms $\overline {u}(t)$ as $x(t) = (B/F)w(t)$ .
$x(t)$ , an internal variable, is the output of the Linear block and has the same dimension as $y(t)$ . $B$ and $F$ are similar to polynomials in a linear Output-Error model.
$B$ and $F$ are similar to polynomials in a linear Output-Error model.
$h$ is a nonlinear function that maps the output of the linear block $x(t)$ to the system output $y(t)$ as $y(t) = h(x(t))$ .

We computes the Hammerstein-Wiener model output y in three stages:

Compute $\overline {u}(t) = f(u(t))$ from the input data. In MATLAB, we can choose sigmoid network, wavelet network, saturation, dead zone, piecewise linear function, one-dimensional polynomial, or a custom network. Those are transfer functions of nonlinear block, $f(:)$ .
Compute the output of the linear block using $\overline {u}(t)$ and initial conditions: $x(t) = (B/F)\overline {u}(t)$ . You can configure the linear block by specifying the orders of numerator $B$ and denominator $F$ .
Compute the model output by transforming the output of the linear block $x(t)$ using the nonlinear function $h(:)$ as $y(t) = h(x(t))$ . Similar to the input nonlinearity, the output nonlinearity is a static function. We can configure the output nonlinearity in the same way as the input nonlinearity. You can also remove the output nonlinearity, such that $y(t) = x(t)$ , then we can call it Hammerstein system.

Hammerstein ARIMX model

Structure of a Hammerstein ARIMAX model

For the above Hammerstein-Wiener model, we specify that the $h(:)$ is empty and $G(z)$ is the structure of linear model ARIMX, Furthmore, we choose the first nonlinear block $f(:)$ is s expressed by a linear combination of known basis functions with unknown coefficients, or is a piecewise linear function with unknown joints and slopes, and hence, identification of the nonlinear block in this case is equivalent to estimating unknown parameters.The nonlinear part is considered as a known basis $( \omega _ { 1 } , \omega _ { 2 } , \ldots , \omega _ { n _ { k } } )$ with coefficients $(k _ { 1 } , k_ { 2 } , \ldots , k _ { n _ { k } } )$ .

arimax:
$A ( q ) y ( t ) = B ( q ) u ( t - n _ { k } ) + C ( q ) e ( t )$
polyest:
$A ( q ) y ( t ) = \frac { B ( q ) } { F ( q ) } u ( t - n _{k} ) + \frac { C ( q ) } { D ( q ) } e ( t )$

Refer to the above formulas, we the detail exprssion of Hammerstein ARIMAX model.
$\left. \begin{array} { l } { y ( t ) = x ( t ) + \nu ( t ) } \\ { x ( t ) = G ( z ) \overline { u } ( t ) = \frac { B ( z ) } { A ( z ) } \overline { u } ( t ) } \\ { \nu ( t ) = N ( z ) v ( t ) = \frac { C ( z ) } { A ( z ) } e ( t ) } \\ { A ( z ) = 1 + a _ { 1 } z ^ { - 1 } + a _ { 2 } z ^ { - 2 } + \cdots + a _ { n _ { a } } z ^ { - n _ { a } } } \\ { B ( z ) = b _ { 1 } z ^ { - 1 } + b _ { 2 } z ^ { - 2 } + \cdots + b _ { n _ { b } } z ^ { - n _ { b } } } \\ { C ( z ) = 1 + c _ { 1 } z ^ { - 1 } + c _ { 2 } z ^ { - 2 } + \cdots + c _ { n _ { c } } z ^ { - n _ { {c} } } } \end{array} \right.$

This can be transformed as
$\left. \begin{array} { l }{ y ( t ) = - \sum _ { i = 1 } ^ { n _ { a } } a _ { i } y ( t - i ) + \sum _ { i = 1 } ^ { n _ { b } } b _ { i } \overline { u } ( t - i ) }\\{ \qquad \quad + \sum _ { i = 1 } ^ { n _ { c } } c _ { i } e ( t - i ) + e ( t ) } \end{array} \right.$

Then it is the nonlinear block expression:
$\left. \begin{array} { l }{ f ( u ( t ) ) = \overline { u } ( t ) = k _ { 1 } \omega _ { 1 } ( u ( t ) ) + k _ { 2 } \omega _ { 2 } ( u ( t ) ) }\\{\qquad \qquad + \ldots + k _ { n _ { k } } \omega _ { n _ { c\k } } ( u ( t ) ) = \sum _ { i = 1 } ^ { n _ { k } } k_ { i } \omega _ { i } ( u ( t ) ) } \end{array} \right.$
Combine them all, we can get the Hammerstein ARIMX model's expression:
$\left. \begin{array} { l }{ y ( t ) = - \sum _ { i = 1 } ^ { n _ { a } } a _ { i } y ( t - i ) + \sum _ { j = 1 } ^ { n _ { b } } b _ { j } \sum _ { i = 1 } ^ { n _ { k } } k _ { i } \omega _ { i } ( u ( t - j ) ) }\\{ \qquad \quad+ \sum _ { i = 1 } ^ { n _ { d } } d _ { i } e ( t - i ) + e ( t ) } \end{array} \right.$

Deep Learning Toolbox

& Econometrics Toolbox

Econometric Modeler

Wavelet Toolbox

& System Identification Toolbox

System Identification

Phased Array System Toolbox

Pulse Waveform Analyzer

Communications Toolbox

Bit Error Rate Analysis

Radar Toolbox

Radar Designer

$Y_{2}: \Delta \text { slope }=\left\{\begin{array}{c} \operatorname{slope}_{1} : Y_{p}(t), 1 \leq t \leq 30 \\ \operatorname{slope}_{2}:\left\{\begin{array}{c} \exp [\overbrace{p(30) \sim q(38)}^{\text {Relu-Fit }}]-1, 31 \leq t \leq 37 \\ Y_{q}(t), 38 \leq t \leq 60 \end{array}\right. \end{array}\right.$

Time series Models