# What are the differences among Autoregression, Moving-average regression, ARMA, and ARIMA for time series forecasting?

[

]

These are all classical models of forecasting based on time series data, with slight differences. This post focuses on the conceptual differences to gain an intuition and overview.

## Autoregressive model

*Autoregression* refers to a regression model based on itself
(“auto”). Basically, it takes *p* number of time steps before the one
to be forecast and fits a regression model based on that. Or, in other
words, predict the current value based on *p* previous values. Time
steps before the one to be forecast are referred to as *lags*, so one
could also say, predict the current value based on *p* lags. And the
number of *lags* is called the *order* of the model.

- $X_{t}$: value to be forecast at time $t$.
- $c$: some constant value
- $\varepsilon_{t}$: error value at time $t$
- $p$ number of
*lags*, or*order*of the model - $\varphi_{i}$: model parameter at index $i$

More: Autoregressive model - Wikipedia

## Moving-average regression model

The *moving-average regression model* takes a slightly different
approach. It takes the average of the time series, and then predicts
the current value based on the error term of *n* previous time
steps. Similar to above, the *n* is referred to as the *order*.

How does this compare to the autoregressive approach above?

Pros:

- The error terms of the lags are modeled to affect the forecasted value directly, rather than indirectly.
- The error influence from lags is limited to the
*order*, and not infinitely like the autoregressive approach.

Cons:

- Fitting the model can be more complicated, as the error terms are not directly observable (they are included within the lag total values).

- $X_{t}$: value to be forecast at time $t$.
- $\mu$: the average of the series
- $\varepsilon_{t}$: error value at time $t$
- $q$: the number of
*lags*to use, or*order*of the model - $\theta_{t}$: the parameter of the model at time
*t*

This is alternatively written with the summmation operator like so:

\[X_{t}=\mu +\varepsilon _{t}+\sum _{i=1}^{q}\theta _{i}\varepsilon _{t-i}.\,\]Note that this model name sounds similar to the *moving average*
concept from statistics, but is conceptually different.

## Autoregressive moving average model (ARMA)

After reviewing the above, *ARMA* becomes much easier to understand,
as it’s simply a combination of the above two.

*ARMA* forecasts the targeted value as a combination of both *p*
previous lag values and *q* previous error values.

## Autoregressive integrated moving average (ARIMA)

The *ARIMA* approach extends *ARMA* by using *difference* values as
opposed to the lag values themselves. This is the *integrated* part.

What is a *difference* value? A *difference* value for time step *t*
would be the differnce between it and the previous value at time step
*t-1*. Or, formulaically:

Using the differences as opposed to the values themselves is an effort
to
eliminate
*non-stationarity*,
or differences in the data when shifted in time. Difference values may
be calculated repeatedly on a series, also giving them an *order*.

## Archive

chinese

tang-dynasty-poetry

李白

王维

python

rl

pytorch

emacs

杜牧

spinningup

numpy

networking

deep-learning

贺知章

白居易

王昌龄

杜甫

李商隐

tips

reinforcement-learning

macports

jekyll

骆宾王

贾岛

孟浩然

time-series

regression

rails

pandas

math

macosx

lesson-plan

helicopters

flying

fastai

conceptual-learning

command-line

bro

黄巢

韦应物

陈子昂

王翰

王之涣

柳宗元

杜秋娘

李绅

张继

孟郊

刘禹锡

元稹

youtube

visdom

system

sungho

stylelint

softmax

siri

sgd

scikit-learn

scikit

research

qtran

qoe

qmix

pyhton

poetry

pedagogy

papers

paper-review

optimization

openssl

openmpi

nyc

neural-net

multiprocessing

mpi

morl

ml

mdp

marl

mandarin

machine-learning

latex

language-learning

khan-academy

jupyter-notebooks

intuition

homebrew

hacking

google-cloud

github

flashcards

faker

dme

deepmind

dec-pomdp

data-wrangling

craftsman

congestion-control

coding

books

book-review

atari

anki

analogy

3brown1blue

2fa