.. _continuous-random-variables:

====================================
Continuous Statistical Distributions
====================================

Overview
========

All distributions will have location (L) and Scale (S) parameters
along with any shape parameters needed, the names for the shape
parameters will vary. Standard form for the distributions will be
given where :math:`L=0.0` and :math:`S=1.0.` The nonstandard forms can be obtained for the various functions using
(note :math:`U` is a standard uniform random variate).


======================================  ==============================================================================================================================  =========================================================================================================================================
Function Name                           Standard Function                                                                                                               Transformation
======================================  ==============================================================================================================================  =========================================================================================================================================
Cumulative Distribution Function (CDF)  :math:`F\left(x\right)`                                                                                                         :math:`F\left(x;L,S\right)=F\left(\frac{\left(x-L\right)}{S}\right)`
Probability Density Function (PDF)      :math:`f\left(x\right)=F^{\prime}\left(x\right)`                                                                                :math:`f\left(x;L,S\right)=\frac{1}{S}f\left(\frac{\left(x-L\right)}{S}\right)`
Percent Point Function (PPF)            :math:`G\left(q\right)=F^{-1}\left(q\right)`                                                                                    :math:`G\left(q;L,S\right)=L+SG\left(q\right)`
Probability Sparsity Function (PSF)     :math:`g\left(q\right)=G^{\prime}\left(q\right)`                                                                                :math:`g\left(q;L,S\right)=Sg\left(q\right)`
Hazard Function (HF)                    :math:`h_{a}\left(x\right)=\frac{f\left(x\right)}{1-F\left(x\right)}`                                                           :math:`h_{a}\left(x;L,S\right)=\frac{1}{S}h_{a}\left(\frac{\left(x-L\right)}{S}\right)`
Cumulative Hazard Function (CHF)        :math:`H_{a}\left(x\right)=` :math:`\log\frac{1}{1-F\left(x\right)}`                                                            :math:`H_{a}\left(x;L,S\right)=H_{a}\left(\frac{\left(x-L\right)}{S}\right)`
Survival Function (SF)                  :math:`S\left(x\right)=1-F\left(x\right)`                                                                                       :math:`S\left(x;L,S\right)=S\left(\frac{\left(x-L\right)}{S}\right)`
Inverse Survival Function (ISF)         :math:`Z\left(\alpha\right)=S^{-1}\left(\alpha\right)=G\left(1-\alpha\right)`                                                   :math:`Z\left(\alpha;L,S\right)=L+SZ\left(\alpha\right)`
Moment Generating Function (MGF)        :math:`M_{Y}\left(t\right)=E\left[e^{Yt}\right]`                                                                                :math:`M_{X}\left(t\right)=e^{Lt}M_{Y}\left(St\right)`
Random Variates                         :math:`Y=G\left(U\right)`                                                                                                       :math:`X=L+SY`
(Differential) Entropy                  :math:`h\left[Y\right]=-\int f\left(y\right)\log f\left(y\right)dy`                                                             :math:`h\left[X\right]=h\left[Y\right]+\log S`
(Non-central) Moments                   :math:`\mu_{n}^{\prime}=E\left[Y^{n}\right]`                                                                                    :math:`E\left[X^{n}\right]=L^{n}\sum_{k=0}^{N}\left(\begin{array}{c} n\\ k\end{array}\right)\left(\frac{S}{L}\right)^{k}\mu_{k}^{\prime}`
Central Moments                         :math:`\mu_{n}=E\left[\left(Y-\mu\right)^{n}\right]`                                                                            :math:`E\left[\left(X-\mu_{X}\right)^{n}\right]=S^{n}\mu_{n}`
mean (mode, median), var                :math:`\mu,\,\mu_{2}`                                                                                                           :math:`L+S\mu,\, S^{2}\mu_{2}`
skewness, kurtosis                      :math:`\gamma_{1}=\frac{\mu_{3}}{\left(\mu_{2}\right)^{3/2}},\,` :math:`\gamma_{2}=\frac{\mu_{4}}{\left(\mu_{2}\right)^{2}}-3`  :math:`\gamma_{1},\,\gamma_{2}`
======================================  ==============================================================================================================================  =========================================================================================================================================


Moments
-------

Non-central moments are defined using the PDF

.. math::

   \mu_{n}^{\prime}=\int_{-\infty}^{\infty}x^{n}f\left(x\right)dx.

Note, that these can always be computed using the PPF. Substitute :math:`x=G\left(q\right)` in the above equation and get

.. math::

   \mu_{n}^{\prime}=\int_{0}^{1}G^{n}\left(q\right)dq

which may be easier to compute numerically. Note that :math:`q=F\left(x\right)` so that :math:`dq=f\left(x\right)dx.` Central moments are computed similarly :math:`\mu=\mu_{1}^{\prime}`

.. math::
   :nowrap:

    \begin{eqnarray*} \mu_{n} & = & \int_{-\infty}^{\infty}\left(x-\mu\right)^{n}f\left(x\right)dx\\  & = & \int_{0}^{1}\left(G\left(q\right)-\mu\right)^{n}dq\\  & = & \sum_{k=0}^{n}\left(\begin{array}{c} n\\ k\end{array}\right)\left(-\mu\right)^{k}\mu_{n-k}^{\prime}\end{eqnarray*}

In particular

.. math::
   :nowrap:

    \begin{eqnarray*} \mu_{3} & = & \mu_{3}^{\prime}-3\mu\mu_{2}^{\prime}+2\mu^{3}\\  & = & \mu_{3}^{\prime}-3\mu\mu_{2}-\mu^{3}\\ \mu_{4} & = & \mu_{4}^{\prime}-4\mu\mu_{3}^{\prime}+6\mu^{2}\mu_{2}^{\prime}-3\mu^{4}\\  & = & \mu_{4}^{\prime}-4\mu\mu_{3}-6\mu^{2}\mu_{2}-\mu^{4}\end{eqnarray*}

Skewness is defined as

.. math::

     \gamma_{1}=\sqrt{\beta_{1}}=\frac{\mu_{3}}{\mu_{2}^{3/2}}

while (Fisher) kurtosis is

.. math::

     \gamma_{2}=\frac{\mu_{4}}{\mu_{2}^{2}}-3,

so that a normal distribution has a kurtosis of zero.


Median and mode
---------------

The median, :math:`m_{n}` is defined as the point at which half of the density is on one side
and half on the other. In other words, :math:`F\left(m_{n}\right)=\frac{1}{2}` so that

.. math::

     m_{n}=G\left(\frac{1}{2}\right).

In addition, the mode, :math:`m_{d}` , is defined as the value for which the probability density function
reaches it's peak

.. math::

     m_{d}=\arg\max_{x}f\left(x\right).


Fitting data
------------

To fit data to a distribution, maximizing the likelihood function is
common. Alternatively, some distributions have well-known minimum
variance unbiased estimators. These will be chosen by default, but the
likelihood function will always be available for minimizing.

If :math:`f\left(x;\boldsymbol{\theta}\right)` is the PDF of a random-variable where :math:`\boldsymbol{\theta}` is a vector of parameters ( *e.g.* :math:`L` and :math:`S` ), then for a collection of :math:`N` independent samples from this distribution, the joint distribution the
random vector :math:`\mathbf{x}` is

.. math::

     f\left(\mathbf{x};\boldsymbol{\theta}\right)=\prod_{i=1}^{N}f\left(x_{i};\boldsymbol{\theta}\right).

The maximum likelihood estimate of the parameters :math:`\boldsymbol{\theta}` are the parameters which maximize this function with :math:`\mathbf{x}` fixed and given by the data:

.. math::
   :nowrap:

    \begin{eqnarray*} \boldsymbol{\theta}_{es} & = & \arg\max_{\boldsymbol{\theta}}f\left(\mathbf{x};\boldsymbol{\theta}\right)\\  & = & \arg\min_{\boldsymbol{\theta}}l_{\mathbf{x}}\left(\boldsymbol{\theta}\right).\end{eqnarray*}

Where

.. math::
   :nowrap:

    \begin{eqnarray*} l_{\mathbf{x}}\left(\boldsymbol{\theta}\right) & = & -\sum_{i=1}^{N}\log f\left(x_{i};\boldsymbol{\theta}\right)\\  & = & -N\overline{\log f\left(x_{i};\boldsymbol{\theta}\right)}\end{eqnarray*}

Note that if :math:`\boldsymbol{\theta}` includes only shape parameters, the location and scale-parameters can
be fit by replacing :math:`x_{i}` with :math:`\left(x_{i}-L\right)/S` in the log-likelihood function adding :math:`N\log S` and minimizing, thus

.. math::
   :nowrap:

    \begin{eqnarray*} l_{\mathbf{x}}\left(L,S;\boldsymbol{\theta}\right) & = & N\log S-\sum_{i=1}^{N}\log f\left(\frac{x_{i}-L}{S};\boldsymbol{\theta}\right)\\  & = & N\log S+l_{\frac{\mathbf{x}-S}{L}}\left(\boldsymbol{\theta}\right)\end{eqnarray*}

If desired, sample estimates for :math:`L` and :math:`S` (not necessarily maximum likelihood estimates) can be obtained from
samples estimates of the mean and variance using

.. math::
   :nowrap:

    \begin{eqnarray*} \hat{S} & = & \sqrt{\frac{\hat{\mu}_{2}}{\mu_{2}}}\\ \hat{L} & = & \hat{\mu}-\hat{S}\mu\end{eqnarray*}

where :math:`\mu` and :math:`\mu_{2}` are assumed known as the mean and variance of the **untransformed** distribution (when :math:`L=0` and :math:`S=1` ) and

.. math::
   :nowrap:

    \begin{eqnarray*} \hat{\mu} & = & \frac{1}{N}\sum_{i=1}^{N}x_{i}=\bar{\mathbf{x}}\\ \hat{\mu}_{2} & = & \frac{1}{N-1}\sum_{i=1}^{N}\left(x_{i}-\hat{\mu}\right)^{2}=\frac{N}{N-1}\overline{\left(\mathbf{x}-\bar{\mathbf{x}}\right)^{2}}\end{eqnarray*}


Standard notation for mean
--------------------------

We will use

.. math::

    \overline{y\left(\mathbf{x}\right)}=\frac{1}{N}\sum_{i=1}^{N}y\left(x_{i}\right)

where :math:`N` should be clear from context as the number of samples :math:`x_{i}`

References
----------

-  Documentation for ranlib, rv2, cdflib

-  Eric Weisstein's world of mathematics http://mathworld.wolfram.com/,
   http://mathworld.wolfram.com/topics/StatisticalDistributions.html

-  Documentation to Regress+ by Michael McLaughlin item Engineering and
   Statistics Handbook (NIST),
   http://www.itl.nist.gov/div898/handbook/index.htm

-  Documentation for DATAPLOT from NIST,
   http://www.itl.nist.gov/div898/software/dataplot/distribu.htm

-  Norman Johnson, Samuel Kotz, and N. Balakrishnan Continuous
   Univariate Distributions, second edition, Volumes I and II, Wiley &
   Sons, 1994.



Continuous Distributions in `scipy.stats`
-----------------------------------------
.. toctree::
   :maxdepth: 1

   continuous_alpha
   continuous_anglit
   continuous_arcsine
   continuous_beta
   continuous_betaprime
   continuous_bradford
   continuous_burr
   continuous_cauchy
   continuous_chi
   continuous_chi2
   continuous_cosine
   continuous_dgamma
   continuous_dweibull
   continuous_erlang
   continuous_expon
   continuous_exponweib
   continuous_exponpow
   continuous_fatiguelife
   continuous_fisk
   continuous_foldcauchy
   continuous_foldnorm
   continuous_f
   continuous_frechet_r
   continuous_frechet_l
   continuous_gamma
   continuous_genlogistic
   continuous_genpareto
   continuous_genexpon
   continuous_genextreme
   continuous_gengamma
   continuous_genhalflogistic
   continuous_gilbrat
   continuous_gompertz
   continuous_gumbel_r
   continuous_gumbel_l
   continuous_halfcauchy
   continuous_halfnorm
   continuous_halflogistic
   continuous_hypsecant
   continuous_gausshyper
   continuous_invgamma
   continuous_invgauss
   continuous_invweibull
   continuous_johnsonsb
   continuous_johnsonsu
   continuous_ksone
   continuous_kstwobign
   continuous_laplace
   continuous_levy_l
   continuous_levy
   continuous_logistic
   continuous_loglaplace
   continuous_loggamma
   continuous_lognorm
   continuous_maxwell
   continuous_mielke
   continuous_nakagami
   continuous_ncx2
   continuous_ncf
   continuous_nct
   continuous_norm
   continuous_pareto
   continuous_lomax
   continuous_powerlognorm
   continuous_powernorm
   continuous_powerlaw
   continuous_rdist
   continuous_rayleigh
   continuous_rice
   continuous_reciprocal
   continuous_recipinvgauss
   continuous_semicircular
   continuous_t
   continuous_triang
   continuous_truncexpon
   continuous_truncnorm
   continuous_tukeylambda
   continuous_uniform
   continuous_vonmises
   continuous_wald
   continuous_wrapcauchy