.. py:currentmodule:: Orange.preprocess

###################################
Data Preprocessing (``preprocess``)
###################################

.. index:: preprocessing

.. index::
   single: data; preprocessing

Preprocessing module contains data processing utilities like data
discretization, continuization, imputation and transformation.

Impute
======

Imputation replaces missing values with new values (or omits such features).

.. literalinclude:: code/imputation-default.py

There are several imputation methods one can use.

.. literalinclude:: code/imputation-average.py


.. autoclass::Orange.preprocess.Impute

.. index:: discretize data
   single: feature; discretize

Discretization
==============

Discretization replaces continuous features with the corresponding categorical
features:

.. literalinclude:: code/discretization-table.py

The variable in the new data table indicate the bins to which the original
values belong. ::

    Original dataset:
    [5.1, 3.5, 1.4, 0.2 | Iris-setosa]
    [4.9, 3.0, 1.4, 0.2 | Iris-setosa]
    [4.7, 3.2, 1.3, 0.2 | Iris-setosa]
    Discretized dataset:
    [<5.5, >=3.2, <2.5, <0.8 | Iris-setosa]
    [<5.5, [2.8, 3.2), <2.5, <0.8 | Iris-setosa]
    [<5.5, >=3.2, <2.5, <0.8 | Iris-setosa]


Default discretization method (four bins with approximatelly equal number of
data instances) can be replaced with other methods.

.. literalinclude:: code/discretization-table-method.py
    :lines: 3-5

.. autoclass::Orange.preprocess.Discretize

..
    Transformation procedure
    ------------------------

    `Discretization Algorithms`_ return a discretized variable (with fixed
    parameters) that can transform either the learning or the testing data.
    Parameter learning is separate to the transformation, as in machine
    learning only the training set should be used to induce parameters.

    To obtain discretized features, call a discretization algorithm with
    with the data and the feature to discretize. The feature can be given
    either as an index name or :obj:`Orange.data.Variable`. The following
    example creates a discretized feature::

        import Orange
        data = Orange.data.Table("iris.tab")
        disc = Orange.feature.discretization.EqualFreq(n=4)
        disc_var = disc(data, 0)

    The values of the first attribute will be discretized the data is
    transformed to the  :obj:`Orange.data.Domain` domain that includes
    ``disc_var``.  In the example below we add the discretized first attribute
    to the original domain::

      ndomain = Orange.data.Domain([disc_var] + list(data.domain.attributes),
          data.domain.class_vars)
      ndata = Orange.data.Table(ndomain, data)
      print(ndata)

    The printout::

      [[<5.150000, 5.1, 3.5, 1.4, 0.2 | Iris-setosa],
       [<5.150000, 4.9, 3.0, 1.4, 0.2 | Iris-setosa],
       [<5.150000, 4.7, 3.2, 1.3, 0.2 | Iris-setosa],
       [<5.150000, 4.6, 3.1, 1.5, 0.2 | Iris-setosa],
       [<5.150000, 5.0, 3.6, 1.4, 0.2 | Iris-setosa],
       ...
      ]

_`Discretization Algorithms`
----------------------------

.. autoclass:: Orange.preprocess.discretize.EqualWidth

.. autoclass:: Orange.preprocess.discretize.EqualFreq

.. autoclass:: Orange.preprocess.discretize.EntropyMDL

To add a new discretization, derive it from ``Discretization``.

.. autoclass:: Orange.preprocess.discretize.Discretization

Continuization
==============

.. class:: Orange.preprocess.Continuize

    Given a data table, return a new table in which the discretize attributes
    are replaced with continuous or removed.

    * binary variables are transformed into 0.0/1.0 or -1.0/1.0
      indicator variables, depending upon the argument ``zero_based``.

    * multinomial variables are treated according to the argument
      ``multinomial_treatment``.

    * discrete attribute with only one possible value are removed;

    ::

        import Orange
        titanic = Orange.data.Table("titanic")
        continuizer = Orange.preprocess.Continuize()
        titanic1 = continuizer(titanic)

    The class has a number of attributes that can be set either in constructor
    or, later, as attributes.

    .. attribute:: zero_based

        Determines the value used as the "low" value of the variable. When
        binary variables are transformed into continuous or when multivalued
        variable is transformed into multiple variables, the transformed
        variable can either have values 0.0 and 1.0 (default,
        ``zero_based=True``) or -1.0 and 1.0 (``zero_based=False``).

    .. attribute:: multinomial_treatment

       Defines the treatment of multinomial variables.

       ``Continuize.Indicators``

           The variable is replaced by indicator variables, each
           corresponding to one value of the original variable.
           For each value of the original attribute, only the
           corresponding new attribute will have a value of one and others
           will be zero. This is the default behaviour.

           Note that these variables are not independent, so they cannot be
           used (directly) in, for instance, linear or logistic regression.

           For example, dataset "titanic" has feature "status" with
           values "crew", "first", "second" and "third", in that order. Its
           value for the 15th row is "first". Continuization replaces the
           variable with variables "status=crew", "status=first",
           "status=second" and "status=third". After ::

               continuizer = Orange.preprocess.Continuize()
               titanic1 = continuizer(titanic)

           we have ::

               >>> titanic.domain
               [status, age, sex | survived]
               >>> titanic1.domain
               [status=crew, status=first, status=second, status=third,
                age=adult, age=child, sex=female, sex=male | survived]

           For the 15th row, the variable "status=first" has value 1 and the
           values of the other three variables are 0::

               >>> print(titanic[15])
               [first, adult, male | yes]
               >>> print(titanic1[15])
               [0.000, 1.000, 0.000, 0.000, 1.000, 0.000, 0.000, 1.000 | yes]


       ``Continuize.FirstAsBase``
           Similar to the above, except that it creates indicators for all
           values except the first one, according to the order in the variable's
           :obj:`~Orange.data.DiscreteVariable.values` attribute. If all
           indicators in the transformed data instance are 0, the original
           instance had the first value of the corresponding variable.

           Continuizing the variable "status" with this setting gives variables
           "status=first", "status=second" and "status=third". If all of them
           were 0, the status of the original data instance was "crew".

               >>> continuizer.multinomial_treatment = continuizer.FirstAsBase
               >>> continuizer(titanic).domain
               [status=first, status=second, status=third, age=child, sex=male | survived]

       ``Continuize.FrequentAsBase``
           Like above, except that the most frequent value is used as the
           base. If there are multiple most frequent values, the
           one with the lowest index in
           :obj:`~Orange.data.DiscreteVariable.values` is used. The frequency
           of values is extracted from data, so this option does not work if
           only the domain is given.

           Continuizing the Titanic data in this way differs from the above by
           the attributes sex: instead of "sex=male" it constructs "sex=female"
           since there were more females than males on Titanic. ::

                >>> continuizer.multinomial_treatment = continuizer.FrequentAsBase
                >>> continuizer(titanic).domain
                [status=first, status=second, status=third, age=child, sex=female | survived]

       ``Continuize.Remove``
           Discrete variables are removed. ::

               >>> continuizer.multinomial_treatment = continuizer.Remove
               >>> continuizer(titanic).domain
               [ | survived]

       ``Continuize.RemoveMultinomial``
           Discrete variables with more than two values are removed. Binary
           variables are treated the same as in `FirstAsBase`.

            >>> continuizer.multinomial_treatment = continuizer.RemoveMultinomial
            >>> continuizer(titanic).domain
            [age=child, sex=male | survived]

       ``Continuize.ReportError``
           Raise an error if there are any multinomial variables in the data.

       ``Continuize.AsOrdinal``
           Multinomial variables are treated as ordinal and replaced by
           continuous variables with indices within
           :obj:`~Orange.data.DiscreteVariable.values`, e.g. 0, 1, 2, 3...

                >>> continuizer.multinomial_treatment = continuizer.AsOrdinal
                >>> titanic1 = continuizer(titanic)
                >>> titanic[700]
                [third, adult, male | no]
                >>> titanic1[700]
                [3.000, 0.000, 1.000 | no]

       ``Continuize.AsNormalizedOrdinal``
           As above, except that the resulting continuous value will be from
           range 0 to 1, e.g. 0, 0.333, 0.667, 1 for a four-valued variable::

                >>> continuizer.multinomial_treatment = continuizer.AsNormalizedOrdinal
                >>> titanic1 = continuizer(titanic)
                >>> titanic1[700]
                [1.000, 0.000, 1.000 | no]
                >>> titanic1[15]
                [0.333, 0.000, 1.000 | yes]

    .. attribute:: transform_class

        If ``True`` the class is replaced by continuous
        attributes or normalized as well. Multiclass problems are thus
        transformed to multitarget ones. (Default: ``False``)



.. class:: Orange.preprocess.DomainContinuizer

    Construct a domain in which discrete attributes are replaced by
    continuous. ::

        domain_continuizer = Orange.preprocess.DomainContinuizer()
        domain1 = domain_continuizer(titanic)

    :obj:`Orange.preprocess.Continuize` calls `DomainContinuizer` to construct
    the domain.

    Domain continuizers can be given either a dataset or a domain, and return
    a new domain. When given only the domain, use the most frequent value as
    the base value.

    By default, the class does not change continuous and class attributes,
    discrete attributes are replaced with N attributes (``Indicators``) with
    values 0 and 1.

Normalization
=============

.. autoclass:: Orange.preprocess.Normalize


Randomization
=============

.. autoclass:: Orange.preprocess.Randomize


Remove
======

.. autoclass:: Orange.preprocess.Remove

Feature selection
=================

`Feature scoring`
-----------------

Feature scoring is an assessment of the usefulness of features for
prediction of the dependant (class) variable. Orange provides classes
that compute the common feature scores for classification and regression.

The code below computes the information gain of feature "tear_rate"
in the Lenses dataset:

    >>> data = Orange.data.Table("lenses")
    >>> Orange.preprocess.score.InfoGain(data, "tear_rate")
    0.54879494069539858

An alternative way of invoking the scorers is to construct the scoring
object and calculate the scores for all the features at once, like in the
following example:

    >>> gain = Orange.preprocess.score.InfoGain()
    >>> scores = gain(data)
    >>> for attr, score in zip(data.domain.attributes, scores):
    ...     print('%.3f' % score, attr.name)
    0.039 age
    0.040 prescription
    0.377 astigmatic
    0.549 tear_rate

Feature scoring methods work on different feature types (continuous or discrete)
and different types of target variables (i.e. in classification or regression
problems).
Refer to method's `feature_type` and `class_type` attributes for intended type
or employ preprocessing methods (e.g. discretization) for conversion between
data types.

.. autoclass:: Orange.preprocess.score.ANOVA
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.Chi2
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.GainRatio
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.Gini
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.InfoGain
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.UnivariateLinearRegression
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.FCBF
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.ReliefF
   :members: feature_type, class_type

.. autoclass:: Orange.preprocess.score.RReliefF
   :members: feature_type, class_type

Additionally, you can use the ``score_data()`` method of some learners (\
:obj:`Orange.classification.LinearRegressionLearner`,
:obj:`Orange.regression.LogisticRegressionLearner`,
:obj:`Orange.classification.RandomForestLearner`, and
:obj:`Orange.regression.RandomForestRegressionLearner`)
to obtain the feature scores as calculated by these learners. For example:

    >>> learner = Orange.classification.LogisticRegressionLearner()
    >>> learner.score_data(data)
    [0.31571299907366146,
     0.28286199971877485,
     0.67496525667835794,
     0.99930286901257692]


`Feature selection`
-------------------

We can use feature selection to limit the analysis to only the most relevant
or informative features in the dataset.

Feature selection with a scoring method that works on continuous features will
retain all discrete features and vice versa.

The code below constructs a new dataset consisting of two best features
according to the ANOVA method:

    >>> data = Orange.data.Table("wine")
    >>> anova = Orange.preprocess.score.ANOVA()
    >>> selector = Orange.preprocess.SelectBestFeatures(method=anova, k=2)
    >>> data2 = selector(data)
    >>> data2.domain
    [Flavanoids, Proline | Wine]

.. autoclass:: Orange.preprocess.SelectBestFeatures

Preprocessors
=============