Quickstart
==========

.. currentmodule:: patsy

If you prefer to learn by diving in and getting your feet wet, then
here are some cut-and-pasteable examples to play with.

First, let's import stuff and get some data to work with:

.. ipython:: python

   import numpy as np
   from patsy import dmatrices, dmatrix, demo_data
   data = demo_data("a", "b", "x1", "x2", "y", "z column")

:func:`demo_data` gives us a mix of categorical and numerical
variables:

.. ipython:: python

   data

Of course Patsy doesn't much care what sort of object you store
your data in, so long as it can be indexed like a Python dictionary,
``data[varname]``. You may prefer to store your data in a `pandas
<http://pandas.pydata.org>`_ DataFrame, or a numpy
record array... whatever makes you happy.

Now, let's generate design matrices suitable for regressing ``y`` onto
``x1`` and ``x2``.

.. ipython:: python

   dmatrices("y ~ x1 + x2", data)

The return value is a Python tuple containing two DesignMatrix
objects, the first representing the left-hand side of our formula, and
the second representing the right-hand side. Notice that an intercept
term was automatically added to the right-hand side. These are just
ordinary numpy arrays with some extra metadata and a fancy __repr__
method attached, so we can pass them directly to a regression function
like :func:`np.linalg.lstsq`:

.. ipython:: python
   :okwarning:

   outcome, predictors = dmatrices("y ~ x1 + x2", data)
   betas = np.linalg.lstsq(predictors, outcome)[0].ravel()
   for name, beta in zip(predictors.design_info.column_names, betas):
       print("%s: %s" % (name, beta))

Of course the resulting numbers aren't very interesting, since this is just
random data.

If you just want the design matrix alone, without the ``y`` values,
use :func:`dmatrix` and leave off the ``y ~`` part at the beginning:

.. ipython:: python

   dmatrix("x1 + x2", data)

We'll use dmatrix for the rest of the examples, since seeing the
outcome matrix over and over would get boring. This matrix's metadata
is stored in an extra attribute called ``.design_info``, which is a
:class:`DesignInfo` object you can explore at your leisure:

.. ipython::

   In [0]: d = dmatrix("x1 + x2", data)

   @verbatim
   In [0]: d.design_info.<TAB>
   d.design_info.builder              d.design_info.slice
   d.design_info.column_name_indexes  d.design_info.term_name_slices
   d.design_info.column_names         d.design_info.term_names
   d.design_info.describe             d.design_info.term_slices
   d.design_info.linear_constraint    d.design_info.terms

Usually the intercept is useful, but if we don't want it we can get
rid of it:

.. ipython:: python

   dmatrix("x1 + x2 - 1", data)

We can transform variables using arbitrary Python code:

.. ipython:: python

   dmatrix("x1 + np.log(x2 + 10)", data)

Notice that ``np.log`` is being pulled out of the environment where
:func:`dmatrix` was called -- ``np.log`` is accessible because we did
``import numpy as np`` up above. Any functions or variables that you
could reference when calling :func:`dmatrix` can also be used inside
the formula passed to :func:`dmatrix`. For example:

.. ipython:: python

   new_x2 = data["x2"] * 100
   dmatrix("new_x2")

Patsy has some transformation functions "built in", that are
automatically accessible to your code:

.. ipython:: python

   dmatrix("center(x1) + standardize(x2)", data)

See :mod:`patsy.builtins` for a complete list of functions made
available to formulas. You can also define your own transformation
functions in the ordinary Python way:

.. ipython:: python

   def double(x):
       return 2 * x

   dmatrix("x1 + double(x1)", data)

.. currentmodule:: patsy.builtins

This flexibility does create problems in one case, though -- because
we interpret whatever you write in-between the ``+`` signs as Python
code, you do in fact have to write valid Python code. And this can be
tricky if your variable names have funny characters in them, like
whitespace or punctuation. Fortunately, patsy has a builtin
"transformation" called :func:`Q` that lets you "quote" such
variables:

.. ipython::

   In [1]: weird_data = demo_data("weird column!", "x1")

   # This is an error...
   @verbatim
   In [2]: dmatrix("weird column! + x1", weird_data)
   [...]
   PatsyError: error tokenizing input (maybe an unclosed string?)
       weird column! + x1
                   ^

   # ...but this works:
   In [3]: dmatrix("Q('weird column!') + x1", weird_data)

:func:`Q` even plays well with other transformations:

.. ipython:: python

   dmatrix("double(Q('weird column!')) + x1", weird_data)

Arithmetic transformations are also possible, but you'll need to
"protect" them by wrapping them in :func:`I()`, so that Patsy knows
that you really do want ``+`` to mean addition:

.. ipython:: python

   dmatrix("I(x1 + x2)", data)  # compare to "x1 + x2"

.. currentmodule:: patsy

Note that while Patsy goes to considerable efforts to take in data
represented using different Python data types and convert them into a
standard representation, all this work happens *after* any
transformations you perform as part of your formula. So, for example,
if your data is in the form of numpy arrays, "+" will perform
element-wise addition, but if it is in standard Python lists, it will
perform concatenation:

.. ipython:: python

   dmatrix("I(x1 + x2)", {"x1": np.array([1, 2, 3]), "x2": np.array([4, 5, 6])})
   dmatrix("I(x1 + x2)", {"x1": [1, 2, 3], "x2": [4, 5, 6]})

Patsy becomes particularly useful when you have categorical
data. If you use a predictor that has a categorical type (e.g. strings
or bools), it will be automatically coded. Patsy automatically
chooses an appropriate way to code categorical data to avoid
producing a redundant, overdetermined model.

If there is just one categorical variable alone, the default is to
dummy code it:

.. ipython:: python

   dmatrix("0 + a", data)

But if you did that and put the intercept back in, you'd get a
redundant model. So if the intercept is present, Patsy uses
a reduced-rank contrast code (treatment coding by default):

.. ipython:: python

   dmatrix("a", data)

The ``T.`` notation is there to remind you that these columns are
treatment coded.

Interactions are also easy -- they represent the cartesian product of
all the factors involved. Here's a dummy coding of each *combination*
of values taken by ``a`` and ``b``:

.. ipython:: python

   dmatrix("0 + a:b", data)

But interactions also know how to use contrast coding to avoid
redundancy. If you have both main effects and interactions in a model,
then Patsy goes from lower-order effects to higher-order effects,
adding in just enough columns to produce a well-defined model. The
result is that each set of columns measures the *additional*
contribution of this effect -- just what you want for a traditional
ANOVA:

.. ipython:: python

   dmatrix("a + b + a:b", data)

Since this is so common, there's a convenient short-hand:

.. ipython:: python

   dmatrix("a*b", data)

Of course you can use :ref:`other coding schemes
<categorical-coding-ref>` too (or even :ref:`define your own
<categorical-coding>`). Here's :class:`orthogonal polynomial coding
<Poly>`:

.. ipython:: python

   dmatrix("C(c, Poly)", {"c": ["c1", "c1", "c2", "c2", "c3", "c3"]})

You can even write interactions between categorical and numerical
variables. Here we fit two different slope coefficients for ``x1``;
one for the ``a1`` group, and one for the ``a2`` group:

.. ipython:: python

   dmatrix("a:x1", data)

The same redundancy avoidance code works here, so if you'd rather have
treatment-coded slopes (one slope for the ``a1`` group, and a second
for the difference between the ``a1`` and ``a2`` group slopes), then
you can request it like this:

.. ipython:: python

   # compare to the difference between "0 + a" and "1 + a"
   dmatrix("x1 + a:x1", data)

And more complex expressions work too:

.. ipython:: python

   dmatrix("C(a, Poly):center(x1)", data)