\chapter{Masked Arrays}
\label{cha:masked-arrays}

%begin{latexonly}
\makeatletter
\py@reset
\makeatother
%end{latexonly}
\declaremodule{extension}{numarray.ma}
\moduleauthor{The numarray team}{numpy-discussion@lists.sourceforge.net}
\modulesynopsis{Masked Arrays}
\index{MaskedArray|see{numarray.ma}}
\index{observations, dealing with missing}

\begin{quote}
   Masked arrays are arrays that may have missing or invalid entries. Module
   \module{numarray.ma} provides a nearly work-alike replacement for numarray
   that supports data arrays with masks.
\end{quote}

\section{What is a masked array?}
\label{sec:numarray.ma:what-is-a-masked-array}

Masked arrays are arrays that may have missing or invalid entries. Module
\module{numarray.ma} provides a work-alike replacement for \module{\numarray}
that supports data arrays with masks. A mask is either None or an array of ones
and zeros, that determines for each element of the masked array whether or not
it contains an invalid entry.  The package assures that invalid entries are not
used in computations.  A particular element is said to be masked
(\index{numarray.ma!invalid}invalid) if the mask is not None and the
corresponding element of the mask is 1; otherwise it is unmasked
(\index{numarray.ma!valid}valid).

This package was written by \index{Dubois, Paul F.}Paul F.\ Dubois at Lawrence
Livermore National Laboratory. Please see the legal notice in the software and
section \ref{sec:legal-notice} ``License and disclaimer for packages
numarray.ma''. 

\section{Using numarray.ma}
\label{sec:numarray.ma:using}

Use numarray.ma as a replacement for numarray:
\begin{verbatim}
from numarray.ma import *
>>> x = array([1, 2, 3])
\end{verbatim}
To create an array with the second element invalid, we would do:
\begin{verbatim}
>>> y = array([1, 2, 3], mask = [0, 1, 0])
\end{verbatim}
To create a masked array where all values ``near'' 1.e20 are invalid, we can
do:
\begin{verbatim}
>>> z = masked_values([1.0, 1.e20, 3.0, 4.0], 1.e20)
\end{verbatim}
For a complete discussion of creation methods for masked arrays please see
section \ref{sec:numarray.ma:constructing-mask-arrays} ``Constructing masked
arrays''.

The \module{\numarray} module is an attribute in \module{numarray.ma}, so to
execute a method \method{foo} from numarray, you can reference it as
\method{numarray.foo}.

Usually people use both numarray.ma and numarray this way, but of course you can
always fully-qualify the names:
\begin{verbatim}
>>> import numarray.ma
>>> x = numarray.ma.array([1, 2, 3])
\end{verbatim}

The principal feature of module \module{numarray.ma} is class
\class{MaskedArray}, the class whose instances are returned by the array
constructors and most functions in module \module{numarray.ma}. We will discuss
this class first, and later cover the attributes and functions in module
\module{numarray.ma}. For now suffice it to say that among the attributes of
the module are the constants from module \module{\numarray} including those for
declaring typecodes, \constant{NewAxis}, and the mathematical constants such as
\constant{pi} and \constant{e}.  An additional typecode, \class{MaskType}, is
the typecode used for masks.


\section{Class MaskedArray}
\label{sec:numarray.ma:class-maskedarray}
\index{numarray.ma!MaskedArray@\class{MaskedArray}}

In Module \module{numarray.ma}, an array is an instance of class
\class{MaskedArray}, which is defined in the module \module{numarray.ma}. An
instance of class \class{MaskedArray} can be thought of as containing the
following parts:
\begin{itemize}
\item An array of data, of any shape;
\item A mask of ones and zeros of the same shape as the data where a one value
  (true) indicates that the element is masked and the corresponding data is
  invalid.
\item A ``fill value'' --- this is a value that may be used to replace the
   invalid entries in order to return a plain \module{\numarray} array. The
   chief method that does this is the method \method{filled} discussed below.
\end{itemize}
We will use the terms ``invalid value'' and ``invalid entry'' to refer to the
data value at a place corresponding to a mask value of 1. It should be
emphasized that the invalid values are \emph{never} used in any computation,
and that the fill value is not used for \emph{any} computational purpose. When
an instance \var{x} of class \class{MaskedArray} is converted to its string
representation, it is the result returned by \code{filled(x)} that is converted
to a string.


\subsection{Attributes of masked arrays}
\label{sec:numarray.ma:attr-mask-arrays}

\begin{memberdesc}[MaskedArray]{flat}
   (deprecated) \remark{why deprecated in numarray?}
   Returns the masked array as a one-dimensional one. This is
   provided for compatibility with \module{\numarray}. \method{ravel} is
   preferred.  \member{flat} can be assigned to: \samp{x.flat = value} will
   change the values of \var{x}.
\end{memberdesc}

\begin{memberdesc}[MaskedArray]{real}
   Returns the real part of the array if complex. It can be assigned to:
   \samp{x.real = value} will change the real parts of \var{x}.
\end{memberdesc}

\begin{memberdesc}[MaskedArray]{imaginary}
   Returns the imaginary part of the array if complex. It can be assigned to:
   \samp{x.imaginary = value} will change the imaginary parts of x.
\end{memberdesc}

\begin{memberdesc}[MaskedArray]{shape}
   The shape of a masked array can be accessed or changed by using the special
   attribute \member{shape}, as with \module{\numarray} arrays. It can be
   assigned to: \samp{x.shape = newshape} will change the shape of \var{x}. The
   new shape has to describe the same total number of elements.
   \remark{Correct?}
\end{memberdesc}

\begin{memberdesc}[MaskedArray]{shared_data}
   This read-only flag if true indicates that the masked array shared a
   reference with the original data used to construct it at the time of
   construction. Changes to the original array will affect the masked array.
   (This is not the default behavior; see ``Copying or not''.) This flag is
   informational only.
\end{memberdesc}

\begin{memberdesc}[MaskedArray]{shared_mask}
   This read-only flag if true indicates that the masked array \emph{currently}
   shares a reference to the mask used to create it. Unlike
   \member{shared_data}, this flag may change as the result of modifying the
   array contents, as the mask uses copy on write semantics if it is shared.
\end{memberdesc}



\subsection{Methods on masked arrays}
\label{sec:numarray.ma:meth-mask-arrays}

\begin{methoddesc}[MaskedArray]{__array__}
   A special method allows conversion to a \module{\numarray} array if no
   element is actually masked. If there is a masked element, an
   \exception{numarray.maError} exception is thrown. Many \module{\numarray}
   functions, such as \function{numarray.sqrt}, will attempt this conversion on
   their arguments. See also module function \function{filled} in section
   \ref{sec:numarray.ma:meth-mask-arrays}.
\begin{verbatim}
yn = numarray.array(x)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{astype}{type}
   Return \var{self} as array of given \var{type}. 
\begin{verbatim}
y = x.astype(Float32)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{byte_swapped}{}
   Returns the raw data \class{\numarray} byte-swapped; included for
   consistency with \module{\numarray} but probably meaningless. 
\begin{verbatim}
y = x.byte_swapped()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{compressed}{}
   Return an array of the valid elements. Result is one-dimensional.  
\begin{verbatim}
y = x.compressed()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{count}{axis=None}
   If \var{axis} is \constant{None} return the count of non-masked elements in
   the whole array.  Otherwise return an array of such counts along the axis
   given.
\begin{verbatim}
n = x.count()
y = x.count(0)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{fill_value}{}
   Get the current fill value. 
\begin{verbatim}
v = x.fill_value()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{filled}{fill_value=None}
   Returns a \module{\numarray} array with the masked values replaced by the
   fill value.  See also the description of module function filled in section
   \ref{sec:numarray.ma:meth-mask-arrays}.
\begin{verbatim}
yn = x.filled()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{ids}{}
   Return the ids of the data and mask areas. 
\begin{verbatim}
id1, id2 = x.ids()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{iscontiguous}{}
   Is the data area contiguous? See \method{numarray.scontiguous} in section
   \ref{arraymethod:iscontiguous}.
\begin{verbatim}
if x.iscontiguous():
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{itemsize}{}
   Size of individual data items in bytes. \samp{n = x.itemsize()}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{mask}{}
   Return the data mask, or \constant{None}. 
\begin{verbatim}
m = x.mask()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{put}{values}
   Set the value at each non-masked entry to the corresponding entry in
   \var{values}. The mask is unchanged. See also module function
   \function{put}. 
\begin{verbatim}
x.put(values)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{putmask}{values}
   Eliminate any masked values by setting the value at each masked entry to the
   corresponding entry in \var{values}. Set the mask to \constant{None}.
\begin{verbatim}
x.putmask(values)
assert getmask(x) is None
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{raw_data}{}
   A reference to the non-filled data; portions may be meaningless. Expert use
   only. 
\begin{verbatim}
d = x.raw_data ()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{savespace}{v}
   Set the spacesaver attribute to \var{v}. 
\begin{verbatim}
x.savespace (1)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{set_fill_value}{v}
   Set the fill value to \var{v}. Omit v to restore default.
   \samp{x.set_fill_value(1.e21)} \remark{Give correct default value for v.}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{set_shape}{args...}
   Set the shape. 
\begin{verbatim}
x.set_shape (3, 12)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{size}{axis}
   Number of elements in array, or along a particular \var{axis}. 
\begin{verbatim}
totalsize = x.size ()
col_len = x.size (1)
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{spacesaver}{}
   Query the spacesave flag.
\begin{verbatim}
flag = x.spacesaver()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{tolist}{fill_value=None}
   Return the Python \class{list} \code{self.filled(fill_value).tolist()}; note
   that masked values are filled. 
\begin{verbatim}
alist=x.tolist()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{tostring}{fill_value=None}
   Return the string \code{self.filled(fill_value).tostring()s = x.tostring()}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{typecode}{}
   Return the type of the data. See module \module{Precision}, section \ref{TBD}.
\begin{verbatim}
z = x.typecode()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{unmask}{}
   Replaces the mask by \constant{None} if possible. Subsequent operations may
   be faster if the array previously had an all-zero mask.
\begin{verbatim}
x.unmask()
\end{verbatim}
\end{methoddesc}

\begin{methoddesc}[MaskedArray]{unshare_mask}{}
   If shared_mask is currently true, replaces the reference to it with a
   copy. 
\begin{verbatim}
x.unshare_mask()
\end{verbatim}
\end{methoddesc}


\subsection{Constructing masked arrays}
\label{sec:numarray.ma:constructing-mask-arrays}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{array}
   {data, type=None, copy=1, savespace=0, mask=None, fill_value=None}
   Creates a masked array with the given \var{data} and
   \var{mask}.  The name \class{array} is simply an alias for the class name,
   \class{MaskedArray}.  The fill value is set to \var{fill_value}, and the
   \var{savespace} flag is applied. If \var{data} is a \class{MaskedArray}, its
   \constant{mask}, \constant{typecode}, \constant{spacesaver} flag, and
   \constant{fill_value} will be used unless specifically overridden by one of
   the remaining arguments. In particular, if \var{d} is a masked array,
   \code{array(d, copy=0)} is \var{d}.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_array}{data, mask=None, fill_value=None}
   This is an easier-to-use version of \method{array},
   for the common case of \code{typecode = None}, \code{copy = 0}. When
   \var{data} is newly-created this function can be used to make it a masked
   array without copying the data if \var{data} is already a \module{\numarray}
   array.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_values}{data, value, rtol=1.e-5, atol=1.e-8, type=None, copy=1, savespace=0)}
   Constructs a masked array whose mask is set at those places where 
   \begin{equation}
      \abs(\var{data} - \var{value}) < \var{atol} + \var{rtol} * \abs(\var{data})
   \end{equation}
   That is a careful way of saying that those elements of the \var{data} that
   have a value of \var{value} (to within a tolerance) are to be treated as
   invalid.  If data is not of a floating point type, calls
   \method{masked_object} instead.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_object}{data, value, copy=1, savespace=0}
   Creates a masked array with those entries marked invalid that are equal to
   \var{value}. Again, \var{copy} and \var{/savespace} are passed on to the
   \module{\numarray} array constructor.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{asarray}{data, type=None}
   This is the same as \code{array(data, typecode, copy=0)}. It is a short way
   of ensuring that something is an instance of \class{MaskedArray} of a given
   \var{type} before proceeding, as in \samp{data = asarray(data)}.
   
   If \var{data} already is a masked array and \var{type} is \constant{None}
   then the return value is \var{data}; nothing is copied in that case.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_where}{condition, data, copy=1)}
   Creates a masked array whose shape is that of \var{condition}, whose values
   are those of \var{data}, and which is masked where elements of
   \var{condition} are true.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{datadesc}{masked}
   This is a module constant that represents a scalar masked value. For
   example, if \var{x} is a masked array and a particular location such as
   \code{x[1]} is masked, the quantity \code{x[1]} will be this special
   constant. This special element is discussed more fully in section
   \ref{sec:numarray.ma:constant-masked} ``The constant \constant{masked}''.
\end{datadesc}


The following additional constructors are provided for convenience.

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_equal}{data, value, copy=1}
\end{methoddesc} \index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_greater}{data, value, copy=1}
\end{methoddesc} \index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_greater_equal}{data, value, copy=1}
\end{methoddesc} \index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_less}{data, value, copy=1}
\end{methoddesc} \index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_less_equal}{data, value, copy=1}
\end{methoddesc} \index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_not_equal}{data, value, copy=1}
   \method{masked_greater} is equivalent to \code{masked_where(greater(data,
      value), data))}.  Similarly, \method{masked_greater_equal},
   \method{masked_equal}, \method{masked_not_equal}, \method{masked_less},
   \method{masked_less_equal} are called in the same way with the obvious
   meanings.  Note that for floating point data, \method{masked_values} is
   preferable to \method{masked_equal} in most cases.  \remark{because...}
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_inside}{data, v1, v2, copy=1}
   Creates an array with values in the closed interval \code{[v1, v2]} masked.
   \var{v1} and \var{v2} may be in either order.
\end{methoddesc}

\index{numarray.ma!constructor}
\begin{methoddesc}[MaskedArray]{masked_outside}{data, v1, v2, copy=1}
   Creates an array with values outside the closed interval \code{[v1, v2]}
   masked.  \var{v1} and \var{v2} may be in either order.
\end{methoddesc}

On entry to any of these constructors, \var{data} must be any object which the
\module{\numarray} package can accept to create an array (with the desired
\var{type}, if specified). The \var{mask}, if given, must be \constant{None} or
any object that can be turned into a \module{\numarray} array of integer type
(it will be converted to type \class{MaskType}, if necessary), have the same
shape as \var{data}, and contain only values of 0 or 1.

If the \var{mask} is not \constant{None} but its shape does not match that of
\var{data}, an exception will be thrown, unless one of the two is of length 1,
in which case the scalar will be resized (using \method{numarray.resize}) to
match the other.

See section \ref{sec:numarray.ma:copying-or-not} ``Copying or not'' for a
discussion of whether or not the resulting array shares its data or its mask
with the arguments given to these constructors.


\paragraph*{Important Tip} \method{filled} is very important. It converts its
argument to a plain \module{\numarray} array.

\begin{funcdesc}{filled}{x, value=None}
   Returns \var{x} with any invalid locations replaced by a fill \var{value}.
   \function{filled} is guaranteed to return a plain \module{\numarray} array.
   The argument \var{x} does not have to be a masked array or even an array,
   just something that \module{\numarray}/\module{numarray.ma} can turn into
   one.
   \begin{itemize}
   \item If \var{x} is not a masked array, and not a \module{\numarray} array,
      \code{numarray.array(x)} is returned.
   \item If \var{x} is a contiguous \module{\numarray} array then \var{x} is
      returned. (A \module{\numarray} array is contiguous if its data storage
      region is layed out in column-major order; \module{\numarray} allows
      non-contiguous arrays to exist but they are not allowed in certain
      operations).
   \item If \var{x} is a masked array, but the mask is \constant{None}, and
      \var{x}'s data array is contiguous, then it is returned. If the data
      array is not contiguous, a (contiguous) copy of it is returned.
   \item If \var{x} is a masked array with an actual mask, then an array formed
      by replacing the invalid entries with \var{value}, or
      \code{fill_value(x)} if \var{value} is \constant{None}, is returned. If
      the fill value used is of a different type or precision than \var{x}, the
      result may be of a different type or precision than \var{x}.
\end{itemize}
Note that a new array is created only if necessary to create a correctly
filled, contiguous, \module{\numarray} array.

The function \method{filled} plays a central role in our design. It is the
``exit'' back to \module{\numarray}, and is used whenever the invalid values
must be replaced before an operation. For example, adding two masked arrays
\var{a} and \var{b} is roughly:
\begin{verbatim}
masked_array(filled(a, 0) + filled(b, 0), mask_or(getmask(a), getmask(b))
\end{verbatim}
That is, fill the invalid entries of \var{a} and \var{b} with zeros, add them
up, and declare any entry of the result invalid if either \var{a} or \var{b}
was invalid at that spot. The functions \function{getmask} and
\function{mask_or} are discussed later.

\function{filled} also can be used to simply be certain that some expression is
a contiguous \module{\numarray} array at little cost. If its argument is a
\module{\numarray} array already, it is returned without copying.

If you are certain that a masked array \var{x} contains a mask that is None or
is all zeros, you can convert it to a numarray array with the
\method{numarray.array(x)} constructor. If you turn out to be wrong, an
\exception{MAError} exception is raised.
\end{funcdesc}

\begin{funcdesc}{fill_value}{x}
\end{funcdesc}
\begin{methoddesc}[MaskedArray]{fill_value}{}
   \code{fill_value(x)} and the method \code{x.fill_value()} on masked arrays,
   return a value suitable for filling \var{x} based on its type.  If \var{x}
   is a masked array, then \var{x.fill_value()} results. The returned value for
   a given type can be changed by assigning to the following names in module
   \module{numarray.ma}. They should be set to scalars or one element arrays.
   \index{numarray.ma!default_real_fill_value@\constant{default_real_fill_value}}
   \index{numarray.ma!default_complex_fill_value@\constant{default_complex_fill_value}}
   \index{numarray.ma!default_character_fill_value@\constant{default_character_fill_value}}
   \index{numarray.ma!default_integer_fill_value@\constant{default_integer_fill_value}}
   \index{numarray.ma!default_object_fill_value@\constant{default_object_fill_value}}
\begin{verbatim}
default_real_fill_value = numarray.array([1.0e20], Float32)
default_complex_fill_value = numarray.array([1.0e20 + 0.0j], Complex32)
default_character_fill_value = masked
default_integer_fill_value = numarray.array([0]).astype(UnsignedInt8)
default_object_fill_value = masked
\end{verbatim}
   The variable \var{masked} is a module variable of \module{numarray.ma} and
   is discussed in section \ref{sec:numarray.ma:constant-masked}. Calling
   \function{filled} with a \var{fill_value} of \constant{masked} sometimes
   produces a useful printed representation of a masked array.  The function
   \function{fill_value} works on any kind of object.
\end{methoddesc}

\index{numarray.ma!set_fill_value@\method{set_fill_value}}\code{set_fill_value(a,
   fill_value)} is the same as \code{a.set_fill_value (fill_value)} if \var{a}
   is a masked array; otherwise it does nothing. Please note that the fill
   value is mostly cosmetic; it is used when it is needed to convert the masked
   array to a plain \module{\numarray} array but not involved in most
   operations. In particular, setting the \member{fill_value} to
   \constant{1.e20} will \emph{not}, repeat not, cause elements of the array
   whose values are currently 1.e20 to be masked. For that sort of behavior use
   the \method{masked_value} constructor.



\subsection{What are masks?}
\label{sec:numarray.ma:what-are-masks}
\index{masks, description of}
\index{masks, savespace attribute}

Masks are either \constant{None} or 1-byte \module{\numarray} arrays of 1's and
0's. To avoid excessive performance penalties, mask arrays are never checked to
be sure that the values are 1's and 0's, and supplying a \var{mask} argument to
a constructor with an illegal mask will have undefined consequences later.

\emph{Masks have the savespace attribute set.}  This attribute, discussed in
part \ref{part:numerical-python}, may have surprising consequences if you
attempt to do any operations on them other than those supplied by this package.
In particular, do not add or multiply a quantity involving a mask. For example,
if \var{m} is a mask consisting of 1080 1 values, \code{sum(m)} is 56, not
1080. Oops.


\subsection{Working with masks}

\begin{funcdesc}{is_mask}{m}
   Returns true if \var{m} is of a type and precision that would be allowed as
   the mask field of a masked array (that is, it is an array of integers with
   \module{\numarray}'s typecode \class{MaskType}, or it is \constant{None}).
   To be a legal mask, \var{m} should contain only zeros or ones, but this is
   not checked.
\end{funcdesc}

\begin{funcdesc}{make_mask}{m, copy=0, flag=0}
   Returns an object whose entries are equal to \var{m} and for which
   \function{is_mask} would return true. If \var{m} is already a mask or
   \constant{None}, it returns \var{m} or a copy of it. Otherwise it will
   attempt to make a mask, so it will accept any sequence of integers for
   \var{m}. If \var{flag} is true, \method{make_mask} returns \constant{None}
   if its return value otherwise would contain no true elements. To make a
   legal mask, \var{m} should contain only zeros or ones, but this is not
   checked.
\end{funcdesc}

\begin{funcdesc}{make_mask_none}{s}
   Returns a mask of all zeros of shape \var{s} (deprecated name:
   \index{numarray.ma!create_mask@\method{create_mask}
      (deprecated)|see{\method{make_mask_none}}}create_mask).
\end{funcdesc}

\begin{funcdesc}{getmask}{x}
   Returns \index{numarray.ma!mask@\method{mask}}\code{x.mask()}, the mask of
   \var{x}, if \var{x} is a masked array, and \constant{None} otherwise.
   \note{\function{getmask} may return \constant{None} if \var{x} is a masked
   array but has a mask of \constant{None}.  (Please see caution above about
   operating on the result).}
\end{funcdesc}

\begin{funcdesc}{getmaskarray}{x}
   Returns \code{x.mask()} if \var{x} is a masked array and has a mask that is
   not \constant{None}; otherwise it returns a zero mask array of the same
   shape as \var{x}.  Unlike \method{getmask}, \method{getmaskarray} always
   returns an \module{\numarray} array of typecode \class{MaskType}. (Please
   see caution above about operating on the result).
\end{funcdesc}

\begin{funcdesc}{mask_or}{m1, m2}
   Returns an object which when used as a mask behaves like the element-wise
   ``logical or'' of \var{m1} and \var{m2}, where \var{m1} and \var{/m2} are
   either masks or \constant{None} (e.g., they are the results of calling
   \method{getmask}). A \constant{None} is treated as everywhere false. If both
   \var{m1} and \var{m2} are \constant{None}, it returns \constant{None}. If
   just one of them is \constant{None}, it returns the other. If \var{m1} and
   \var{m2} refer to the same object, a reference to that object is returned.
\end{funcdesc}


\subsection{Operations}
\label{sec:numarray.ma:operations}

Masked arrays support the operators $+$, $*$, $/$, $-$, $**$, and unary plus
and minus.  The other operand can be another masked array, a scalar, a
\module{\numarray} array, or something \method{numarray.array} can convert to a
\module{\numarray} array. The results are masked arrays.

In addition masked arrays support the in-place operators $+=$, $-=$, $*=$, and
$/=$.  Implementation of in-place operators differs from \module{\numarray}
semantics in being more generous about converting the right-hand side to the
required type: any kind or lesser type accepted via an \method{astype}
conversion.  In-place operators truly operate in-place when the target is not
masked.



\subsection{Copying or not?}
\label{sec:numarray.ma:copying-or-not}

Depending on the arguments results of constructors may or may not contain a
separate copy of the data or mask arguments. The easiest way to think about
this is as follows: the given field, be it data or a mask, is required to be a
\module{\numarray} array, possibly with a given typecode, and a mask's shape
must match that of the data. If the copy argument is zero, and the candidate
array otherwise qualifies, a reference will be made instead of a copy. If for
any reason the data is unsuitable as is, an attempt will be made to make a copy
that is suitable. Should that fail, an exception will be thrown. Thus, a
\code{copy=0} argument is more of a hope than a command.

If the basic array \index{numarray.ma!constructor}constructor is given a masked
array as the first argument, its mask, typecode, spacesaver flag, and fill
value will be used unless specifically specified by one of the remaining
arguments. In particular, if \var{d} is a masked array, \code{array(d, copy=0)}
is \var{d}.

Since the default behavior for masks is to use a reference if possible, rather
than a copy, which produces a sizeable time and space savings, it is especially
important not to modify something you used as a mask argument to a masked array
creation routine, if it was a \module{\numarray} array of typecode
\class{MaskType}.





\subsection{Behaviors}
\label{sec:numarray.ma:behaviors}
\begin{funcdesc}{float}{a}
\end{funcdesc}
\begin{funcdesc}{int}{a}
  The conversion operators \function{float}, and \function{int} are defined
  to operate on masked arrays consisting of a single unmasked element.
  Masked values and multi-element arrays are not convertible.  
\end{funcdesc}
\begin{funcdesc}{repr}{a}
\end{funcdesc}
\begin{funcdesc}{str}{a}
  A masked array defines the conversion operators \function{str} and
  \function{repr} by applying the corresponding operator to the
  \module{\numarray} array \code{filled(a)}.  
\end{funcdesc}


\subsection{Indexing and Slicing}
\label{sec:numarray.ma:indexing-slicing}

Indexing and slicing differ from Numeric: while generally the same, they return
a copy, not a reference, when used in an expression that produces a non-scalar
result. Consider this example:
\begin{verbatim}
from Numeric import *
x = array([1.,2.,3.])
y = x[1:]
y[0] = 9.
print x
\end{verbatim}
This will print \code{[1., 9., 3.]} since \code{x[1:]} returns a reference to a
portion of \var{x}.  Doing the same operation using \module{numarray.ma},
\begin{verbatim}
from numarray.ma import *
x = array([1.,2.,3.])
y = x[1:]
y[0] = 9.
print x
\end{verbatim}
will print \code{[1., 2., 3.]}, while \var{y} will be a separate array whose
present value would be \code{[9., 3.]}. While sentiment on the correct
semantics here is divided amongst the Numeric Python community as a whole, it
is not divided amongst the author's community, on whose behalf this package is
written.


\subsection{Indexing in assignments}
\label{sec:numarray.ma:indexing-assignments}

Using multiple sets of square brackets on the left side of an assignment
statement will not produce the desired result:
\begin{verbatim}
x = array([[1,2],[3,4]])
x[1][1] = 20.                           # Error, does not change x
x[1,1] = 20.                            # Correct, changes x
\end{verbatim}
The reason is that \code{x[1]} is a copy, so changing it changes that copy, not
\var{x}.  Always use just one single square bracket for assignments.


\subsection{Operations that produce a scalar result}
\label{sec:numarray.ma:operations-producing-scalars}

If indexing or another operation on a masked array produces a scalar result,
then a scalar value is returned rather than a one-element masked array. This
raises the issue of what to return if that result is masked. The answer is that
the module constant
\index{numarray.ma!masked@\constant{masked}}\constant{masked} is returned. This
constant is discussed in section \ref{sec:numarray.ma:constant-masked}.  While
this most frequently occurs from indexing, you can also get such a result from
other functions. For example, averaging a 1-D array, all of whom's values are
invalid, would result in \constant{masked}.


\subsection{Assignment to elements and slices}
\label{sec:numarray.ma:assignments-elements-slices}

Assignment of a normal value to a single element or slice of a masked array has
the effect of clearing the mask in those locations. In this way previously
\index{numarray.ma!invalid}invalid elements become
\index{numarray.ma!valid}valid. The value being assigned is filled first, so
that you are guaranteed that all the elements on the left-hand side are now
valid.  \remark{???}

Assignment of \constant{None} to a single element or slice of a masked array
has the effect of setting the mask in those locations, and the locations become
invalid.

Since these operations change the mask, the result afterwards will no longer
share a mask, since masks have copy-on-write semantics.



\section{MaskedArray Attributes}
\label{sec:numarray.ma:attributes}

\begin{datadesc}{e}
\end{datadesc}
\begin{datadesc}{pi}
\end{datadesc}
\begin{datadesc}{NewAxis}
   Constants \constant{e}, \constant{pi}, \constant{NewAxis} from
   \module{\numarray}, and the constants from module \module{Precision} that
   define nice names for the typecodes.
\end{datadesc}

The special variables \index{numarray.ma!masked@\constant{masked}}\constant{masked} and
\index{numarray.ma!masked@\constant{masked}}masked_print_option are discussed in section
\ref{sec:numarray.ma:constant-masked}.

The module \module{\numarray} is an element of \module{numarray.ma}, so after \samp{from
   numarray.ma import *}, you can refer to the functions in \module{\numarray} such as
\constant{numarray.ones}; see part \ref{part:numerical-python} for the
constants available in \module{\numarray}.




\section{MaskedArray Functions}
\label{sec:numarray.ma:functions}

Each of the operations discussed below returns an instance of \module{numarray.ma} class
\index{numarray.ma!MaskedArray@\class{MaskedArray}}\class{MaskedArray}, having performed
the desired operation element-wise.  In most cases the array arguments can be
masked arrays or \module{\numarray} arrays or something that \module{\numarray}
can turn into a \module{\numarray} array, such as a list of real numbers.

In most cases, if \module{\numarray} has a function of the same name, the
behavior of the one in \module{numarray.ma} is the same, except that it ``respects'' the
mask.


\subsection{Unary functions}
\label{sec:numarray.ma:unary-functions}

The result of a unary operation will be masked wherever the original operand
was masked. It may also be masked if the argument is not in the domain of the
function.  The following functions have their standard meaning:
\begin{quote}
   \index{absolute@\function{absolute} (in module numarray.ma)}\function{absolute}, 
   \index{arccos@\function{arccos} (in module numarray.ma)}\function{arccos}, 
   \index{arcsin@\function{arcsin} (in module numarray.ma)}\function{arcsin}, 
   \index{arctan@\function{arctan} (in module numarray.ma)}\function{arctan}, 
   \index{around@\function{around} (in module numarray.ma)}\function{around}, 
   \index{conjugate@\function{conjugate} (in module numarray.ma)}\function{conjugate}, 
   \index{cos@\function{cos} (in module numarray.ma)}\function{cos}, 
   \index{cosh@\function{cosh} (in module numarray.ma)}\function{cosh}, 
   \index{exp@\function{exp} (in module numarray.ma)}\function{exp},
   \index{floor@\function{floor} (in module numarray.ma)}\function{floor},
   \index{log@\function{log} (in module numarray.ma)}\function{log}, 
   \index{log10@\function{log10} (in module numarray.ma)}\function{log10}, 
   \index{negative@\function{negative} (in module numarray.ma)}\function{negative}
   (also as operator \index{- (in module numarray.ma)}\index{numarray.ma!-}-),
   \index{nonzero@\function{nonzero} (in module numarray.ma)}\function{nonzero}, 
   \index{sin@\function{sin} (in module numarray.ma)}\function{sin}, 
   \index{sinh@\function{sinh} (in module numarray.ma)}\function{sinh}, 
   \index{sqrt@\function{sqrt} (in module numarray.ma)}\function{sqrt}, 
   \index{tan@\function{tan} (in module numarray.ma)}\function{tan}, 
   \index{tanh@\function{tanh} (in module numarray.ma)}\function{tanh}.
\end{quote}

\begin{funcdesc}{fabs}{x}
   The absolute value of \var{x} as a \constant{Float32} array.
   \remark{What happens when you pass \constant{Float64} ?}
\end{funcdesc}


\subsection{Binary functions}
\label{sec:numarray.ma:binary-functions}

Binary functions return a result that is masked wherever either of the operands
were masked; it may also be masked where the arguments are not in the domain of
the function.

\begin{quote}
   \index{add@\function{add} (in module numarray.ma)}\function{add}
   (also as operator \index{+}\index{numarray.ma!+}+),
   \index{subtract@\function{subtract} (in module numarray.ma)}\function{subtract}
   \index{- (in module numarray.ma)}\index{numarray.ma!-}(also as operator -),
   \index{multiply@\function{multiply} (in module numarray.ma)}\function{multiply}
   \index{* (in module numarray.ma)}\index{numarray.ma!*}(also as operator *), 
   \index{divide@\function{divide} (in module numarray.ma)}\function{divide}
   \index{/ (in module numarray.ma)}\index{numarray.ma!/}(also as operator / ), 
   \index{power@\function{power} (in module numarray.ma)}\function{power}
   \index{** (in module numarray.ma)}\index{numarray.ma!**}(also as operator **), 
   \index{remainder@\function{remainder} (in module numarray.ma)}\function{remainder},
   \index{fmod@\function{fmod} (in module numarray.ma)}\function{fmod},
   \index{hypot@\function{hypot} (in module numarray.ma)}\function{hypot},
   \index{arctan2@\function{arctan2} (in module numarray.ma)}\function{arctan2},
   \index{bitwise_and@\function{bitwise_and} (in module numarray.ma)}\function{bitwise_and},
   \index{bitwise_or@\function{bitwise_or} (in module numarray.ma)}\function{bitwise_or},
   \index{bitwise_xor@\function{bitwise_xor} (in module numarray.ma)}\function{bitwise_xor}.
\end{quote}



\subsection{Comparison operators}

To compare arrays, use the following binary functions. Each of them returns a
masked array of 1's and 0's.

\begin{quote}
   \index{equal@\function{equal} (in module numarray.ma)}\function{equal},
   \index{greater@\function{greater} (in module numarray.ma)}\function{greater},
   \index{greater_equal@\function{greater_equal} (in module numarray.ma)}\function{greater_equal},
   \index{less@\function{less} (in module numarray.ma)}\function{less},
   \index{less_equal@\function{less_equal} (in module numarray.ma)}\function{less_equal},
   \index{not_equal@\function{not_equal} (in module numarray.ma)}\function{not_equal}.
\end{quote}

Note that as in \module{\numarray}, you can use a scalar for one argument and
an array for the other. \note{See section \ref{TBD} why operators and comparison
   functions are not excatly equivalent.}



\subsection{Logical operators}

Arrays of logical values can be manipulated with:

\begin{quote}
   \index{logical_and@\function{logical_and} (in module numarray.ma)}\function{logical_and},
   \index{logical_not@\function{logical_not} (in module numarray.ma)}\function{logical_not (unary)},
   \index{logical_or@\function{logical_or} (in module numarray.ma)}\function{logical_or},
   \index{logical_xor@\function{logical_xor} (in module numarray.ma)}\function{logical_xor}.
\end{quote}

\begin{funcdesc}{alltrue}{x}
   Returns 1 if all elements of \var{x} are true. Masked elements are treated
   as true.
\end{funcdesc}

\begin{funcdesc}{sometrue}{x}
   Returns 1 if any element of \var{x} is true. Masked elements are treated as
   false.
\end{funcdesc}



\subsection{Special array operators}

\begin{funcdesc}{isarray}{x}
   Return true \var{x} is a masked array.
   \remark{What is about \numarray's?}
\end{funcdesc}

\begin{funcdesc}{rank}{x} 
   The number of dimensions in \var{x}.
\end{funcdesc}

\begin{funcdesc}{shape}{x}
   Returns the shape of \var{x}, a tuple of array extents.
\end{funcdesc}

\begin{funcdesc}{resize}{x, shape}
   Returns a new array with specified \var{shape}.
\end{funcdesc}

\begin{funcdesc}{reshape}{x, shape}
   Returns a copy of \var{x} with the given new \var{shape}.
\end{funcdesc}

\begin{funcdesc}{ravel}{x}
   Returns \var{x} as one-dimensional \class{MaskedArray}.
\end{funcdesc}

\begin{funcdesc}{concatenate}{(a0, ... an), axis=0}
   Concatenates the arrays \code{a0, ... an} along the specified \var{axis}.
\end{funcdesc}

\begin{funcdesc}{repeat}{a, repeats, axis=0}
   Repeat elements \var{i} of \var{a} \code{repeats[i]} times along \var{axis}.
   \var{repeats} is a sequence of length \code{a.shape[axis]} telling how many
   times to repeat each element.
\end{funcdesc}

\begin{funcdesc}{identity}{n}
   Returns the identity matrix of shape \var{n} by \var{n}.
\end{funcdesc}

\begin{funcdesc}{indices}{dimensions, type=None}
   Returns an array representing a grid of indices with row-only and
   column-only variation.
\end{funcdesc}

\begin{funcdesc}{len}{x}
   This is defined to be the length of the first dimension of \var{x}. This
   definition, peculiar from the array point of view, is required by the way
   Python implements slicing. Use \function{size} for the total length of
   \var{x}.
\end{funcdesc}

\begin{funcdesc}{size}{x, axis=None}
   This is the total size of \var{x}, or the length of a particular dimension
   \var{axis} whose index is given. When axis is given the dimension of the
   result is one less than the dimension of \var{x}.
\end{funcdesc}

\begin{funcdesc}{count}{x, axis=None}
   Count the number of (non-masked) elements in the array, or in the array
   along a certain \var{axis}.  When \var{axis} is given the dimension of the
   result is one less than the dimension of \var{x}.
\end{funcdesc}

\begin{funcdesc}{arange}{}
\end{funcdesc}
\begin{funcdesc}{arrayrange}{}
\end{funcdesc}
\begin{funcdesc}{diagonal}{}
\end{funcdesc}
\begin{funcdesc}{fromfunction}{}
\end{funcdesc}
\begin{funcdesc}{ones}{}
\end{funcdesc}
\begin{funcdesc}{zeros}{}
   are the same as in numarray, but return masked arrays.
\end{funcdesc}

\begin{funcdesc}{sum}{}
\end{funcdesc}
\begin{funcdesc}{product}{}
   are called the same way as count; the difference is that the result is the
   sum or product of the unmasked element.
\end{funcdesc}

\begin{funcdesc}{average}{x, axis=0, weights=None, returned=0}
   Compute the average value of the non-masked elements of \var{x} along the
   selected \var{axis}. If \var{weights} is given, it must match the size and
   shape of \var{x}, and the value returned is:
   \begin{equation}
      \text{average} = \frac{\sum{}weights_i\cdot{}x_i}{\sum{}weights_i}
   \end{equation}
   In computing these sums, elements that correspond to those that are masked
   in \var{x} or \var{weights} are ignored. If successful a 2-tuple consisting
   of the average and the sum of the weights is returned.
\end{funcdesc}

\begin{funcdesc}{allclose}{x, y, fill_value=1, rtol=1.e-5, atol=1.e-8}
   Test whether or not arrays \var{x} and \var{y} are equal subject to the
   given relative and absolute tolerances. If \var{fill_value} is 1, masked
   values are considered equal, otherwise they are considered different. The
   formula used for elements where both \var{x} and \var{y} have a valid value
   is:
   \begin{equation}
      |x-y| < \var{atol} + \var{rtol} \cdot{} |y|
   \end{equation}
   This means essentially that both elements are small compared to \var{atol}
   or their difference divided by their value is small compared to \var{rtol}.
\end{funcdesc}

\begin{funcdesc}{allequal}{x, y, fill_value=1}
   This function is similar to \function{allclose}, except that exact equality
   is demanded. \note{Consider the problems of floating-point representations
      when using this function on non-integer numbers arrays.}
\end{funcdesc}

\begin{funcdesc}{take}{a, indices, axis=0}
   Returns a selection of items from \var{a}. See the documentation of
   \function{numarray.take} in section \ref{sec:array-functions:take}.
\end{funcdesc}

\begin{funcdesc}{transpose}{a, axes=None}
   Performs a reordering of the axes depending on the tuple of indices
   \var{axes}; the default is to reverse the order of the axes.
\end{funcdesc}

\begin{funcdesc}{put}{a, indices, values}
   The opposite of \function{take}. The values of the array \var{a} at the
   locations specified in \var{indices} are set to the corresponding value of
   \var{values}.  The array \var{a} must be a contiguous array. The argument
   \var{indices} can be any integer sequence object with values suitable for
   indexing into the flat form of \var{a}.  The argument \var{values} must be
   any sequence of values that can be converted to the typecode of \var{a}.
\begin{verbatim}
>>> x = arange(6)
>>> put(x, [2,4], [20,40])
>>> print x
[ 0  1 20  3 40  5 ]
\end{verbatim}
   Note that the target array \var{a} is not required to be one-dimensional.
   Since it is contiguous and stored in row-major order, the array indices can
   be treated as indexing \var{a}s elements in storage order.
   
   The wrinkle on this for masked arrays is that if the locations being set by
   \function{put} are masked, the mask is cleared in those locations.
\end{funcdesc}

\begin{funcdesc}{choose}{condition, t}
   This function has a result shaped like \var{condition}. \var{t} must be a
   tuple. Each element of the tuple can be an array, a scalar, or the constant
   element \constant{masked} (See section \ref{sec:numarray.ma:constant-masked}). Each
   element of the result is the corresponding element of \code{t[i]} where
   \var{condition} has the value \var{i}. The result is masked where
   \var{condition} is masked or where the selected element is masked or the
   selected element of \var{t} is the constant \constant{masked}.
\end{funcdesc}

\begin{funcdesc}{where}{condition, x, y}
   Returns an array that is \code{filled(x)} where \var{condition} is true,
   \code{filled(y)} where the condition is false. One of \var{x} or \var{y} can
   be the constant element \constant{masked} (See section
   \ref{sec:numarray.ma:constant-masked}). The result is masked where \var{condition} is
   masked, where the element selected from \var{x} or \var{y} is masked, or
   where \var{x} or \var{y} itself is the constant \constant{masked} and it is
   selected.
\end{funcdesc}

\begin{funcdesc}{innerproduct}{a, b}
\end{funcdesc}
\begin{funcdesc}{dot}{a, b}
   These functions work as in \module{\numarray}, but missing values don't
   contribute. The result is always a masked array, possibly of length one,
   because of the possibility that one or more entries in it may be invalid
   since all the data contributing to that entry was invalid.
\end{funcdesc}

\begin{funcdesc}{outerproduct}{a, b}
   Produces a masked array such that \code{result[i, j] = a[i] * b[j]}. The
   result will be masked where \code{a[i]} or \code{b[j]} is masked.
\end{funcdesc}

\begin{funcdesc}{compress}{condition, x, dimension=-1}
   Compresses out only those valid values where \var{condition} is true. Masked
   values in \var{condition} are considered false.
\end{funcdesc}

\begin{funcdesc}{maximum}{x, y=None}
\end{funcdesc}
\begin{funcdesc}{minimum}{x, y=None}
   Compute the maximum (minimum) valid values of \var{x} if \var{y} is
   \constant{None}; with two arguments, they return the element-wise larger or
   smaller of valid values, and mask the result where either \var{x} or \var{y}
   is masked.  If both arguments are scalars a scalar is returned.
\end{funcdesc}

\begin{funcdesc}{sort}{x, axis=-1, value=None}
   Returns the array \var{x} sorted along the given axis, with masked values
   treated as if they have a sort value of \var{value} but locations containing
   \var{value} are masked in the result if \var{x} had a mask to start with.
   \note{Thus if \var{x} contains \var{value} at a non-masked spot, but has
      other spots masked, the result may not be what you want.}
\end{funcdesc}

\begin{funcdesc}{argsort}{x, axis=-1, fill_value=None}
   This function is unusual in that it returns a \module{\numarray} array,
   equal to \code{numarray.argsort(filled(x, fill_value), axis)}; this is an
   array of indices for sorting along a given axis.
\end{funcdesc}



\subsection{Controlling the size of the string representations}
\label{sec:numarray.ma:contr-size-string}


\begin{funcdesc}{get_print_limit}{}
\end{funcdesc}
\begin{funcdesc}{set_print_limit}{n=0}
   These functions are used to limit printing of large arrays; query and set
   the limit for converting arrays using \function{str} or \function{repr}.
   
   If an array is printed that is larger than this, the values are not printed;
   rather you are informed of the type and size of the array. If \var{n} is
   zero, the standard \module{\numarray} conversion functions are used.
   
   When imported, \module{numarray.ma} sets this limit to 300, and the limit is also
   made to apply to standard \module{\numarray} arrays as well.
\end{funcdesc}



\section{Helper classes}
\label{sec:numarray.ma:helper-classes}

\begin{quote}
   This section discusses other classes defined in module numarray.ma.
\end{quote}

\begin{classdesc}{MAError}
   This class inherits from Exception, used to raise exceptions in the
   \module{numarray.ma} module. Other exceptions are possible, such as errors from the
   underlying \module{\numarray} module.
\end{classdesc}


\subsection{The constant masked}
\label{sec:numarray.ma:constant-masked}
\index{numarray.ma!masked@\constant{masked} (constant)}

A constant named \index{numarray.ma!masked@\constant{masked}}\constant{masked} in
\module{numarray.ma} serves several purposes.
\begin{enumerate}
\item When a indexing operation on an \class{MaskedArray} instance returns a
   scalar result, but the location indexed was masked, then \constant{masked}
   is returned. For example, given a one-dimensional array \var{x} such that
   \code{x.mask()[3]} is 1, then \code{x[3]} is \constant{masked}.
\item When \constant{masked} is assigned to elements of an array via indexing
   or slicing, those elements become masked. So after \code{x[3] = masked},
   \code{x[3]} is masked.
\item Some other operations that may return scalar values, such as
   \function{average}, may return \constant{masked} if given only invalid data.
\item To test whether or not a variable is this element, use the \function{is}
   or \function{is not} operator, not \code{==} or \code{!=}.
\item Operations involving the constant \constant{masked} may result in an
   exception.  In operations, \constant{masked} behaves as an integer array of
   shape \code{()} with one masked element. For example, using \code{+} for
   illustration,
   \begin{itemize}
   \item \constant{masked} + \constant{masked} is \constant{masked}.
   \item \constant{masked} + numeric scalar or numeric scalar +
      \constant{masked} is \constant{masked}.
   \item \constant{masked} + array or array + \constant{masked} is a masked
      array with all elements \constant{masked} if array is of a numeric type.
      The same is true if array is a \module{\numarray} array.
   \end{itemize}
\end{enumerate}



\subsection{The constant masked_print_option}
\index{numarray.ma!masked_print_option@\constant{masked_print_option} (constant)}


Another constant, \constant{masked_print_option} controls what happens when
masked arrays and the constant
\index{numarray.ma!masked@\constant{masked}}\constant{masked} are printed:

\begin{methoddesc}[masked_print_option]{display}{} 
   Returns a string that may be used to indicate those elements of an array
   that are masked when the array is converted to a string, as happens with the
   print statement.
\end{methoddesc}

\begin{methoddesc}[masked_print_option]{set_display}{string} 
   This functions can be used to set the string that is used to indicate those
   elements of an array that are masked when the array is converted to a
   string, as happens with the print statement.
\end{methoddesc}

\begin{methoddesc}[masked_print_option]{enable}{flag}
   can be used to enable (\var{flag} = 1, default) the use of the display
   string. If disabled (\var{flag} = 0), the conversion to string becomes
   equivalent to \code{str(self.filled())}.
\end{methoddesc}

\begin{methoddesc}[masked_print_option]{enabled}{}
   Returns the state of the display-enabling flag.
\end{methoddesc}


\paragraph*{Example of masked behavior}
\label{sec:numarray.ma:example-mask-behavior}
\begin{verbatim}
>>> from numarray.ma import *
>>> x=arange(5)
>>> x[3] = masked
>>> print x
[0 ,1 ,2 ,-- ,4 ,]
>>> print repr(x)
array(data = 
 [0,1,2,0,4,],
      mask = 
 [0,0,0,1,0,],
      fill_value=[0,])
>>> print x[3]
--
>>> print x[3] + 1.0
--
>>> print masked + x
[-- ,-- ,-- ,-- ,-- ,]
>>> masked_print_option.enable(0)
>>> print x
[0,1,2,0,4,]
>>> print x + masked
[0,0,0,0,0,]
>>> print filled(x+masked, -99)
[-99,-99,-99,-99,-99,]
\end{verbatim}


\begin{classdesc}{masked_unary_function}{f, fill=0, domain=None}
   Given a \index{unary}unary array function \function{f}, give a function
   which when applied to an argument \var{x} returns \function{f} applied to
   the array \code{filled(x, fill)}, with a mask equal to
   \code{mask_or(getmask(x), domain(x))}.
   
   The argument domain therefore should be a callable object that returns true
   where \var{x} is not in the domain of \function{f}. 
\end{classdesc}

The following domains are also supplied as members of module \module{numarray.ma}:
\begin{classdesc}{domain_check_interval}{a, b)(x}
   Returns true where \code{x < a or y > b}.
\end{classdesc}

\begin{classdesc}{domain_tan}{eps}{x}
   This is true where \code{abs(cos (x)) < eps}, that is, a domain suitable for
   the tangent function.
\end{classdesc}

\begin{classdesc}{domain_greater}{v)(x}
   True where \code{x <= v}.
\end{classdesc}

\begin{classdesc}{domain_greater_equal}{v)(x}
   True where x < v.
\end{classdesc}


\begin{classdesc}{masked_binary_function}{f, fillx=0, filly=0}
   Given a binary array function \function{f}, \code{masked_binary_function(f,
      fillx=0, filly=0)} defines a function whose value at \var{x} is
   \code{f(filled(x, fillx), filled (y, filly))} with a resulting mask of
   \code{mask_or(getmask (x), getmask(y))}. The values \var{fillx} and
   \var{filly} must be chosen so that \code{(fillx, filly)} is in the domain of
   \function{f}.
\end{classdesc}

In addition, an instance of
\index{numarray.ma!masked_binary_function@\class{masked_binary_function}}\class{masked_binary_function}
has two methods defined upon it:

\begin{methoddesc}[masked_binary_function]{reduce}{target, axis = 0}
\end{methoddesc}

\begin{methoddesc}[masked_binary_function]{accumulate}{target, axis = 0}
\end{methoddesc}

\begin{methoddesc}[masked_binary_function]{outer}{a, b}
   These methods perform reduction, accumulation, and applying the function in
   an outer-product-like manner, as discussed in the section
   \ref{sec:ufuncs-have-special-methods}.
\end{methoddesc}


\begin{classdesc}{domained_binary_function}{}
   This class exists to implement division-related operations. It is the same
   as \class{masked_binary_function}, except that a new second argument is a
   domain which is used to mask operations that would otherwise cause failure,
   such as dividing by zero. The functions that are created from this class are
   \function{divide}, \function{remainder} (\function{mod}), and
   \function{fmod}.
\end{classdesc}

The following domains are available for use as the domain argument:

\begin{classdesc}{domain_safe_divide}{)(x, y}
   True where \code{absolute(x)*divide_tolerance > absolute(y)}.  As the
   comments in the code say, \emph{better ideas welcome}. The constant
   \index{numarray.ma!divide_tolerance@\constant{divide_tolerance}}\constant{divide_tolerance}
   is set to \constant{1.e-35} in the source and can be changed by editing its
   value in \file{MA.py} and reinstalling. This domain is used for the divide
   operator.
\end{classdesc}


\section{Examples of Using numarray.ma}
\label{sec:numarray.ma:examples-using-ma}


\subsection{Data with a given value representing missing data}
\label{sec:numarray.ma:data-with-given-repr-miss-data}

Suppose we have read a one-dimensional list of elements named \var{x}. We also
know that if any of the values are \constant{1.e20}, they represent missing
data. We want to compute the average value of the data and the vector of
deviations from average.
\begin{verbatim}
>>> from numarray.ma import *
>>> x = array([0.,1.,2.,3.,4.])
>>> x[2] = 1.e20
>>> y = masked_values (x, 1.e20)
>>> print average(y)
2.0
>>> print y-average(y)
[ -2.00000000e+00, -1.00000000e+00,  --,  1.00000000e+00,
        2.00000000e+00,]
\end{verbatim}


\subsection{Filling in the missing data}
\label{sec:numarray.ma:filling-missing-data}

Suppose now that we wish to print that same data, but with the missing values
replaced by the average value.
\begin{verbatim}
>>> print filled (y, average(y))
\end{verbatim}


\subsection{Numerical operations}
\label{sec:numarray.ma:numerical-operations}

We can do numerical operations without worrying about missing values, dividing
by zero, square roots of negative numbers, etc.
\begin{verbatim}
>>> from numarray.ma import *
>>> x=array([1., -1., 3., 4., 5., 6.], mask=[0,0,0,0,1,0])
>>> y=array([1., 2., 0., 4., 5., 6.], mask=[0,0,0,0,0,1])
>>> print sqrt(x/y)
[  1.00000000e+00,  --,  --,  1.00000000e+00, --,  --,]
\end{verbatim}
Note that four values in the result are invalid: one from a negative square
root, one from a divide by zero, and two more where the two arrays \var{x} and
\var{y} had invalid data. Since the result was of a real type, the print
command printed \code{str(filled(sqrt (x/y)))}.



\subsection{Seeing the mask}
\label{sec:numarray.ma:seeing-mask}

There are various ways to see the mask. One is to print it directly, the other
is to convert to the \function{repr} representation, and a third is get the
mask itself.  Use of \function{getmask} is more robust than \code{x.mask()},
since it will work (returning \constant{None}) if \var{x} is a
\module{\numarray} array or list.
\begin{verbatim}
>>> x = arange(10)
>>> x[3:5] = masked
>>> print x
[0 ,1 ,2 ,-- ,-- ,5 ,6 ,7 ,8 ,9 ,]
>>> print repr(x)
*** Masked array, mask present ***
Data:
[0 ,1 ,2 ,-- ,-- ,5 ,6 ,7 ,8 ,9 ,]
Mask (fill value [0,])
[0,0,0,1,1,0,0,0,0,0,]
>>> print getmask(x)
[0,0,0,1,1,0,0,0,0,0,]
\end{verbatim}



\subsection{Filling it your way}
\label{sec:numarray.ma:filling-it-your-way}

If we want to print the data with \constant{-1}'s where the elements are
masked, we use \function{filled}.
\begin{verbatim}
>>> print filled(z, -1)
[ 1.,-1.,-1., 1.,-1.,-1.,]
\end{verbatim}



\subsection{Ignoring extreme values}
\label{sec:numarray.ma:ignore-extreme-values}

Suppose we have an array \var{d} and we wish to compute the average of the
values in \var{d} but ignore any data outside the range -100. to 100.
\begin{verbatim}
v = masked_outside(d, -100., 100.)
print average(v)
\end{verbatim}


\subsection{Averaging an entire multidimensional array}
\label{sec:numarray.ma:averaging-an-entire}

The problem with averaging over an entire array is that the average function
only reduces one dimension at a time. So to average the entire array,
\function{ravel} it first.
\begin{verbatim}
>>> x
*** Masked array, no mask ***
Data:
[[ 0, 1, 2,]
 [ 3, 4, 5,]
 [ 6, 7, 8,]
 [ 9,10,11,]]
>>> average(x)
*** Masked array, no mask ***
Data:
[ 4.5, 5.5, 6.5,]
>>> average(ravel(x))
5.5
\end{verbatim}




%% Local Variables:
%% mode: LaTeX
%% mode: auto-fill
%% fill-column: 79
%% indent-tabs-mode: nil
%% ispell-dictionary: "american"
%% reftex-fref-is-default: nil
%% TeX-auto-save: t
%% TeX-command-default: "pdfeLaTeX"
%% TeX-master: "numarray"
%% TeX-parse-self: t
%% End: