A User's Guide to the differences between numarray and Numeric.

For the most part, one can use the existing Numeric manual
as a manual for numarray (we are in the process of editing
the Numeric manual for use with numarray). The following
should be read to understand what differences there are between
numarray and Numeric. The emphasis is given to differences
between common capabilities; at the end, there are explanations
of new capabilities.

*****
Types
*****

In Numeric there are two ways of referring to the type of an array,
either by using a character code (one character string) or a type
name. For example, one can specify a Numeric float as either
Float, Float32, or 'f'. In fact Float and Float32 are simply
module variables that have value 'f'.

In numarray, types are represented by type objects and not
character codes. As with Numeric there is a module variable Float32,
but now it represents an instance of a FloatingType class.
For example, if x is a Float32 array

x.type() will return a FloatingType instance associated with 32 bit
floats (instead of using x.typecode() as is done in Numeric).
So the following will not work in numarray:

if x.typecode() == 'f':

rather, use:

if x.type() == Float32

[all examples presume "from numarray import *" has been used instead
of "import numarray"]

The advantage of the new scheme is that other kinds of tests become
simpler. The type classes are heirarchical so one can easiy test to
see if the array is an integer array. For example:

if isinstance(x.type(), IntegralType):

or

if isinstance(x.type(), UnsignedIntegralType):

This is the type heirarchy


                                 NumericType
                                 |  |  |  |
                                /   |   \  \______________________
                    ___________/    |    \_________               |
                   |                |              |              |
              IntegralType     BooleanType    FloatingType   ComplexType
               |        |           |           |       |      |     |
              /          \        (Bool)   (Float32) (Float64) |     |
             /            \                                    |     |
            |              |                          (Complex64)
(Complex128)
UnsignedIntegralType   SignedIntegralType
   |         |          |       |       |
(UInt8)   (UInt16)   (Int8)  (Int16) (Int32)
   |         |          |       |       |
    \       /            \      |      /
   UnsignedType            SignedType


In the above, parentheseis indicate instances of a type class. All of
of the Int type classes inherit from Numeric type and one of SignedType
or UnsignedType (multiple inheritance). Thus one can use isinstance()
to see if an array is in a more general class of types.

Numarray defines a number of aliases for the above types. In particular:

              Aliases
Bool
Int8          '1', "i1", "Byte"
Int16         's', "i2", "Short"
Int32         'i', "i4", "Int"
UInt8              "u1", "UByte"
UInt16             "u2", "UShort"
Float32       'f', "f4", "Float"
Float64       'd', "f8", "Double"
Complex64     'F', "c8", "Complex"
Complex128    'D', "c16"

(This is a bit of a lie, the single character "Numeric" codes are not
implemented, but are very easily added)

The aliases are generally accepted whereever a type class is when
used as an argument for a type parameter or keyword. For example
the following are all equivalent:

x = array([2,3], 'f')      # doesnt' work yet
x = array([2,3], Float32)
x = array([2,3], "Float")
x = array([2,3], "f4")

The unsigned integer types have no corresponding type in Numeric,
nor does Bool.

*************
Type Coercion
*************

In expressions involving only arrays, the normal coercion rules
apply (i.e., the same as Numeric). However, the same rules do
not apply to binary operations between arrays and Python scalars
in certain cases. If the kind of number is the same for the array
and scalar (e.g., both are integer types or both are float types),
then the type of the output is determined by the precision of the
array, not the scalar. Some examples will best illustrate:

Scalar type  *  Array type        Numeric result type    numarray result
type
   Int            Int16                Int32                    Int16
   Int            Int8                 Int32                    Int8
   Float          Int8                 Float64                  Float64
   Float          Float32              Float64                  Float32

The change in the rules was made so that it would be easy to preserve the
precision of arrays in expressions involving scalars. Previous solutions
with Numeric were either quite awkward (using a function to create a rank-0
array from a scalar with the desired type) or surprising (the savespace
attribute, that never allowed type coercion). The problem arises because
Python has a limited selection of scalar types. This appears to be the
best solution though it admittedy may surprise some who are used to the
classical type coercion model.

Another twist on type coercion appears when combining a signed int with
an unsigned in. For example, combining an Int16 with an UInt16 results
in an Int32 since neither type can hold the range of the other (we have not
implemented UInt32 since we haven't implemented Int64)

****************
Array Attributes
****************

Numeric arrays have some public attributes. Numarray arrays
have none. All changes to an array's state must be made
through accessor methods. Specifically, Numeric has the following
attributes:

Numeric        numarray accessor method(s)
Attribute

shape      --> shape()  (i.e., specify new shape through method arg instead
of
                         assigning to shape attribute)
flat       --> flat()
real       --> real()   (set capability not implemented yet, but will be)
imag       --> imag()   (ditto)
savespace  --> not used, no equivalent functionality (xxx check this)

Private attributes:

Numarray arrays have lots of them (all preceded by underscores).
Don't mess with them. Changing them in ways inconsistent with
other attributes can result in Python crashing. Only those
contributing to the underlying system code should access these
attributes. Even for read-only purposes you should rely on the
public accesor methods (lest we change the details of the
underlying attributes on you; you were warned!)

*****************
Other differences
*****************

Warning and error messages may have changed.

There are no doubt many other differences (mostly minor we hope)
that we have not discovered (or have forgotten).  Please let us
know about them so we can properly document them.

****************
New capabilities
****************

Index Arrays:

Arrays supplied as arguments to subscripts have
special meaning. If the array is of Bool type, then the indexing will be
treated as the equivalent of the compress function. If the array is of
an Integer type, then a take or put operation is implied. We will
generalize the existing take and put as follows:

If ind1, ind2,...indN are index arrays whose values indicate the index
into another array then

x[ind1, ind2]

forms a new array with the same shape as ind1, ind2 (they all must be
broadcastable to the same shape) and values such:

result[i,j,k] = x[ind1[i,j,k], ind2[i,j,k]]

In this example, ind1, ind2 are index arrays with 3 dimensions (but they
could have an arbitrary number of dimensions).

To illustrate with some specific examples:


>>> # simple index array example
>>> x = 2*arange(10)
>>> ind = array([3,6,2,4,4])
>>> x[ind]

array([ 6, 12,  4,  8,  8])


>>> # index a 2-d array
>>> x = arange(12)
>>> x = reshape(x,(3,4))
>>> x

array([[ 0,  1,  2,  3],
       [ 4,  5,  6,  7],
       [ 8,  9, 10, 11]])

>>> ind1 = array([2,1])
>>> ind2 = array([0,2])
>>> x[ind1, ind2]

array([8, 6])


>>> # multidimensional index arrays
>>> ind1 = array([[2,2],[1,0]])
>>> ind2 = array([[2,1],[0,1]])
>>> x[ind1, ind2]

array([[10,  9],
       [ 4,  1]])


>>> # Mindblowing combination of multidimensional index arrays with
>>> # partial indexing. Strap on your seatbelts.
>>> x[ind1]

array([[[ 8,  9, 10, 11],
   [ 8,  9, 10, 11]],

  [[ 4,  5,  6,  7],
   [ 0,  1,  2,  3]]])

Note that in this last example, each index in the single index array
(ind1) is treated as though x were given only one index. For each
of these 'single' indices, a 1-d array is returned, thus the combination
of the 2 dimensions in the index array combined with the leftover
dimension in the array being indexed produces a 3 dimensional
array.

When using constants for
some of the index positions, then the result uses that constant for all
values. Slices and strides (at least initially) will not be permitted in
the same subscript as index arrays. So


>>> x[ind1, 2]

array([[10, 10],
       [ 6,  2]])

would be legal, but


>>> x[ind1, 1:3]

Traceback (most recent call last):
[...]
    raise IndexError("Cannot mix arrays and slices as indices")
IndexError: Cannot mix arrays and slices as indices

would not be.

Similarly for assignment:

x[ind1, ind2, ind3] = values

will form a new array such that:

x[ind1[i,j,k], ind2[i,j,k], ind3[i,j,k]] = values[i,j,k]

The index arrays and the value array must be broadcast consistent.
(As an example: ind1.shape()=(5,4), ind2.shape()=(5,),
ind3.shape()=(1,4), and values.shape()=(1).

# Index put example, using broadcasting and illustrating that Python
# integer sequences work as indices also.

>>> x = zeros((10,10))
>>> x[[2,5,6],array([0,1,9,3])[:,NewAxis]] = 111
>>> x

array([[  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [111, 111,   0, 111,   0,   0,   0,   0,   0, 111],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [111, 111,   0, 111,   0,   0,   0,   0,   0, 111],
  [111, 111,   0, 111,   0,   0,   0,   0,   0, 111],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0],
  [  0,   0,   0,   0,   0,   0,   0,   0,   0,   0]])


If indices are repeated, the last value encountered will be stored. When
index values are out of range they will be clipped to the appropriate
range. That is to say, negative indices will not have the same meaning
by default [This will change!]. Use of the equivalent take and put
functions will allow other interpretations of the indices (raise
exceptions for out of bounds indices, allow negative indices to work
backwards as they do when used individually, or for indices to wrap
around). The same behavior applies for functions such as choose and
where. [We are planning to change indexing so that negative indices
have the traditional Python interpretation]


>>> x = 2*arange(10)
>>> x[[0, 5, 100, 5]] = [1000, 1005, 1100, 2005]
>>> x

array([1000,    2,    4,    6,    8, 2005,   12,   14,   16, 1100])

Output arguments for Ufunc methods.

Reduce, accumulate, and outer accept an array as the ouput argument.
Such arguments must be of the same type as the expected output
(xxx is this correct)

Arbitrary types for Ufunc output arrays.

Like Numeric, numarray accepts an output array as an output argument
for Ufuncs. Unlike Numeric, the array may have any type (automatic
conversion is performed on the output).

tofile and fromfile capability

There is a fromfile function that creates an array directly
from a file (based on the undocumented file method "readinto").
This avoids the need to read data in through a string. Likewise,
there is a tofile method for arrays to do the reverse.

Examples:

x = fromfile("greatdata.dat", Float32, (100,100))

will create a Float32 100x100 data from the file greatdata.dat.
If one want to read an array offset into the file, open the
file first, seek or otherwise move to the beginning of the
data and then call the function.

f = open("greatdata.dat")
f.seek(2500)
x = from file(f, Float32, (100,100))

(2*x).tofile("doubledgreatdata.dat")

Likewise, tofile will work with fileobjects and one can write multiple
arrays to a single output file that way.

Memory-mapped files:

It is possible to memory map a file and refer to its contents
with Numarray

[Details to be provided later.]

Record Arrays:

[Documentation forthcoming]

Character Arrays:

[Documentation forthcoming]

New Array properties:

It is possible to create an array where the values are intrinsically
byteswapped. Normally we expect that this property will be set by
a function that takes its data from a file and recognizes that the
data are byteswapped and decides for whatever reason (e.g., memory
mapping) that it should not byteswap the data in place. This is not
a property we expect most users to be concerned with explicitly, but
primarily for those that write software that creates arrays from
memory mapped files or read only sources.

It is possible to create arrays with arbitray byte offests and strides
between elements. Since such arrays may have data elements that are
"nonaligned". As with byteswapping, we do not expect users to deal
with this issue explicitly much; it is more for those that write
function that create record or other inhomogeneous, but regular, arrays.