User Manual
===========


Quick start
-----------

PyEPR_ provides Python_ bindings for the ENVISAT Product Reader C API
(`EPR API`_) for reading satellite data from ENVISAT_ ESA_ (European
Space Agency) mission.

PyEPR_, as well as the `EPR API`_ for C, supports ENVISAT_ MERIS, AATSR
Level 1B and Level 2 and also ASAR data products. It provides access to
the data either on a geophysical (decoded, ready-to-use pixel samples)
or on a raw data layer. The raw data access makes it possible to read
any data field contained in a product file.

Full access to the Python EPR API is provided by the :mod:`epr` module that
have to be imported by the client program e-g- as follows::

    import epr

The following snippet open an ASAR product and dumps the "Main Processing
Parameters" record to the standard output::

    import epr

    product = epr.Product(
        'ASA_IMP_1PNUPA20060202_062233_000000152044_00435_20529_3110.N1')
    dataset = product.get_dataset('MAIN_PROCESSING_PARAMS_ADS')
    record = dataset.read_record(0)
    print record

.. _PyEPR: https://github.com/avalentino/pyepr
.. _Python: http://www.python.org
.. _`EPR API`: https://github.com/bcdev/epr-api
.. _ENVISAT: http://envisat.esa.int
.. _ESA: http://earth.esa.int


Requirements
------------

In order to use PyEPR it is needed that the following software are
correctly installed and configured:

* Python2_ >= 2.6 or Python3_ >= 3.1
* numpy_ >= 1.3.0
* `EPR API`_ >= 2.2
* a reasonably updated C compiler [#]_ (build only)
* Cython_ >= 0.13 [#]_ (build only)

.. note:: In order to build PyEPR_ for Python3_ it is required Cython_ >= 0.15


.. [#] PyEPR_ has been developed and tested with gcc_ 4.
.. [#] The source tarball of official releases also includes the C extension
       code generated by cython_ so users don's strictly need cython_ to
       install PyEPR_.

       It is only needed to re-generate the C extension code (e.g. if one
       wants to build a development version of PyEPR_).

.. _Python2: Python_
.. _Python3: Python_
.. _numpy: http://www.numpy.org
.. _gcc: http://gcc.gnu.org
.. _Cython: http://cython.org


Download
--------

.. highlight:: sh

Official source tarballs can be downloaded form PyPi_:

    http://pypi.python.org/pypi/pyepr

The source code of the development versions is available on the GitHub_
project page

    https://github.com/avalentino/pyepr

To clone the git_ repository the following command can be used::

    $ git clone https://github.com/avalentino/pyepr.git

.. _PyPi: http://pypi.python.org
.. _GitHub: https://github.com
.. _git: http://git-scm.com


Installation
------------

The easier way to install PyEPR_ is using tools like easy_install_, pip_::

    $ easy_install pyepr

or::

    $ easy_install -U --prefix=<TARGET DIRECTORY>

PyEPR_ uses the standard Python_ distutils_ so it can be installed from
sources using the following command::

    $ python setup.py install

For a user specific installation use::

    $ python setup.py install --user

To install PyEPR_ in a non-standard path::

    $ python setup.py install --prefix=<TARGET_PATH>

just make sure that :file:`<TARGET_PATH>/lib/pythonX.Y/site-packages` is in
the :envvar:`PYTHONPATH`.

For all installation methods described above it is assumed that the `EPR API`_
C library is properly installed in the system (see the Requirements_ section).

It is also possible to use the `EPR API`_ C sources directly to build PyEPR by
specifying the  :option:`--epr-api-src` option::

    $ python setup.py install --epr-api-src=../epr-api/src

.. _easy_install: http://pypi.python.org/pypi/setuptools#using-setuptools-and-easyinstall
.. _pip: http://pypi.python.org/pypi/pip
.. _distutils: http://docs.python.org/distutils


Testing
-------

PyEPR_ package comes with a complete test suite but in order to run it
the ENVISAT sample product used for testing
MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1__
have to be downloaded from the ESA_ website, saved in the :file:`test`
directory and decompressed.

__ http://earth.esa.int/services/sample_products/meris/LRC/L2/MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz

On GNU Linux platforms the following shell commands can be used::

    $ cd pyepr-0.X/test
    $ wget http://earth.esa.int/services/sample_products/meris/LRC/L2/\
      MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz
    $ gunzip MER_LRC_2PTGMV20000620_104318_00000104X000_00000_00000_0001.N1.gz

After installation the test suite can be run using the following command
in the :file:`test` directory::

    $ python test_all.py


Python vs C API
---------------

The Python_ EPR API is fully object oriented.
The main structures of the C API have been implemented as objects while
C function have been logically grouped and mapped onto object methods.

The entire process of defining an object oriented API for Python_ has
been quite easy and straightforward thanks to the good design of the C
API,

Of course there are also some differences that are illustrated in the
following sections.


Memory management
-----------------

.. highlight:: python

Being Python_ a very high level language uses have never to worry about
memory allocation/deallocation. They simply have to instantiate objects::

    product = epr.Product('filename.N1')

and use them freely.

Objects are automatically destroyed when there are no more references to
them and memory is deallocated automatically.

Even better, each object holds a reference to other objects it depends
on so the user never have to worry about identifiers validity or about
the correct order structures have to be feed.

For example: the C `EPR_DatasetId` structure has a field (`product_id`)
that points to the *product* descriptor `EPR_productId` to which it
belongs to.

The reference to the parent product is used, for example, when one wants
to read a record using the `epr_read_record` function:

.. code-block:: c

    EPR_SRecord* epr_read_record(EPR_SDatasetId* dataset_id, ...);

The function takes a `EPR_SDatasetId` as a parameter and assumes all
fields (including ``dataset->product_id``) are valid.
It is responsibility of the programmer to keep all structures valid and
free them at the right moment and in the correct order.

This is the standard way to go in C but not in Python_.

In Python_ all is by far simpler, and the user can get a *dateset*
object instance::

    dataset = product.get_dataset('MAIN_PROCESSING_PARAMS_ADS')

and then forget about the *product* instance it depends on.
Even if the *product* variable goes out of scope and is no more directly
accessible in the program the *dataset* object keeps staying valid since
it holds an internal reference to the *product* instance it depends on.

When *record* is destroyed automatically also the parent :class:`epr.Product`
object is destroyed (assumed there is no other reference to it).

The entire machinery is completely automatic and transparent to the user.


Arrays
------

PyEPR_ uses numpy_ in order to manage efficiently the potentially large
amount of data contained in ENVISAT_ products.

* :meth:`epr.Field.get_elems` return an 1D array containing elements of
  the field
* the `Raster.data` property is a 2D array exposes data contained in the
  :class:`epr.Raster` object in form of :class:`numpy.ndarray`

  .. note::

        :attr:`epr.Raster.data` directly exposes :class:`epr.Raster`
        i.e. shares the same memory buffer with :class:`epr.Raster`::

            >>> raster.get_pixel(i, j)
            5
            >>> raster.data[i, j]
            5
            >>> raster.data[i, j] = 3
            >>> raster.get_pixel(i, j)
            3

* :meth:`epr.Band.read_as_array` is an additional method provided by
  the Python_ EPR API (does not exist any correspondent function in the
  C API). It is mainly a facility method that allows users to get access
  to band data without creating an intermediate :class:`epr.Raster` object.
  It read a slice of data from the :class:`epr.Band` and returns it as a
  2D :class:`numpy.ndarray`.


Enumerators
-----------

Python_ does not have *enumerators* at language level.
Enumerations are simply mapped as module constants that have the same
name of the C enumerate but are spelled all in capital letters.

For example:

============ ============
    C           Pythn
============ ============
e_tid_double E_TID_DOUBLE
e_smod_1OF1  E_SMOD_1OF1
e_smid_log   E_SMID_LOG
============ ============


Error handling and logging
--------------------------

Currently error handling and logging functions of the EPR C API are not
exposed to python.

Internal library logging is completely silenced and errors are converted
to Python_ exceptions.
Where appropriate standard Python_ exception types are use in other cases
custom exception types (e.g. :exc:`epr.EPRError`,
:exc:`epr.EPRValueError`) are used.


Library initialization
----------------------

Differently from the C API library initialization is not needed: it is
performed internally the first time the :mod:`epr` module is imported
in Python_.


High level API
--------------

PyEPR_ provides some utility method that has no correspondent in the C API:

* :meth:`epr.Record.fields`
* :meth:`epr.Record.get_field_names`
* :meth:`epr.Dataset.records`
* :meth:`epr.Product.get_dataset_names`
* :meth:`epr.Product.get_band_names`
* :meth:`epr.Product.datasets`
* :meth:`epr.Product.bands`

Example::

    for dataset in product.datasets():
        for record in dataset.records():
            print record
            print

Another example::

    if 'proc_data' in product.band_names():
        band = product.get_band('proc_data')
        print band


Special methods
---------------

The Python_ EPR API also implements some `special method`_ in order to make
EPR programming even handy and, in short, pythonic_.

The ``__repr__`` methods have been overridden to provide a little more
information with respect to the standard implementation.

In some cases ``__str__`` method have been overridden to output a verbose
string representation of the objects and their contents.

If the EPR object has a ``print_`` method (like e.g. :meth:`epr.Record.print_`
and :meth:`epr.Field.print_`) then the string representation of the object
will have in the same format used by the ``print_`` method.
So writing::

    fd.write(str(record))

giver the same result of::

    record.print_(fd)

Of course the :meth:`epr.Record.print_` method is more efficient for writing
to file.

Also :class:`epr.Dataset` and :class:`epr.Record` classes implement the
``__iter__`` `special method`_ for iterating over records and fields
respectively.
So it is possible to write code like the following::

    for record in dataset:
        for index, field in enumerate(record):
            print index, field

:class:`epr.DSD` and :class:`epr.Filed` classes implement the ``__eq__``
and ``__ne__`` methods for objects comparison::

    if filed1 == field2:
        print 'field 1 and field2 are equal'
        print field1
    else:
        print 'field1:', field1
        print 'field2:', field2

:class:`epr.Field` object also implement the ``__len__`` special method
that returns the number of elements in the field::

    if field.get_type() != epr.E_TID_STRING:
        assert field.get_num_elems() == len(field)
    else:
        assert len(field) == len(field.get_elem())

.. note:: differently from the :meth:`epr.Field.get_num_elems` method
          ``len(field)`` return the number of elements if the field
          type is not :data:`epr.E_TID_STRING`.
          If the field contains a string then the string length is
          returned.

.. _`special method`: http://docs.python.org/reference/datamodel.html
.. _pythonic: http://www.cafepy.com/article/be_pythonic