CODA Python

The CODA Python interface consists of a Python package 'coda' containing several functions that allow the user to easily access data inside product files.

The CODA Python interface contains both a wrapping of the CODA C interface as well as a version of the higher level CODA functions as can be found in the IDL and MATLAB interfaces of CODA.

Contents

CODA Definition Path

Note that in order to access products whose formats are defined using .codadef files, you should let CODA now where these .codadef files are stored. By default the CODA Python interface will look for .codadef files in a directory relative to the location of the CODA Python package (../../../../share/coda/definitions).

You can override the default location by setting the CODA_DEFINITION environment variable. This environment variable should be a ':' separated (';' on Windows) list of absolute paths to directories containing .codadef files or absolute paths to .codadef files themselves (or a mix of those).

When you import the CODA Python package this will trigger an initialisation of CODA, so you should make sure that the CODA_DEFINITION environment variable is set before you import the CODA Python package. Setting the environment variable can be performed from within Python using:

import os

os.putenv('CODA_DEFINITION', '<your codadef search path>')

# ...

import coda

High level CODA Data Types

When reading data from a product file, CODA will use the following mapping to translate the ingested data into Python data structures:

CODA classCODA read type / CODA special typePython data type
record coda.Record
array This will be a numpy array (numpy.array) object. The following table relates the CODA array base type to the numpy base type:
CODA classCODA read type / CODA special typenumpy base type
record Python object (coda.Record)
array Python object (a numpy.array object)
integerint8int8
integeruint8uint8
integerint16int16
integeruint16uint16
integerint32int32
integeruint32uint32
integerint64int64
integeruint64uint64
realfloatfloat32
realdoublefloat64
textcharPython object (the Python object is a Python String of length 1)
textstringPython object (Python String)
rawbytesPython object (numpy.array object with base type uint8)
specialno_dataPython object (None)
specialtimefloat64
specialcomplexcomplex64
integerint8Python Integer
integeruint8Python Integer
integerint16Python Integer
integeruint16Python Integer
integerint32Python Integer
integeruint32Python Long
integerint64Python Long
integeruint64Python Long
realfloatPython Float
realdoublePython Float
textcharPython String
textstringPython String
rawbytesnumpy.array object with base type uint8
specialno_dataNone
specialtimePython Float
specialcomplexPython Complex Number

High level CODA Functions

Most high level CODA functions require a start and path parameter. These two parameters together define the location of a data item in a product.

The start parameter determines the offset in the product from which the path parameter is expanded. It can either be a file handle or a CODA Cursor (see Low level CODA Data Types). When a file handle is passed we start at the root of the product and when it is a cursor we start at the position of the cursor.

The path argument is actually a series of parameters (which can also be empty). Starting from the start position, the path parameters should provide valid fieldnames and array indices to navigate deeper into the product. For example, suppose we have a product that has a measurements data set with 100 data set records and in each data set record there is a time field containing a time value of type double. If we want to read the time value in the first data set record (using the file handle filehandle as start parameter) we would use: coda.fetch(filehandle, "measurements", 0, "time"). You can provide each path argument as a separate parameter or combine them together into a single string argument. Alternative calls that have the same result could thus be coda.fetch(filehandle, "measurements[0]", "time") or coda.fetch(filehandle, "measurements[0]/time").

It is also possible to read several data elements at once. We can for instance read the full data set record using coda.fetch(filehandle, "measurements", 0) or even read the whole product using coda.fetch(filehandle). When you read a group of data at once, CODA will create a dynamic data structure in Python (consisting of coda.Record and numpy.array objects to represent records and arrays) for the product data that is read.

The types of arguments that you can use in the list of arguments for path are:

Just as the low level functions, the high level CODA Python functions will throw an exception when an error condition occurs. For the high level functions the exception will be of type coda.CodaError.

coda.open(filename)

This function opens a file and returns a handle to the opened product file.

The high and lowel level coda.open functions are actually one and the same. You can thus use the product file handle that is returned by coda.open both as start parameter in the high level CODA functions mentioned below as well as pf parameter in the low level CODA functions.

coda.open_as(filename, product_class, product_type, version)

This function will try to open the specified file for reading similar to coda.open, but instead of trying to automatically recognise the applicable product class/type/version as coda.open does, this function will impose the format definition that is associated with the given product_class, product_type, and version parameters.

Note that you normally won't need this function as CODA will be able to recognize which format definition to use automatically. However, for the rare occasions where coda.open is not sufficient, you can use this function to force the use of a specific format definition.

You can specify -1 for the version to request the latest available version of the format definition.

The high and lowel level coda.open_as functions are one and the same, just as for coda.open.

coda.close(filehandle)

This function closes the file associated with the file handle filehandle.

Just as the coda.open function, the high and lowel level coda.close functions are one and the same.

coda.get_attributes(start, *path)

Retrieve the attributes of the specified data item.

This function returns a coda.Record containing the attributes of the specified data item.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

This function is deprecated. You can replace it by using coda.fetch(start, *path, "@").

coda.get_description(start, *path)

Retrieve the description of a field.

This function returns a string containing the description in the CODA product format definition of the specified data element.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.fetch(start, *path)

Retrieve data from a product file.

Reads the specified data element from the product file. For instance if pf is a product file handle obtained by calling coda.open() and the product contains a dataset of records then you can retrieve a time field from the first record using:

>>> value = coda.fetch(pf, "datasetname", 0, "time")

You can also combine the path into a single string parameter:

>>> value = coda.fetch(pf, "datasetname[0]/time")

Which path to provide depends on the format of the product you are trying to access.

Instead of just reading individual values, like strings, integers, doubles, etc. it is also possible to read complete arrays or records of data. For instance, you could read the whole first record of the dataset using:

>>> record = coda.fetch(pf, "datasetname", 0)

This gives you a coda.Record containing all the record fields.

It is also possible to read an entire product at once by leaving the data specification argument list empty:

>>> product = coda.fetch(pf)

To read attributes, you can pass '@' as path element. For instance, the following command reads the 'units' attribute value from a temperature variable:

>>> unit = coda.fetch(pf, "temperature@units")

For the coda.fetch function there is an additional feature. If you provide a -1 for one or more of the dimensions of an array you will fetch all elements in the specified dimension(s). For example, with coda.fetch(pf, "datasetname", -1, "dsr_time") you can fetch all dsr_time values for all measurements into a single array. Note that passing -1 only works when passing it as an explicit argument (i.e. calling coda.fetch(pf, "dataset[-1]/dsr_time"), where the -1 index is part of a string argument, will not work).

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.get_field_available(start, *path)

Find out whether a dynamically available record field is available or not.

This function returns True if the record field is available and False if it is not. The last item of the path argument should point to a record field. An empty path is considered an error, even if the start argument is a CODA cursor.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.get_field_count(start, *path)

Retrieve the number of fields in a record.

This function returns the number of fields in the coda.Record instance that will be returned if coda.fetch() is called with the same arguments. The last node on the path should reference a record.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.get_field_names(start, *path)

Retrieve the names of the fields in a record.

This function returns the names of the fields in the coda.Record instance that will be returned if coda.fetch() is called with the same arguments. The last node on the path should reference a record.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.get_size(start, *path)

Retrieve the dimensions of the specified array.

This function returns the dimensions of the array that will be returned if coda.fetch() is called with the same arguments. Thus, you can check what the dimensions of an array are without having to retrieve the entire array with coda.fetch(). The last node on the path should reference an array.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.time_to_string(n_seconds_since_2000)

Convert a number of seconds since 2000-01-01 to a human readable string format. For example:

>>> coda.time_to_string(68260079.0)

would return the string '2002-03-01 01:07:59.000000'.

It is possible to input a list or tuple of doubles, in which case a list of strings will be returned.

coda.time_to_utcstring(n_seconds_since_2000)

Convert a TAI number of seconds since 2000-01-01 (TAI) to a human readable string format in UTC format. For example:

>>> coda.time_to_utcstring(68260111.0)

would return the string '2002-03-01 01:07:59.000000'.

It is possible to input a list or tuple of doubles, in which case a list of strings will be returned.

coda.get_unit(start, *path)

Retrieve unit information.

This function returns a string containing the unit information which is stored in the CODA product format definition for the specified data item.

The start parameter must be a valid CODA file handle that was retrieved with coda.open() or a valid CODA Cursor. If the start argument is a cursor, then the specified path is traversed starting from the position represented by the cursor. The format of the path argument is described at the top of this section.

coda.version()

Retrieve CODA version information.

This function returns a string containing the version number of CODA. The version number is always of the format 'x.y.z', i.e., major, minor, and revision numbers, separated by dots.

coda.set_option_filter_record_fields(enable)

Records like for instance a Main Product Header contain fields that have a fixed value (fieldnames like 'PRODUCT=', quote characters, end of line characters, etc.) or are spare fields. If this option is set to 1 then these kinds of fields will be filtered out when retrieving a record from a product file (using e.g. coda.fetch). If this option is set to 0 then all fields will be returned.

The default value for this option is: 1

This option only effects the higher level CODA Python functions. The lower level functions do not perform filtering on record fields.

coda.get_option_filter_record_fields()

Retrieve the current setting for filtering of record fields.

See also coda.set_option_filter_record_fields(enable).

Low level CODA Data Types

Just as in the C interface the coda_product, coda_type, and coda_cursor types are opaque types. This means that you can not print or inspect these types, but can only pass them around.

To create a new CODA Cursor there is a special coda.Cursor class from which you instantiate new Cursor objects (these objects are opaque wrappers of the underlying cursors in the C domain). You can create a new cursors with: cursor = coda.Cursor(). After creation you will have to initialize it using the coda.cursor_set_product function (just like in C). It is possible to create a copy of a CODA Cursor by using a so-called deep copy:

import copy
cursor = coda.Cursor()
cursor2 = copy.deepcopy(cursor)

Low level CODA Functions

For a description of all low level CODA functions please consult the CODA C interface documentation. There are a few differences between the Python and C interface with respect to certain parameters and error handling. Below you will find an overview of the calling signature for each of the supported low level CODA functions. You'll notice that most differences are rather straightforward (parameters have moved from the parameter list to the list of return values or have been removed). If a change requires more explanation a comment is added to the function definition.

The low level CODA Python functions do not return error codes to indicate succes or failure (as is the case for the C functions). If an error condition occurs, an exception (of type coda.CodacError) will be thrown. You can catch this exception using e.g.:

try:
    # call your CODA function(s) here
except coda.CodacError, e:
    # handle CODA-specific exception
    print "ERROR: %s" % e

If you do not catch the exception, the error message will be printed to the console.

coda.init()

You do not have to call this function yourself to initialize CODA. When CODA is imported in Python the init function will already be called for you. If, however, you call coda.done at any time, you can use this function to re-initialize CODA again.

Note that the coda_set_definition_path function is not provided in the CODA Python interface. You should use the CODA_DEFINITION environment variable to set the definition path as explained in the CODA Definition Path section.

coda.done()

If, for some reason, you want to unload the CODA package, you should first clean up CODA by calling this function. However, unloading a Python package is not a common activity, so you should rarely have to call coda.done().

coda.set_option_bypass_special_types(enable)
coda.get_option_bypass_special_types()
coda.set_option_perform_boundary_checks(enable)
coda.get_option_perform_boundary_checks()
coda.set_option_perform_conversions(enable)
coda.get_option_perform_conversions()
coda.set_option_use_fast_size_expressions(enable)
coda.get_option_use_fast_size_expressions()
coda.set_option_use_mmap(enable)
coda.get_option_use_mmap()
coda.NaN()
coda.isNaN(x)
coda.PlusInf()
coda.MinInf()
coda.isInf(x)
coda.isPlusInf(x)
coda.isMinInf(x)
coda.c_index_to_fortran_index(num_dims, dim, index)
coda.match_filefilter(filter, filepaths, callbackfunc)

The callbackfunc parameter should be a Python function that accepts filepath, status and error as parameters. For example:

>>> def findhelper(filepath, status, error):
...     if status == coda.coda_ffs_match:
...         print "File %s matches filter!" % filepath    
...     elif ((status == coda.coda_ffs_unsupported_file)
...         or (status == coda.coda_ffs_no_match)):
...         # don't print anything if the file does not positively match the filter
...         pass
...     else:
...         print "ERROR: %s (%s)" % (error, filepath)
...     return 0
>>> coda.match_filefilter('', '/home/codauser', findhelper)
[YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC] = coda.time_double_to_parts(datetime)
[YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC] = coda.time_double_to_parts_utc(datetime)
datetime = coda.time_parts_to_double(YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC)
datetime = coda.time_parts_to_double_utc(YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC)
str = coda.time_parts_to_string(YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC, format)
[YEAR, MONTH, DAY, HOUR, MINUTE, SECOND, MUSEC] = coda.time_string_to_parts(format, str)
str = coda.time_double_to_string(datetime, format)
str = coda.time_double_to_string_utc(datetime, format)
datetime = coda.time_string_to_double(format, str)
datetime = coda.time_string_to_double_utc(format, str)
[file_size, format, product_class, product_type, version] = coda.recognize_file(filename)
pf = coda.open(filename)

The product file handle returned by this function can also be used with the high level CODA functions.

pf = coda.open_as(filename, product_class, product_type, version)
coda.close(pf)
filename = coda.get_product_filename(pf)
file_size = coda.get_product_file_size(pf)
format = coda.get_product_format(pf)
product_class = coda.get_product_class(pf)
product_type = coda.get_product_type(pf)
version = coda.get_product_version(pf)
type = coda.get_product_root_type(pf)
backend = coda.get_product_backend(pf)
value = coda.get_product_variable_value(pf, variable, index)
coda.type_get_format_name(type_class)
coda.type_get_class_name(type_class)
coda.type_get_native_type_name(native_type)
coda.type_get_special_type_name(special_type)
has_attributes = coda.type_has_attributes(type)
format = coda.type_get_format(type)
type_class = coda.type_get_class(type)
read_type = coda.type_get_read_type(type)
length = coda.type_get_string_length(type)
bit_size = coda.type_get_bit_size(type)
name = coda.type_get_name(type)
description = coda.type_get_description(type)
unit = coda.type_get_unit(type)
fixed_value = coda.type_get_fixed_value(type)
attributes = coda.type_get_attributes(type)
num_fields = coda.type_get_num_record_fields(type)
index = coda.type_get_record_field_index_from_name(type, name)
index = coda.type_get_record_field_index_from_real_name(type, real_name)
field_type = coda.type_get_record_field_type(type, index)
name = coda.type_get_record_field_name(type, index)
name = coda.type_get_record_field_real_name(type, index)
hidden = coda.type_get_record_field_hidden_status(type, index)
available = coda.type_get_record_field_available_status(type, index)
is_union = coda.type_get_record_union_status(type)
num_dims = coda.type_get_array_num_dims(type)
dim = coda.type_get_array_dim(type)
base_type = coda.type_get_array_base_type(type)
special_type = coda.type_get_special_type(type)
base_type = coda.type_get_special_base_type(type)
coda.cursor_set_product(cursor, product)

You can create a new cursor with cursor = coda.Cursor(). With coda.cursor_set_product this cursor can then be initialized to the root of a product. It is also possible to use a cursor as start parameter in the high level CODA functions.

coda.cursor_goto(cursor, path)
coda.cursor_goto_first_record_field(cursor)
coda.cursor_goto_next_record_field(cursor)
coda.cursor_goto_record_field_by_index(cursor, index)
coda.cursor_goto_record_field_by_name(cursor, name)
coda.cursor_goto_available_union_field(cursor)
coda.cursor_goto_first_array_element(cursor)
coda.cursor_goto_next_array_element(cursor)
coda.cursor_goto_array_element(cursor, subs)

It is not needed to provide the number of dimensions as a parameter (num_subs) since this value is determined from inspecting the length of subs.

coda.cursor_goto_array_element_by_index(cursor, index)
coda.cursor_goto_attributes(cursor)
coda.cursor_goto_root(cursor)
coda.cursor_goto_parent(cursor)
coda.cursor_use_base_type_of_special_type(cursor)
has_ascii_content = coda.cursor_has_ascii_content(cursor)
has_attributes = coda.cursor_has_attributes(cursor)
length = coda.cursor_get_string_length(cursor)
bit_size = coda.cursor_get_bit_size(cursor)
byte_size = coda.cursor_get_byte_size(cursor)
num_elements = coda.cursor_get_num_elements(cursor)
pf = coda.cursor_get_product_file(cursor)
depth = coda.cursor_get_depth(cursor)
index = coda.cursor_get_index(cursor)
bit_offset = coda.cursor_get_file_bit_offset(cursor)
byte_offset = coda.cursor_get_file_byte_offset(cursor)
format = coda.cursor_get_format(cursor)
type_class = coda.cursor_get_type_class(cursor)
read_type = coda.cursor_get_read_type(cursor)
special_type = coda.cursor_get_special_type(cursor)
type = coda.cursor_get_type(cursor)
index = coda.cursor_get_record_field_index_from_name(cursor, name)
available = coda.cursor_get_record_field_available_status(cursor, index)
index = coda.cursor_get_available_union_field_index(cursor)
dim = coda.cursor_get_array_dim(cursor)
dst = coda.cursor_read_int8(cursor)
dst = coda.cursor_read_uint8(cursor)
dst = coda.cursor_read_int16(cursor)
dst = coda.cursor_read_uint16(cursor)
dst = coda.cursor_read_int32(cursor)
dst = coda.cursor_read_uint32(cursor)
dst = coda.cursor_read_int64(cursor)
dst = coda.cursor_read_uint64(cursor)
dst = coda.cursor_read_float(cursor)
dst = coda.cursor_read_double(cursor)
dst = coda.cursor_read_char(cursor)

Since Python does not have a native char type the character data will be returned as a string of length 1.

dst = coda.cursor_read_string(cursor)
dst = coda.cursor_read_bits(cursor, bit_offset, bit_length)
dst = coda.cursor_read_bytes(cursor, offset, length)
dst = coda.cursor_read_int8_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_uint8_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_int16_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_uint16_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_int32_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_uint32_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_int64_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_uint64_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_float_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_double_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

dst = coda.cursor_read_char_array(cursor)

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

Since Python does not have a native char type the character array will be returned as an array of signed 8 bit integers.

dst = coda.cursor_read_int8_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_uint8_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_int16_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_uint16_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_int32_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_uint32_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_int64_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_uint64_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_float_partial_array(curso, offset, lengthr)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_double_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

dst = coda.cursor_read_char_partial_array(cursor, offset, length)

CODA will always return the array data in a flat numpy array object using C array ordering.

Since Python does not have a native char type the character array will be returned as an array of signed 8 bit integers.

dst = coda.cursor_read_complex_double_pair()
dst = coda.cursor_read_complex_double_pairs_array()

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

[dst_re, dst_im] = coda.cursor_read_complex_double_split()
[dst_re, dst_im] = coda.cursor_read_complex_double_split_array()

No array_ordering parameter is required; CODA will always return the array data in a numpy array object using C array ordering.

stringvalue = coda.expression_get_type_name(expression_type)
expr = coda.expression_from_string(exprstring)
coda.expression_delete(expr)
expression_type = coda.expression_get_type(expr)
is_constant = coda.expression_is_constant(expr)
is_equal = coda.expression_is_equal(expr1, expr2)
boolvalue = coda.expression_eval_bool(expr, cursor=None)
integervalue = coda.expression_eval_integer(expr, cursor=None)
doublevalue = coda.expression_eval_float(expr, cursor=None)
stringvalue = coda.expression_eval_string(expr, cursor=None)
coda.expression_eval_node(expr, cursor)

Copyright © 2007-2020 s[&]t, The Netherlands.