File: field_manipulation.rst

package info (click to toggle)
cf-python 1.3.2+dfsg1-4
  • links: PTS, VCS
  • area: main
  • in suites: sid, stretch
  • size: 7,996 kB
  • sloc: python: 51,733; ansic: 2,736; makefile: 78; sh: 2
file content (1088 lines) | stat: -rw-r--r-- 36,613 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
.. set tocdepth in sidebar

:tocdepth: 2

.. currentmodule:: cf
.. default-role:: obj

.. _manipulating-fields:

Manipulating `cf.Field` objects
===============================

Manipulating a field generally involves operating on its data array
and making any necessary changes to the field's domain to make it
consistent with the new array.


Data array
----------

Conversion to a numpy array
^^^^^^^^^^^^^^^^^^^^^^^^^^^

A field's data array may be converted to either an independent numpy
array or a numpy array view (`numpy.ndarray.view`) with its
`~Field.array` and `~Field.varray` attributes respectively:

>>> a = f.array
>>> print a
[[2 -- 4 -- 6]]
>>> a[0, 0] = 999
>>> print a
[[999 -- 4 -- 6]]
>>> print f.array
[[2 -- 4 -- 6]]

Changing the numpy array view in place will also change the
field's data array in-place:

>>> v = f.varray
>>> print v
[[2 -- 4 -- 6]]
>>> v[0, 0] = 999
>>> print f.array
[[999 -- 4 -- 6]]

A field exposes the numpy array interface and so may be used as input
to any of the `numpy array creation functions
<http://docs.scipy.org/doc/numpy/reference/routines.array-creation.html#from-existing-data>`_:

>>> print f.array
[[2 -- 4 -- 6]]
>>> numpy.all(f.array)
True
>>> numpy.all(f)
True

.. note::

   The numpy array created by the `~Field.varray` or `~Field.array`
   attributes forces all of the data to be read into memory at the
   same time, which may not be possible for very large arrays.

Data mask
^^^^^^^^^

A copy of a field's missing data mask is returned by its
`~cf.Field.mask` attribute.

This mask is an independent field in its own right, and so changes to
it will not be seen by the field which generated it. See the
:ref:`assignment section <fm_assignment>` for details on how to edit
the field's mask in place.


Copying
-------

A deep copy of a field may be created with its `~Field.copy` method,
which is functionally equivalent to, but faster than, using the
:py:obj:`copy.deepcopy` function:

>>> g = f.copy()
>>> import copy
>>> g = copy.deepcopy(f)

Copying utilizes :ref:`LAMA copying functionality <LAMA_copying>`.

.. _Subspacing:

Subspacing
----------

Subspacing a field means subspacing its data array and its domain in a
consistent manner.

A field may be subspaced via its `~Field.subspace` attribute. This
attribute returns an object which may be :ref:`indexed <indexing>` to
select a subspace by data array index values (``f.subspace[indices]``)
or :ref:`called <calling>` to select a subspace by dimension
coordinate array values (``f.subspace(**coordinate_values)``):

>>> g = f.subspace[0, ...]
>>> g = f.subspace(latitude=30, longitude=cf.wi(0, 90, 'degrees'))

The result of subspacing a field is a new, independent field whose
data array and, crucially, any data arrays within the field's metadata
(such as coordinates, ancillary variables, coordinate references,
*etc.*) are appropriate subspaces of their originals:

>>> print f
air_temperature field summary
-----------------------------
Data           : air_temperature(time(12), latitude(73), longitude(96)) K
Cell methods   : time: mean
Axes           : time(12) = [1860-01-16 12:00:00, ..., 1860-12-16 12:00:00]
               : latitude(73) = [-90, ..., 90] degrees_north
               : longitude(96) = [0, ..., 356.25] degrees_east
               : height(1) = [2] m
>>> g = f.subspace[-1, :, 48::-1]
>>> print g
air_temperature field summary
-----------------------------
Data           : air_temperature(time(1), latitude(73), longitude(49)) K
Cell methods   : time: mean
Axes           : time(1) = [1860-12-16 12:00:00]
               : latitude(73) = [-90, ..., 90] degrees_north
               : longitude(49) = [180, ..., 0] degrees_east
               : height(1) = [2] m

Subspacing utilizes :ref:`LAMA subspacing functionality
<LAMA_subspacing>`.

.. _indexing:

Indexing
^^^^^^^^

Subspacing by axis indices uses an extended Python slicing syntax,
which is similar to :ref:`numpy array indexing
<numpy:arrays.indexing>`:

>>> f.shape
(12, 73, 96)
>>> f.subspace[...].shape
(12, 73, 96)
>>> f.subspace[slice(0, 12), :, 10:0:-2].shape
(12, 73, 5)
>>> f.subspace[..., f.coord('longitude')<180].shape
(12, 73, 48)

There are three extensions to the numpy indexing functionality:

* Size 1 axes are never removed.

  An integer index *i* takes the *i*-th element but does not reduce
  the rank of the output array by one:

  >>> f.shape
  (12, 73, 96)
  >>> f.subspace[0].shape
  (1, 73, 96)
  >>> f.subspace[3, slice(10, 0, -2), 95:93:-1].shape
  (1, 5, 2)

* The indices for each axis work independently.

  When more than one axis’s slice is a 1-d boolean sequence or 1-d
  sequence of integers, then these indices work independently along
  each axis (similar to the way vector subscripts work in Fortran),
  rather than by their elements:

  >>> f.shape
  (12, 73, 96)
  >>> f.subspace[:, [0, 72], [5, 4, 3]].shape
  (12, 2, 3)

  Note that the indices of the last example would raise an error when
  given to a numpy array.

* Boolean indices may be any object which exposes the numpy array
  interface, such as the field's coordinate objects:

  >>> f.subspace[:, f.coord('latitude')<0].shape
  (12, 36, 96)


.. _calling:

Coordinate values
^^^^^^^^^^^^^^^^^

Subspacing by values of 1-d coordinates allows a subspaced field to be
defined via coordinate values of its domain. The benefits of
subspacing in this fashion are:

* The axes to be subspaced may identified by name.

* The position in the data array of each axis need not be known and
  the axes to be subspaced may be given in any order.

* Axes for which no subspacing is required need not be specified.

* Size 1 axes of the domain which are not spanned by the data array
  may be specified.

Coordinate values are provided as keyword arguments to a call to the
`~Field.subspace` attribute. Coordinates are identified by their
`~Coordinate.identity` or their axis's identifier in the field's
domain.

>>> f.subspace().shape
(12, 73, 96)
>>> f.subspace(latitude=0).shape
(12, 1, 96)
>>> f.subspace(latitude=cf.wi(-30, 30)).shape
(12, 25, 96)
>>> f.subspace(long=cf.ge(270, 'degrees_east'), lat=cf.set([0, 2.5, 10])).shape
(12, 3, 24)
>>> f.subspace(latitude=cf.lt(0, 'degrees_north'))
(12, 36, 96)
>>> f.subspace(latitude=[cf.lt(0, 'degrees_north'), 90])
(12, 37, 96)
>>> import math
>>> f.subspace('exact', longitude=cf.lt(math.pi, 'radian'), height=2)
(12, 73, 48)
>>> f.subspace(height=cf.gt(3))
IndexError: No indices found for 'height' values gt 3
>>> f.subspace(dim2=3.75).shape
(12, 1, 96)
>>> f.subspace(time=cf.le(cf.dt('1860-06-16 12:00:00')).shape
(6, 73, 96)
>>> f.subspace(time=cf.gt(cf.dt(1860, 7)),shape
(5, 73, 96)

Note that if a comparison function (such as `cf.wi`) does not specify
any units, then the units of the named coordinate are assumed.

.. _fm_cyclic_axes:

Cyclic axes
-----------


>>> f.subspace[..., -10, 10]
(12, 25, 96)
>>> f.subspace(longitude=cf.wi(-30, 30))
(12, 3, 24)
>>> f.subspace(long=cf.ge(270, 'degrees_east'), lat=cf.set([0, 2.5, 10])).shape
(12, 3, 24)


.. _fm_assignment:

Assignment
----------

Elements of a field's data array may be changed by assigning values
directly to a subspace of the field defined by the
`~cf.Field.subspace` attribute or by using the `~cf.Field.where`
method.

Assignment uses :ref:`LAMA functionality <LAMA>`, so it is possible to
assign to fields which are larger than the available memory.

Array elements may be set from a field or logically scalar object,
using the same :ref:`metadata-aware broadcasting rules <broadcasting>`
as for field arithmetic and comparison operations. In the
`~cf.Field.subspace` case, the object attribute must be broadcastable
to the defined subspace, whilst in the `~cf.Field.where` case the
object must be broadcastable to the field itself.

The treatment of missing data elements depends on the value of field's
`~cf.Field.hardmask` attribute. If it is True then masked elements
will not unmasked, otherwise masked elements may be set to any
value. In either case, unmasked elements may be set to any value
(including missing data).

Set all values to 273.15:

>>> f.subspace[...] = 273.15

or equivalently:

>>> f.where(True, 273.15, None, i=True)

Set all negative data array values to zero and leave all other
elements unchanged:

>>> g = f.where(f<0, 0)

Double the values in the northern hemisphere:

>>> index = f.indices(longitude=cf.ge(0))
>>> f.subspace[index] *= 2

See `cf.Field.where` for more examples.

Selection
---------

Field selection
^^^^^^^^^^^^^^^

Fields from field lists may be selected according to conditions on
their metadata with the `cf.FieldList.select` method (as well as the
`cf.Field.select` method). Conditions may be given on attributes and
CF properties, domain items of the field (dimension coordinate,
auxiliary coordinate, cell measure or coordinate reference objects),
the number of field domain axes and the number of field data array
axes. For example:

>>> f
[<CF Field: eastward_wind(grid_latitude(110), grid_longitude(106)) m s-1>,
 <CF Field: air_temperature(time(12), latitude(73), longitude(96)) K>]
>>> f.select('air_temperature')
<CF Field: air_temperature(time(12), latitude(73), longitude(96)) K>]

>>>  f.select('air_temperature', rank=2)
[]
>>>  f.select('air_temperature', items={'latitude': cf.gt(0)}, rank=cf.ge(3))
<CF Field: air_temperature(time(12), latitude(73), longitude(96)) K>

Any of the `~FieldList.select` arguments may also be used with
`cf.read` to select fields when reading from files:

>>> f = cf.read('file*.nc', select='air_temperature')
>>> f = cf.read('file*.nc', select_options={'rank': cf.gt(2)})
>>> f = cf.read('file*.nc', select='air_temperature', select_options={'rank': cf.gt(2)})

This may be faster than reading all fields and then selecting afterwards. 

Domain item selection
^^^^^^^^^^^^^^^^^^^^^

Domain items may be retrieved with a variety of methods, some specific
to each item type (such as `cf.Field.dim`) and some more generic (such
as `cf.Field.coords` and `cf.Field.item`):

===========================  ==================================================================
Item                         Field retrieval methods
===========================  ==================================================================
Dimension coordinate object  `~Field.dim`, `~Field.dims`, `~Field.coord`, `~Field.coords`
	                     `~Field.item`, `~Field.items`
Auxiliary coordinate object  `~Field.aux`, `~Field.auxs`, `~Field.coord`, `~Field.coords`
	                     `~Field.item`, `~Field.items`
Cell measure object          `~Field.measure`, `~Field.measures`, `~Field.item`, `~Field.items`
Coordinate reference object  `~Field.ref`, `~Field.refs`, `~Field.item`, `~Field.items`
===========================  ==================================================================

In each case the singular method form (such as `~Field.aux`) returns
an actual domain item whereas the plural method form (such as
`~Field.auxs`) returns a dictionary whose keys are the domain item
identifiers with corresponding values of the items themselves.

For example, to retrieve a unique dimension coordinate object with a
standard name of "time":

>>> f.dim('time')
<CF DimensionCoordinate: time(12) noleap>

To retrieve all coordinate objects and their domain identifiers:

>>> f.coords()
{'dim0': <CF DimensionCoordinate: time(12) noleap>,
 'dim1': <CF DimensionCoordinate: latitude(64) degrees_north>,
 'dim2': <CF DimensionCoordinate: longitude(128) degrees_east>,
 'dim3': <CF DimensionCoordinate: height(1) m>}

To retrieve all domain items and their domain identifiers:

>>> f.items()
{'dim0': <CF DimensionCoordinate: time(12) noleap>,
 'dim1': <CF DimensionCoordinate: latitude(64) degrees_north>,
 'dim2': <CF DimensionCoordinate: longitude(128) degrees_east>,
 'dim3': <CF DimensionCoordinate: height(1) m>}

In this example, all of the items happen to be coordinates.

Aggregation
-----------

Fields are aggregated into as few multidimensional fields as possible
with the `cf.aggregate` function, which implements the `CF aggregation
rules
<http://www.met.reading.ac.uk/~david/cf_aggregation_rules.html>`_.

>>> f
[<CF Field: air_temperature(time(12), latitude(73), longitude(96)) K>,
 <CF Field: air_temperature(latitude(73), longitude(96)) K @ 273.15>]
>>> print f
air_temperature field summary
-----------------------------
Data           : air_temperature(time(12), latitude(73), longitude(96)) K
Cell methods   : time: mean
AXes           : time(12) = [1860-01-16 12:00:00, ..., 1860-12-16 12:00:00]
               : latitude(73) = [-90, ..., 90] degrees_north
               : longitude(96) = [0, ..., 356.25] degrees_east
               : height(1) = [2] m
air_temperature field summary
-----------------------------
Data           : air_temperature(latitude(73), longitude(96)) K @ 273.15
Cell methods   : time: mean
Axes           : time(12) = [1859-12-16 12:00:00]
               : longitude(96) = [356.25, ..., 0] degrees_east
               : latitude(73) = [-90, ..., 90] degrees_north
               : height(1) = [2] m
...
>>> g = cf.aggregate(f)
>>> g
[<CF Field: air_temperature(time(13), latitude(73), longitude(96)) K>]
>>> print g
air_temperature field summary
-----------------------------
Data           : air_temperature(time(13), latitude(73), longitude(96)) K
Cell methods   : time: mean
Axes           : time(13) = [1859-12-16 12:00:00, ..., 1860-12-16 12:00:00]
               : latitude(73) = [-90, ..., 90] degrees_north
               : longitude(96) = [0, ..., 356.25] degrees_east
               : height(1) = [2] m

By default, the fields returned by `cf.read` have been aggregated:

>>> f = cf.read('file*.nc')
>>> len(f)
1
>>> f = cf.read('file*.nc', aggregate=False)
>>> len(f)
12


.. _Arithmetic-and-comparison:

Arithmetic and comparison
-------------------------

Arithmetic, bitwise and comparison operations are defined on a field
as element-wise operations on its data array which yield a new
`cf.Field` object or, for augmented assignments, modify the field's
data array in-place.

A field's data array is modified in a very similar way to how a numpy
array would be modified in the same operation, i.e. :ref:`broadcasting
<broadcasting>` ensures that the operands are compatible and the data
array is modified element-wise.

Broadcasting is metadata-aware and will automatically account for
arbitrary configurations, such as axis order, but will not allow
fields with incompatible metadata to be combined, such as adding a
field of height to one of temperature.

The :ref:`resulting field's metadata <resulting_metadata>` will be
very similar to that of the operands which are also
fields. Differences arise when the existing metadata can not correctly
describe the newly created field. For example, when dividing a field
with units of *metres* by one with units of *seconds*, the resulting
field will have units of *metres per second*.

Arithmetic and comparison utilizes :ref:`LAMA functionality <LAMA>` so
data arrays larger than the available physical memory may be operated
on.

.. _broadcasting:

Broadcasting
^^^^^^^^^^^^

The term broadcasting describes how data arrays of the operands with
different shapes are treated during arithmetic, comparison and
assignment operations. Subject to certain constraints, the smaller
array is "broadcast" across the larger array so that they have
compatible shapes.

The general broadcasting rules are similar to the :mod:`broadcasting
rules implemented in numpy <numpy.doc.broadcasting>`, the only
difference occurring when both operands are fields, in which case the
fields are temporarily conformed so that:

* The fields have matching units.

* Axes are aligned according to their coordinates' metadata to ensure
  that matching axes are broadcast against each other.

* Common axes have matching axis directions.

This restructuring of the field ensures that the matching axes are
broadcast against each other.

Broadcasting is done without making needless copies of data and so is
usually very efficient.


Valid operands
^^^^^^^^^^^^^^

A field may be combined or compared with the following objects:

+----------------+----------------------------------------------------+
| Object         | Description                                        |
+================+====================================================+
|:py:obj:`int`,  | The field's data array is combined with            |
|:py:obj:`long`, | the python scalar                                  |
|:py:obj:`float` |                                                    |
+----------------+----------------------------------------------------+
|`cf.Data`       | The field's data array                             |
|with size 1     | is combined with the `cf.Data` object's scalar     |
|                | value, taking into account:                        |
|                |                                                    |
|                | * Different but equivalent units                   |
+----------------+----------------------------------------------------+
|`cf.Field`      | The two field's must satisfy the field combination |
|                | rules. The fields' data arrays and domains are     |
|                | combined taking into account:                      |
|                |                                                    |
|                | * Axis identities                                  |
|                | * Array units                                      |
|                | * Axis orders                                      |
|                | * Axis directions                                  |
|                | * Missing data values		              |
+----------------+----------------------------------------------------+


A field may appear on the left or right hand side of an operator.

.. warning::

   Combining a numpy array on the *left* with a field on the *right*
   does work, but will give generally unintended results -- namely a
   numpy array of fields.


.. _resulting_metadata:

Resulting metadata
^^^^^^^^^^^^^^^^^^

When creating a new field which has different physical properties to
the input field(s) the units will also need to be changed:

>>> f.units
'K'
>>> f += 2
>>> f.units
'K'

>>> f.units
'K'
>>> f **= 2
>>> f.units
'K2'

>>> f.units, g.units
('m', 's')
>>> h = f / g
>>> h.units
'm s-1'

When creating a new field which has a different domain to the input
fields, the new domain will in general contain the superset of the
axes of the two input fields, but may not have some of either input
field's auxiliary coordinates or size 1 dimension coordinates. Refer
to the field combination rules for details.


.. _floating_point_errors:

Floating point errors
^^^^^^^^^^^^^^^^^^^^^

It is possible to set the action to take when an arithmetic operation
produces one of the following floating-point errors:

.. tabularcolumns:: |l|l|

=================  =================================
Error              Description                      
=================  =================================
Division by zero   Infinite result obtained from    
                   finite numbers.

Overflow           Result too large to be expressed.

Invalid operation  Result is not an expressible     
                   number, typically indicates that 
                   a NaN was produced.

Underflow          Result so close to zero that some
                   precision was lost.
=================  =================================

For each type of error, one of the following actions may be chosen:

* Take no action. Allows invalid values to occur in the result data
  array.

* Print a `RuntimeWarning` (via the Python `warnings` module). Allows
  invalid values to occur in the result data array.

* Raise a `FloatingPointError` exception.

The treatment of floating-point errors is set with
`cf.Data.seterr`. Converting invalid numbers to masked values after an
arithmetic operation may be done with the `cf.Field.mask_invalid`
method. It is also possible to mask invalid numbers during arithmetic
operations (see `cf.Data.mask_fpe`).

Note that these setting apply to all data array arithmetic within the
`cf` package.

Statistical operations
----------------------

Axes of a field may be collapsed by statistical methods with the
`cf.Field.collapse` method. Collapsing an axis involves reducing its
size with a given (typically statistical) method.

By default all axes with size greater than 1 are collapsed completely
with the given method. For example, to find the minumum of the data
array:

>>> g = f.collapse('min')

By default the calculations of means, standard deviations and
variances use a combination of volume, area and linear weights based
on the field's metadata. For example to find the mean of the data
array, weighted where possible:

>>> g = f.collapse('mean')

Specific weights may be forced with the weights parameter. For example
to find the variance of the data array, weighting the X and Y axes by
cell area, the T axis linearly and leaving all other axes unweighted:

>>> g = f.collapse('variance', weights=['area', 'T'])

A subset of the axes may be collapsed. For example, to find the mean
over the time axis:

>>> f
<CF Field: air_temperature(time(12), latitude(73), longitude(96) K>
>>> g = f.collapse('T: mean')
>>> g
<CF Field: air_temperature(time(1), latitude(73), longitude(96) K>

For example, to find the maximum over the time and height axes:

>>> g = f.collapse('T: Z: max')

or, equivalently:

>>> g = f.collapse('max', axes=['T', 'Z'])

An ordered sequence of collapses over different (or the same) subsets
of the axes may be specified. For example, to first find the mean over
the time axis and subequently the standard deviation over the latitude
and longitude axes:

>>> g = f.collapse('T: mean area: sd')

or, equivalently, in two steps:

>>> g = f.collapse('mean', axes='T').collapse('sd', axes='area')

Grouped collapses are possible, whereby groups of elements along an
axis are defined and each group is collapsed independently. The
collapsed groups are concatenated so that the collapsed axis in the
output field has a size equal to the number of groups. For example, to
find the variance along the longitude axis within each group of size
10 degrees:

>>> g = f.collapse('longitude: variance', group=cf.Data(10, 'degrees'))

Climatological statistics (a type of grouped collapse) as defined by
the CF conventions may be specified. For example, to collapse a time
axis into multiannual means of calendar monthly minima:

>>> g = f.collapse('time: minimum within years T: mean over years',
...                 within_years=cf.M())

In all collapses, missing data array elements are accounted for in the
calculation.

The following collapse methods are available, over any subset of the
axes:

=========================  =====================================================
Method                     Notes
=========================  =====================================================
Maximum                    The maximum of the values.
                           
Minimum                    The minimum of the values.
                                    
Sum                        The sum of the values.
                           
Mid-range                  The average of the maximum and the minimum of the
                           values.
                           
Range                      The absolute difference between the maximum and
                           the minimum of the values.
                           
Mean                       The unweighted mean, :math:`m`, of :math:`N`
                           values :math:`x_i` is
                           
                           .. math:: m=\frac{1}{N}\sum_{i=1}^{N} x_i
                           
                           The weighted mean, :math:`\tilde{m}`, of :math:`N`
                           values :math:`x_i` with corresponding weights
                           :math:`w_i` is
                           

                           .. math:: \tilde{m}=\frac{1}{\sum_{i=1}^{N} w_i}
                                               \sum_{i=1}^{N} w_i x_i
                           
Standard deviation         The unweighted standard deviation, :math:`s`, of
                           :math:`N` values :math:`x_i` with mean :math:`m`
                           and with :math:`N-ddof` degrees of freedom
                           (:math:`ddof\ge0`) is
                           
                           .. math:: s=\sqrt{\frac{1}{N-ddof}
                                       \sum_{i=1}^{N} (x_i - m)^2}
                           
                           The weighted standard deviation,
                           :math:`\tilde{s}_N`, of :math:`N` values
                           :math:`x_i` with corresponding weights
                           :math:`w_i`, weighted mean
                           :math:`\tilde{m}` and with :math:`N`
                           degrees of freedom is
                           
                           .. math:: \tilde{s}_N=\sqrt{\frac{1}
                                         {\sum_{i=1}^{N} w_i}
                                         \sum_{i=1}^{N} w_i(x_i -
                                         \tilde{m})^2}
                           
                           The weighted standard deviation,
                           :math:`\tilde{s}`, of :math:`N` values
                           :math:`x_i` with corresponding weights
                           :math:`w_i` and with :math:`N-ddof` degrees
                           of freedom :math:`(ddof>0)` is
                           
                           .. math:: \tilde{s}=\sqrt{ \frac{a
                                     \sum_{i=1}^{N} w_i}{a
                                     \sum_{i=1}^{N} w_i - ddof}}
                                     \tilde{s}_N
                           
                           where :math:`a` is the smallest positive
                           number whose product with each weight is an
                           integer. :math:`a \sum_{i=1}^{N} w_i` is
                           the size of a new sample created by each
                           :math:`x_i` having :math:`aw_i` repeats. In
                           practice, :math:`a` may not exist or may be
                           difficult to calculate, so :math:`a` is
                           either set to a predetermined value or an
                           approximate value is calculated (see
                           `cf.Field.collapse` for details).
                           
Variance                   The variance is the square of the standard
                           deviation.
                           
Sample size                The sample size, :math:`N`, as would be used for 
                           other statistical calculations.
                           
Sum of weights             The sum of sample weights,
                           :math:`\sum_{i=1}^{N} w_i`, as would be
                           used for other statistical calculations.

Sum of squares of weights  The sum of squares of sample weights,
                           :math:`\sum_{i=1}^{N} {w_i}^{2}`,
                           as would be used for other statistical
                           calculations.
=========================  =====================================================

See `cf.Field.collapse` for more details.

Regridding operations
---------------------

A field may be regridded onto a new latitude-longitude grid:

>>> f
<CF Field: air_temperature(time(12), latitude(73), longitude(96) K>
>>> g
<CF Field: precipitation(time(24), longitude(128), latitude(64)) kg m-2 s-1>
>>> h = f.regrids(g)
>>> h
<CF Field: air_temperature(time(12), longitude(128), latitude(64) K>

By default the interpolation is first-order conservative, but bilinear
interpolation is also possible. The missing data masks of the field
and the new grid are aslo taken into account. 

See `cf.Field.regrids` for more details.

.. _units:

Units
-----

A field (as well as any other object which :ref:`inherits
<inheritance_diagrams>` from `cf.Variable`) always contains a
`cf.Units` object which gives the physical units of the values
contained in its data array.

The `cf.Units` object is stored in the field's `~Field.Units`
attribute but may also be accessed through the field's `~Field.units`
and `~Field.calendar` CF properties, which may take any value allowed
by the `CF conventions
<http://cf-pcmdi.llnl.gov/documents/cf-conventions/latest-cf-conventions-document-1>`_. In
particular, the value of the `~Field.units` CF property is a string
that can be recognized by `UNIDATA's Udunits-2 package
<http://www.unidata.ucar.edu/software/udunits/>`_, with a few
exceptions for greater consistency with CF. These are detailed by the
`cf.Units` object.

Assignment
^^^^^^^^^^

The Field's units may be assigned directly to its `cf.Units` object:

>>> f.Units.units = 'days since 1-1-1'
>>> f.Units.calendar = 'noleap'

>>> f.Units = cf.Units('metre')

But the same result is achieved by assigning to the field's
`~Field.units` and `~Field.calendar` CF properties:

>>> f.units = 'days since 1-1-1'
>>> f.calendar = 'noleap'
>>> f.Units
<CF Units: days since 1-1-1 calendar=noleap>
>>> f.units
'days since 1-1-1'
>>> f.calendar
'noleap'


Time units
^^^^^^^^^^

Time units may be given as durations of time or as an amount of time
since a reference time:

>>> f.units = 'day'
>>> f.units = 'seconds since 1992-10-8 15:15:42.5 -6:00'

.. note::

   It is recommended that the units ``'year'`` and ``'month'`` be used
   with caution, as explained in the following excerpt from the CF
   conventions: "The Udunits package defines a year to be exactly
   365.242198781 days (the interval between 2 successive passages of
   the sun through vernal equinox). It is not a calendar year. Udunits
   includes the following definitions for years: a common_year is 365
   days, a leap_year is 366 days, a Julian_year is 365.25 days, and a
   Gregorian_year is 365.2425 days. For similar reasons the unit
   ``'month'``, which is defined to be exactly year/12, should also be
   used with caution."

The date given in reference time units is always associated with one
of the calendars recognized by the CF conventions and may be set with
the *calendar* CF property (on the field or Units object).

If the calendar is not set then, as in the CF conventions, for the
purposes of calculation and comparison, it defaults to the mixed
Gregorian/Julian calendar as defined by Udunits:

>>> f.units = 'days since 2000-1-1'
>>> f.calendar
AttributeError: Can't get 'Field' attribute 'calendar'
>>> g.units = 'days since 2000-1-1'
>>> g.calendar = 'gregorian'
>>> g.Units.equals(f.Units)
True

The calendar is ignored for units other than reference time units.

Changing units
^^^^^^^^^^^^^^

Changing units to equivalent units causes the variable's data array
values to be modified in place (if required) when they are next
accessed, and not before:

>>> f.units
'metre'
>>> f.array
array([    0.,  1000.,  2000.,  3000.,  4000.])
>>> f.units = 'kilometre'
>>> f.units
'kilometre'
>>> f.array
array([ 0.,  1.,  2.,  3.,  4.])

>>> f.units
'hours since 2000-1-1'
>>> f.array
array([-1227192., -1227168., -1227144.])
>>> f.units = 'days since 1860-1-1'
>>> f.array
array([ 1.,  2.,  3.])

The `cf.Units` object may be operated on with augmented arithmetic
assignments and binary arithmetic operations:

>>> f.units
'kelvin'
>>> f.array
array([ 273.15,  274.15,  275.15,  276.15,  277.15])

>>> f.Units -= 273.15
>>> f.units
'K @ 273.15'
>>> f.array
array([ 0.,  1.,  2.,  3.,  4.])

>>> f.Units = f.Units + 273.15
>>> f.units
'K'
>>> f.array
array([ 273.15,  274.15,  275.15,  276.15,  277.15])

>>> f.units = 'K @ 237.15'
'K @ 273.15'
>>> f.array
array([ 0.,  1.,  2.,  3.,  4.])

If the field has a data array and its units are changed to
non-equivalent units then a :py:mod:`TypeError` will be raised when
the data are next accessed:

>>> f.units
'm s-1'
>>> f.units = 'K'
>>> f.array
TypeError: Units are not convertible: <CF Units: m s-1>, <CF Units: K>

Overriding units
^^^^^^^^^^^^^^^^

If the units are incorrect, either due to a data manipulation or
an incorrect encoding, it is possible to replace the existing units with
new units, which don't have to be equivalent, without altering the
data values:

>>> f.units
'mm/day'
>>> f.mean()
<CF Data: 3.3455467 mm/day>
>>> g = f.override_units('kg m-2 s-1')
>>> g.mean()
<CF Data: 3.3455467 kg m-2 s-1>
>>> g.override_units('watts m-2', i=True)
>>> g.mean()
<CF Data: 3.3455467 watts m-2>

Overriding the calendar of reference time units is done in a similar manner:

>>> f.calendar
'360_day'
>>> f.array.min()
59.0
>>> f.min()
<CF Data: 1960-02-30 00:00:00 360_day>
>>> g = f.override_calandar('gregorian')
>>> g.array.min()
59.0
>>> g.min()
<CF Data: 1960-02-29 00:00:00 gregorian>

Note that in this case the data values have remained unchanged, but
their date-time interpretation has been redefined.

See `cf.Field.override_units` and `cf.Field.override_calendar` for details.


Equality and equivalence of units
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The `cf.Units` object has methods for assessing whether two units are
equivalent or equal, regardless of their exact string representations.

Two units are equivalent if and only if numeric values in one unit are
convertible to numeric values in the other unit (such as
``'kilometres'`` and ``'metres'``). Two units are equal if and only if
they are equivalent and their conversion is a scale factor of 1 (such
as ``'kilometres'`` and ``'1000 metres'``). Note that equivalence and
equality are based on internally stored binary representations of the
units, rather than their string representations.

>>> f.units = 'm/s'

>>> g.units = 'm s-1'
>>> f.Units == g.Units
True
>>> f.Units.equals(g.Units)
True

>>> g.units = 'km s-1'
>>> f.Units.equivalent(g.Units)
False

>>> f.units = 'days since 1987-12-3'
>>> g.units = 'hours since 2000-12-1'	
>>> f.Units == g.Units
False
>>> f.Units.equivalent(g.Units)
True


Coordinate units
^^^^^^^^^^^^^^^^

The units of a coordinate's bounds are always the same as the
coordinate itself, and the units of the bounds automatically change
when a coordinate's units are changed:

>>> c.units
'degrees'
>>> c.bounds.units
'degrees'
>>> c.bounds.array
array([  0.,  90.])
>>> c.units = 'radians'
>>> c.bounds.units
'radians'
>>> c.bounds.array
array([ 0.        ,  1.57079633])

Manipulating other variables
----------------------------

A field is a subclass of `cf.Variable`, and that class and other
subclasses of `cf.Variable` share generally similar behaviours and
methods:

========================  ===============================================
Class		          Description
========================  ===============================================
`cf.AuxiliaryCoordinate`  A CF auxiliary coordinate construct.

`cf.CellMeasure`          A CF cell measure construct containing
                          information that is needed about the size,
                          shape or location of the field's cells.
		          
`cf.Coordinate`           Base class for storing a coordinate.

`cf.CoordinateBounds`     A CF coordinate's bounds object containing cell
                          boundaries or intervals of climatological time.
	          
`cf.DimensionCoordinate`  A CF dimension coordinate construct.
		          
`cf.Variable`             Base class for storing a data array with
                          metadata.
========================  ===============================================

In general, different axis identities, different axis orders and
different axis directions are not considered, since these objects do
not contain a coordinate system required to define these properties
(unlike a field).

Coordinates
^^^^^^^^^^^

Coordinates are a special case as they may contain a data array for
their coordinate bounds which needs to be treated consistently with
the main coordinate array. If a coordinate has bounds then all
coordinate methods also operate on the bounds in a consistent manner:

>>> c
<CF Coordinate: latitude(73, 96)>
>>> c.bounds
<CF CoordinateBounds: (73, 96, 4)>
>>> d = c.subspace[0:10]
>>> d.shape
(10, 96)
>>> d.bounds.shape
(10, 96, 4)
>>> d.transpose([1, 0])
>>> d.shape
(96, 10)
>>> d.bounds.shape
(96, 10, 4)

.. note:: 

   If the coordinate bounds are operated on independently, care should
   be taken not to break consistency with the parent coordinate.