File: explain_precision.rst_

package info (click to toggle)
gmt 5.3.1%2Bdfsg-2
  • links: PTS, VCS
  • area: main
  • in suites: stretch
  • size: 150,460 kB
  • ctags: 18,539
  • sloc: ansic: 194,217; sh: 7,349; xml: 149; makefile: 72; fortran: 49; lisp: 41; csh: 5
file content (11 lines) | stat: -rw-r--r-- 665 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
ASCII Format Precision
----------------------

The ASCII output formats of numerical data are controlled by parameters
in your :doc:`gmt.conf` file. Longitude and latitude are formatted
according to :ref:`FORMAT_GEO_OUT <FORMAT_GEO_OUT>`, whereas other values are formatted
according to :ref:`FORMAT_FLOAT_OUT <FORMAT_FLOAT_OUT>`. Be aware that the format in effect
can lead to loss of precision in the output, which can lead to various
problems downstream. If you find the output is not written with enough
precision, consider switching to binary output (**-bo** if available) or
specify more decimals using the :ref:`FORMAT_FLOAT_OUT <FORMAT_FLOAT_OUT>` setting.