File: Precision-issues.html

package info (click to toggle)
octave 10.3.0-1
  • links: PTS, VCS
  • area: main
  • in suites:
  • size: 145,388 kB
  • sloc: cpp: 335,976; ansic: 82,241; fortran: 20,963; objc: 9,402; sh: 8,756; yacc: 4,392; lex: 4,333; perl: 1,544; java: 1,366; awk: 1,259; makefile: 659; xml: 192
file content (102 lines) | stat: -rw-r--r-- 4,944 bytes parent folder | download | duplicates (2)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
<!DOCTYPE html>
<html>
<!-- Created by GNU Texinfo 7.1.1, https://www.gnu.org/software/texinfo/ -->
<head>
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<title>Precision issues (GNU Octave (version 10.3.0))</title>

<meta name="description" content="Precision issues (GNU Octave (version 10.3.0))">
<meta name="keywords" content="Precision issues (GNU Octave (version 10.3.0))">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta name="viewport" content="width=device-width,initial-scale=1">

<link href="index.html" rel="start" title="Top">
<link href="Concept-Index.html" rel="index" title="Concept Index">
<link href="index.html#SEC_Contents" rel="contents" title="Table of Contents">
<link href="Graphics-Toolkits.html" rel="up" title="Graphics Toolkits">
<link href="Hardware-vs-Software-Rendering.html" rel="prev" title="Hardware vs Software Rendering">
<style type="text/css">
<!--
a.copiable-link {visibility: hidden; text-decoration: none; line-height: 0em}
div.center {text-align:center}
div.example {margin-left: 3.2em}
span:hover a.copiable-link {visibility: visible}
-->
</style>
<link rel="stylesheet" type="text/css" href="octave.css">


</head>

<body lang="en">
<div class="subsubsection-level-extent" id="Precision-issues">
<div class="nav-panel">
<p>
Previous: <a href="Hardware-vs-Software-Rendering.html" accesskey="p" rel="prev">Hardware vs Software Rendering</a>, Up: <a href="Graphics-Toolkits.html" accesskey="u" rel="up">Graphics Toolkits</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<h4 class="subsubsection" id="Precision-issues-1"><span>15.4.8.3 Precision issues<a class="copiable-link" href="#Precision-issues-1"> &para;</a></span></h4>
<a class="index-entry-id" id="index-opengl-single-precision-date-time"></a>

<p>The OpenGL graphics toolkits (<code class="code">&quot;qt&quot;</code> and <code class="code">&quot;fltk&quot;</code>) use single
precision for rendering.  This limitation in particular applies to plots of
time series against serial dates as used by the <code class="code">datenum</code>, <code class="code">datestr</code>,
<code class="code">datestruct</code>, and <code class="code">datetick</code> functions.
</p>
<p>Serial dates encode timestamps as days elapsed since the year zero with hours,
minutes, seconds as the fractional part.  On December 31st 1999, the serial
date representation was 730485.  A double precision variable with this integer
part allows for a resolution in its fractional part of 1.2e-10, representing
about 5 microseconds.  But with single precision, the resolution is reduced to
about 0.06, representing 45 minutes.  Any attempt to plot timestamped data
with finer granularity will result in a distorted graph.
</p>
<p>As a workaround, it is possible to use the <code class="code">&quot;gnuplot&quot;</code> graphics toolkit
or subtract 2000 years&mdash;i.e., <code class="code">datenum (2000, 0, 0)</code> or 730485&mdash;from the
time values.  Due to the fact that the calendar structure repeats every 2000
years, the relation between year, month, day of month and day of week will stay
unchanged and the ticks and ticklabels produced by the <code class="code">datetick</code> function
will still be correct.  Only years will lack the millennium digit.  Thus,
&quot;2020&quot; will be printed as &quot;20&quot;.  For example:
</p>
<div class="example">
<div class="group"><pre class="example-preformatted"># timestamps of 24 hours in one minute steps
t = datenum (2020, 1, 1):(1/1440):datenum (2020, 1, 2);

# some example time series data
x = -cos (2*pi*t) + rand (size (t)) / 10;

subplot (1, 2, 1);
plot (t, x);
datetick (&quot;x&quot;);
xlabel (&quot;serial date&quot;);
title (&quot;problem&quot;);

subplot (1, 2, 2);
plot (t - 730485, x);
datetick (&quot;x&quot;);
xlabel (&quot;2000 years off&quot;);
title (&quot;workaround&quot;);
</pre></div></div>

<p>The result of which can be seen in <a class="ref" href="#fig_003aprecisiondate">Figure 15.8</a>.
</p>
<div class="float" id="fig_003aprecisiondate">
<div class="center"><img class="image" src="precisiondate.png" alt="precisiondate">
</div><div class="caption"><p><strong class="strong">Figure 15.8: </strong>Single precision issues with OpenGL graphics toolkits</p></div></div>
<p>Similarly, other data can be translated or re-scaled to work around this issue.
</p>

</div>
<hr>
<div class="nav-panel">
<p>
Previous: <a href="Hardware-vs-Software-Rendering.html">Hardware vs Software Rendering</a>, Up: <a href="Graphics-Toolkits.html">Graphics Toolkits</a> &nbsp; [<a href="index.html#SEC_Contents" title="Table of contents" rel="contents">Contents</a>][<a href="Concept-Index.html" title="Index" rel="index">Index</a>]</p>
</div>



</body>
</html>