1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
|
<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
<title>The Lanczos Approximation</title>
<link rel="stylesheet" href="../../../../../../../doc/html/boostbook.css" type="text/css">
<meta name="generator" content="DocBook XSL Stylesheets Vsnapshot_2006-12-17_0120">
<link rel="start" href="../../index.html" title="Math Toolkit">
<link rel="up" href="../backgrounders.html" title="Backgrounders">
<link rel="prev" href="relative_error.html" title="Relative Error">
<link rel="next" href="remez.html" title="The Remez Method">
</head>
<body bgcolor="white" text="black" link="#0000FF" vlink="#840084" alink="#0000FF">
<table cellpadding="2" width="100%"><tr>
<td valign="top"><img alt="Boost C++ Libraries" width="277" height="86" src="../../../../../../../boost.png"></td>
<td align="center"><a href="../../../../../../../index.html">Home</a></td>
<td align="center"><a href="../../../../../../../libs/libraries.htm">Libraries</a></td>
<td align="center"><a href="http://www.boost.org/people/people.htm">People</a></td>
<td align="center"><a href="http://www.boost.org/more/faq.htm">FAQ</a></td>
<td align="center"><a href="../../../../../../../more/index.htm">More</a></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="relative_error.html"><img src="../../../../../../../doc/html/images/prev.png" alt="Prev"></a><a accesskey="u" href="../backgrounders.html"><img src="../../../../../../../doc/html/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../../doc/html/images/home.png" alt="Home"></a><a accesskey="n" href="remez.html"><img src="../../../../../../../doc/html/images/next.png" alt="Next"></a>
</div>
<div class="section" lang="en">
<div class="titlepage"><div><div><h3 class="title">
<a name="math_toolkit.backgrounders.lanczos"></a><a href="lanczos.html" title="The Lanczos Approximation"> The Lanczos Approximation</a>
</h3></div></div></div>
<a name="math_toolkit.backgrounders.lanczos.motivation"></a><h5>
<a name="id785869"></a>
<a href="lanczos.html#math_toolkit.backgrounders.lanczos.motivation">Motivation</a>
</h5>
<p>
<span class="emphasis"><em>Why base gamma and gamma-like functions on the Lanczos approximation?</em></span>
</p>
<p>
First of all I should make clear that for the gamma function over real numbers
(as opposed to complex ones) the Lanczos approximation (See <a href="http://en.wikipedia.org/wiki/Lanczos_approximation" target="_top">Wikipedia
or </a> <a href="http://mathworld.wolfram.com/LanczosApproximation.html" target="_top">Mathworld</a>)
appears to offer no clear advantage over more traditional methods such as
<a href="http://en.wikipedia.org/wiki/Stirling_approximation" target="_top">Stirling's
approximation</a>. <a href="lanczos.html#pugh">Pugh</a> carried out an extensive
comparison of the various methods available and discovered that they were
all very similar in terms of complexity and relative error. However, the
Lanczos approximation does have a couple of properties that make it worthy
of further consideration:
</p>
<div class="itemizedlist"><ul type="disc">
<li>
The approximation has an easy to compute truncation error that holds for
all <span class="emphasis"><em>z > 0</em></span>. In practice that means we can use the
same approximation for all <span class="emphasis"><em>z > 0</em></span>, and be certain
that no matter how large or small <span class="emphasis"><em>z</em></span> is, the truncation
error will <span class="emphasis"><em>at worst</em></span> be bounded by some finite value.
</li>
<li>
The approximation has a form that is particularly amenable to analytic
manipulation, in particular ratios of gamma or gamma-like functions are
particularly easy to compute without resorting to logarithms.
</li>
</ul></div>
<p>
It is the combination of these two properties that make the approximation
attractive: Stirling's approximation is highly accurate for large z, and
has some of the same analytic properties as the Lanczos approximation, but
can't easily be used across the whole range of z.
</p>
<p>
As the simplest example, consider the ratio of two gamma functions: one could
compute the result via lgamma:
</p>
<pre class="programlisting"><span class="identifier">exp</span><span class="special">(</span><span class="identifier">lgamma</span><span class="special">(</span><span class="identifier">a</span><span class="special">)</span> <span class="special">-</span> <span class="identifier">lgamma</span><span class="special">(</span><span class="identifier">b</span><span class="special">));</span>
</pre>
<p>
However, even if lgamma is uniformly accurate to 0.5ulp, the worst case relative
error in the above can easily be shown to be:
</p>
<pre class="programlisting"><span class="identifier">Erel</span> <span class="special">></span> <span class="identifier">a</span> <span class="special">*</span> <span class="identifier">log</span><span class="special">(</span><span class="identifier">a</span><span class="special">)/</span><span class="number">2</span> <span class="special">+</span> <span class="identifier">b</span> <span class="special">*</span> <span class="identifier">log</span><span class="special">(</span><span class="identifier">b</span><span class="special">)/</span><span class="number">2</span>
</pre>
<p>
For small <span class="emphasis"><em>a</em></span> and <span class="emphasis"><em>b</em></span> that's not a
problem, but to put the relationship another way: <span class="emphasis"><em>each time a and
b increase in magnitude by a factor of 10, at least one decimal digit of
precision will be lost.</em></span>
</p>
<p>
In contrast, by analytically combining like power terms in a ratio of Lanczos
approximation's, these errors can be virtually eliminated for small <span class="emphasis"><em>a</em></span>
and <span class="emphasis"><em>b</em></span>, and kept under control for very large (or very
small for that matter) <span class="emphasis"><em>a</em></span> and <span class="emphasis"><em>b</em></span>.
Of course, computing large powers is itself a notoriously hard problem, but
even so, analytic combinations of Lanczos approximations can make the difference
between obtaining a valid result, or simply garbage. Refer to the implementation
notes for the <a href="../special/sf_beta/beta_function.html" title="Beta">beta</a>
function for an example of this method in practice. The incomplete <a href="../special/sf_gamma/igamma.html" title="Incomplete Gamma Functions">gamma_p gamma</a> and
<a href="../special/sf_beta/ibeta_function.html" title="Incomplete Beta Functions">beta</a> functions
use similar analytic combinations of power terms, to combine gamma and beta
functions divided by large powers into single (simpler) expressions.
</p>
<a name="math_toolkit.backgrounders.lanczos.the_approximation"></a><h5>
<a name="id786247"></a>
<a href="lanczos.html#math_toolkit.backgrounders.lanczos.the_approximation">The
Approximation</a>
</h5>
<p>
The Lanczos Approximation to the Gamma Function is given by:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos0.png"></span>
</p>
<p>
Where S<sub>g</sub>(z) is an infinite sum, that is convergent for all z > 0, and
<span class="emphasis"><em>g</em></span> is an arbitrary parameter that controls the "shape"
of the terms in the sum which is given by:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos0a.png"></span>
</p>
<p>
With individual coefficients defined in closed form by:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos0b.png"></span>
</p>
<p>
However, evaluation of the sum in that form can lead to numerical instability
in the computation of the ratios of rising and falling factorials (effectively
we're multiplying by a series of numbers very close to 1, so roundoff errors
can accumulate quite rapidly).
</p>
<p>
The Lanczos approximation is therefore often written in partial fraction
form with the leading constants absorbed by the coefficients in the sum:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos1.png"></span>
</p>
<p>
where:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos2.png"></span>
</p>
<p>
Again parameter <span class="emphasis"><em>g</em></span> is an arbitrarily chosen constant,
and <span class="emphasis"><em>N</em></span> is an arbitrarily chosen number of terms to evaluate
in the "Lanczos sum" part.
</p>
<div class="note"><table border="0" summary="Note">
<tr>
<td rowspan="2" align="center" valign="top" width="25"><img alt="[Note]" src="../../../../../../../doc/html/images/note.png"></td>
<th align="left">Note</th>
</tr>
<tr><td align="left" valign="top"><p>
Some authors choose to define the sum from k=1 to N, and hence end up with
N+1 coefficients. This happens to confuse both the following discussion
and the code (since C++ deals with half open array ranges, rather than
the closed range of the sum). This convention is consistent with <a href="lanczos.html#godfrey">Godfrey</a>, but not <a href="lanczos.html#pugh">Pugh</a>,
so take care when referring to the literature in this field.
</p></td></tr>
</table></div>
<a name="math_toolkit.backgrounders.lanczos.computing_the_coefficients"></a><h5>
<a name="id786510"></a>
<a href="lanczos.html#math_toolkit.backgrounders.lanczos.computing_the_coefficients">Computing
the Coefficients</a>
</h5>
<p>
The coefficients C0..CN-1 need to be computed from <span class="emphasis"><em>N</em></span>
and <span class="emphasis"><em>g</em></span> at high precision, and then stored as part of
the program. Calculation of the coefficients is performed via the method
of <a href="lanczos.html#godfrey">Godfrey</a>; let the constants be contained
in a column vector P, then:
</p>
<p>
P = B D C F
</p>
<p>
where B is an NxN matrix:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos4.png"></span>
</p>
<p>
D is an NxN matrix:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos3.png"></span>
</p>
<p>
C is an NxN matrix:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos5.png"></span>
</p>
<p>
and F is an N element column vector:
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos6.png"></span>
</p>
<p>
Note than the matrices B, D and C contain all integer terms and depend only
on <span class="emphasis"><em>N</em></span>, this product should be computed first, and then
multiplied by <span class="emphasis"><em>F</em></span> as the last step.
</p>
<a name="math_toolkit.backgrounders.lanczos.choosing_the_right_parameters"></a><h5>
<a name="id786715"></a>
<a href="lanczos.html#math_toolkit.backgrounders.lanczos.choosing_the_right_parameters">Choosing
the Right Parameters</a>
</h5>
<p>
The trick is to choose <span class="emphasis"><em>N</em></span> and <span class="emphasis"><em>g</em></span>
to give the desired level of accuracy: choosing a small value for <span class="emphasis"><em>g</em></span>
leads to a strictly convergent series, but one which converges only slowly.
Choosing a larger value of <span class="emphasis"><em>g</em></span> causes the terms in the
series to be large and/or divergent for about the first <span class="emphasis"><em>g-1</em></span>
terms, and to then suddenly converge with a "crunch".
</p>
<p>
<a href="lanczos.html#pugh">Pugh</a> has determined the optimal value of <span class="emphasis"><em>g</em></span>
for <span class="emphasis"><em>N</em></span> in the range <span class="emphasis"><em>1 <= N <= 60</em></span>:
unfortunately in practice choosing these values leads to cancellation errors
in the Lanczos sum as the largest term in the (alternating) series is approximately
1000 times larger than the result. These optimal values appear not to be
useful in practice unless the evaluation can be done with a number of guard
digits <span class="emphasis"><em>and</em></span> the coefficients are stored at higher precision
than that desired in the result. These values are best reserved for say,
computing to float precision with double precision arithmetic.
</p>
<div class="table">
<a name="id786798"></a><p class="title"><b>Table45.Optimal choices for N and g when computing with
guard digits (source: Pugh)</b></p>
<div class="table-contents"><table class="table" summary="Optimal choices for N and g when computing with
guard digits (source: Pugh)">
<colgroup>
<col>
<col>
<col>
<col>
</colgroup>
<thead><tr>
<th>
<p>
Significand Size
</p>
</th>
<th>
<p>
N
</p>
</th>
<th>
<p>
g
</p>
</th>
<th>
<p>
Max Error
</p>
</th>
</tr></thead>
<tbody>
<tr>
<td>
<p>
24
</p>
</td>
<td>
<p>
6
</p>
</td>
<td>
<p>
5.581
</p>
</td>
<td>
<p>
9.51e-12
</p>
</td>
</tr>
<tr>
<td>
<p>
53
</p>
</td>
<td>
<p>
13
</p>
</td>
<td>
<p>
13.144565
</p>
</td>
<td>
<p>
9.2213e-23
</p>
</td>
</tr>
</tbody>
</table></div>
</div>
<br class="table-break"><p>
The alternative described by <a href="lanczos.html#godfrey">Godfrey</a> is to
perform an exhaustive search of the <span class="emphasis"><em>N</em></span> and <span class="emphasis"><em>g</em></span>
parameter space to determine the optimal combination for a given <span class="emphasis"><em>p</em></span>
digit floating-point type. Repeating this work found a good approximation
for double precision arithmetic (close to the one <a href="lanczos.html#godfrey">Godfrey</a>
found), but failed to find really good approximations for 80 or 128-bit long
doubles. Further it was observed that the approximations obtained tended
to optimised for the small values of z (1 < z < 200) used to test the
implementation against the factorials. Computing ratios of gamma functions
with large arguments were observed to suffer from error resulting from the
truncation of the Lancozos series.
</p>
<p>
<a href="lanczos.html#pugh">Pugh</a> identified all the locations where the theoretical
error of the approximation were at a minimum, but unfortunately has published
only the largest of these minima. However, he makes the observation that
the minima coincide closely with the location where the first neglected term
(a<sub>N</sub>) in the Lanczos series S<sub>g</sub>(z) changes sign. These locations are quite
easy to locate, albeit with considerable computer time. These "sweet
spots" need only be computed once, tabulated, and then searched when
required for an approximation that delivers the required precision for some
fixed precision type.
</p>
<p>
Unfortunately, following this path failed to find a really good approximation
for 128-bit long doubles, and those found for 64 and 80-bit reals required
an excessive number of terms. There are two competing issues here: high precision
requires a large value of <span class="emphasis"><em>g</em></span>, but avoiding cancellation
errors in the evaluation requires a small <span class="emphasis"><em>g</em></span>.
</p>
<p>
At this point note that the Lanczos sum can be converted into rational form
(a ratio of two polynomials, obtained from the partial-fraction form using
polynomial arithmetic), and doing so changes the coefficients so that <span class="emphasis"><em>they
are all positive</em></span>. That means that the sum in rational form can
be evaluated without cancellation error, albeit with double the number of
coefficients for a given N. Repeating the search of the "sweet spots",
this time evaluating the Lanczos sum in rational form, and testing only those
"sweet spots" whose theoretical error is less than the machine
epsilon for the type being tested, yielded good approximations for all the
types tested. The optimal values found were quite close to the best cases
reported by <a href="lanczos.html#pugh">Pugh</a> (just slightly larger <span class="emphasis"><em>N</em></span>
and slightly smaller <span class="emphasis"><em>g</em></span> for a given precision than <a href="lanczos.html#pugh">Pugh</a> reports), and even though converting to rational
form doubles the number of stored coefficients, it should be noted that half
of them are integers (and therefore require less storage space) and the approximations
require a smaller <span class="emphasis"><em>N</em></span> than would otherwise be required,
so fewer floating point operations may be required overall.
</p>
<p>
The following table shows the optimal values for <span class="emphasis"><em>N</em></span> and
<span class="emphasis"><em>g</em></span> when computing at fixed precision. These should be
taken as work in progress: there are no values for 106-bit significand machines
(Darwin long doubles & NTL quad_float), and further optimisation of the
values of <span class="emphasis"><em>g</em></span> may be possible. Errors given in the table
are estimates of the error due to truncation of the Lanczos infinite series
to <span class="emphasis"><em>N</em></span> terms. They are calculated from the sum of the
first five neglected terms - and are known to be rather pessimistic estimates
- although it is noticeable that the best combinations of <span class="emphasis"><em>N</em></span>
and <span class="emphasis"><em>g</em></span> occurred when the estimated truncation error almost
exactly matches the machine epsilon for the type in question.
</p>
<div class="table">
<a name="id787120"></a><p class="title"><b>Table46.Optimum value for N and g when computing at fixed
precision</b></p>
<div class="table-contents"><table class="table" summary="Optimum value for N and g when computing at fixed
precision">
<colgroup>
<col>
<col>
<col>
<col>
<col>
</colgroup>
<thead><tr>
<th>
<p>
Significand Size
</p>
</th>
<th>
<p>
Platform/Compiler Used
</p>
</th>
<th>
<p>
N
</p>
</th>
<th>
<p>
g
</p>
</th>
<th>
<p>
Max Truncation Error
</p>
</th>
</tr></thead>
<tbody>
<tr>
<td>
<p>
24
</p>
</td>
<td>
<p>
Win32, VC++ 7.1
</p>
</td>
<td>
<p>
6
</p>
</td>
<td>
<p>
1.428456135094165802001953125
</p>
</td>
<td>
<p>
9.41e-007
</p>
</td>
</tr>
<tr>
<td>
<p>
53
</p>
</td>
<td>
<p>
Win32, VC++ 7.1
</p>
</td>
<td>
<p>
13
</p>
</td>
<td>
<p>
6.024680040776729583740234375
</p>
</td>
<td>
<p>
3.23e-016
</p>
</td>
</tr>
<tr>
<td>
<p>
64
</p>
</td>
<td>
<p>
Suse Linux 9 IA64, gcc-3.3.3
</p>
</td>
<td>
<p>
17
</p>
</td>
<td>
<p>
12.2252227365970611572265625
</p>
</td>
<td>
<p>
2.34e-024
</p>
</td>
</tr>
<tr>
<td>
<p>
116
</p>
</td>
<td>
<p>
HP Tru64 Unix 5.1B / Alpha, Compaq C++ V7.1-006
</p>
</td>
<td>
<p>
24
</p>
</td>
<td>
<p>
20.3209821879863739013671875
</p>
</td>
<td>
<p>
4.75e-035
</p>
</td>
</tr>
</tbody>
</table></div>
</div>
<br class="table-break"><p>
Finally note that the Lanczos approximation can be written as follows by
removing a factor of exp(g) from the denominator, and then dividing all the
coefficients by exp(g):
</p>
<p>
<span class="inlinemediaobject"><img src="../../../equations/lanczos7.png"></span>
</p>
<p>
This form is more convenient for calculating lgamma, but for the gamma function
the division by <span class="emphasis"><em>e</em></span> turns a possibly exact quality into
an inexact value: this reduces accuracy in the common case that the input
is exact, and so isn't used for the gamma function.
</p>
<a name="math_toolkit.backgrounders.lanczos.references"></a><h5>
<a name="id787399"></a>
<a href="lanczos.html#math_toolkit.backgrounders.lanczos.references">References</a>
</h5>
<a name="godfrey"></a><a name="pugh"></a><div class="orderedlist"><ol type="1">
<li>
Paul Godfrey, <a href="http://my.fit.edu/~gabdo/gamma.txt" target="_top">"A note
on the computation of the convergent Lanczos complex Gamma approximation"</a>.
</li>
<li>
Glendon Ralph Pugh, <a href="http://bh0.physics.ubc.ca/People/matt/Doc/ThesesOthers/Phd/pugh.pdf" target="_top">"An
Analysis of the Lanczos Gamma Approximation"</a>, PhD Thesis November
2004.
</li>
<li>
Viktor T. Toth, <a href="http://www.rskey.org/gamma.htm" target="_top">"Calculators
and the Gamma Function"</a>.
</li>
<li>
Mathworld, <a href="http://mathworld.wolfram.com/LanczosApproximation.html" target="_top">The
Lanczos Approximation</a>.
</li>
</ol></div>
</div>
<table xmlns:rev="http://www.cs.rpi.edu/~gregod/boost/tools/doc/revision" width="100%"><tr>
<td align="left"></td>
<td align="right"><div class="copyright-footer">Copyright 2006 -2007 John Maddock, Paul A. Bristow, Hubert Holin
and Xiaogang Zhang<p>
Distributed under the Boost Software License, Version 1.0. (See accompanying
file LICENSE_1_0.txt or copy at <a href="http://www.boost.org/LICENSE_1_0.txt" target="_top">http://www.boost.org/LICENSE_1_0.txt</a>)
</p>
</div></td>
</tr></table>
<hr>
<div class="spirit-nav">
<a accesskey="p" href="relative_error.html"><img src="../../../../../../../doc/html/images/prev.png" alt="Prev"></a><a accesskey="u" href="../backgrounders.html"><img src="../../../../../../../doc/html/images/up.png" alt="Up"></a><a accesskey="h" href="../../index.html"><img src="../../../../../../../doc/html/images/home.png" alt="Home"></a><a accesskey="n" href="remez.html"><img src="../../../../../../../doc/html/images/next.png" alt="Next"></a>
</div>
</body>
</html>
|
