File: Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt-with-Geodesic-Acceleration.html

package info (click to toggle)
gsl-ref-html 2.3-1
  • links: PTS
  • area: non-free
  • in suites: bullseye, buster, sid
  • size: 6,876 kB
  • ctags: 4,574
  • sloc: makefile: 35
file content (105 lines) | stat: -rw-r--r-- 6,110 bytes parent folder | download
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd">
<html>
<!-- Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010, 2011, 2012, 2013, 2014, 2015, 2016 The GSL Team.

Permission is granted to copy, distribute and/or modify this document
under the terms of the GNU Free Documentation License, Version 1.3 or
any later version published by the Free Software Foundation; with the
Invariant Sections being "GNU General Public License" and "Free Software
Needs Free Documentation", the Front-Cover text being "A GNU Manual",
and with the Back-Cover Text being (a) (see below). A copy of the
license is included in the section entitled "GNU Free Documentation
License".

(a) The Back-Cover Text is: "You have the freedom to copy and modify this
GNU Manual." -->
<!-- Created by GNU Texinfo 5.1, http://www.gnu.org/software/texinfo/ -->
<head>
<title>GNU Scientific Library &ndash; Reference Manual: Nonlinear Least-Squares TRS Levenberg-Marquardt with Geodesic Acceleration</title>

<meta name="description" content="GNU Scientific Library &ndash; Reference Manual: Nonlinear Least-Squares TRS Levenberg-Marquardt with Geodesic Acceleration">
<meta name="keywords" content="GNU Scientific Library &ndash; Reference Manual: Nonlinear Least-Squares TRS Levenberg-Marquardt with Geodesic Acceleration">
<meta name="resource-type" content="document">
<meta name="distribution" content="global">
<meta name="Generator" content="makeinfo">
<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
<link href="index.html#Top" rel="start" title="Top">
<link href="Function-Index.html#Function-Index" rel="index" title="Function Index">
<link href="Nonlinear-Least_002dSquares-TRS-Overview.html#Nonlinear-Least_002dSquares-TRS-Overview" rel="up" title="Nonlinear Least-Squares TRS Overview">
<link href="Nonlinear-Least_002dSquares-TRS-Dogleg.html#Nonlinear-Least_002dSquares-TRS-Dogleg" rel="next" title="Nonlinear Least-Squares TRS Dogleg">
<link href="Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt.html#Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt" rel="previous" title="Nonlinear Least-Squares TRS Levenberg-Marquardt">
<style type="text/css">
<!--
a.summary-letter {text-decoration: none}
blockquote.smallquotation {font-size: smaller}
div.display {margin-left: 3.2em}
div.example {margin-left: 3.2em}
div.indentedblock {margin-left: 3.2em}
div.lisp {margin-left: 3.2em}
div.smalldisplay {margin-left: 3.2em}
div.smallexample {margin-left: 3.2em}
div.smallindentedblock {margin-left: 3.2em; font-size: smaller}
div.smalllisp {margin-left: 3.2em}
kbd {font-style:oblique}
pre.display {font-family: inherit}
pre.format {font-family: inherit}
pre.menu-comment {font-family: serif}
pre.menu-preformatted {font-family: serif}
pre.smalldisplay {font-family: inherit; font-size: smaller}
pre.smallexample {font-size: smaller}
pre.smallformat {font-family: inherit; font-size: smaller}
pre.smalllisp {font-size: smaller}
span.nocodebreak {white-space:nowrap}
span.nolinebreak {white-space:nowrap}
span.roman {font-family:serif; font-weight:normal}
span.sansserif {font-family:sans-serif; font-weight:normal}
ul.no-bullet {list-style: none}
-->
</style>


</head>

<body lang="en" bgcolor="#FFFFFF" text="#000000" link="#0000FF" vlink="#800080" alink="#FF0000">
<a name="Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt-with-Geodesic-Acceleration"></a>
<div class="header">
<p>
Next: <a href="Nonlinear-Least_002dSquares-TRS-Dogleg.html#Nonlinear-Least_002dSquares-TRS-Dogleg" accesskey="n" rel="next">Nonlinear Least-Squares TRS Dogleg</a>, Previous: <a href="Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt.html#Nonlinear-Least_002dSquares-TRS-Levenberg_002dMarquardt" accesskey="p" rel="previous">Nonlinear Least-Squares TRS Levenberg-Marquardt</a>, Up: <a href="Nonlinear-Least_002dSquares-TRS-Overview.html#Nonlinear-Least_002dSquares-TRS-Overview" accesskey="u" rel="up">Nonlinear Least-Squares TRS Overview</a> &nbsp; [<a href="Function-Index.html#Function-Index" title="Index" rel="index">Index</a>]</p>
</div>
<hr>
<a name="Levenberg_002dMarquardt-with-Geodesic-Acceleration"></a>
<h4 class="subsection">39.2.2 Levenberg-Marquardt with Geodesic Acceleration</h4>
<a name="index-Levenberg_002dMarquardt-algorithm_002c-geodesic-acceleration"></a>
<a name="index-nonlinear-least-squares_002c-levenberg_002dmarquardt_002c-geodesic-acceleration"></a>

<p>This method applies a so-called geodesic acceleration correction to
the standard Levenberg-Marquardt step <em>\delta_k</em> (Transtrum et al, 2011).
By interpreting <em>\delta_k</em> as a first order step along a geodesic in the
model parameter space (ie: a velocity <em>\delta_k = v_k</em>), the geodesic
acceleration <em>a_k</em> is a second order correction along the
geodesic which is determined by solving the linear least squares system
</p>
<div class="example">
<pre class="example">[J_k; sqrt(mu_k) D_k] a_k = - [f_vv(x_k); 0]
</pre></div>

<p>where <em>f_{vv}</em> is the second directional derivative of
the residual vector in the velocity direction <em>v</em>,
<em>f_{vv}(x) = D_v^2 f = \sum_{\alpha\beta} v_{\alpha} v_{\beta} \partial_{\alpha} \partial_{\beta} f(x)</em>,
where <em>\alpha</em> and <em>\beta</em> are summed over the <em>p</em>
parameters. The new total step is then <em>\delta_k' = v_k + {1 \over 2}a_k</em>.
The second order correction <em>a_k</em> can be calculated with a modest additional
cost, and has been shown to dramatically reduce the number of iterations
(and expensive Jacobian evaluations) required to reach convergence on a variety
of different problems. In order to utilize the geodesic acceleration, the user must supply a
function which provides the second directional derivative vector
<em>f_{vv}(x)</em>, or alternatively the library can use a finite
difference method to estimate this vector with one additional function
evaluation of <em>f(x + h v)</em> where <em>h</em> is a tunable step size
(see the <code>h_fvv</code> parameter description).
</p>



</body>
</html>