# Gauss–Newton algorithm Fitting of a noisy curve by an asymmetrical peak model, using the Gauss–Newton algorithm with variable damping factor α.
Top: raw data and model.
Bottom: evolution of the normalised sum of the squares of the errors.

The Gauss–Newton algorithm is used to solve non-linear least squares problems. It is a modification of Newton's method for finding a minimum of a function. Unlike Newton's method, the Gauss–Newton algorithm can only be used to minimize a sum of squared function values, but it has the advantage that second derivatives, which can be challenging to compute, are not required.

Non-linear least squares problems arise, for instance, in non-linear regression, where parameters in a model are sought such that the model is in good agreement with available observations.

The method is named after the mathematicians Carl Friedrich Gauss and Isaac Newton, and first appeared in Gauss' 1809 work Theoria motus corporum coelestium in sectionibus conicis solem ambientum.

## Description

Given m functions r = (r1, …, rm) (often called residuals) of n variables β = (β1, …, βn), with m ≥ n, the Gauss–Newton algorithm iteratively finds the value of the variables that minimizes the sum of squares

$S({\boldsymbol {\beta }})=\sum _^r_^({\boldsymbol {\beta }}).$ Starting with an initial guess ${\boldsymbol {\beta }}^{(0)}$ for the minimum, the method proceeds by the iterations

${\boldsymbol {\beta }}^{(s+1)}={\boldsymbol {\beta }}^{(s)}-\left(\mathbf } ^{\mathsf }\mathbf } \right)^{-1}\mathbf } ^{\mathsf }\mathbf \left({\boldsymbol {\beta }}^{(s)}\right),$ where, if r and β are column vectors, the entries of the Jacobian matrix are

$\left(\mathbf } \right)_={\frac {\partial r_\left({\boldsymbol {\beta }}^{(s)}\right)}{\partial \beta _}},$ and the symbol $^{\mathsf }$ denotes the matrix transpose.

If m = n, the iteration simplifies to

${\boldsymbol {\beta }}^{(s+1)}={\boldsymbol {\beta }}^{(s)}-\left(\mathbf } \right)^{-1}\mathbf \left({\boldsymbol {\beta }}^{(s)}\right),$ which is a direct generalization of Newton's method in one dimension.

In data fitting, where the goal is to find the parameters β such that a given model function y = f(x, β) best fits some data points (xi, yi), the functions ri are the residuals:

$r_({\boldsymbol {\beta }})=y_-f\left(x_,{\boldsymbol {\beta }}\right).$ Then, the Gauss–Newton method can be expressed in terms of the Jacobian Jf of the function f as

${\boldsymbol {\beta }}^{(s+1)}={\boldsymbol {\beta }}^{(s)}+\left(\mathbf } ^{\mathsf }\mathbf } \right)^{-1}\mathbf } ^{\mathsf }\mathbf \left({\boldsymbol {\beta }}^{(s)}\right).$ Note that $\left(\mathbf } ^{\mathsf }\mathbf } \right)^{-1}\mathbf } ^{\mathsf }$ is the left pseudoinverse of $\mathbf }$ .