Algorithms - Gauss Marquardt Levenberg (GML)

The OSTRICH Gauss Marquardt Levenberg (GML) optimization algorithm

Initial Publication:

Modified:

The following optional group will configure the Gauss-Marquardt-Levenberg algorithm and will be processed if [ProgramType] is set to “Levenberg-Marquardt”. 

BeginLevMar
InitialLambda              [init_lambda]
LambdaScaleFactor          [lambda_sf]
MoveLimit                  [move_limit]
AlgorithmConvergenceValue  [conv_val]
LambdaPhiRatio             [phi_ratio]
LambdaRelReduction         [rel_reduce]
MaxLambdas                 [max_lambda]
MaxIterations              [max_iters]
EndLevMar

Figure 1: General Format of the Gauss-Marquardt-Levenberg Group

Where BeginLevMar and EndLevMar are parsing tags that wrap the following set of algorithm configuration variables:

  • InitialLambda: The initial Marquardt λ. The λ variable controls the algorithm’s transition from using a Steepest-Descent approach to using a Taylor Series approximation. The default is 10.00.
  • LambdaScaleFactor: The Marquardt λ scale factor – this is the factor by which λ is multiplied or divided during λ adjustment. The default is 1.10.
  • MoveLimit: Parameter move limits – the maximum adjustment of a parameter (relative to the range of the parameter) that is allowed in a single iteration. The default is 0.10, or 10%.
  • AlgorithmConvergenceValue: The algorithm convergence value – regression will stop when the relative reduction in the objective function (i.e. phi, Φ) over two iterations is less than this value. The default value is 1.00E-4.
  • LambdaPhiRatio: This is the reduction criteria for deciding on optimal adjustments of the λ term. Adjustments for the given iteration are complete when the relative reduction in Φ is greater than this value. The default value is 0.30.
  • LambdaRelReduction: This is the reduction criteria for abandoning λ adjustment. Adjustments for the given iteration are halted when the relative reduction in Φ is less than this value. The default value is 0.01.
  • MaxLambdas: The maximum number of λ adjustments per iteration. The default value is 10.
  • MaxIterations: The maximum iterations in the overall method. The default value is 30.

References

Levenberg, K. 1944. A Method for the Solution of Certain Problems in Least Squares. Quarterly of Applied Mathematics 2, 164-168.

Marquardt, D. 1963. An Algorithm for Least-Squares Estimation of Nonlinear Parameters. SIAM Journal on Applied Mathematics 11, 431-441.