Levenberg-Marquardt Algorithm for Generalized Non-linear Regression
 

A part of my research on network failure modeling requires developing a generalized non-linear regression module for non-linear functions of N independent variables. The functionality has to be necessarily developed within an object-oriented frame work. I implemented the Levenberg-Marquardt algorithm for generalized non-linear regression, which is a "half Newton" method. The resulting C++ class works fine for the product of N polynomials, but refuses to converge for simple non-linear functional forms. Does anyone have any suggestions in this regard ? Is any ready-made class commercially available on generalized non-linear regression ?

Thanks

Re: Levenberg-Marquardt Algorithm for Generalized Non-linear Regression
 

There are more advanced implementations of Levenberg-Marquardt algorithm such as More's MINPACK algorithm which utilises a scaled truct region and is "globally" convergent. I used MINPACK in my Ph.D. research and found it to be quite robust. It is also fairly compact. MINPACK can be obtained at the following website:

http://netlib2.cs.utk.edu/liblist.html

Although MINPACK is globally convergent, it's rate of convergence is only linear for large residual problems. For large residual problems, I'd suggest NL2SOL which is a full-Newton type method, i.e. includes the Hessian. NL2SOL is also employs the model trust region concept and is globally convergent. I believe an early copy of this routine can be found at the website above. More recent implementation of NL2SOL can be obtained from Lucent technologies.

Good luck

NOTE: the term globally convergent means the algorithm will converge to a local minima - global minimization is another story.