Automatic Differentiation
 

Can anyone recommend a tool for Automatic Differentiation? I need extremely accurate Jacobians (and possibly Hessian) for use in an implicit flux scheme. Performance is the most important criterion and to lesser degree usability. Any suggestions or comments are welcome.

Re: Automatic Differentiation
 

What is the language of your code ?

If performance is most important then you should use a source transformation tool. For fortran Tapenade is a good choice. For C I dont know of any free tool but there is a commercial one called TAMC.

Re: Automatic Differentiation
 

I can highly recommend the AD tools ADF (for Fortran) and ADC (for C/C++) (from vivlabs). For the Jacobian, these tools produce AD code that is only 20-30% slower than hand code. In my application with 100,000+ equations it also scales well (no performance degrading when going from 1,000 to 100,000 equations). ADF and ADC use the operator overloading methods, thus only minimal code changes are necessary.

It's a commercial tool, but I believe they offer academic licenses too (which applies to me). You can check out their web site at http://www.vivlabs.com.

Re: Automatic Differentiation
 

Tools which use operator overloading are very slow and require large memory. ADF/ADC also uses overloading but they claim to get good performance. But I did not see any examples. Do you know of any papers which contain application of this tool ?

For C/C++, adolc is pretty easy to use but I found it is very slow compared to a source transformation approach (Tapenade).

Re: Automatic Differentiation
 

I have not yet compared ADF/ADC to source transformation tools, but I've tested adolc and CppAd. I agree with you that adolc is pretty easy to use but also found it is very slow, especially compared to ADF/ADC.

I can only report about my own application, where I need to get fast derivatives for large and sparse systems. I find that 30% performance degradation compared to hand code is quite acceptable provided there is no performance penalty if you increase the number of equations.

Last year they were giving out trial versions, thus you might be able to test ADF/ADC on your own application for free.

Re: Automatic Differentiation
 

Thank you very much for all the help, it is really much appreciated.

I am mostly looking to apply automatic differentiation to Fortran codes. Another requirement I forgot to write about is that I need it for commercial purposes. It seems that Tapenade is not free for commercial use.

I am relatively new to this field and got a bit confused about the differences between the source transformation and the operator overloading approach.

I will probably give ADF a try, if it is not too expensive.

Re: Automatic Differentiation
 

Does ADF/ADC provide reverse mode of differentiation ? I could not find this information on their website which is rather strange. But digging up a paper written by the company founder, I suspect it has only forward mode of differentiation. So that is one more aspect you must keep in mind. Reverse mode is very efficient when you have small number of dependent variables and large number of independent variables. While Tapenade is not free for commercial use you can buy a license for your use, I think it would be cheaper than buying a commercial product, since Tapenade is developed by a government lab. Another tool you can look at ADIFOR but I think it also provides forward mode only. For you application forward mode would be good enough I guess. You want to use AD for an implicit scheme. So what you need is not the jacobian but the product of a jacobian with a given vector. You can then use matrix-free methods like GMRES to solve your equation. What is the size of your jacobians ?

Re: Automatic Differentiation
 

Do you know the cost of Tapenade for commercial use? A tool developed by a govermant lab may not be cheaper.

In any case, I get your point about source transformation tools. Yes, they probably give you the very best performance, but you really have to factor in the additional programming effort. Why would you be willing to go this extra mile if operator overloading techniques have matured to a point where performance difference is on the order of a few ten percents? Within this range other factors such as the choice of the compiler have to be considered too.

Like most other AD tools ADF/ADC provide utility functions for efficient computation of a jacobian times a given vector.

Re: Automatic Differentiation
 

As far as I know there is no fixed price for Tapenade. You must contact them and find out. But I am sure it will be cheaper than commercial tools.

In most cases, Tapenade does not require rewriting your code. You may have to rewrite your code only if there are some pathological situations like some severe non-differentiability. Otherwise the code remains untouched. In ADF/ADC you still have to add some extra code. In Tapenade the specification of dependent, independent, etc. is external so you do not modify your code at all. Everything can be automated using makefiles. If your code changes then all you have to do is run make to differentiate and generate new executable.