Automatic differentiation of computer programs in a parallel computing environment

Abstract

This paper gives an introduction to a number of methods collectively known as Automatic Differentiation (AD). AD is the systematic application of the rules of calculus to computer programs yielding, as output, programs which accurately compute numerical values of first, second, or higher derivatives. AD is not symbolic differentiation nor a divided difference method. It is a process for evaluating derivatives which depends only on the algorithmic specification of the entire program or part of the program. AD software tools are now available for most languages employed in scientific computing including Fortran, Fortran90, HPF, C++, etc. The tools have reached a sufficient maturity that they should be confidently used in applications which include processes which rely upon the generation of derivative information or error analysis. This paper will describe the basic ideas of AD and show how it may be extended for use in a parallel computing environment.