Automatic Hessians by reverse accumulation

Abstract

Let n be the number of independent variables of a function f, and let W and S respectively be the time and space bounds for the joint evaluation of {f, f} using automatic differentiation with reverse accumulation. In this note, we examine an extension of the technique of reverse accumulation which allows the automatic extraction of the Hessian of f. The method allows the parallel evaluation of all rows of the Hessian matrix in about 2W time units and 3S space units on each of n processors, or sequential row-by-row evaluation in about 2nW time units and 3S space units on a single processor. The approach described here is intended for use with operator overloading (for example in Ada) and allows the conventional coding of the target function f.