Continuous steepest descent path for traversing non-convex regions

Abstract

This paper revisits the ideas of seeking unconstrained minima by following a continuous steepest descent path (CSDP). We are especially interested in the merits of such an approach in regions where the objective function is non-convex and Newton-like methods become ineffective. The paper combines ODE-trajectory following with trust-region ideas to give an algorithm which performs curvilinear searches on each iteration. Progress along the CSDP is governed both by the decrease in function value and measures of the accuracy of a local quadratic model. Experience with a prototype implementation of the algorithm is promising and it is shown to be competitive with more conventional line search and trust region approaches. In particular, it is also shown to perform well in comparison with the, superficially similar, gradient-flow method proposed by Behrman.