Iterative Methods for Optimization

This book is suitable for use as a tutorial, a reference, or a textbook in an introductory optimization course. It presents a carefully selected group of methods for unconstrained and bound-constrained optimization problems and analyzes them in depth, both theoretically and algorithmically. Readers should be familiar with the material in an elementary graduate-level course in numerical analysis, and with local convergence results for systems of nonlinear equations. The book focuses on clarity in algorithmic description and analysis rather than generality. It covers more than the traditional gradient-based opitimization; it treats sampling methods, including the Hooke-Jeeves, implicit filtering, MDS, and Nelder-Mead schemes, in a unified way, and also makes connections between sampling methods and the traditional gradient methods.