Numerical Optimization Jorge Nocedal Stephen J. Wright Springer Springer Series in Operations Research Editors: Peter Glynn Stephen M. Robinson Springer New York Berlin Heidelberg Barcelona Hong Kong London Milan Paris Singapore Tokyo Jorge Nocedal Stephen J. Wright Numerical Optimization With 85 Illustrations 1 3 Jorge Nocedal Stephen J. Wright ECE Department Mathematics and Computer Northwestern University Science Division Evanston, IL 60208-3118 Argonne National Laboratory USA 9700 South Cass Avenue Argonne, IL 60439-4844 USA Series Editors: Peter Glynn Stephen M. Robinson Department of Operations Research Department of Industrial Engineering Stanford University University of Wisconsin–Madison Stanford, CA 94305 1513 University Avenue USA Madison, WI 53706-1572 USA Cover illustration is from Pre-Hispanic Mexican Stamp Designs by Frederick V. Field, courtesy of Dover Publi- cations, Inc. Library of Congress Cataloging-in-Publication Data Noce