Exercise 6.3: Finding Zero of a Continuously Differentiable Function
Let F: R^r -> R^n be a continuously differentiable function. Consider a method for finding the zero of F by minimizing the function f(w) = ||F(w)|| = sqrt(sum((F_i(w))^2)), where i = 1 to n.
Given a direction of decrease pk for ||F|| at wk, the Armijo step length criterion requires that the reduction in ||F(w)|| be no worse than 7a times the reduction in ||Mk(w)||, the norm of the affine model. In other words:
||F(wk) - F(wk + apk)||^2 <= m * ||Mk(wk) - Mk(wk + apk)||^2 (6.1)
where a is a pre-assigned constant such that 0 < a < 1.
Show that if pk is any nonzero vector such that J(wk)pk ~ F(wk), the backtracking condition (6.1) is equivalent to requiring that wk + apk satisfies ||F(wk + apk)|| < (1 - a * a) * ||F(wk)||.