Exercises 6.8 Exercises
1.
Find and classify all critical points of the function \(f(x,y)=x^3-y^3+6xy\text{.}\)2.
Find and classify all critical points of the function \(f(x,y)=x^4-8xy-4x^2+16y^2\text{.}\)3.
Let \(f(x,y)=x^3-y^3+6xy\text{.}\) Plot the region of the square \(-3\leq x\leq 3, -3\leq y\leq 3\) containing initial conditions such that the steepest descent method with fixed stepsize \(s=0.01\) converges to the minimum.4.
Write python code for minimizing the function \(f(x,y)=x^3-y^3+6xy\) using the steepest descent method with line search. Set any reasonable threshold and find the fixed stepsize for which the steepest descent method converges with the smallest possible number of iterations and compare it with the number of iterates needed with the line search.5.
Newton's method converges fast but requires the evaluation of the Hessian at every step, that is computationally heavy. The following method is an interesting deformation of Newton's method that does not use the Hessian. Implement the following optimization algorithm for a function \(f(\boldsymbol{x})\text{.}\)- choose an initial point \(\boldsymbol{x_0}\text{;}\)
- choose an initial symmetric matrix \(B\) with positive eigenvalues (the identity matrix is usually a good choice);
-
for all \(k\) (include some condition to stop the loop when you reach some desired precision or when iterations are too many) repeat the following instructions:
- set \(\boldsymbol{s}_k\) as the solution of the linear system \(B\boldsymbol{s}_k=-\nabla_{\boldsymbol{x}_k}f\text{;}\)
- set \(\boldsymbol{x}_{k+1}\) to \(\boldsymbol{x}_k+\boldsymbol{s}\text{;}\)
- set \(\boldsymbol{y_k}\) to \(\nabla_{\boldsymbol{x}_{k+1}}f-\nabla_{\boldsymbol{x}_{k}}f\text{;}\)
- set \(B\) to \(B + \frac{\boldsymbol{y_k} \boldsymbol{y_k}^T}{\boldsymbol{y_k}^T\boldsymbol{s}_k} - \frac{B\boldsymbol{s}_k(B\boldsymbol{s}_k)^T}{\boldsymbol{s}_k^TB\boldsymbol{s}_k}\text{.}\)
6. Effects of the floating point calculations.
- Find the minimum of the quadratic function