Descent method — Steepest descent and conjugate gradient in Python
Python implementation
Let’s start with this equation and we want to solve for x:
The solution x the minimize the function below when A is symmetric positive definite (otherwise, x could be the maximum). It is because the gradient of f(x), ∇f(x) = Ax- b. And when Ax=b, ∇f(x)=0 and thus x is the minimum of the function.
In this article, I am going to show you two ways to find the solution x — method of Steepest Descent and method of Conjugate Gradient.
Method of Steepest Descent in Python
Now let’s use this steepest_descent
function to calculate
With the steepest_descent
method, we get a value of (-4,5) and a wall time 2.01ms.