Gradient Descent With Interactive Python Code Visualisation

Blog Contents:

• What is Gradient Descent?
• Understanding Gradient Descent?
• How Gradient Descent find?
• Interactive Python Code Visualisation  to find Gradient Descent

What is Gradient Descent?

It is an optimization algorithm that is used in machine learning to find the minimum value of the cost function.

Gradient descent is based on convex function and it is used to find the local minimum of a differential function(cost function).

My Github Link . If anyone can provide any kind of opportunity to me as a fresher machine learning or in data science field I will be thankfull

Understanding  Gradient Descent?

Consider a bowl. The inside curve of the bowl is your cost function and every point of the bowl curve surface is the coefficient (cost). The bottom of the bowl is the point where cost is minimum and give the lowest value of the cost function.

How do we find Gradient Descent?

To find the gradient descent first we need to assume the starting the value for the coefficient of the function. Lets say starting coefficient value is 0

coefficient = 0

Now we will use derivate our cost function to find the gradient of it. Derivate of the cost function tells the direction of cost function.Lets

derivative = differentiation(cost function)

Now we know the derivative(gradient) we can update the coefficient value with a new parameter is called learning rate (learning rate can be high or low . but low learning rate recommend for better results . The major drawback of low learning rate is that it is time consuming. you can see in the video how low and high learning effect the graph)

coefficient =  coefficient - (derivative*learning rate)

This process will go on until we get the lowest cost function value

Interactive Python Code Visualisation of Gradient Descent

import numpy as np

import matplotlib.pyplot as plt

x_poly = np.linspace(-3,5,81)

print(x_poly[:5],'....',x_poly[-5:])

def cost_function(X):

    return 2*X**2-4*X

def gradient(X):

    return  (4*X) -4

y_poly = cost_function(x_poly)

from ipywidgets import interactive

def f(iterations,learning_rate):

    #iterations = 15

    x_path = np.empty(iterations,)

    x_path[0] = x_start

    for i in range (1,iterations):

        derivative =gradient(x_path[i-1])

        x_path[i] = x_path[i-1]-(derivative*learning_rate)

    x_path

    plt.plot(x_poly,y_poly)

    plt.plot(x_path,cost_function(x_path),'-o')

interactive_plot = interactive(f,iterations = (1,20),learning_rate= (0.01,1,.1))

interactive_plot

Other post that you might be interested:

Understanding Hypothesis testing and its Application Using Python

Interactive polynomial and Linear Regression using Jupyter Notebook