Note
Click here to download the full example code
Warm-up: numpy¶
Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024
A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.
This implementation uses numpy to manually compute the forward pass, loss, and backward pass.
A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.
99 625.6180673440844
199 439.2741136280522
299 309.447969034437
399 218.92549346112696
499 155.759265389961
599 111.64950494145245
699 80.8252411278727
799 59.270336036054026
899 44.18752857094402
999 33.62693308857416
1099 26.228283017710723
1199 21.041926056674768
1299 17.40439731958987
1399 14.85185374380926
1499 13.05979793151108
1599 11.801073048615276
1699 10.916567269540062
1799 10.294766681193368
1899 9.857473621793368
1999 9.549824393823018
Result: y = -0.027652688581905414 + 0.8498561231156678 x + 0.004770548923102859 x^2 + -0.09235109757922562 x^3
import numpy as np
import math
# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)
# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
# y = a + b x + c x^2 + d x^3
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = np.square(y_pred - y).sum()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')
Total running time of the script: ( 0 minutes 0.233 seconds)
