Note
Click here to download the full example code
Warm-up: numpy¶
Created On: Dec 03, 2020 | Last Updated: Dec 03, 2020 | Last Verified: Nov 05, 2024
A third order polynomial, trained to predict \(y=\sin(x)\) from \(-\pi\) to \(pi\) by minimizing squared Euclidean distance.
This implementation uses numpy to manually compute the forward pass, loss, and backward pass.
A numpy array is a generic n-dimensional array; it does not know anything about deep learning or gradients or computational graphs, and is just a way to perform generic numeric computations.
99 259.32136455014984
199 174.48468952019508
299 118.38137532613197
399 81.27821341848298
499 56.74032018775961
599 40.51221381745156
699 29.779663424823287
799 22.68155122959751
899 17.98706625671968
999 14.882230882969736
1099 12.828728935535704
1199 11.470546875154195
1299 10.572233986074473
1399 9.978072476247
1499 9.585075558303155
1599 9.325130204123404
1699 9.153187466414273
1799 9.039452214839464
1899 8.964217811006574
1999 8.914449979849916
Result: y = -0.0008873827303141185 + 0.84718756983399 x + 0.0001530882870915524 x^2 + -0.09197151911287338 x^3
import numpy as np
import math
# Create random input and output data
x = np.linspace(-math.pi, math.pi, 2000)
y = np.sin(x)
# Randomly initialize weights
a = np.random.randn()
b = np.random.randn()
c = np.random.randn()
d = np.random.randn()
learning_rate = 1e-6
for t in range(2000):
# Forward pass: compute predicted y
# y = a + b x + c x^2 + d x^3
y_pred = a + b * x + c * x ** 2 + d * x ** 3
# Compute and print loss
loss = np.square(y_pred - y).sum()
if t % 100 == 99:
print(t, loss)
# Backprop to compute gradients of a, b, c, d with respect to loss
grad_y_pred = 2.0 * (y_pred - y)
grad_a = grad_y_pred.sum()
grad_b = (grad_y_pred * x).sum()
grad_c = (grad_y_pred * x ** 2).sum()
grad_d = (grad_y_pred * x ** 3).sum()
# Update weights
a -= learning_rate * grad_a
b -= learning_rate * grad_b
c -= learning_rate * grad_c
d -= learning_rate * grad_d
print(f'Result: y = {a} + {b} x + {c} x^2 + {d} x^3')
Total running time of the script: ( 0 minutes 0.237 seconds)
