Stochastic Gradient Descent
Next topic
Video
Notes
Code
y' = a x + b
x
y
a:
-2
-1
0
1
2
3
b:
-2
-1
0
1
2
3
alpha:
0.001
0.005
0.01
0.05
0.1
0.5
1.0
5.0
10
50
batch size:
1
2
3
6
with Momentum
Initialize
Next
\(x\)
\(y\)
\(ŷ\)
\(ŷ-y\)
L
\(\partial L \over \partial a\)
\(\partial L \over \partial b\)
a
b
Epoch:
Batch:
L (loss):
velocity a:
velocity b:
a:
b:
chart:
X-Y Plot
Loss
a
b