This answer key contains 4 multiple-choice, 4 fill-in-the-blank, and 4 output-prediction questions for Python gradient descent, step by step, with the exact answer and a short explanation for every question.
The answers are shown in full, so use this page after making an attempt. The practice page is the better first pass.
What does a gradient-descent update subtract from the parameter?
Answer: The gradient times the learning rate
Why: The gradient gives the direction of increasing loss; scaling it by the learning rate controls the step size.
If the gradient is negative, what happens to w after a normal update?
Answer: It increases
Why: Subtracting a negative gradient adds a positive amount to w.
What is the main risk of a learning rate that is too large?
Answer: The updates can overshoot and diverge
Why: A large step can jump past the minimum and make the next gradient even larger.
What is the output?
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
print(gradient(0, [1, 2], [1, 2]))Answer: -5.0
Why: At w = 0, each residual is negative, so the gradient is negative.
Complete the gradient-descent update.
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
w = 1.0
w -= 0.1 * ____(w, [1, 2], [2, 4])
print(w)Answer: gradient
Why: The update subtracts the learning rate times the gradient from w.
Complete the factor from differentiating the squared loss.
def gradient(w, xs, ys):
return ____ * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
print(gradient(1, [1, 2], [2, 4]))Answer: 2
Why: Differentiating (w*x - y) squared brings down a factor of 2.
Complete the mean in the loss.
def loss(w, xs, ys):
return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / ____(xs)
print(loss(1, [1, 2], [2, 4]))Answer: len
Why: The total squared error is divided by the number of observations.
Complete the loop so it takes three updates.
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
w = 1.0
for _ in range(____):
w -= 0.1 * gradient(w, [1, 2], [2, 4])
print(round(w, 3))Answer: 3
Why: range(3) produces three iterations.
What does this code print?
def loss(w, xs, ys):
return sum((w * x - y) ** 2 for x, y in zip(xs, ys)) / len(xs)
print(loss(2, [1, 2], [2, 4]))Answer: 0.0
Why: At w = 2 every prediction matches the target, so the loss is 0.0.
What does the program output?
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
print(gradient(2, [1, 2], [2, 4]))Answer: 0.0
Why: The model already fits the data, so every residual is zero and the gradient is 0.0.
What appears on the screen?
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
w = 1.0
w -= 0.1 * gradient(w, [1, 2], [2, 4])
print(w)Answer: 1.5
Why: The gradient is -5.0, so subtracting 0.1 times -5.0 raises w from 1.0 to 1.5.
What is printed?
def gradient(w, xs, ys):
return 2 * sum((w * x - y) * x for x, y in zip(xs, ys)) / len(xs)
w = 1.0
for _ in range(2):
w -= 0.1 * gradient(w, [1, 2], [2, 4])
print(w)Answer: 1.75
Why: The first update reaches 1.5 and the second reaches 1.75.
Try the questions first
Open the practice page for Python gradient descent, step by step and answer the same 12 questions with instant feedback.