Whats the horizontal and vertical axis denoting in the below SGD contour

Question

What is that oscillation in the y axis in gradient descent contour.

Answer 1

The idea of the contour plot is to slice the 3D graph with multiple planes parallel to x,y plane. If we connect all the points where a plane cuts into the 3D graph, we will get contour line.

The oscillating lines show the converging of weights to the global minima. The blue line is taking more steps to converge and light green line is taking fewer steps.

Watch the following video on contour plots from Khan Academy for better understanding using visuals:

https://www.khanacademy.org/math/multivariable-calculus/thinking-about-multivariable-function/visualizing-scalar-valued-functions/v/contour-plots

I hope this helps.

Answer 2

In the example, we have $2$ variables / parameters we can tune, which is x and y axes.

The idea is that, any point in the 2D plane is a solution of the problem. Some solutions are good some solutions are bad.

The best solution is in the middle and the curve is showing how are we using an algorithm to get there.

Check the my code in other answer to see details.

How could stochastic gradient descent save time comparing to standard gradient descent?

---

BTW the oscillation in your plot is more like a gradient decent, but not SGD.