9

how exactly is partial derivative different from gradient of a function?

In both the case, we are computing the rate of change of a function with respect to some independent variable. While I was going through Gradient Descent, there also the partial derivative term and the gradient were written and used separately.

What's the catch?

Upendra01
  • 1,566
  • 4
  • 18
  • 28
  • 1
    It's a theorem. The gradient is the derivative of a function $\mathbb{R}^m\to\mathbb{R}$, as explained at https://stats.stackexchange.com/a/257616/919. As such it is a linear form on $\mathbb{R}^m$. It is defined without reference to any particular basis of $\mathbb{R}^m$. The partial derivatives are the derivatives of functions $\mathbb{R}\to\mathbb{R}$ defined by holding all but one variable fixed. They depend on the basis chosen for $\mathbb{R}^m$. The theorem asserts that the components of the gradient with respect to that basis are the partial derivatives. – whuber Jun 16 '17 at 14:26

2 Answers2

7

Gradient is the partial derivatives :

$$\nabla f = \left(\frac{\partial f}{\partial x_1};\frac{\partial f}{\partial x_2};...;\frac{\partial f}{\partial x_n}\right)$$

Eg : $f=x^2y$

$$\nabla f =(2xy;x^2)$$

Gradient gives the rate of change in every direction $e$ ($e$ is a unit vector) thanks to the dot product $\nabla f.e$ :

Eg :$\nabla f.(0;1)=\frac{\partial f}{\partial y}$

Benoit Sanchez
  • 7,377
  • 21
  • 43
4

If a function $f$ takes the parameters $x_1, \ldots, x_n$, then the partial derivatives w.r.t. the $x_i$ determine the gradient:

\begin{equation} \nabla f = \frac{\partial f}{\partial x_1 }\mathbf{e}_1 + \cdots + \frac{\partial f}{\partial x_n }\mathbf{e}_n. \end{equation}

If you look at the definition of the gradient-descent method, it is completely defined in terms of the gradient.

how exactly is partial derivative different from gradient of a function?

A partial derivative may be taken also w.r.t. a different variable, e.g.,

\begin{equation} \frac{\partial f}{\partial z }, \end{equation}

where $z = z(x_1, \ldots, x_n)$ is some function of the xs.

Ami Tavory
  • 4,410
  • 11
  • 20
  • can gradient be defined as partial derivatives w.r.t say yi's instead of xi's. Is it allowed? – Upendra01 Jun 16 '17 at 08:52
  • @Snowbell You could form a vector composed of partial derivatives w.r.t. yis, but, in general, it will not correspond to the direction of greatest change. – Ami Tavory Jun 16 '17 at 08:54