3

My question comes from a comment in this question Vector Jacobian product in automatic differentiation

The question states...

$$ t = Wz, \,\,\, z\in \mathbb{R}^{m\times 1}, t \in \mathbb{R}^{n \times 1}, W\in\mathbb{R}^{n \times m} $$

$$ \frac{\partial t}{\partial z} = W $$

Which is all good but then a comment states an observation that a different Jacobian, $\frac{\partial t}{\partial W} \in \mathbb{R}^{n \times n \times m}$. I cannot justify to myself wwhy that Jacobian would have three axes. Can anyone explain this?

Joff
  • 599
  • 2
  • 13

1 Answers1

2

Well, $t$ is a vector, and $W$ is a matrix. Think about differentiating everything element-wise, i.e. $$D_{ijk}=\frac{\partial t_i}{\partial W_{jk}}$$ and $D$ automatically becomes a 3D tensor.

gunes
  • 49,700
  • 3
  • 39
  • 75