2

I am a beginner of the machine learning. And at the same time, this is the first time to ask a question in this site, so I might have a problem with this question in terms of understandability. And when I learned backpropagation, I saw this type of formula.

E: scholar function(error function), $\mathbf{W}$: matrix(Weight matrix), $\mathbf{p}$: vector, $\mathbf{x}$: input vector, $\mathbf{b}$: bias vector, $\mathbf{W}^T$: transposed matrix of $\mathbf{W}$

$\mathbf{p} := \mathbf{Wx} + \mathbf{b}$

$\frac {\partial E}{\partial \mathbf{W}} = \frac{\partial E}{\partial \mathbf{p}} \cdot \frac{\partial \mathbf{p}}{\partial \mathbf{W}^T} = \frac{\partial E}{\partial \mathbf{p}} \mathbf{x}^T$

This is the first time for me to see the differentiation of vector by the matrix. But, how is it defined?

Althogh I coundn't find any useful information, could anyone explain this?

Frans Rodenburg
  • 10,376
  • 2
  • 25
  • 58
Kazuya Tomita
  • 139
  • 7
  • Read this https://atmos.washington.edu/~dennis/MatrixCalculus.pdf – Carl Oct 07 '17 at 04:34
  • Accounts of differentiation (involving even more complicated functions) can be found at https://stats.stackexchange.com/questions/257579 and https://stats.stackexchange.com/questions/246738. You may [search this site](https://stats.stackexchange.com/search?q=+vector+matrix+derivative) for more solutions. – whuber Oct 07 '17 at 16:47
  • Although I checked those topics, I haven't found useful information. There are just related information. I just would like to know exact definition of the differentiation of vector by a matrix. – Kazuya Tomita Oct 14 '17 at 02:55
  • this is the wiki:https://en.wikipedia.org/wiki/Matrix_calculus. And according to it, the differentiation of vector by matrix is not defined. So, I assume it is not common. But my book says and you appear to know it. Could you explain? – Kazuya Tomita Oct 14 '17 at 03:16

0 Answers0