Suggest edit — matrix calculus

Title

Name

Note

---
title: "matrix calculus"
type: concept
source: https://www.jemoka.com/posts/kbhmatrix_calculus/
confidence: high
status: active
---

Transpose Rules \(\qty(AB)^{T} = B^{T}A^{T}\) \(\qty(a^{T}Bc)^{T} = c^{T} B^{T}a\) \(a^{T}b = b^{T}a\) \(\qty(A+B)C = AC + BC\) \(\qty(a+b)^{T}C = a^{T}C + b^{T}C\) \(AB \neq BA\) Derivative Scalar derivative Vector derivative \(f\qty(x) \to \pdv{f}{x}\) \(f\qty(x) \to \pdv{f}{x}\) \(bx \to b\) \(x^{T}B \to B\) \(bx \to b\) \(x^{T}b \to b\) \(x^{2} \to 2x\) \(x^{T}x \to 2x\) \(bx^{2} \to 2bx\) \(x^{T}Bx \to 2Bx\) Products \begin{equation} \pdv{AB}{A} = B^{T}, \pdv{AB}{B} = A^{T} \end{equation}
\begin{equation} \pdv{Ax}{A} = x^{T}, \pdv{Ax}{x}= A \end{equation}
Vector and Quadratic Forms \begin{equation} \pdv{y^{T} x}{x} = y, \pdv{y^{T} x}{y} = x \end{equation}
\begin{equation} \pdv{x^{T}Ax}{x} = \qty(A+A^{T})x = 2Ax \end{equation}
for symmetric A
Chain Rule for Matrix Multiplication Suppose:
\begin{equation} z = Wu + b \end{equation}
\begin{equation} J = J\qty(z) \end{equation}
then: \(\pdv{J}{W} = \pdv{J}{z} u^{T}\).
Rest of them