[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / raw/concept/kbhmatrix_calculus.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "matrix calculus" source: https://www.jemoka.com/posts/kbhmatrix_calculus/ --- Transpose Rules \(\qty(AB)^{T} = B^{T}A^{T}\) \(\qty(a^{T}Bc)^{T} = c^{T} B^{T}a\) \(a^{T}b = b^{T}a\) \(\qty(A+B)C = AC + BC\) \(\qty(a+b)^{T}C = a^{T}C + b^{T}C\) \(AB \neq BA\) Derivative Scalar derivative Vector derivative \(f\qty(x) \to \pdv{f}{x}\) \(f\qty(x) \to \pdv{f}{x}\) \(bx \to b\) \(x^{T}B \to B\) \(bx \to b\) \(x^{T}b \to b\) \(x^{2} \to 2x\) \(x^{T}x \to 2x\) \(bx^{2} \to 2bx\) \(x^{T}Bx \to 2Bx\) Products \begin{equation} \pdv{AB}{A} = B^{T}, \pdv{AB}{B} = A^{T} \end{equation} \begin{equation} \pdv{Ax}{A} = x^{T}, \pdv{Ax}{x}= A \end{equation} Vector and Quadratic Forms \begin{equation} \pdv{y^{T} x}{x} = y, \pdv{y^{T} x}{y} = x \end{equation} \begin{equation} \pdv{x^{T}Ax}{x} = \qty(A+A^{T})x = 2Ax \end{equation} for symmetric A Chain Rule for Matrix Multiplication Suppose: \begin{equation} z = Wu + b \end{equation} \begin{equation} J = J\qty(z) \end{equation} then: \(\pdv{J}{W} = \pdv{J}{z} u^{T}\). Rest of them