[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / wiki/concepts/xavier_initialization.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "xavier initialization" type: concept related: [Neural Network] source: https://www.jemoka.com/posts/kbhxavier_initialization/ confidence: high status: active --- An neural network initialization scheme that tries to avoid Vanishing Gradients. Consider \(Wx\) step in a neural network: \begin{equation} o_{i} = \sum_{j=1}^{n_{\text{in}}} w_{ij} x_{j} \end{equation} The variance of this: \begin{equation} \text{Var}\qty [o_{i}] = n_{\text{in}} \sigma^{2} v^{2} \end{equation}