[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jeremynixon / Thinking / daily/2017-10-14-interesting-facts-in-machine-learning-neural-networks.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "17-10-14 Interesting Facts in Machine Learning (Neural Networks)" visibility: public --- # 17-10-14 Interesting Facts in Machine Learning (Neural Networks) Category: [[idea-lists-upon-request|Idea Lists (Upon Request)]] [Read the original document](https://docs.google.com/document/d/1NTpMGZG45aHJqUNpAtmpvsPlUClrDHIFHC3TTTVHQeo/edit?usp=drivesdk&sa=D&ust=1596495076884000&usg=AOvVaw323F_Th6pv-cQmpK-IjVD0) <!-- gdoc-inlined --> --- 1. Learns compositional (bottom-up) hierarchical structure 2. Model complexity overcomes the curse of dimensionality 1. Combinatorial in depth and in width 3. Requires high signal-to-noise ratio 4. ‘Just’ adaptive basis function regression 5. Optimizer improved by exponentially weighted average of the gradient, learning rate 6. Covariate Shift 7. Close-to-linear model leads to failure to generalize, ex. adversarial examples 8. Dimensionality of the representation increases with depth of a convnet. 9. Softmax leads to extreme solutions 10. Non-convex optimization surface is dominated by saddle points. 11. Convnets are: 1. Parameter Sharing leads to translation equivariance 2. Locality (Sparse Connectivity) 3. Composition 4. Not equivariant to scale or rotation. 12. Many machine learning libraries implement cross-correlation but call it convolution 13. Achieving the global minimum would overfit the training data. --- *Source: [Original Google Doc](https://docs.google.com/document/d/1NTpMGZG45aHJqUNpAtmpvsPlUClrDHIFHC3TTTVHQeo/edit?usp=drivesdk&sa=D&ust=1596495076884000&usg=AOvVaw323F_Th6pv-cQmpK-IjVD0)*