[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / raw/concept/kbhderivatives_descent_and_approximation.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "SU-CS361: Derivatives, Bracketing, Descent, and Approximation Index" source: https://www.jemoka.com/posts/kbhderivatives_descent_and_approximation/ --- Formal Formulation of Optimization constraint types of conditions FONC and SONC Derivatives Directional Derivatives numerical methods Finite-Difference Method Forward Difference Central Difference Backward Difference Complex-Difference Method exact methods: autodiff Forward Accumulation cooool: Dual Number Method Bracketing (one dimensional optimization schemes) Fibonacci Search Quadratic Search Shubert-Piyavskill Method Descent Direction Iteration Line Search Approximate Line Search Sufficient Decrease Condition Curvature Condition Trust Region Methods First-Order Methods good ol gradient descent Conjugate Gradient Hyper-gradient Descent Second-Order Methods Newton’s Method or approximate it using Secant Method Direct Methods Cyclic Coordinate Search Accelerated Coordinate Search Powell’s Method Hooke-Jeeves Search Generalized Pattern Search opportunistic search dynamic ordering Nelder-Mead Simplex Method