[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / wiki/concepts/optimization_math.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "optimization (math)" type: concept related: [Convex Optimization, Non Linear Optimization, Worst Case Analysis] source: https://www.jemoka.com/posts/kbhoptimization_math/ confidence: high status: active --- constituents where \(x \in \mathbb{R}^{n}\) is a vector of variables \(f_{0}\) is the objective function, “soft” to be minimized \(f_{1} … f_{m}\) are the inequality constraints \(g_{1} … g_{p}\) are the equality constraints requirements Generally of structure: \begin{equation} \min f_{0}\qty(x) \end{equation} subject to: \begin{align} f_{i} \qty(x) \leq 0, i = 1 \dots m \\ g_{i}\qty(x) = 0, i = 1 \dots p \end{align} solving optimization problems You can’t generally solve optimization problems… Some types convex optimization problems, you can solve these non-linear optimization additional information optimization terms feasible \(x \in \mathbb{R}^{n}\) is feasible if \(x \in \text{dom } f_{0}\) and it satisfies constraitns optimal value \begin{equation} p^{*} = \text{inf}\qty {f_{0}\qty(x) \mid f_{i}\qty(x) \leq 0, i = 1, \dots, m, h_{i}\qty(x) = 0, i = 1 \dots p} \end{equation} \(p^{*} = \infty\) if a problem is infeasible \(p^{*} = -\infty\) if a problem is unbounded below a feasible \(x\) is optimal if \(f_{0}\qty(x) = p^{*}\) locally optimal optimal with additional constraint: \begin{equation} \norm{z-x}_{2} \leq R \end{equation} implicit constraints Of course, all optimization problems have the following constraint implicitly: \begin{equation} \end{equation} why optimization Headline: instead of saying how to choose some action/model \(x\), you articulate what you want out of the properties of \(x\), then let an algorithm decide on action/model \(x\). optimization for decision making “Its a mathematical formulation of making good choices.” trades in a portfolio airplane controls assignment / schedule resource allocation The smaller the objective \(f_{0}\qty(x)\), the better. Constraints limit action space or impose conditions on the outcome. \(x\) represents some kind of actions. optimization for modeling Instead of \(x\) representing an action, \(x\) represents the parameters. Constraints impose model parameter requirements; objective \(f_{0}\qty(x)\) is a sum of…. a prediction error (loss) on observed data a (regularization) term that penalizes model complexity optimization for worst-case analysis variables are actions or parameters out of our control constraints limit the possible range of parameters minimizing \(-f_{0}\qty(x)\) finds worst possible parameter values for your system optimization-based models Simulate the dynamics of a system (i.e. what it will do) by giving it the same signals. i.e. model cells by constraints of reactions and optimizing for e.g. growth. commentary “Its not interesting to have bigger “margins” to the constraints.”