[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / raw/concept/probability/kbhprobability_of_failure.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "Probability of Failure" source: https://www.jemoka.com/posts/kbhprobability_of_failure/ --- \begin{align} p_{\text{fail}} &= \mathbb{E}_{\tau \sim p\qty(\cdot)} \qty [1 \qty{\tau \not \in \psi}] \\ &= \int 1 \qty {\tau \not\in \psi} p\qty(\tau) \dd{\tau } \end{align} that is, the Probability of Failure is just the normalizing constant of the Failure Distribution. Like with Failure Distribution itself, computing this is quite intractable. We have a few methods to solve this, namely: direct estimation: directly approximate your failure probability from nominal distribution \(p\) — \(\tau_{j} \sim p\qty(\cdot)\), \(\hat{p}_{\text{fail}} = \frac{1}{m} \sum_{i=1}^{m} 1\qty {\tau_{i} \not \in \psi}\) Importance Sampling: design a distribution to probe failure, namely proposal distribution \(q\), and then reweight by how different it is from \(p\) — \(\tau_{j} \sim q\qty(\cdot)\), \(\hat{p}_{\text{fail}} = \frac{1}{m}\sum_{i=1}^{m} w_{i} \mathbb{1} \qty {\tau_{i}\not \in \psi}\), call \(w_{i} = \frac{p\qty(\tau_{i})}{q\qty(\tau_{i})}\) (the “importance weight”) adaptive importance sampling multiple importance sampling sequential monte-carlo How do you pick a proposal distribution? See proposal distribution. evaluating estimators bias: an estimator is unbiased if it predicts the true value in expectation; namely \(\mathbb{E}\qty [\hat{p}_{\text{fail}}] = p_{\text{fail}}\) consistency: an estimator is consistent if it converges to infinity at infinite samples, namely that \(\text{lim}_{m \to \infty} \hat{p}_{\text{fail}} = p_{\text{fail}}\) variance: we want the variance of the estimates around the true value to be low evaluation of direct estimation \begin{equation} \text{Var}\qty [\hat{p}] = \frac{p_{\text{fail}} \qty(1-p_{\text{fail}})}{m} \end{equation} as the number of samples increases, the variance in the estimate decreases (yay!) as the probability of failure deceases, the variance in the failure actually icreases!!! baaaad vibes!