[[
wikihub
]]
Search
⌘K
Explore
People
For Agents
Sign in
Explore
People
For Agents
Sign in
@jemoka / Jemoka Knowledge Base / raw/concept/kbhmle_for_gaussian.md
Suggest edit
Cancel
Submit suggestion
Title
Name
Note
--- title: "MLE for Conditional Gaussian" source: https://www.jemoka.com/posts/kbhmle_for_gaussian/ --- Let’s say we want to find MLE parameters \(\theta\) for a conditional Gaussian with constant variance. That is: \begin{equation} p\qty(y_{i} | x_{i}) = \mathcal{N} \qty(y_{i}|f_{\theta } \qty(x_{i}), \sigma^{2}) \end{equation} and we have a corresponding dataset: \(\qty(x_1, y_1), …, \qty(x_{m}, y_{m})\). where: \begin{align} \hat{\theta} &= \arg\max_{\theta} \sum_{i=1}^{m} \log p\qty(y_{i}|x_{i}) \\ &= \arg\max_{\theta} \sum_{i=1}^{m} \log \mathcal{N} \qty(y_{i}| f_{\theta} \qty(x_{i}), \sigma^{2}) \\ &= \arg\max_{\theta } \sum_{i=1}^{m} \log \frac{1}{\sqrt{{2 \pi \sigma^{2}}}} \exp \qty(- \frac{\qty(y_{i}- f_{\theta }\qty(x_{i}))^{2}}{2\sigma^{2}}) \end{align} taking the $log $ of an $exp $, and removing constants (since they don’t affect optimization), this gives us: \begin{equation} \arg\max_{\theta} \sum_{i=1}^{m} - \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation} which is the… \begin{equation} \arg\min_{\theta} \sum_{i=1}^{m} \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation} woah, least-squares error!