Suggest edit — MLE for Conditional Gaussian

Title

Name

Note

---
title: "MLE for Conditional Gaussian"
type: concept
related: [Maximum Likelihood Parameter Learning]
source: https://www.jemoka.com/posts/kbhmle_for_gaussian/
confidence: high
status: active
---

Let&rsquo;s say we want to find MLE parameters $\theta$ for a conditional Gaussian with constant variance. That is:
\begin{equation} p\qty(y_{i} | x_{i}) = \mathcal{N} \qty(y_{i}|f_{\theta } \qty(x_{i}), \sigma^{2}) \end{equation}
and we have a corresponding dataset: $\qty(x_1, y_1), &hellip;, \qty(x_{m}, y_{m})$.
where:
\begin{align} \hat{\theta} &amp;= \arg\max_{\theta} \sum_{i=1}^{m} \log p\qty(y_{i}|x_{i}) \\ &amp;= \arg\max_{\theta} \sum_{i=1}^{m} \log \mathcal{N} \qty(y_{i}| f_{\theta} \qty(x_{i}), \sigma^{2}) \\ &amp;= \arg\max_{\theta } \sum_{i=1}^{m} \log \frac{1}{\sqrt{{2 \pi \sigma^{2}}}} \exp \qty(- \frac{\qty(y_{i}- f_{\theta }\qty(x_{i}))^{2}}{2\sigma^{2}}) \end{align}
taking the $log $ of an $exp $, and removing constants (since they don&rsquo;t affect optimization), this gives us:
\begin{equation} \arg\max_{\theta} \sum_{i=1}^{m} - \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation}
which is the&hellip;
\begin{equation} \arg\min_{\theta} \sum_{i=1}^{m} \qty(y_{i} - f_{\theta} (x_{i}))^{2} \end{equation}
woah, least-squares error!