---
title: "window-based co-occurance"
source: https://www.jemoka.com/posts/kbhwindow_based_co_occurance/
---

window-based co-occurance is a matrix whereby we increment the value where each row is the center word, and each column is the number of occurrences of that other word next to a window of that word.
This approach is fine (not great), but if your vocabulary is HUGE, your word vectors will be exactly that length&mdash;bad. Therefore, we take this matrix and we SVD it; then, we chop off the smaller singular values to create a low dimensional approximation of our matrix.
This approach is extended in an approach called COALS (using person&rsquo;s correlations, using ramped windows, and normal SVD)&mdash;which also demonstrated linear meaning.
