Suggest edit — linear-algebra-as-thinking

Title

Name

Note

---
visibility: public-edit
---

# linear algebra as thinking

linear algebra is the mathematics of directions, transformations, and projections. every idea can be a vector. every process can be a matrix. and the deep structures — eigenvalues, rank, nullspace — reveal things about systems that are invisible from any other angle.

## everything is a vector

a vector is just a list of numbers — but that framing undersells it. a vector is a *point in a space*, and the space can represent anything:

- a color is a vector in RGB space: (255, 128, 0) = orange
- a location is a vector in geographic space: (40.7, -74.0) = new york
- a person's test scores are a vector: (SAT: 1520, GPA: 3.9, AP count: 12)
- a word's meaning is a vector in semantic space: "king" = (0.2, -0.5, 0.8, ...)

once something is a vector, you can do vector operations on it. you can compute distances (how similar are these two things?), averages (what's the "center" of this group?), and projections (how much of this is in that direction?). at its core, this is [[arithmetic-everywhere|arithmetic]] lifted into higher dimensions — addition, subtraction, scaling — but now operating on objects far richer than single numbers.

the semantic space example from my essay: word embeddings represent meanings as vectors in 300-dimensional space. the dot product of two word vectors measures their semantic similarity. and vector arithmetic captures analogies:

vector("king") - vector("man") + vector("woman") ≈ vector("queen")

the direction from "man" to "king" (roughly: royalty) is the same as the direction from "woman" to "queen." this isn't a coincidence or a trick — it reflects genuine structure in how language encodes meaning. linear algebra made that structure visible and computable.

## every process is a matrix

a matrix is a linear transformation — it takes vectors in one space and maps them to vectors in another. but the conceptual reach is much broader:

- google's PageRank algorithm represents the web as a matrix (pages × links) and finds the dominant eigenvector
- a markov chain represents state transitions as a matrix: what's the probability of going from state A to state B?
- a rotation, reflection, or scaling in any dimension is a matrix multiplication
- a system of equations is a matrix equation: Ax = b

the power is that all of these — web ranking, probability, geometry, equation-solving — use the same mathematical machinery. once you know how to work with matrices, you can work with all of them.

## eigenvalues: what stays stable under transformation

this is the deepest idea in linear algebra. an eigenvector of a matrix is a direction that doesn't change under the transformation — it just gets scaled. the eigenvalue tells you by how much.

why does this matter? because eigenvectors reveal the *essential behavior* of a system:

- in PageRank, the dominant eigenvector is the steady-state importance of each page
- in principal component analysis (PCA), eigenvectors are the directions of maximum variance in data — the "most important" dimensions
- in dynamical systems, eigenvalues tell you whether the system is stable (|λ| < 1), unstable (|λ| > 1), or oscillating (complex λ)
- in quantum mechanics, eigenvalues of an operator are the possible measurement outcomes

eigenvectors answer: "what is the intrinsic structure of this transformation?" strip away coordinates, strip away representation — what's left is the eigenstructure.

## projection: seeing components

projecting a vector onto a direction extracts "how much of this is in that direction." in 3D, projecting onto the x-axis gives you the x-component.

but projection is conceptually richer than that:
- in statistics, linear regression is projection: you're projecting data onto the best-fit line
- in signal processing, fourier analysis is projection: you're projecting a signal onto each frequency component (see [[engineering-and-modeling|engineering]])
- in machine learning, dimensionality reduction is projection: compressing high-dimensional data into a lower-dimensional space while preserving the most important structure

any time you're decomposing something into components, you're projecting.

## rank and dimensionality

the rank of a matrix tells you how many independent dimensions the transformation actually uses. a 1000×1000 matrix might have rank 3 — meaning all that apparent complexity lives in just 3 dimensions.

this is profound for data analysis. real-world data is almost always lower-dimensional than it appears. a dataset with 100 features might effectively live in a 5-dimensional subspace. finding that subspace (via PCA, SVD, or other methods) is one of the most powerful techniques in data science and [[computer-science|computer science]] more broadly.

the concept generalizes: when something seems complex, ask "what's the effective dimensionality?" how many independent factors actually drive this system? that's the rank. this move — from surface complexity to underlying simplicity — is a form of [[abstraction-as-power|abstraction]]: ignoring the noise to find the signal.

## the deep point

linear algebra is the mathematics of structure in spaces. once you start seeing things as vectors in spaces — meanings, states, data points, possibilities — you gain access to a powerful toolkit: distances, transformations, decompositions, and the deep structural insights that eigenvalues and rank provide.

[[the-organizational-lens|the organizational lens]] applied here asks: "what are the dimensions? what are the transformations? what stays stable? what's the effective dimensionality?" these questions apply to far more than matrices.