Orthogonal Projections Explained: The Intuitive Guide for Developers

How to Deconstruct Any Vector (A Developer's Guide to Orthogonal Projections)

Finding the coordinates of a vector seems simple. In a standard Cartesian grid, the vector `v = [3, 2]` clearly has coordinates (3, 2). But what happens when our world is skewed? What if our basis vectors aren't at a perfect 90-degree angle? Suddenly, a simple question becomes a much harder puzzle.

This puzzle—finding the components of a vector in an arbitrary coordinate system—is at the heart of many applications in engineering and data science. The tool we use to solve it is one of the most powerful and intuitive concepts in linear algebra: the Orthogonal Projection.

In this post, we'll decode the math of projections not by memorizing a formula, but by deriving it from a single, powerful geometric clue.

Watch the video for the full visual story, then scroll down for the detailed derivation and code.

The Mystery: Finding the "Shadow"

Let's simplify our problem. We have a vector v, and we want to find out "how much" of it points in an arbitrary direction, represented by a vector b.

Geometrically, we are looking for the point on the line defined by b that is closest to the tip of v. This "closest point" defines a vector that looks like the "shadow" of v cast onto b. We call this shadow the projection vector, which we'll label p.

The key insight is that because p lies on the line of b, it must be a scaled version of b. Our entire mission is to find that scaling factor, `c`.

p = c * b

The Clue That Unlocks Everything

The fact that the projection is the "closest" point gives us the crucial clue we need. The line connecting the shadow p back to the original vector v must be perfectly perpendicular (orthogonal) to the direction b.

Let's define this connecting vector as the "error" vector, e = v - p. The geometric definition of an orthogonal projection tells us that:

b ⋅ e = 0

This is it. This is the single piece of information we need to solve the entire puzzle. By substituting our definitions for e and p, we can now deduce the formula for the unknown scalar `c`.

1. b ⋅ (v - p) = 0      (Substitute e = v - p)
2. b ⋅ (v - cb) = 0    (Substitute p = cb)
3. (b⋅v) - c*(b⋅b) = 0   (Using the distributive property of the dot product)
4. c = (b ⋅ v) / (b ⋅ b)       (Solve for c)

And there we have it. The unknown coordinate `c` is simply the dot product of the original vector and the axis, divided by the dot product of the axis with itself. Now we can write the full formula for the projection vector p:

p = ( (v ⋅ b) / (b ⋅ b) ) * b

The Payoff: The Orthonormal Shortcut

This formula is our universal "information extractor." But it gets even better. What happens if our direction vector b is a unit vector from an orthonormal basis?

In that case, the length (or norm) of b is 1. The dot product of a vector with itself, b ⋅ b, is equal to the square of its norm. So, for a unit vector, b ⋅ b = 1² = 1.

The entire denominator in our formula disappears!

p = (v ⋅ b) * b      // When ||b|| = 1

This is a massive engineering win. It's why we care so much about orthonormal bases: they simplify the math of projections, leading to huge gains in speed and numerical stability in algorithms that perform billions of these calculations.

In Code: The NumPy Way

The formula we derived translates directly into clean, simple Python code using NumPy.

import numpy as np

v = np.array([2, 3])
b = np.array([4, 0]) # The direction vector we are projecting onto

# Calculate the scaling factor 'c' using our derived formula
c = np.dot(v, b) / np.dot(b, b)

# Calculate the projection vector 'p'
p = c * b

print(f"The original vector v is: {v}")
print(f"The projection of v onto b is: {p}")
# Output: The projection of v onto b is: [2. 0.]
    

Conclusion

By starting with a simple but frustrating problem, we've uncovered the logic behind one of linear algebra's most fundamental tools. The Orthogonal Projection is our method for deconstructing any vector into its components along any set of axes.

Now that we have this powerful tool, it opens up a new question: How can we use it, over and over, to take a messy, skewed basis and systematically "clean it up" into a perfect orthonormal one? That's the topic for our next deep dive.

Your Turn...

Can you think of another real-world scenario, besides a GPS, where deconstructing a vector into its components would be useful? Share your ideas in the comments!

This post is part of the "Linear Algebra: The Core of Machine Learning" series. For the previous part, check out: Upgrade Your Basis: The Power of Orthogonality.