The Math That Powers AI: The Dot Product Explained

The Dot Product

In previous blog posts, we've seen how to add vectors and how to scale them. But what happens when we multiply one vector by another? The most important way to do this is called the Dot Product, and understanding it is the key to unlocking how things like neural networks actually work.

Watch the video for a quick visual walkthrough, then read on for the details.

The Intuition: How Aligned Are Two Vectors?

So, what is the dot product, really? It just tells us how much two vectors are pointing in the same direction.

  • If two vectors point in a similar direction, their dot product will be a large positive number. This means, they're "working together."
  • If they point in opposite directions, their dot product will be a large negative number. Meaning, they're "working against each other."
  • And if they're at a 90-degree angle (orthogonal), their dot product is zero. They have no effect on each other.
The dot product

The dot product changes based on the angle between the vectors.

The Math

The math to calculate this is super simple. You just multiply the corresponding elements of the two vectors and then add up the results.

a = [a1, a2]
b = [b1, b2]
a · b = (a1 * b1) + (a2 * b2)

Let's prove the concept with our examples. Let's take vector a = [2, 3]:

  • If b = [3, 2] (similar direction), the dot product is (2*3) + (3*2) = 12.
  • If b = [-3, -2] (opposite direction), the dot product is (2*-3) + (3*-2) = -12.
  • If b = [-3, 2] (orthogonal), the dot product is (2*-3) + (3*2) = 0.

The result is just a single number (a scalar), not another vector.

The Code: Using NumPy

In NumPy, this is even easier. We can define two NumPy arrays and use the .dot() method to calculate the result. There are three different ways we can do that:

  • By calling the .dot() method from vector 'a' and passing 'b' as an argument.
  • By calling the .dot() method from vector 'b' and passing 'a' as an argument.
  • By calling the np.dot() method and passing both vectors 'a' and 'b' as arguments.
import numpy as np

a = np.array([2, 3])
b = np.array([3, 2])

# Use the .dot() method
dot_product_1 = a.dot(b)
print(dot_product_1)
# Output: 12

dot_product_2 = b.dot(a)
print(dot_product_2)
# Output: 12

dot_product_3 = np.dot(a, b)
print(dot_product_3)
# Output: 12

This simple operation is used everywhere in machine learning, from calculating the similarity between two items in a recommendation system to being the fundamental calculation in every single neuron of a deep learning network.

Conclusion

The dot product is a simple but incredibly powerful tool. It's a quick way to see how two vectors relate to each other, and it's one of the most common operations we'll encounter as we go deeper into machine learning.

Your Turn...

What's the dot product of a vector with itself? What does that number represent? (Hint: think about our video on magnitude). Share your thoughts in the comments!

This post is part of the "Decoding the Math of ML" series. For the previous part, check out: Scalar Multiplication (Scaling Vectors Explained).

Comments

Post a Comment

Popular posts from this blog

Clear Data Queue from SQL - IBM i

Image
Clear Data Queue: Data Queues (DTAQ) has been very useful for communication between different Jobs on IBM i.  There are multiple APIs to work with Data Queues. In the last post we have seen How to Retrieve data from Data Queue using SQL .  In this post, we will see how to clear data from Data Queue in SQL using ' CLEAR_DATA_QUEUE ' Procedure in QSYS2 with some examples. This would work similar to API ' QCLRDTAQ '.  Let's look at an example of clearing all the entries from a Standard Data Queue.  E.g.:  In the above statement, We are passing two parameters. 1st Parameter: Data Queue - Name of the Data Queue that needs to be cleared.  2nd Parameter: Data Queue Library - Name of the Library Data Queue is present in. This parameter accepts special variables like ' *LIBL ' or ' *CURLIB '.  Let's look at another example of Keyed Data Queue and clearing only set of records based on the Key.  E.g.:  Before we actually clear the Data Queue, Below ar...
Read more

How to Scale a Vector (Scalar Multiplication Explained)

Image
Scalar Multiplication (Scaling Vectors Explained) We know how to add vectors together. But what happens if we just multiply a vector by a single number? This is called scalar multiplication , and a " scalar " is just the formal name for a regular number like 2, -1, or 0.5. It’s one of the most fundamental operations in linear algebra. Watch the video for a quick visual walkthrough, then read on for the details. The Intuition: Scaling an Arrow Scalar multiplication is all about scaling or resizing a vector. Let's take our vector v = [2, 1] . Multiplying it by a scalar changes its magnitude, and sometimes its direction. Multiplying by 2 makes the arrow twice as long, but keeps the same direction. Multiplying by 0.5 makes it half as long. Multiplying by -1 keeps the length but completely flips its direction by 180 degrees. Visualizing how different scalars affect the vector [2, 1]. ...
Read more

Different Ways of Sorting Data in a List - Python

Image
Sorting Data in a List List is a collection of data (of different data types), much like an array. Like any data structure or data set, data in the list doesn't necessarily be in any specific order.  At times, depending on the use case we get to order the data in a specific sequence (Ascending, Descending or Reverse). There are couple of methods and functions for sorting the data in a list, which can be used depending on the use case.  sort() method  sorted() function reverse() method reversed() function  Let's have a look at each of these in detail.  sort() method Sort() method by default sorts the data in an ascending order. One important thing to remember about sort() method is that, it sorts the list in-place, meaning the list on which method is called would be sorted and the method would return None (no sorted list would be returned).  Syntax           list.sort(<key=None>, <reverse=False>) The two parameters ...
Read more