Decoding Complexities

The Silent Bug That Makes Your Model Wrong, Not Broken Two customers. One is 25, the other is 30 — five years apart. Another pair: 25 and 70 — forty-five years apart. Ask a K-Nearest Neighbors model which pair is more similar, using age and income, and it tells you the 45-year age gap pair is the closer match. No error. No crash. Just a wrong answer, delivered with total confidence. Why It Happens Income ranges from roughly zero to a hundred thousand. Age ranges from zero to about a hundred. When a model measures "distance" between two people, it just does math on the raw numbers — and a $1,000 income difference completely swamps a 45-year age difference, because the numbers are so much bigger to begin with. The model isn't wrong about the arithmetic. It's just never been told these two numbers live on completely different scales. The Fix, in Real Numbers Scale those same two customers pro...

Your model works perfectly in testing. You deploy it. It crashes on the first real prediction. ValueError: The feature names should match those that were passed during fit. Feature names seen at fit time, yet now missing: - Delhi If you've seen an error like this — a feature that's missing, or a mismatch in feature names — it almost always traces back to one decision: how a categorical column got converted into numbers. What Actually Happened Somewhere in the pipeline, a column like city — Bangalore, Mumbai, Delhi — got converted into numbers. That conversion is called one-hot encoding . Instead of one column holding text labels, you get one new column per category , each holding a 1 or a 0. Bangalore becomes city_Bangalore = 1 , everything else 0. Mumbai becomes city_Mumbai = 1 , everything else 0. Why Not Just Use 1, 2, 3? Models only understand numbers. They can't do math on the word "Bangalore....

Here's a question. Given a matrix, how do you know upfront whether it can be inverted at all — before you go through the trouble of actually calculating the inverse? The answer is a single number called the determinant. If it's zero, the matrix has no inverse. If it's anything else, it does. That's the whole claim. Now let's take it apart and see why it's true. Calculating the Determinant of a 2×2 Matrix For a matrix A = [ a b ] [ c d ] the determinant is a times d, minus b times c . Let's prove it with real numbers. Take: A = [ 2 3 ] [ 1 4 ] The determinant is 2 × 4 − 3 × 1 = 8 − 3 = 5 . Since this isn't zero, A can be inverted. Now take: C = [ 2 4 ] [ 1 2 ] The determinant is 2 × 2 − 4 × 1 = 4 − 4 = 0 . Look closely — row two is exactly half of row one. The rows are linearly dependent. Whenever the determinant is zero, the rows or c...

We keep talking about Gradient Descent when it comes to training a machine learning model. We calculate the gradient, adjust weights to minimize the loss. That sounds simple enough. But what does it mean to train a large language model with billions of parameters? Gradient descent still works exactly as it should — you compute gradients with respect to every single parameter and store optimizer states for all of them. What's the problem then? It works perfectly, but requires a lot of compute power and tens of gigabytes just to hold the training state. It requires GPUs, possibly a connection to a data center. What if you are a student or a researcher with just a laptop? Do you give up on fine-tuning a model with billions of parameters? What if you could train a model where, instead of modifying billions of weights, you only tune a few parameters — and still get nearly the same result? The answer is Low Rank Approximation . And to understand it, ...

If you are standing on a mountain, you check the slope and start walking to reach the bottom of the mountain. How does the machine do it? If the input data contains just one feature, then the machine will adjust its weight to minimize the loss. It means it either has to move right or left depending upon the slope. What if there are more than one feature? ChatGPT has 175 billion parameters. Training it means finding the lowest point in a 175 billion dimensional space. That is the mathematical problem every AI trainer faces. And, there is a solution for this. The Single Parameter Case If we just have a single parameter, the problem gets easy — we calculate the slope and decide which way to move. The moment we have another parameter, just slope doesn't work anymore. Imagine standing on a mountain or somewhere on a bowl — you have infinite directions to move. And which direction to choose is the key. We can't c...

I was checking what tools were available to GitHub Copilot from the MCP server I had just built. It showed a list. Each entry had a tool name at the top — the actual function name exported as an MCP tool. Next to it, in slightly smaller text, a description. Wait,  I hadn't written any descriptions. Not in the MCP configuration, not in any separate documentation file. I had written nothing that I thought of as a description. Then I realized. That smaller text was my docstring. The comment I wrote at the top of the function — the one explaining what the procedure does, what arguments it takes, what it returns — was sitting right there, being shown to Copilot as the tool's documentation. The AI was reading my code comments. Not inferring from them. Using them directly, word for word, to understand what the tool does and how to use it. That was the moment I realized something had quietly shifted about what a code comment actually is. The Old ...