Decoding Complexities

Why Neural Networks use Cross-Entropy: The Math of "Surprise" Every Machine Learning tutorial tells you the same thing: If you're predicting a continuous number (like a house price), use Mean Squared Error (MSE) . But if you are classifying an image (like Cats vs. Dogs), you must use Cross-Entropy Loss . But why is that? What actually happens if you try to use Squared Error to classify a cat? Your Neural Network just gives up. It doesn't get trained properly. In this Applied Engineering Lab, we decode exactly why MSE fails for classification, how Information Theory solves it using the math of "Surprise," and how to fix a critical math bug that will crash your production servers. The Problem: Why MSE Fails for Classification Regression is about how far off you are. Squaring the error is great for punishing a $50,000 mistake on a house price prediction. But Classification is just Tru...

Building a Spam Detector from Scratch: The Naive Bayes Classifier Three billion emails are sent every hour. And nearly half of them are spam. "Congratulations! You won free money. Click here now." If we look at this email, we immediately realize it is spam. But how does the machine realize it? It's an engineering problem . An email has thousands of words, and each word changes the probability of it being spam or not spam. Checking how these words relate to each other is computationally impossible. We need to make a naive assumption to solve it. This assumption leads us to one of the most effective classifiers: The Naive Bayes Classifier . The "Naive" Assumption The Naive Bayes classifier assumes that all features are conditionally independent given the class . What does this mean in simple terms? It assumes that the presence of one word does not affect the probability of another wo...

Why MLE Fails: Decoding Bayesian Inference (Python Simulation) Traditional statistical analysis (like Maximum Likelihood Estimation) relies solely on data. It does not consider any prior knowledge we might have about the environment. This works perfectly when we have a massive dataset. But in reality, data is expensive. We often have very little data, yet we do have some prior knowledge about the problem we are trying to solve. To build robust models, it is best to utilize what we already know along with the data we observe. This is achieved using a concept called Bayesian Inference . Understanding Bayes Theorem Before we code it, we need to understand the mathematics. Bayes Theorem is a way of calculating the probability of an event based on prior knowledge of conditions related to that event. P(A|B) = [P(B|A) * P(A)] / P(B) Let's break down the terminology: P(A|B) (Posterior Probability): ...

Why We Use Mean Squared Error: Decoding Maximum Likelihood Estimation In our previous exploration of the Central Limit Theorem , we established that the error in Linear Regression follows a Normal Distribution. This is a huge win—it gives us the shape of the error. But there is no single distribution that fits every dataset. There are infinite Bell Curves—some wide, some narrow, some shifted to the left or right. This raises a critical question: How do we find the specific distribution parameters that best fit our data? In simple words, we need to find the parameters (like Mean or Slope) that minimize the error. We do this using a technique called Maximum Likelihood Estimation (MLE) . Likelihood vs. Probability: What's the Difference? In English, "Likelihood" and "Probability" are synonyms. In Mathematics, they are opposites. Probability is the art of estimating an event (or data) o...

Why Linear Regression Actually Works: Simulating the Central Limit Theorem In Linear Regression, we almost always use Mean Squared Error (MSE) as our loss function. When we apply Gradient Descent, we assume this function forms a nice, convex shape. We find the bottom of this bowl, minimize the error, and assume our parameters are optimal. But there is a hidden assumption here: We assume the noise (error) in our data follows a Normal Distribution. In the real world, data is messy. It rarely follows a perfect Bell Curve. It might be uniform, skewed, or exponential. This leads to a critical question: Why does minimizing squared errors work even when the original data is NOT normal? The answer lies in a statistical law so powerful it feels like a cheat code: The Central Limit Theorem (CLT) . What is the Central Limit Theorem? The Central Limit Theorem states that if you extract sufficiently large samples (ideally n ≥ 30) fr...

What are the Measures of Variability in Statistics? Data is everything in AI today. Knowing the middle value—like mean or median—is good, but we also need to know how much the data spreads. Let's take a look about different measures of variability: range, variance, standard deviation, and interquartile range. This is in continuation to understanding Central Tendency of data (If you have not already read that post, Have a quick look at “ What is Central Tendency? ”).  What is Variability? Variability shows how far data points are from each other or the center. For example, If we say a data set has a mean of 11.4 doesn’t tell us if numbers are close to it or all over. In AI, spread matters—big swings might mean bad data for a model. For example, In any application different job run times spread could show system issues. We measure variability with: Range Variance Standard Deviation Interquartile Range Let’s use...

What is Central Tendency? In the age of Artificial Intelligence, World revolves around data. Understanding the data is a key. When working with numerical data, at times we would want to represent whole data set with a single value (like average of the data) that we may call "Central Value" or "Middle Value". How do we measure Central Value or Central Tendency? There are different ways to measure the central value of the data provided. This could vary depending on the how we want to look at the data. Like, Do we need to calculate an average of the data? Do we need to split the data into two equal parts? Do we need to identify the value that occurred most of the time in data? Based on what we need to look at, Below are the different measures. Mean Median Mode Let's now look at what each measure is and how do we calculate mean with an example. Mean Mean ...