Probability and Statistic-From Multiplication to Factorial
I believe every subject in the world boils down to a few core principles. A full grasp of them will navigate you to any derivatives. Previously, I claimed that statistics is the core of Machine Learning. And then I found my self stuck unless I explain some core concepts for statistics. So naturally, a wise idea is to start another series on Statistic and Probability. Let’s start with Factorial.
You are probably somewhat familiar with Factorial. But how is it a core concept in statistics? Here is a short answer: Factorial is a function that returns all possible events without replacement (the last word is important). This is 1 way to find our sample space, which is 50% of the equation to find the probability of anything. Combining that with the other 50% of the equation, number of desire events, we have everything that’s needed to compute the probability as illustrated below (i.e. the probability of rolling a 2 or 3 on a die is 2/6).
All factorial is doing is counting. And all counting in statistics is done with multiplication. Wait, why? To answer this question, we need to dig a little further down to how we count, and follow by a tree branch analogy to imagine Factorial.
Counting
If I ask you to count the eggs in egg carton, what would you do? The most natural way is to add them 1 by 1. Can we do better? Yes, we can rearrange the eggs inside an egg carton into a perfect rectangle and multiply the rows with the columns of that rectangle and add the result with the number of eggs that does not fit the in the rectangle (If you ever wonder the order of operation, this is why).
So instead of adding 1 by 1, we are adding 3 groups of 4s or vice versa. You can think of adding 1 by 1 as a vector and multiplication as matrix (I didn’t put the cool matrix photo just to catch your attention).
Ok, it’s easy to imagine eggs, but why is it that our brain turns into scramble eggs (pun intended) when we are trying to imagine number of possible poker hands? There are 3 reasons.
Psychology
All possible events look different, and our brain naturally rejects counting things that are different. For example, it’s a lot easier for our brain to hear “how many bananas” than “how many things are in your bag”, which doesn’t even sound right. There is something odd about the latter, but we can nevertheless still do it. And yes, when we call them things, we naturally already categorized them under 1 umbrella, as “things”. And that can be true to everything. However, the point I am making here is that counting things that look different is unintuitive, even though we can categorize them all into 1 category: all possible events.
Relation between rows and columns
When we are counting eggs, there is no relationship between rows and column, but there is for possible poker hands. What do I mean by relationship? You cannot derive the egg carton columns from rows, or vice versa, with 100% certainty. But when counting possible 2 hand cards, the column must be 1 less than the row. This is because when we are expanding out all possible hands with each of the 52 cards as the first card, we cannot use that card that we have already place in one of the scenario or rows again since each card only exists once.
Formally it is known as none replaceable, and Factorial is perfect for counting things of that nature, but this example just fall short of being a factorial. It’s actually a permutation that I will cover in the next blog. For now we kept the illustration to 2 cards hand because it’s easier to imagine. But what happen when we do have more cards?
5th dimension?
Counting all possible hands can be exactly like counting eggs if we limit the dimension to 2, shown in the above illustration. But our analogy quickly breaks down when we increase that dimension to more than 3 (In case you wonder what a 3 dimensional egg carton look like, I have a photo below.), which makes it incredibly difficult to imagine.
Since spatial analogy breaks down at this stage, we have to make a new analogy: tree branches.
Tree Branches
Unlike spatial analogy, tree branch analogy produces all possible outcomes in a linear fashion indefinitely. 2 things to note in the below illustration.
One
I know that the tree branch diagram expands in 2 dimensions, but we are measuring the possible outcomes, which is the lowest bottom nodes. Every other nodes above that are incomplete hypothetical directions that eventually leads to all the possible hands. So as you can see, even as this tree branch analogy expands in 2 dimension, length and width, we really only care about the width, specifically the most bottom nodes. This is what I mean when I say tree branch analogy produces possible outcomes in a linear fashion that expands indefinitely.
Two
Even though there are more bottom nodes than top overall, notice each nodes’ possible outcomes decreases. The first layer consist of 5 nodes (blue/purple circle), and each of those 5 nodes now only split into 4 nodes. This is a decrease pattern, hence Factorial becomes very useful to count things of that nature. But you have to remember that unless it goes all the way down to 1, it is not factorial (n * n-1 * n-2…1) it’s premutation.
Conclusion
Probability and Statistics is a tough subject because it confront our understandings of the most basic mathematical principles right from the start. That does not mean just knowing how to do text book problems robotically, but to understand why. Today it confront us on our understands of multiplication, tomorrow it will test our intuition in division. Thank you for reading, stay tuned.
