Lesson 54 – The Bernoulli gene – dataanalysisclassroom

D: That thought you have going must be bothering you. It gave you a long face.

J: Hello. Yes, it is bugging me. We are racing ahead with the lessons, rolling on one distribution after another. I follow each lesson and can understand the distributions as they are. But there are many, and I am guessing these are just the major ones we are learning. How do I make sense of them? Is there any way to summarize the distributions we learned so far?

D: Hmm. If I tell you that there is a Bernoulli connection to the major distributions we learned, can you trace it back?

J: I am not following.

D: Okay, let’s try this. I will show you a graphic that connects the distributions we learned thus far. Can you work with me to trace the links? That will be an efficient way, both to summarize and remember the distributions.

J: I like the thought and the exercise. Let’s do it. Show me the graphic.

D: Here. I put the distributions in one place and connected the dots. It’s a high-resolution image. Click on it to open a full page graphic. I want you to walk me through it.

J: Cool. The Bernoulli trial (0 1) is the origin. Lesson 31. There are two possibilities; event occurred – success or event did not occur – failure. There is a sequence (Bernoulli gene code) of 0’s and 1’s (010110010000), n Bernoulli trials. There is a probability of occurrence of p that is constant over all the trials, and the trials are assumed to be independent.

D: Good. Now, what is the first distribution we learned?

J: The binomial distribution. Lesson 32. The number of successes in a Bernoulli sequence of n trials is a binomial distribution.

D: Correct. Can you write down the probability distribution function for the Binomial distribution?

J: Sure. $P(X = x) = \frac{n!}{(n-x)!x!}p^{x}(1-p)^{n-x}$

D: Now, refer to the recent lesson we did on the normal distribution and the central limit theorem. It is lesson 48.

J: Ah, I see how you connected the Binomial to the Normal distribution in the graphic. The Binomial distribution can be estimated very accurately using the normal density function. I went through the looong derivation you had.

$P(X = x) = \frac{n!}{(n-x)!x!}p^{x}(1-p)^{n-x} = f(x) = \frac{1}{\sqrt{2 \pi \sigma^{2}}} e^{\frac{-1}{2}(\frac{x-\mu}{\sigma})^{2}}$

Then, from the recent lessons, I know that the standard normal, the log-normal and the Chi-square distributions are transformations of the normal distribution.

The standard normal distribution is derived by subtracting from the distribution X, the mean ( $\mu$ ) and dividing by the standard deviation ( $\sigma$ ). Lesson 50.

$Z = \frac{X - \mu}{\sigma}$

$Y = e^{Z}$ is a log-normal distribution because the log of Y is normal, standard normal. Lesson 52.

Last week was Chi-square distribution. Lesson 53. If Z follows a normal distribution, then, $\chi = Z^{2}$ is a Chi-square distribution with one degree of freedom. If there are n standard normal random variables, $Z_{1}, Z_{2}, ..., Z_{n}$ , their sum of squares is a Chi-square distribution with n degrees of freedom.

$\chi = Z_{1}^{2}+Z_{2}^{2}+ ... + Z_{n}^{2}$

I now see the flow from Bernoulli to Binomial to Normal to Standard Normal to Log-normal and Chi-square. Amazing how deep the Bernoulli gene code is.

D: Indeed. Now go through the rest of the graphic. You will see the connections of the other distributions.

J: Yes. I am looking at the Poisson distribution now. If we are counting the number of 1’s of the Bernoulli sequence in an interval, then these counts follow a Poisson distribution. Lessons 36 and Lesson 37.

$P(X = x) = \frac{e^{-\lambda t}(\lambda t)^{x}}{x!}$

$\lambda$ is the rate of occurrence; the average number of 1’s per interval. This function is derived as an approximate function from the Binomial distribution with a large number of trials in an interval. Lesson 37.

D: That is correct. Now, do you remember how the Poisson distribution is connected to the Exponential distribution?

J: Yes. The Poisson distribution represents the number of events in an interval of time, and the exponential distribution represents the time between these events.

The time to arrival exceeds some value t, only if $N = 0$ within t, i.e., if there are no events in an interval [0, t].

$P(T > t) = P(N = 0) = \frac{e^{-\lambda t}(\lambda t)^{0}}{0!} = e^{-\lambda t}$

The probability density function f(t) is

$f(t) = \frac{d}{dt}F(t) = \frac{d}{dt}(1-e^{-\lambda t}) = \lambda e^{-\lambda t}$

Let me keep going.

On the discrete sense, if you measure the number of trials it takes to see the next 1 in the Bernoulli sequence, then it is a Geometric distribution. Lesson 33.

$P(1^{st} \hspace{5}1\hspace{5}on\hspace{5} k^{th}\hspace{5}trial) = (1-p)^{k-1}p$

The exponential distribution is the continuous analog of the discrete Geometric distribution.

Sticking to that thought, if we measure the trials or time to $r^{th}$ 1 or event, we are looking at the Negative Binomial distribution (Lesson 35) for discrete and Gamma distribution (Lesson 45) for the continuous case.

$P(r^{th} \hspace{5}1\hspace{5}on\hspace{5} k^{th}\hspace{5}trial) = \binom{k-1}{r-1}(1-p)^{k-r}p^{r}$ for the Negative Binomial, and

$f(t) = \frac{\lambda e^{-\lambda t}(\lambda t)^{r-1}}{(r-1)!}$ for the Gamma distribution.

The time to ‘r’th arrival $T_{r} = t_{1} + t_{2} + t_{3} + ... + t_{r}$ , (sum of exponentials) and we derived the probability density function (Gamma) of $T_{r}$ using the convolution of the individual exponential random variables $t_{1}, t_{2}, ... t_{r}$ .

Like the Geometric and Exponential, the Negative Binomial and Gamma are discrete and continuous analogs.

D: And, what did we learn last week about Chi-square?

J: That its probability density function is a Gamma density function with $\lambda=1/2$ and $r=n/2$ .

$f(\chi)=\frac{\frac{1}{2}*(\frac{1}{2} \chi)^{\frac{n}{2}-1}*e^{-\frac{1}{2}*\chi}}{(\frac{n}{2}-1)!}$ for $\chi > 0$ and 0 otherwise.

Wow.

D: See how it all comes together. Bernoulli gene is running through the distribution line to connect them.

J: Yes. I can see that very clearly, now that I walked through each one in this order. Thank you for creating this fantastic graphic. It helps me in remembering the distributions.

D: You are welcome. Anything to make data analysis easy and fun.

J: What is up next? I am Filled with new energy.

Dust off your computer. We will learn some tricks in R for the continuous distributions. Time to crank some code.

If you find this useful, please like, share and subscribe.
You can also follow me on Twitter @realDevineni for updates on new lessons.

Enjoy this blog? Please spread the word :)