Inverse Transform Method

proof of inverse transform theorem

This time I will explain a beautiful theorem in probability theory is used in several application from making a library that generate pseudo random number with specifict distribution to advanced deep learning concept. Also in this article I will write a code for sampling from random particular distribution using inverse transform theorem

Theorem 1. Suppose that we have a uniformlly distributed random variable over zero to one $U \sim \mathcal{U}(0,1)$ and another random variable $Y = CDF^{-1}_{X}{(U)}$ then $CDF_X$ is CDF for $Y$

Proof :

Since $U \sim \mathcal{U}(0,1)$ then

$$\begin{equation}CDF_{U}(u)=\mathbb{P}(U\leq u)=u\end{equation}$$

Then by equation(1) we get $$\begin{aligned}CDF_{Y}(y)&=\mathbb{P}(Y \leq y)\\&=\mathbb{P}(CDF^{-1}_{X}(U)\leq y)\\&=\mathbb{P}(U \leq CDF_X(y))\\ &=CDF_X(y)\end{aligned}$$

Q.E.D

Theorem 1 tell us that we can sample any random variable by sampling random uniform first then then feed the value to its inverse of target CDF we want to sample, later I will write a code to sample from Gaussian distribution by using this theorem