Transformations of Random Variables

Change of Variables

In one dimension

Let $X$ be a continuous r.v. with PDF $f_X$, and let $Y=g(X)$, where $g$ is differentiable and strictly increasing (or strictly decreasing). Then the PDF of $Y$ is given by

$$f_Y(y)=f_X(x)\left|\frac{\mathrm{d}x}{\mathrm{d}y}\right|$$

where $x=g^{-1}(y)$. The support of $Y$ is all $g(x)$ with $x$ in the support of $X$.

Tip

Do not forget specifying the support of $Y$

General case

Let $\text{X}=(X_1,\dots ,X_n)$ be a continuous random vector with joint PDF $f_{\text{X}}$. Let $g:A_0\to B_0$ be an invertible function, where $A_0$ and $B_0$ are open subsets of ${\mathbb{R}}^n$, $A_0$ contains the support of $\text{X}$, and $B_0$ is the range of $g$.

Let $\text{Y}=g(\text{X})$. and mirror this by letting $\mathbf{y}=g(\mathbf{x})$. Since $g$ is invertible, we also have $\text{X}=g^{-1}(\text{Y})$ and $\mathbf{x}=g^{-1}(\mathbf{y})$.

Suppose that all the partial derivatives $\frac{\partial x_i}{\partial y_i}$ exist and are continuous, so we can form the Jacobian matrix

$$\frac{\partial \mathbf{x}}{\partial \mathbf{y}}=\left(\begin{array}{cccc}\frac{\partial x_1}{\partial y_1}& \frac{\partial x_1}{\partial y_2}& \cdots & \frac{\partial x_1}{\partial y_n}\\ ⋮& & & ⋮\\ \frac{\partial x_n}{\partial y_1}& \frac{\partial x_n}{\partial y_2}& \cdots & \frac{\partial x_n}{\partial y_n}\end{array}\right)$$

Also assume that the determinant of this Jacobian matrix is never $0$. Then the joint PDF of $\text{Y}$ is

$$f_{\text{Y}}(\mathbf{y})=f_{\text{X}}\left({\mathbf{g}}^{-1}(\mathbf{y})\right)\cdot \left|\left|\frac{\partial \mathbf{x}}{\partial \mathbf{y}}\right|\right|\text{for}\mathbf{y}\in B_0$$

This relationship means that

$${\int }_A{f}_{\text{X}}(\mathbf{x})\mathrm{d}\mathbf{x}={\int }_B{f}_{\text{Y}}(\mathbf{y})\mathrm{d}\mathbf{y}$$

Convolution

Let $X\sim f_X(x)$ and $Y\sim f_Y(y)$ are independent, then the PDF of $Z=X+Y$ is the convolution of $f_X,f_Y$:

$$g(z)={\int }_{-\infty }^{\infty }{f}_X(x)f_Y(z-x)\mathrm{d}x={\int }_{-\infty }^{\infty }{f}_X(z-x)f_Y(x)\mathrm{d}x$$

Regenerative Property of Binomial Distribution

For Binomial Distribution, let $X\sim \mathrm{Bin}(n,p)$, $Y\sim \mathrm{Bin}(m,p)$, we compute the PDF of $Z=X+Y$

$$\begin{array}{rl}P(Z=z)& =\sum _{x=-\infty }^{\infty }P(X=x)P(Y=z-x)\\ & =\sum _{x=0}^z\left(\genfrac{}{}{0ex}{}{n}{x}\right)p^x(1-p{)}^{n-x}(\genfrac{}{}{0ex}{}{m}{z-x})p^{z-x}(1-p{)}^{m-z+x}\\ & =p^z(1-p{)}^{m+n-z}\sum _{x=0}^z(\genfrac{}{}{0ex}{}{n}{x}\left)\right(\genfrac{}{}{0ex}{}{m}{z-x})\\ & =p^z(1-p{)}^{m+n-z}(\genfrac{}{}{0ex}{}{m+n}{z})\end{array}$$

Therefore, $Z\sim \mathrm{Bin}(m+n,p)$. This is a discrete convolution

Regenerative Property of Poisson Distribution

For Poisson Distribution, let $X\sim \mathrm{Pois}({\lambda }_1),Y\sim \mathrm{Posi}({\lambda }_2)$, then if $X,Y$ are independent, we have $X+Y=\mathrm{Pois}({\lambda }_1+{\lambda }_2)$

Tip

By replacing $Y$ by $-Y$, the convolution formula can also be used for $Z=X-Y$

Product

Let the PDF of $(X,Y{)}^T$ be $f(x,y)$, then the PDF of $Z=XY$ is

$$g(z)={\int }_{-\infty }^{\infty }\frac{1}{\left|x\right|}f\left(x,\frac{z}{x}\right)\mathrm{d}x$$

This is by

$$\begin{array}{rl}P(Z\le z)& =P\left(XY\le z\right)\\ & =P\left(Y\le \frac{z}{X},X\ge 0\right)+P\left(Y\ge \frac{z}{X},X\le 0\right)\\ & ={\int }_0^{\infty }{\int }_{-\infty }^{z/x}f(x,y)\mathrm{d}y\mathrm{d}x+{\int }_{-\infty }^0{\int }_{z/x}^{\infty }f(x,y)\mathrm{d}y\mathrm{d}x\end{array}$$

Therefore

$$\begin{array}{rl}g(z)=\frac{\partial }{\partial z}P(Z\le z)& ={\int }_0^{\infty }\frac{1}{x}f\left(x,\frac{z}{x}\right)\mathrm{d}x-{\int }_{-\infty }^0\frac{1}{x}f\left(x,\frac{z}{x}\right)\mathrm{d}x\\ & ={\int }_{-\infty }^{\infty }\frac{1}{\left|x\right|}f\left(x,\frac{z}{x}\right)\mathrm{d}x\end{array}$$

Division

Let the PDF of $(X,Y{)}^T$ be $f(x,y)$, then the PDF of $Z=\frac{X}{Y}$ is

$$g(z)={\int }_{-\infty }^{\infty }\left|u\right|f(zu,u)\mathrm{d}u$$

This is by

$$\begin{array}{rl}P(Z\le z)& =P\left(\frac{X}{Y}\le z\right)\\ & =P(X\le Yz,Y>0)+P(X>Yz,Y<0)\\ & ={\int }_0^{\infty }{\int }_{-\infty }^{yz}f(x,y)\mathrm{d}x\mathrm{d}y+{\int }_{-\infty }^0{\int }_{yz}^{\infty }f(x,y)\mathrm{d}x\mathrm{d}y\end{array}$$

Therefore

$$\begin{array}{rl}g(z)=\frac{\partial }{\partial z}P(Z\le z)& ={\int }_0^{\infty }yf(yz,y)\mathrm{d}y+{\int }_{-\infty }^0(-y)f(yz,y)\mathrm{d}y\\ & ={\int }_{-\infty }^{\infty }\left|y\right|f(yz,y)\mathrm{d}y\end{array}$$