【概率论基础进阶】大数定律和中心极限定理

文章目录

    • 切比雪夫不等式
    • 大数定律
    • 中心极限定理

切比雪夫不等式

切比雪夫不等式:设随机变量 X X X的数学期望 E ( X ) E(X) E(X)和方差 D ( X ) D(X) D(X)存在,则对任意的 ϵ > 0 \epsilon >0 ϵ>0,总有
P { ∣ X − E ( X ) ∣ ≥ ϵ } ≤ D ( X ) ϵ 2 P \left\{|X-E(X)|\geq \epsilon \right\}\leq \frac{D(X)}{\epsilon ^{2}} P{XE(X)ϵ}ϵ2D(X)
这个不等式称为切比雪夫不等式

例1:设随机变量 X X X的概率密度 f ( x ) = { 2 e − 2 x x > 0 0 x ≤ 0 f(x)=\left\{\begin{aligned}&2e^{-2x}&x>0\\&0&x \leq 0\end{aligned}\right. f(x)={2e2x0x>0x0

  • 根据切比雪夫不等式估计 P { X ≥ 3 2 } ≤ A , A = ( ) \begin{aligned} P \left\{X \geq \frac{3}{2}\right\}\leq A,A=()\end{aligned} P{X23}A,A=()

X ∼ E ( 2 ) X \sim E(2) XE(2),因此
P { X ≥ 3 2 } = P { X − 1 2 ≥ 1 } = P { X − 1 2 ≥ 1 } + P { X − 1 2 ≤ − 1 } ⏟ 0 = P { ∣ X − 1 2 ∣ ≤ 1 } = P { ∣ X − E X ∣ ≥ 1 } ≤ D X 1 2 = 1 4 \begin{aligned} P \left\{X \geq \frac{3}{2}\right\}&=P \left\{X - \frac{1}{2}\geq 1\right\}\\ &=P \left\{X- \frac{1}{2}\geq 1\right\}+\underbrace{P \left\{X- \frac{1}{2}\leq -1\right\}}_{0}\\ &=P \left\{\left|X- \frac{1}{2}\right|\leq 1\right\}\\ &=P \left\{|X-EX| \geq 1\right\}\leq \frac{DX}{1^{2}}=\frac{1}{4} \end{aligned} P{X23}=P{X211}=P{X211}+0 P{X211}=P{ X21 1}=P{XEX1}12DX=41

  • 直接计算 P { X ≥ 3 2 } = B , B = ( ) \begin{aligned} P \left\{X \geq \frac{3}{2}\right\}=B,B=()\end{aligned} P{X23}=B,B=()

根据指数分布
P { X > t } = e λ t , t > 0 P \left\{X>t\right\}=e^{\lambda t},t>0 P{X>t}=eλt,t>0
因此
P { X ≥ 3 2 } = e − 2 ⋅ 3 2 = e − 3 P \left\{X \geq \frac{3}{2}\right\}=e^{-2 \cdot \frac{3}{2}}=e^{-3} P{X23}=e223=e3

e − 3 ≈ 0.05 \begin{aligned} e^{-3}\approx 0.05\end{aligned} e30.05,而切比雪夫不等式估计的是 0.25 \begin{aligned} 0.25\end{aligned} 0.25,显然是由一定差距的

例2:设 X X X的密度为 f ( x ) , D X = 1 f(x),DX=1 f(x),DX=1,而 Y Y Y的密度为 f ( − y ) f(-y) f(y),且 X X X Y Y Y的相关系数为 − 1 4 \begin{aligned} - \frac{1}{4}\end{aligned} 41,用切比雪夫不等式估计 P { ∣ X + Y ∣ ≥ 2 } ≤ ( ) P \left\{|X+Y| \geq 2\right\}\leq () P{X+Y2}()

Z = X + Y Z=X+Y Z=X+Y,由切比雪夫不等式
P { ∣ Z − E Z ∣ ≤ ϵ } ≤ D X ϵ 2 P { ∣ ( X + Y ) − E ( X + Y ) ∣ ≤ ϵ } ≤ D ( X + Y ) ϵ 2 \begin{aligned} P \left\{|Z-EZ|\leq \epsilon \right\}&\leq \frac{DX}{\epsilon ^{2}}\\ P \left\{|(X+Y)-E(X+Y)|\leq \epsilon \right\}&\leq \frac{D(X+Y)}{\epsilon ^{2}} \end{aligned} P{ZEZϵ}P{(X+Y)E(X+Y)ϵ}ϵ2DXϵ2D(X+Y)
E Y = ∫ − ∞ + ∞ y f ( − y ) d y = ∫ − ∞ + ∞ ( − y ) f ( − y ) d ( − y ) = − y = t − ∫ − ∞ + ∞ t f ( t ) d t = − E X \begin{aligned} EY&=\int_{-\infty}^{+\infty}yf(-y)dy\\ &=\int_{-\infty}^{+\infty}(-y)f(-y)d(-y)\\ &\overset{-y=t}{=}- \int_{-\infty}^{+\infty}tf(t)dt\\ &=-EX \end{aligned} EY=+yf(y)dy=+(y)f(y)d(y)=y=t+tf(t)dt=EX
因此 E Z = E ( X + Y ) = E X + E Y = 0 EZ=E(X+Y)=EX+EY=0 EZ=E(X+Y)=EX+EY=0
D Y = E ( Y 2 ) − ( E Y ) 2 = ∫ − ∞ + ∞ y 2 f ( − y ) d y − ( − E X ) 2 = − y = t ∫ + ∞ − ∞ ( − t ) 2 f ( t ) d ( − t ) − ( E X ) 2 = E ( X 2 ) − ( E X ) 2 = D X D ( X + Y ) = D X + D Y + 2 cov ( X , Y ) = D X + D Y + 2 D X D Y ⋅ ρ X Y = 1 + 1 − 1 2 = 3 2 \begin{aligned} DY&=E(Y^{2})-(EY)^{2}\\ &=\int_{-\infty}^{+\infty}y^{2}f(-y)dy-(-EX)^{2}\\ &\overset{-y=t}{=}\int_{+\infty}^{-\infty}(-t)^{2}f(t)d(-t)-(EX)^{2}\\ &=E(X^{2})-(EX)^{2}=DX\\ D(X+Y)&=DX+DY+2\text{cov}(X,Y)\\ &=DX+DY+2\sqrt{DX}\sqrt{DY}\cdot \rho_{XY}\\ &=1+1- \frac{1}{2}=\frac{3}{2} \end{aligned} DYD(X+Y)=E(Y2)(EY)2=+y2f(y)dy(EX)2=y=t+(t)2f(t)d(t)(EX)2=E(X2)(EX)2=DX=DX+DY+2cov(X,Y)=DX+DY+2DX DY ρXY=1+121=23
因此
P { ∣ X + Y ∣ ≤ 2 } = P { ∣ ( X + Y ) − E ( X + Y ) ∣ ≤ 2 } ≤ D ( X + Y ) 2 2 = D ( X + Y ) 4 = 3 8 \begin{aligned} P \left\{|X+Y|\leq 2\right\}&=P \left\{|(X+Y)-E(X+Y)|\leq 2\right\}\\ &\leq \frac{D(X+Y)}{2^{2}}=\frac{D(X+Y)}{4}=\frac{3}{8}\end{aligned} P{X+Y2}=P{(X+Y)E(X+Y)2}22D(X+Y)=4D(X+Y)=83

大数定律

定义:设 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,是一个随机变量序列, A A A是一个常数,如果对任意 ϵ > 0 \epsilon >0 ϵ>0,有
lim ⁡ n → ∞ P { ∣ X n − A ∣ < ϵ } = 1 \lim\limits_{n\to\infty}P \left\{|X_{n}-A|<\epsilon \right\}=1 nlimP{XnA<ϵ}=1
则称随机变量序列 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,依概率收敛域常数 A A A,记作 X n → P A X_{n}\overset{P}{\rightarrow }A XnPA

随机变量加减一个常数还是随机变量

切比雪夫大数定律:设 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,两两不相关的随机变量序列,存在常数 C C C使 D ( X 1 ) ≤ C ( i = 1 , 2 , ⋯   ) D(X_{1})\leq C(i=1,2,\cdots ) D(X1)C(i=1,2,),则对任意 ϵ > 0 \epsilon >0 ϵ>0,有
lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i − 1 n ∑ i = 1 n E ( X i ) ∣ < ϵ } = 1 \lim\limits_{n\to\infty}P\left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}- \frac{1}{n}\sum\limits_{i=1}^{n}E(X_{i})\right|< \epsilon \right\}=1 nlimP{ n1i=1nXin1i=1nE(Xi) <ϵ}=1

不太严谨的做题时可以写成 1 n ∑ i = 1 n x i → P 1 n ∑ i = 1 n E ( X i ) \begin{aligned} \frac{1}{n}\sum\limits_{i=1}^{n}x_{i}\overset{P}{\rightarrow } \frac{1}{n}\sum\limits_{i=1}^{n}E(X_{i})\end{aligned} n1i=1nxiPn1i=1nE(Xi)

伯努利大数定理:设随机变量 X n ∼ B ( n , p ) , n = 1 , 2 , ⋯ X_{n}\sim B(n,p),n=1,2,\cdots XnB(n,p),n=1,2,,则对于任意 ϵ > 0 \epsilon >0 ϵ>0,有
lim ⁡ n → ∞ P { ∣ X n n − p ∣ < ϵ } = 1 \lim\limits_{n\to\infty}P \left\{\left| \frac{X_{n}}{n}-p\right|<\epsilon \right\}=1 nlimP{ nXnp <ϵ}=1

X n = Y 1 + Y 2 + ⋯ + Y n , Y i ∼ B ( 1 , p ) X_{n}=Y_{1}+Y_{2}+\cdots +Y_{n},Y_{i}\sim B(1,p) Xn=Y1+Y2++Yn,YiB(1,p),则有 1 n ∑ i = 1 n Y i = X n n → P p \begin{aligned} \frac{1}{n}\sum\limits_{i=1}^{n}Y_{i}=\frac{X_{n}}{n}\overset{P}{\rightarrow }p\end{aligned} n1i=1nYi=nXnPp

辛钦大数定律:设随机变量 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,独立同分布,具有数学期望 E ( X i ) = μ , i = 1 , 2 , ⋯ E(X_{i})=\mu,i=1,2,\cdots E(Xi)=μ,i=1,2,,则对于任意 ϵ > 0 \epsilon >0 ϵ>0
lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i − μ ∣ < ϵ } = 1 \lim\limits_{n\to\infty}P \left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}-\mu\right|<\epsilon \right\}=1 nlimP{ n1i=1nXiμ <ϵ}=1

1 n ∑ i = 1 n x i → P μ \begin{aligned} \frac{1}{n}\sum\limits_{i=1}^{n}x_{i}\overset{P}{\rightarrow } \mu\end{aligned} n1i=1nxiPμ

切比雪夫大数定律要求随机变量两两不相关,且方差存在
辛钦大数定律要求随机变量独立同分布,且期望存在
做题时看题目给出条件符合哪个,用哪个

例3:设随机变量 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,相互独立,均服从分布函数
F ( x , θ ) = { 1 − e − x 2 θ x ≥ 0 0 x < 0 , θ > 0 F(x,\theta )=\left\{\begin{aligned}&1-e^{- \frac{x^{2}}{\theta }}&x \geq 0\\&0&x<0\end{aligned}\right.,\theta >0 F(x,θ)= 1eθx20x0x<0,θ>0
是否存在实数 a a a,使得对任何 ϵ > 0 \epsilon >0 ϵ>0都有 lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0 \begin{aligned} \lim\limits_{n\to\infty}P \left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{2}-a\right|\geq \epsilon \right\}=0\end{aligned} nlimP{ n1i=1nXi2a ϵ}=0

由题目可知
lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0 ⇒ lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ < ϵ } = 1 \begin{aligned} \lim\limits_{n\to\infty}P \left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{2}-a\right|\geq \epsilon \right\}=0\Rightarrow \lim\limits_{n\to\infty}P \left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{2}-a\right|< \epsilon \right\}=1\end{aligned} nlimP{ n1i=1nXi2a ϵ}=0nlimP{ n1i=1nXi2a <ϵ}=1

f ( x ; θ ) = F ′ ( x ; θ ) = { 2 x θ e − x 2 θ x ≥ 0 0 x < 0 E X i 2 = ∫ − ∞ + ∞ x 2 f ( x ; θ ) d x = ∫ 0 + ∞ x 2 ⋅ 2 x θ e − x 2 θ = x 2 θ = t θ ∫ 0 + ∞ t e − t d t = θ \begin{aligned} f(x;\theta )&=F'(x;\theta )=\left\{\begin{aligned}& \frac{2x}{\theta }e^{- \frac{x^{2}}{\theta }}&x \geq 0\\&0&x<0\end{aligned}\right.\\ EX_{i}^{2}&=\int_{-\infty}^{+\infty}x^{2}f(x;\theta )dx\\ &=\int_{0}^{+\infty}x^{2}\cdot \frac{2x}{\theta }e^{- \frac{x^{2}}{\theta }}\\ &\overset{ \frac{x^{2}}{\theta }=t}{=}\theta \int_{0}^{+\infty}t e^{-t}dt=\theta \end{aligned} f(x;θ)EXi2=F(x;θ)= θ2xeθx20x0x<0=+x2f(x;θ)dx=0+x2θ2xeθx2=θx2=tθ0+tetdt=θ
因此期望 E X 1 2 = θ EX_{1}^{2}=\theta EX12=θ存在,又因为 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,相互独立,均服从同一分布函数,因此 X 1 2 , X 2 2 , ⋯   , X n 2 , ⋯ X_{1}^{2},X_{2}^{2},\cdots ,X_{n}^{2},\cdots X12,X22,,Xn2,独立同分布。根据辛钦大数定律,当 n → ∞ n \to \infty n时,
1 n ∑ i = 1 n X i 2 → P E X i 2 = θ \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{2}\overset{P}{\rightarrow }EX^{2}_{i}=\theta n1i=1nXi2PEXi2=θ
即对任何 ϵ > 0 \epsilon >0 ϵ>0,都有 lim ⁡ n → ∞ P { ∣ 1 n ∑ i = 1 n X i 2 − a ∣ ≥ ϵ } = 0 \begin{aligned} \lim\limits_{n\to\infty}P \left\{\left| \frac{1}{n}\sum\limits_{i=1}^{n}X_{i}^{2}-a\right|\geq \epsilon \right\}=0\end{aligned} nlimP{ n1i=1nXi2a ϵ}=0

中心极限定理

列维-林德伯格中心极限定理:设随机变量 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,独立同分布,具有数学期望与方差, E ( X n ) = μ , D ( X n ) = σ 2 , n = 12 , ⋯ E(X_{n})=\mu,D(X_{n})=\sigma^{2},n=12,\cdots E(Xn)=μ,D(Xn)=σ2,n=12,,则对于任意实数 x x x,有
lim ⁡ n → ∞ P { ∑ i = 1 n X i − n μ n σ ≤ x } = Φ ( x ) \lim\limits_{n\to\infty}P \left\{ \frac{\sum\limits_{i=1}^{n}X_{i}-n \mu}{\sqrt{n}\sigma}\leq x\right\}=\Phi(x) nlimP n σi=1nXinμx =Φ(x)

定理表明当 n n n充分大时 ∑ i = 1 n X i \sum\limits_{i=1}^{n}X_{i} i=1nXi的标准化 ∑ i = 1 n X i − n μ n σ \begin{aligned} \frac{\sum\limits_{i=1}^{n}X_{i}-n \mu}{\sqrt{n}\sigma}\end{aligned} n σi=1nXinμ近似服从标准正态分布 N ( 0 , 1 ) N(0,1) N(0,1),或者说 ∑ i = 1 n X i \sum\limits_{i=1}^{n}X_{i} i=1nXi近似的服从 N ( n μ , n σ 2 ) N(n \mu,n \sigma^{2}) N(nμ,nσ2)

棣莫弗-拉普拉斯中心极限定理:设随机变量 X n ∼ B ( n , p ) ( n = 1 , 2 , ⋯   ) X_{n}\sim B(n,p)(n=1,2,\cdots ) XnB(n,p)(n=1,2,),则对于任意实数 x x x,有
lim ⁡ n → ∞ P { X n − n p n p ( 1 − p ) ≤ x } = Φ ( x ) \lim\limits_{n\to\infty}P \left\{\frac{X_{n}-np}{\sqrt{np(1-p)}}\leq x\right\}=\Phi (x) nlimP{np(1p) Xnnpx}=Φ(x)
其中 Φ ( x ) \Phi (x) Φ(x)是标准正态的分布函数

定理表明当 n n n充分大时,服从 B ( n , p ) B(n,p) B(n,p)的随机变量 X n X_{n} Xn经标准化后得 X n − n p n p ( 1 − p ) \begin{aligned} \frac{X_{n}-np}{\sqrt{np(1-p)}}\end{aligned} np(1p) Xnnp近似服从标准正态分布 N ( 0 , 1 ) N(0,1) N(0,1),或者说 X n X_{n} Xn近似的服从 N ( n p , n p ( 1 − p ) ) N(np,np(1-p)) N(np,np(1p))

例4:设 X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,为独立同分布的随机变量列,且均服从参数为 1 1 1的指数分布,则 lim ⁡ n → ∞ P { ∑ i = 1 n X i ≤ n } = ( ) \lim\limits_{n\to\infty}P \left\{\sum\limits_{i=1}^{n}X_{i}\leq n\right\}=() nlimP{i=1nXin}=()

X 1 , X 2 , ⋯   , X n , ⋯ X_{1},X_{2},\cdots ,X_{n},\cdots X1,X2,,Xn,独立同分布,且 E ( X i ) = 1 , D ( X i ) = 1 E(X_{i})=1,D(X_{i})=1 E(Xi)=1,D(Xi)=1,根据列维-林德伯格中心极限定理, ∑ i = 1 n X i \sum\limits_{i=1}^{n}X_{i} i=1nXi近似的服从 N ( n ⋅ 1 λ , n ⋅ 1 λ 2 ) \begin{aligned} N(n \cdot \frac{1}{\lambda},n \cdot \frac{1}{\lambda^{2}})\end{aligned} N(nλ1,nλ21) N ( n , n ) N(n,n) N(n,n),所以 lim ⁡ n → ∞ P { X n − n p n p ( 1 − p ) ≤ x } = Φ ( x ) \begin{aligned} \lim\limits_{n\to\infty}P \left\{\frac{X_{n}-np}{\sqrt{np(1-p)}}\leq x\right\}=\Phi (x)\end{aligned} nlimP{np(1p) Xnnpx}=Φ(x),因此
lim ⁡ n → ∞ P { ∑ i = 1 n X i ≤ n } = lim ⁡ n → ∞ P { ∑ i = 1 n X i − n n ≤ 0 } = Φ ( 0 ) = 1 2 \lim\limits_{n\to\infty}P \left\{\sum\limits_{i=1}^{n}X_{i}\leq n\right\}=\lim\limits_{n\to\infty}P \left\{\frac{\sum\limits_{i=1}^{n}X_{i}-n}{\sqrt{n}}\leq 0\right\}=\Phi (0)=\frac{1}{2} nlimP{i=1nXin}=nlimP n i=1nXin0 =Φ(0)=21

CSDN话题挑战赛第2期
参赛话题:学习笔记

你可能感兴趣的