# 随机变量
定义 ( Ω , F , P ) (\Omega, \mathcal F, P) ( Ω , F , P ) X : Ω → R X:\Omega \to \mathbb R X : Ω → R ( Ω , F , P ) (\Omega, \mathcal F, P) ( Ω , F , P ) 随机变量 (Random Variable)「確率変数」 ，当且仅当对于任意实数 x x x
{ ω ∈ Ω ∣ X ( ω ) ≤ x } ∈ F \{\omega \in \Omega \mid X(\omega) \leq x\} \in \mathcal F
{ ω ∈ Ω ∣ X ( ω ) ≤ x } ∈ F
如果记概率
p j = P ( X = x j ) p_j = P(X = x_j)
p j = P ( X = x j )
那么组
( x 1 , p 1 ) , ( x 2 , p 2 ) , … (x_1, p_1), (x_2, p_2), \ldots
( x 1 , p 1 ) , ( x 2 , p 2 ) , …
称为随机变量 X X X 概率分布 (Probability Distribution)「確率分布」
若 X X X X X X 离散型随机变量 (Discrete Random Variable)「離散型確率変数」
若存在 f : R → [ 0 , ∞ ) f:\mathbb R \to [0,\infty) f : R → [ 0 , ∞ )
P ( a ≤ X ≤ b ) = ∫ a b f ( x ) d x , ( a ≤ b ) P(a \leq X \leq b) = \int_a^b f(x)dx,\ (a \leq b)
P ( a ≤ X ≤ b ) = ∫ a b f ( x ) d x , ( a ≤ b )
则称 X X X 连续型随机变量 (Continuous Random Variable)「連続型確率変数」 f f f X X X 概率密度函数 (Probability Density Function)「確率密度関数」
在随机变量的基础上，可以定义出表示概率分布情况的函数
定义 ( Ω , F , P ) (\Omega, \mathcal F, P) ( Ω , F , P ) X X X
F X ( x ) = P ( X ≤ x ) F_X(x) = P(X \leq x)
F X ( x ) = P ( X ≤ x )
为 X X X 累积分布函数 (Cumulative Distribution Function)
研究 PDF（概率密度函数）和 CDF（累积分布函数）是研究概率分布情况的核心
# 期望值与方差期望值和方差是描述概率分布情况的重要参数。
期望值指示了如字面意思的，” 我们可以如何期待结果 “。例如用期望值来预测彩票收入X X X 期望值 (Expected Value)「期待値」
E [ X ] = { ∑ x i ∈ X ( Ω ) x i P ( X = x i ) , X is Discrete ∫ − ∞ ∞ x f ( x ) d x , X is Continuous E[X] = \begin{cases}
\displaystyle\sum_{x_i \in X(\Omega)} x_i P(X = x_i), & \text{X is Discrete} \\[16pt]
\displaystyle\int_{-\infty}^{\infty} x f(x) dx, & \text{X is Continuous}
\end{cases} E [ X ] = ⎩ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎧ x i ∈ X ( Ω ) ∑ x i P ( X = x i ) , ∫ − ∞ ∞ x f ( x ) d x , X is Discrete X is Continuous
方差描述数据的离散程度，方差越大意味着数据越分散。例如一个班的学习成绩如果方差越小，说明大家的差距越小X X X 方差 (Variance)「分散」
V [ X ] = E [ ( X − E [ X ] ) 2 ] = { ∑ x i ∈ X ( Ω ) ( x i − E [ X ] ) 2 P ( X = x i ) , X is Discrete ∫ − ∞ ∞ ( x − E [ X ] ) 2 f ( x ) d x , X is Continuous V[X] = E[(X - E[X])^2] = \begin{cases}
\displaystyle\sum_{x_i \in X(\Omega)} (x_i - E[X])^2 P(X = x_i), & \text{X is Discrete} \\[16pt]
\displaystyle\int_{-\infty}^{\infty} (x - E[X])^2 f(x) dx, & \text{X is Continuous}
\end{cases} V [ X ] = E [ ( X − E [ X ] ) 2 ] = ⎩ ⎪ ⎪ ⎪ ⎪ ⎨ ⎪ ⎪ ⎪ ⎪ ⎧ x i ∈ X ( Ω ) ∑ ( x i − E [ X ] ) 2 P ( X = x i ) , ∫ − ∞ ∞ ( x − E [ X ] ) 2 f ( x ) d x , X is Discrete X is Continuous
在消除正负号所带来的影响后，也可以通过根号来还原平方带来的量纲变化X X X 标准差 (Standard Deviation)「標準偏差」
σ = V [ X ] \sigma = \sqrt{V[X]}
σ = V [ X ]
通常，习惯上记
期望值为 E [ X ] = μ E[X] = \mu E [ X ] = μ
方差为 V [ X ] = σ 2 V[X] = \sigma^2 V [ X ] = σ 2
标准差为 σ \sigma σ
两种算子具有以下基本计算性质
命题
E [ a X + b ] = a E [ X ] + b E[aX + b] = aE[X] + b E [ a X + b ] = a E [ X ] + b V [ a X + b ] = a 2 V [ X ] V[aX + b] = a^2 V[X] V [ a X + b ] = a 2 V [ X ]
证明
由定义，对于离散型随机变量
E [ a X + b ] = ∑ x i ∈ X ( Ω ) ( a x i + b ) P ( X = x i ) = a ∑ x i ∈ X ( Ω ) x i P ( X = x i ) + b ∑ x i ∈ X ( Ω ) P ( X = x i ) = a E [ X ] + b V [ a X + b ] = E [ ( a X + b − E [ a X + b ] ) 2 ] = E [ ( a X + b − a E [ X ] − b ) 2 ] = E [ a 2 ( X − E [ X ] ) 2 ] = a 2 E [ ( X − E [ X ] ) 2 ] = a 2 V [ X ] \begin{aligned}
E[aX + b] &= \sum_{x_i \in X(\Omega)} (a x_i + b) P(X = x_i) \\
&= a \sum_{x_i \in X(\Omega)} x_i P(X = x_i) + b \sum_{x_i \in X(\Omega)} P(X = x_i) \\
&= a E[X] + b \\
V[aX + b] &= E[(aX + b - E[aX + b])^2] \\
&= E[(aX + b - aE[X] - b)^2] \\
&= E[a^2 (X - E[X])^2] \\
&= a^2 E[(X - E[X])^2] \\
&= a^2 V[X]
\end{aligned} E [ a X + b ] V [ a X + b ] = x i ∈ X ( Ω ) ∑ ( a x i + b ) P ( X = x i ) = a x i ∈ X ( Ω ) ∑ x i P ( X = x i ) + b x i ∈ X ( Ω ) ∑ P ( X = x i ) = a E [ X ] + b = E [ ( a X + b − E [ a X + b ] ) 2 ] = E [ ( a X + b − a E [ X ] − b ) 2 ] = E [ a 2 ( X − E [ X ] ) 2 ] = a 2 E [ ( X − E [ X ] ) 2 ] = a 2 V [ X ]
同样地，对于连续型随机变量
E [ a X + b ] = ∫ − ∞ ∞ ( a x + b ) f ( x ) d x = a ∫ − ∞ ∞ x f ( x ) d x + b ∫ − ∞ ∞ f ( x ) d x = a E [ X ] + b V [ a X + b ] = E [ ( a X + b − E [ a X + b ] ) 2 ] = ∫ − ∞ ∞ ( a x + b − a E [ X ] − b ) 2 f ( x ) d x = ∫ − ∞ ∞ a 2 ( x − E [ X ] ) 2 f ( x ) d x = a 2 E [ ( X − E [ X ] ) 2 ] = a 2 V [ X ] \begin{aligned}
E[aX + b] &= \int_{-\infty}^{\infty} (a x + b) f(x) dx \\
&= a \int_{-\infty}^{\infty} x f(x) dx + b \int_{-\infty}^{\infty} f(x) dx \\
&= a E[X] + b \\
V[aX + b] &= E[(aX + b - E[aX + b])^2] \\
&= \int_{-\infty}^{\infty} (a x + b - a E[X] - b)^2 f(x) dx \\
&= \int_{-\infty}^{\infty} a^2 (x - E[X])^2 f(x) dx \\
&= a^2 E[(X - E[X])^2] \\
&= a^2 V[X]
\end{aligned} E [ a X + b ] V [ a X + b ] = ∫ − ∞ ∞ ( a x + b ) f ( x ) d x = a ∫ − ∞ ∞ x f ( x ) d x + b ∫ − ∞ ∞ f ( x ) d x = a E [ X ] + b = E [ ( a X + b − E [ a X + b ] ) 2 ] = ∫ − ∞ ∞ ( a x + b − a E [ X ] − b ) 2 f ( x ) d x = ∫ − ∞ ∞ a 2 ( x − E [ X ] ) 2 f ( x ) d x = a 2 E [ ( X − E [ X ] ) 2 ] = a 2 V [ X ]
□ \square □
实际计算中，方差往往用如下等价计算式
命题
V [ X ] = E [ X 2 ] − ( E [ X ] ) 2 V[X] = E[X^2] - (E[X])^2 V [ X ] = E [ X 2 ] − ( E [ X ] ) 2
证明
由定义
V [ X ] = E [ ( X − E [ X ] ) 2 ] = E [ X 2 − 2 X E [ X ] + ( E [ X ] ) 2 ] = E [ X 2 ] − 2 E [ X ] E [ X ] + ( E [ X ] ) 2 = E [ X 2 ] − ( E [ X ] ) 2 \begin{aligned}
V[X] &= E[(X - E[X])^2] \\
&= E[X^2 - 2X E[X] + (E[X])^2] \\
&= E[X^2] - 2E[X]E[X] + (E[X])^2 \\
&= E[X^2] - (E[X])^2
\end{aligned} V [ X ] = E [ ( X − E [ X ] ) 2 ] = E [ X 2 − 2 X E [ X ] + ( E [ X ] ) 2 ] = E [ X 2 ] − 2 E [ X ] E [ X ] + ( E [ X ] ) 2 = E [ X 2 ] − ( E [ X ] ) 2
□ \square □
# 矩母函数矩母函数像是概率分布的一个身份证X X X
M X ( t ) = E [ e t X ] M_X(t) = E[e^{tX}]
M X ( t ) = E [ e t X ]
为 X X X 矩母函数 (Moment Generating Function)「積率母関数」 M X ( t ) M_X(t) M X ( t )
命题 X X X M X ( t ) M_X(t) M X ( t )
E [ X n ] = M X ( n ) ( 0 ) E[X^n] = M_X^{(n)}(0)
E [ X n ] = M X ( n ) ( 0 )
由此
E [ X ] = M X ′ ( 0 ) , V [ X ] = M X ′ ′ ( 0 ) − ( M X ′ ( 0 ) ) 2 E[X] = M_X'(0),\quad V[X] = M_X''(0) - (M_X'(0))^2
E [ X ] = M X ′ ( 0 ) , V [ X ] = M X ′ ′ ( 0 ) − ( M X ′ ( 0 ) ) 2
证明
期望值算子的线性性质可知
d n d t n M X ( t ) = d n d t n E [ e t X ] = E [ d n d t n e t X ] = E [ X n e t X ] \frac{d^n}{dt^n} M_X(t) = \frac{d^n}{dt^n} E[e^{tX}] = E\left[\frac{d^n}{dt^n} e^{tX}\right] = E[X^n e^{tX}]
d t n d n M X ( t ) = d t n d n E [ e t X ] = E [ d t n d n e t X ] = E [ X n e t X ]
代入 t = 0 t=0 t = 0
E [ X n ] = M X ( n ) ( 0 ) E[X^n] = M_X^{(n)}(0)
E [ X n ] = M X ( n ) ( 0 )
□ \square □
# 概率不等式第一个不等式揭示了期望值是如何控制随机变量的分布的
定理 Markov 不等式 X X X ε > 0 \varepsilon \gt 0 ε > 0
P ( ∣ X ∣ ≥ ε ) ≤ E [ ∣ X ∣ ] ε P(|X| \geq \varepsilon) \leq \frac{E[|X|]}{\varepsilon}
P ( ∣ X ∣ ≥ ε ) ≤ ε E [ ∣ X ∣ ]
证明
令 f f f X X X 离散型
E [ ∣ X ∣ ] = ∑ x i ∈ X ( Ω ) ∣ x i ∣ f ( x i ) ≥ ∑ ∣ x i ∣ ≥ ε ∣ x i ∣ f ( x i ) ≥ ∑ ∣ x i ∣ ≥ ε ε f ( x i ) = ε P ( ∣ X ∣ ≥ ε ) \begin{aligned}
E[|X|] &= \sum_{x_i \in X(\Omega)} |x_i| f(x_i) \\
&\geq \sum_{|x_i| \geq \varepsilon} |x_i| f(x_i) \\
&\geq \sum_{|x_i| \geq \varepsilon} \varepsilon f(x_i) \\
&= \varepsilon P(|X| \geq \varepsilon)
\end{aligned} E [ ∣ X ∣ ] = x i ∈ X ( Ω ) ∑ ∣ x i ∣ f ( x i ) ≥ ∣ x i ∣ ≥ ε ∑ ∣ x i ∣ f ( x i ) ≥ ∣ x i ∣ ≥ ε ∑ ε f ( x i ) = ε P ( ∣ X ∣ ≥ ε )
连续型
E [ ∣ X ∣ ] = ∫ − ∞ ∞ ∣ x ∣ f ( x ) d x ≥ ∫ ∣ x ∣ ≥ ε ∣ x ∣ f ( x ) d x ≥ ∫ ∣ x ∣ ≥ ε ε f ( x ) d x = ε P ( ∣ X ∣ ≥ ε ) \begin{aligned}
E[|X|] &= \int_{-\infty}^{\infty} |x| f(x) dx \\
&\geq \int_{|x| \geq \varepsilon} |x| f(x) dx \\
&\geq \int_{|x| \geq \varepsilon} \varepsilon f(x) dx \\
&= \varepsilon P(|X| \geq \varepsilon)
\end{aligned} E [ ∣ X ∣ ] = ∫ − ∞ ∞ ∣ x ∣ f ( x ) d x ≥ ∫ ∣ x ∣ ≥ ε ∣ x ∣ f ( x ) d x ≥ ∫ ∣ x ∣ ≥ ε ε f ( x ) d x = ε P ( ∣ X ∣ ≥ ε )
由此得
P ( ∣ X ∣ ≥ ε ) ≤ E [ ∣ X ∣ ] ε P(|X| \geq \varepsilon) \leq \frac{E[|X|]}{\varepsilon}
P ( ∣ X ∣ ≥ ε ) ≤ ε E [ ∣ X ∣ ]
□ \square □
方差表示的是数据的分散程度，所以方差比较小的概率分布中，数据会集中在期望值附近
定理 Chebyshev 不等式 X X X E [ X ] = μ E[X] = \mu E [ X ] = μ V [ X ] = σ 2 V[X] = \sigma^2 V [ X ] = σ 2 ε > 0 \varepsilon \gt 0 ε > 0
P ( ∣ X − μ ∣ ≥ ε ) ≤ σ 2 ε 2 P(|X - \mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2}
P ( ∣ X − μ ∣ ≥ ε ) ≤ ε 2 σ 2
证明
令 f f f X X X 离散型
σ 2 = E [ ( X − μ ) 2 ] = ∑ x i ∈ X ( Ω ) ( x i − μ ) 2 f ( x i ) ≥ ∑ ∣ x i − μ ∣ ≥ ε ( x i − μ ) 2 f ( x i ) ≥ ∑ ∣ x i − μ ∣ ≥ ε ε 2 f ( x i ) = ε 2 P ( ∣ X − μ ∣ ≥ ε ) \begin{aligned}
\sigma^2 &= E[(X - \mu)^2] = \sum_{x_i \in X(\Omega)} (x_i - \mu)^2 f(x_i) \\
&\geq \sum_{|x_i - \mu| \geq \varepsilon} (x_i - \mu)^2 f(x_i) \\
&\geq \sum_{|x_i - \mu| \geq \varepsilon} \varepsilon^2 f(x_i) \\
&= \varepsilon^2 P(|X - \mu| \geq \varepsilon)
\end{aligned} σ 2 = E [ ( X − μ ) 2 ] = x i ∈ X ( Ω ) ∑ ( x i − μ ) 2 f ( x i ) ≥ ∣ x i − μ ∣ ≥ ε ∑ ( x i − μ ) 2 f ( x i ) ≥ ∣ x i − μ ∣ ≥ ε ∑ ε 2 f ( x i ) = ε 2 P ( ∣ X − μ ∣ ≥ ε )
连续型
σ 2 = E [ ( X − μ ) 2 ] = ∫ − ∞ ∞ ( x − μ ) 2 f ( x ) d x ≥ ∫ ∣ x − μ ∣ ≥ ε ( x − μ ) 2 f ( x ) d x ≥ ∫ ∣ x − μ ∣ ≥ ε ε 2 f ( x ) d x = ε 2 P ( ∣ X − μ ∣ ≥ ε ) \begin{aligned}
\sigma^2 &= E[(X - \mu)^2] = \int_{-\infty}^{\infty} (x - \mu)^2 f(x) dx \\
&\geq \int_{|x - \mu| \geq \varepsilon} (x - \mu)^2 f(x) dx \\
&\geq \int_{|x - \mu| \geq \varepsilon} \varepsilon^2 f(x) dx \\
&= \varepsilon^2 P(|X - \mu| \geq \varepsilon)
\end{aligned} σ 2 = E [ ( X − μ ) 2 ] = ∫ − ∞ ∞ ( x − μ ) 2 f ( x ) d x ≥ ∫ ∣ x − μ ∣ ≥ ε ( x − μ ) 2 f ( x ) d x ≥ ∫ ∣ x − μ ∣ ≥ ε ε 2 f ( x ) d x = ε 2 P ( ∣ X − μ ∣ ≥ ε )
由此得
P ( ∣ X − μ ∣ ≥ ε ) ≤ σ 2 ε 2 P(|X - \mu| \geq \varepsilon) \leq \frac{\sigma^2}{\varepsilon^2}
P ( ∣ X − μ ∣ ≥ ε ) ≤ ε 2 σ 2
□ \square □
这意味着，均值的 ε \varepsilon ε σ 2 ε 2 \dfrac{\sigma^2}{\varepsilon^2} ε 2 σ 2
定理 Jensen 不等式 X X X φ : R → R \varphi:\mathbb R \to \mathbb R φ : R → R E [ ∣ X ∣ ] , E [ ∣ φ ( X ) ∣ ] < ∞ E[|X|], E[|\varphi(X)|] \lt \infty E [ ∣ X ∣ ] , E [ ∣ φ ( X ) ∣ ] < ∞
φ ( E [ X ] ) ≤ E [ φ ( X ) ] \varphi(E[X]) \leq E[\varphi(X)]
φ ( E [ X ] ) ≤ E [ φ ( X ) ]
证明（暂时省略）
