机器学习--高数概率论第一章

2023-03-19 12:20 作者:圣母和正负喜欢没办法 0人读过 | 我要投稿

一、微积分

1 夹逼定理

$x%5Cin%20U(x_%7B0%7D%20)%E6%97%B6%EF%BC%8Cg(x)%5Cleq%20f(x)%5Cleq%20h(x)%E6%88%90%E7%AB%8B%EF%BC%8C%E4%B8%94%5Clim_%7Bx%5Cto0%7D%20g(x)%3DA%2C%5Clim_%7Bx%5Cto0%7D%20h(x)%3DA%EF%BC%8C%E5%88%99%5Clim_%7Bx%5Cto0%7D%20f(x)%3DA$

$%5Csin%20x%20%3C%20x%20%3C%20%5Ctan%20x%20%20$ $x%20%5Cin%20U(0%2C%5Cvarepsilon%20)$

$%5Clim_%7Bx%5Cto0%7D%20%5Cfrac%7Bsinx%7D%7Bx%7D%20%3D%201$

2 导数

即是曲线的斜率，是曲线变化的快慢。

路程是原始函数

一阶导数------速度

二阶导数------加速度

$(a%5Ex)'%3Da%5Ex%5Cln%20a$

题目:

1 $f(x)%3Dx%5Ex$ 的最小值

两边同时取对数然后求导，

令t'=0带入计算求出t。

之后做最小优化时要用，基本也是求最小值

2 泰勒公式

$f(x)%3Df(x_%7B0%7D%20)%2Bf'(x_%7B0%7D%20)(x-x_%7B0%7D%20)%2B%5Cfrac%7Bf'(x_%7B0%7D%20)%7D%7B2!%7D(x-x_%7B0%7D%20)%5E2%2B...%2B%5Cfrac%7Bf'(x_%7B0%7D%20)%7D%7Bn!%7D(x-x_%7B0%7D)%5En%2BR_%7B0%7D(x)%20$

其实多项式求解要容易一些。

$%20e%5Ex%20%20%20%20%20$ $sinx%0A$ 都是一样计算，可以理解成无限逼近。

3 基尼指数

$f(x)%3D-lnx%E5%9C%A8x%3D1%E5%A4%84%E7%9A%84%E4%B8%80%E9%98%B6%E5%B1%95%E5%BC%80$

$H(x)%3D-%5Csum_%7Bk%3D1%7D%5EKp_%7Bk%7Dln%5Cln%20p_%7Bk%7D%3D%20%5Csum_%7Bk%3D1%7D%5EKp_%7Bk%7D(1-%20p_%7Bk%7D)$

4 方向导数

$z%3Df(x%2Cy)%E5%9C%A8%E7%82%B9P(x%2Cy)%E5%8F%AF%E5%BE%AE%EF%BC%8C%E5%88%99%E5%87%BD%E6%95%B0%E5%9C%A8%E8%AF%A5%E7%82%B9%E6%B2%BF%E4%BB%BB%E6%84%8F%E6%96%B9%E5%90%91%E5%AF%BC%E6%95%B0%E9%83%BD%E5%AD%98%E5%9C%A8%0A%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20l%7D%20%3D%20%20%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20x%7Dcos%5Cvarphi%20%2B%20%5Cfrac%7B%5Cvartheta%20f%7D%7B%5Cvartheta%20y%7Dsin%5Cvarphi%20$

$%5Cpsi%20$ 是x轴到L的转角

5 梯度

梯度的方向是函数在该方向变化最快的方向

即：解析式z=H(x,y)的山，（ $x_%7B0%7D%EF%BC%8Cy_%7B0%7D%20$ ）的梯度变化最快

梯度下降法：

考虑自己下山方向和梯度呈 $%5Ctheta%20$ 夹角，下降速度是多少？

6 凹函数(二阶导数大于0)

（碗状函数）

$f(%5Ctheta%20x%2B(1-%5Ctheta)y%20)%5Cleq%20%5Ctheta%20f(x)%2B(1-%5Ctheta)f(y)%20%20%E5%AF%B9%E4%BA%8E%5Cforall%20x%2Cy%5Cin%20dom%20f%20%2C%20%200%20%5Cleq%20%5Ctheta%5Cleq1$

也有： $f(%5Ctheta_%7B1%7D%20x_%7B1%7D%20%2B...%2B%5Ctheta_%7Bn%7D%20x_%7Bn%7D%20%20)%5Cleq%20%5Ctheta_%7B1%7Df(%5Ctheta_%7B1%7D)%2B...%2B%5Ctheta_%7Bn%7Df(%5Ctheta_%7Bn%7D)%2C%E5%85%B6%E4%B8%AD0%5Cleq%5Ctheta_%7Bi%7D%5Cleq1%2C%5Ctheta_%7B1%7D%2B...%2B%5Ctheta_%7Bn%7D%3D1$

有最小值，便于优化。

应用：最大熵模型---互相损失

$D(p%7C%7Cq)%3D%5Csum_%7Bx%7Dp(x)%5Clog%20%5Cfrac%7Bp(x)%7D%7Bq(x)%7D%3DE_%7Bp(x)%7D%5Clog%5Cfrac%7Bp(x)%7D%7Bq(x)%7D%20%0A$

证明D(p||q)?

$%E5%B0%B1%E6%B1%82-%5Clog%5Csum_%7Bx%7D(p(x)%5Cfrac%7Bq(x)%7D%7Bp(x)%7D%20)%3D-%5Clog%5Csum_%7Bx%7Dq(x)%3D0$

7概率论

事件和概率没有必然关系

概率为0不代表事件就不发生

1>累计分布：

$%5Cphi%20(x)%E5%8D%95%E5%A2%9E%EF%BC%8Cmin(%5Cphi(x))%3D0%2Cmax(%5Cphi(x))%3D1$

$%E5%80%BC%E5%9F%9F%5Cin%20%5B0%2C1%5D%E4%B8%8Ay%3Df(x)%E7%9C%8B%E6%88%90y%E7%9A%84%E4%BA%8B%E4%BB%B6%E7%B4%AF%E8%AE%A1%E6%A6%82%E7%8E%87$

$y%E5%A6%82%E6%9E%9C%E5%8F%AF%E5%AF%BC%EF%BC%8Cp%3Df'(x)%E7%9C%8B%E6%88%90%E5%85%B6%E6%A6%82%E7%8E%87%E5%AF%86%E5%BA%A6%E5%87%BD%E6%95%B0%E3%80%82$

2>古典概率

n个不同球放入N（N>n）个盒子，盒子不限，求事件A={每个盒子最多有1个球}

$P(A)%3D%5Cfrac%7BC_%7BN%7D%5En%20%7D%7BN%5En%20%7D%20%EF%BC%8C%E5%9F%BA%E6%9C%AC%E4%BA%8B%E4%BB%B6N%5En%E4%B8%AA%EF%BC%8C%E5%85%B1N(N-1)...(N-N%2B1)%E7%A7%8D$

3>生日悖论

套用上诉公式

会发现人数越多，概率越大

4>古典概率

$%E9%BA%BB%E5%B0%86136%E5%BC%A0%EF%BC%8C%E9%9A%8F%E6%9C%BA%E9%80%894%E5%BC%A0%EF%BC%8C%E5%8F%96%E6%B3%95C_%7B136%7D%5E%7B14%7D$

$%E5%BA%84%E5%AE%B614%E5%BC%A0%EF%BC%8C%E5%85%B6%E4%BB%9613%E5%BC%A0$

$p%3D%5Cfrac%7BC_%7B34%7D%5E%7B14%7D%204%5E%7B14%7D%20%7D%7BC_%7B136%7D%5E%7B14%7D%20%7D%20%3D%200.0879$

5>装箱问题

$12%E4%BB%B6%E6%AD%A3%E5%93%81%E3%80%813%E4%BB%B6%E6%AC%A1%E5%93%81%EF%BC%8C%E9%9A%8F%E6%9C%BA%E8%A3%85%E5%85%A53%E4%B8%AA%E7%AE%B1%E5%AD%90%E3%80%81%E6%AF%8F%E7%AE%B1%E8%A3%855%E4%BB%B6%EF%BC%8C%E6%AF%8F%E7%AE%B1%E6%81%B0%E8%A3%851%E4%BB%B6%E6%AC%A1%E5%93%81%E6%A6%82%E7%8E%87$

$%E5%85%B1%E6%9C%89%EF%BC%9A15!%2F(5!5!5!)%E8%A3%85%E6%B3%95$

$%E6%AC%A1%E5%93%81%E8%A3%85%E6%B3%95%3A%203%EF%BC%81$

$%E6%AD%A3%E5%93%81%E8%A3%85%E6%B3%95%EF%BC%9A12!%2F(4!4!4!)$

$P(A)%3D(3!*12!(4!4!4!))%2F(15!%2F(5!5!5!))%3D25%2F91$

6>和组合数关系

n个物品分成k组，每组物品个数n1,n2,n3,n4...nk,(n1+...+nk=n), $%E5%88%86%E7%BB%84%E6%96%B9%E6%B3%95%5Cfrac%7Bn!%7D%7Bn_%7B1%7D%20n_%7B2%7D%20...n_%7Bk%7D%20%7D%20$

优化：物品分组：第一组m个，第二组n-m个， $%5Cfrac%7Bn!%7D%7Bm!(n-m)!%7D%20%3D%20C_%7Bn%7D%5Em%20$

7>推荐系统

惊喜度、喜爱度

A和B两个商品和用户匹配度为0.8和0.2，系统将随机为A生成一个均匀分布在0-0.8之间。B在0-0.2之间，计算B最终分数大于A的概率。

$A%3DB%E7%9A%84%E7%9B%B4%E7%BA%BF%E4%B8%8A%E6%96%B9%E5%8C%BA%E5%9F%9F%EF%BC%8CB%3EA%E7%9A%84%E6%83%85%E5%86%B5$

$S_%7BA%7D%3D0.02%20%2CS_%7BB%7D%3D0.16%20$

$p%3D0.02%2F0.16%3D0.125$

8> 概率公式

1 条件概率

在B发生条件下A发生的概率

$P(A%7CB)%3D%5Cfrac%7BP(AB)%7D%7BP(B)%7D%20$

2 全概率公式

$P(A)%3D%5Csum_%7Bi%7DP(A%7CB_%7Bi%7D%20)P(B_%7Bi%7D)$

3 贝叶斯公式

$P(B_%7Bi%7D%7CA)%3D%20%5Cfrac%7BP(A%7CB_%7Bi%7D)P(B_%7Bi%7D)%7D%20%7B%5Csum_%7Bj%7DP(A%7CB_%7Bj%7D%20)P(B_%7Bj%7D)%7D$

用于反推

例程：

8支枪，5支校准，3支没校准，校准射中靶概率0.8，没校准的0.3，从8支任取一把射击中靶，这把是校准的概率。

典型反推用贝叶斯，已知结果求概率

$P(G%3D1)%3D%5Cfrac%7B5%7D%7B8%7D%20%20%2C%20P(G%3D0)%3D%5Cfrac%7B3%7D%7B8%7D%20$

$P(A%3D1%7CG%3D1)%3D0.8%20%2C%20P(A%3D0%7CG%3D1)%3D0.2%20$

$P(A%3D1%7CG%3D0)%3D0.3%20%2C%20P(A%3D0%7CG%3D0)%3D0.7%20$

$P(G%3D1%7CA%3D1)%3D%3F$

$P(G%3D1%7CA%3D1)%3D%5Cfrac%7BP(A%3D1%7CG%3D1)P(G%3D1)%7D%20%7B%5Csum_%7Bi%7DP(A%3D1%7CG%3Di%20)P(G%3Di)%7D%3D0.8163$

两大学派：

频率学派：假定参数是某个未知定值，求这些参数如何取值，能达到目标函数极大、极小取值

贝叶斯派：假定参数可变，服从某个分布，求这些分布下某个目标函数极大、极小

大数据：属于频率学派

9 常见分布

1 0-1分布

$E(X)%3D1*p%2B0*q%3Dp$

$D(X)%3DE(x%5E2%20)-%5BE(X)%5D%5E2%3D1*p%2B0*(1-p)-p%5E2$

2 二项分布（伯努利分布）

服从参数为n，概率为p的分布

比如抛硬币

$X%3D%5Csum_%7Bi%3D1%7D%5EnX_%7Bi%7D%20$

$E(X)%3D%5Csum_%7Bi%3D1%7D%5EnE(X_%7Bi%7D)%3Dnp$

$D(X)%3D%5Csum_%7Bi%3D1%7D%5EnD(X_%7Bi%7D)%3Dnp(1-p)$

分布律

$P(X%3Dk)%3DC_%7Bn%7D%5Ek%20p%5Ek(1-p)%5E%7Bn-k%7D$

$%E5%88%99E(X)%3D%5Csum_%7Bk%3D0%7D%5EnkP(X%3Dk)%3D%5Csum_%7Bk%3D0%7D%5EnkC_%7Bn%7D%5E%7Bk%7Dp%5E%7Bk%7D(1-p)%5E%7Bn-k%7D$

$%20%3D%5Csum_%7Bk%3D0%7D%5En%20%5Cfrac%7Bkn!%7D%7Bk!(n-k)!%7Dp%5Ek(1-p)%5E%7Bn-k%7D$

$%3D%5Csum_%7Bk%3D1%7D%5E%7Bn%7D%5Cfrac%7Bnp(n-1)!%7D%7B(k-1)!%5B(n-1)-(k-1)%5D!%7Dp%5E%7Bk-1%7D(1-p)%5E%7B(n-1)-(k-1)%7D%20$

$%3Dnp%5Bp%2B(1-p)%5D%5E%7Bn-1%7D%3Dnp$

3 泊松分布

$e%5Ex%3D1%2Bx%2B%5Cfrac%7Bx%5E2%7D%7B2!%7D%2B...%2B%5Cfrac%7Bx%5Ek%7D%7Bk!%7D%2BR_%7Bk%7D$

$1%3D1*e%5E%7B-x%7D%2Bx%2B%5Cfrac%7Bx%5E2%7D%7B2!%7De%5E%7B-x%7D%2B...%2B%5Cfrac%7Bx%5Ek%7D%7Bk!%7De%5E%7B-x%7D%2BR_%7Bk%7De%5E%7B-x%7D$

$%E5%90%83%E5%B1%8E%EF%BC%9A%E5%B0%86x%E7%9C%8B%E6%88%90%5Clambda%20$

$%5Cfrac%7Bx%5Ek%7D%7Bk!%7D*e%5E%7B-x%7D----%3E%20%5Cfrac%7B%5Clambda%5Ek%7D%7Bk!%7D*e%5E%7B-%5Clambda%20%7D$

$%E5%88%86%E5%B8%83%E5%BE%8B%EF%BC%9AP%7B(X%3Dk)%7D%3D%5Cfrac%7B%5Clambda%5E%7Bk%7D%7D%7Bk!%7De%5E%7B-%5Clambda%7D$

$E(X)%3D%5Csum_%7Bk%3D0%7D%5Enk%5Cfrac%7B%5Clambda%5Ek%7D%7Bk!%7D*e%5E%7B-%5Clambda%7D%3De%5E%7B-%5Clambda%7D%5Csum_%7Bk%3D1%7D%5En%5Cfrac%7B%5Clambda%5E%7Bk-1%7D%7D%7B(k-1)!%7D*%5Clambda%20%3D%5Clambda*e%5E%7B-%5Clambda%7D*e%5E%7B%5Clambda%7D%3D%5Clambda$

应用：机器故障、产品缺陷、细菌分布、放射性物质单位时间发射粒子数、火车客户、次数。

4 均匀分布

$E(X)%3D%5Cint_%7B%5Cvarpi%7D%5E%7B%5Cvarpi%20%7D%20xf(x)dx%3D%5Cint_%7Ba%7D%5E%7Bb%7D%20%5Cfrac%7B1%7D%7Bb-a%7D%20xdx%3D%5Cfrac%7B1%7D%7B2%7D(a%2Bb)%20$

$D(x)%3DE(X%5E2)-%5BE(X)%5D%5E2%3D%5Cint_%7Ba%7D%5E%7Bb%7D%20x%5E2%5Cfrac%7B1%7D%7Bb-a%7Ddx-(%5Cfrac%7Ba%2Bb%7D%7B2%7D)%5E2%3D%5Cfrac%7B(b-a)%5E2%7D%7B12%7D$

5 指数分布

$E(X)%3D%5Cint_%7B%5Cvarpi%7D%5E%7B%5Cvarpi%7D%20xf(x)dx%3D%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7Dx%5Cfrac%7B1%7D%7B%5Ctheta%7De%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7Ddx%3D-xe%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7D%5Cvert%5E%7B%2B%7B%5Cvarpi%7D%20%7D_%7B0%7D%2B%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7De%5E%7B-%7B%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7D%7Ddx%3D%5Ctheta%20$

D() $D(X)%3DE(X%5E2)-%5BE(X)%5D%5E2%3D%5Cint_%7B0%7D%5E%7B%2B%5Cvarpi%7D%20x%5E2*%5Cfrac%7B1%7D%7B%5Ctheta%7De%5E%7B-%5Cfrac%7Bx%7D%7B%5Ctheta%7D%7Ddx-%5Ctheta%5E2%3D2%5Ctheta%5E2-%5Ctheta%5E2%3D%5Ctheta%5E2$

无记忆性

$P(x%3Es%2Bt%7Cx%3Es)%3DP(x%3Et)$

6 正态分布

$f(x)%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%20%7D%20%7De%5E%7B-%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%20%7D%7D%20%20%2C%20%5Csigma%3E0%2C%20-%E2%88%9E%3Cx%3C%2B%E2%88%9E$

$E(x)%3D%5Cint_%7B-%E2%88%9E%7D%5E%7B%2B%E2%88%9E%7D%20xf(x)dx%0A%3D%5Cint_%7B-%E2%88%9E%7D%5E%7B%2B%E2%88%9E%7Dx%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%7D%7De%5E%7B%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%7D%7D%20dx$

$%5Cfrac%7Bx-%5Cmu%7D%7B%5Csigma%7D%3Dt---%3Ex%3D%5Cmu%2B%5Csigma*t$

$E(X)%3D%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%7D%7D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(%5Cmu%2B%5Csigma*t)e%5E%7B-%5Cfrac%7Bt%5E2%7D%7B2%7D%7Ddt%3D%5Cmu$

$D(X)%3D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(x-%5Cmu)%5E2f(x)dx%3D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7D(x-%5Cmu)%5E2*%5Cfrac%7B1%7D%7B%5Csqrt%7B2%5Cpi%5Csigma%7D%7De%5E-%5Cfrac%7B(x-%5Cmu)%5E2%7D%7B2%5Csigma%5E2%7Ddx$

$%5Cfrac%7Bx-%5Cmu%7D%7B%5Csigma%7D%3Dt---%3Ex%3D%5Cmu%2B%5Csigma*t%0A$

$D(X)%3D%5Cfrac%7B%5Csigma%5E2%7D%7B%5Csqrt%7B2%5Cpi%7D%7D%5Cint%5E%7B%2B%E2%88%9E%7D_%7B-%E2%88%9E%7Dt%5E2e%5E%7B%5Cfrac%7Bt%5E2%7D%7B2%7D%7Ddt%3D0%2B%5Cfrac%7B%5Csigma%5E2%7D%7B%5Csqrt%7B2%5Cpi%7D%7D*%5Csqrt%7B2%5Cpi%7D%3D%5Csigma%5E2$