1.2 MSE

Root Mean Squared Errod 均方根误差
08 模型正则化 - 图6
这里换一种写法
08 模型正则化 - 图7

2.岭回归

2.1 原理

岭回归是模型正则化的一种方式。
目标：使 08 模型正则化 - 图8 尽可能小。
08 模型正则化 - 图9 这个式子考虑了 08 模型正则化 - 图10 的影响， 08 模型正则化 - 图11 如果越大，要让整个式子越小， 08 模型正则化 - 图12 就应该越小

所以，新的 08 模型正则化 - 图13 要同时考虑方差和 08 模型正则化 - 图14 两个因素，这样就能做到平衡。

方差：过拟合，偏大
模型正则化：新的越小，越小
2.2 编码
2.2.1 准备数据
```python import numpy as np import matplotlib.pyplot as plt

np.random.seed(42) x = np.random.uniform(-3.0, 3.0, size=100) X = x.reshape(-1, 1) y = 0.5 * x + 3 + np.random.normal(0, 1, size=100)

plt.scatter(x, y) plt.show()

![image.png](https://cdn.nlark.com/yuque/0/2021/png/12405790/1639821683034-327f0ff1-77a0-4d04-a7d2-bb3959228c2e.png#clientId=udb1a98dd-80ee-4&from=paste&id=ua0dd822e&margin=%5Bobject%20Object%5D&name=image.png&originHeight=252&originWidth=366&originalType=url&ratio=1&size=6297&status=done&style=none&taskId=ua8f891a7-4a55-4b59-955a-a5a32c2230a)
<a name="hGZO7"></a>
### 2.2.2 多项式建模
```python
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import PolynomialFeatures
from sklearn.preprocessing import StandardScaler
from sklearn.linear_model import LinearRegression
# 分割数据
from sklearn.model_selection import train_test_split
np.random.seed(666)
X_train, X_test, y_train, y_test = train_test_split(X, y)
# 多项式参数配置
def PolynomialRegression(degree):
    return Pipeline([
        ("poly", PolynomialFeatures(degree=degree)),
        ("std_scaler", StandardScaler()),
        ("lin_reg", LinearRegression())
    ])
# 建模
from sklearn.metrics import mean_squared_error
poly_reg = PolynomialRegression(degree=20)
poly_reg.fit(X_train, y_train)
y_poly_predict = poly_reg.predict(X_test)
# 衡量MSE
mean_squared_error(y_test, y_poly_predict)  # 167.94010866878617
# 预测
X_plot = np.linspace(-3, 3, 100).reshape(100, 1)
y_plot = poly_reg.predict(X_plot)
plt.scatter(x, y)
plt.plot(X_plot[:,0], y_plot, color='r')
plt.axis([-3, 3, 0, 6])
plt.show()

2.2.3 使用岭回归

from sklearn.linear_model import Ridge
def RidgeRegression(degree, alpha):
    return Pipeline([
        ("poly", PolynomialFeatures(degree=degree)),
        ("std_scaler", StandardScaler()),
        ("ridge_reg", Ridge(alpha=alpha))
    ])
ridge1_reg = RidgeRegression(20, 0.0001)
ridge1_reg.fit(X_train, y_train)
y1_predict = ridge1_reg.predict(X_test)
mean_squared_error(y_test, y1_predict)  # 1.3233492754151852
plot_model(ridge1_reg)

# 修改alpha值
ridge2_reg = RidgeRegression(20, 1)
ridge2_reg.fit(X_train, y_train)
y2_predict = ridge2_reg.predict(X_test)
mean_squared_error(y_test, y2_predict)  # 1.1888759304218448
plot_model(ridge2_reg)

# 修改alpha值
ridge3_reg = RidgeRegression(20, 100)
ridge3_reg.fit(X_train, y_train)
y3_predict = ridge3_reg.predict(X_test)
mean_squared_error(y_test, y3_predict)  # 1.31964561130862
plot_model(ridge3_reg)

# 修改alpha值
ridge4_reg = RidgeRegression(20, 10000000)  
ridge4_reg.fit(X_train, y_train)
y4_predict = ridge4_reg.predict(X_test)
mean_squared_error(y_test, y4_predict)  # 1.8408455590998372
plot_model(ridge4_reg)

08 模型正则化 - 图24 值越大，模型的过拟合程度越小。

3. LASSO回归

3.1 原理

Least Absolute Shrinkage and Selection Operator Regression
LASSO回归是模型正则化的一种方式。
目标：使 08 模型正则化 - 图25 尽可能小。

3.2 编码

from sklearn.linear_model import Lasso
def LassoRegression(degree, alpha):
    return Pipeline([
        ("poly", PolynomialFeatures(degree=degree)),
        ("std_scaler", StandardScaler()),
        ("lasso_reg", Lasso(alpha=alpha))
    ])
lasso1_reg = LassoRegression(20, 0.01)  # 这里的alpha是theta绝对值
lasso1_reg.fit(X_train, y_train)
y1_predict = lasso1_reg.predict(X_test)
mean_squared_error(y_test, y1_predict)  # 1.1496080843259968
plot_model(lasso1_reg)

lasso2_reg = LassoRegression(20, 0.1)
lasso2_reg.fit(X_train, y_train)
y2_predict = lasso2_reg.predict(X_test)
mean_squared_error(y_test, y2_predict)  # 1.1213911351818648
plot_model(lasso2_reg)

lasso3_reg = LassoRegression(20, 1)
lasso3_reg.fit(X_train, y_train)
y3_predict = lasso3_reg.predict(X_test)
mean_squared_error(y_test, y3_predict)  # 1.8408939659515595
plot_model(lasso3_reg)