Linear Regression with MSE

💡

MSE = Mean Squared Error 均方差

这个就是最常用的拟合线性回归的方法

Compute MSE

In this exercise, you will implement a method to compute the mean squared error for a set of inputs , measurements , and slope estimate Here, and are vectors of data points.

As a reminder, the equation for computing the estimated y for a single data point is:

and for mean squared error is:


def mse(x, y, theta_hat):
  """Compute the mean squared error
  Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
    theta_hat (float): An estimate of the slope parameter
  Returns:
    float: The mean squared error of the data with the estimated parameter.
  """

  # Compute the estimated y
  y_hat = theta_hat * x

  # Compute mean squared error
  mse = np.mean((y - y_hat)**2)

  return mse


theta_hats = [0.75, 1.0, 1.5]
for theta_hat in theta_hats:
  print(f"theta_hat of {theta_hat} has an MSE of {mse(x, y, theta_hat):.2f}")

Least-squares optimization

💡

采用MSE拟合的模型，最优解由最小方差时的theta决定

💡

由MSE的计算公式可知，MSE为一个关于theta的凸函数，可以通过求导找到最优解

We can do this by minimizing the cost function. Mean squared error is a convex objective function, therefore we can compute its minimum using calculus.

where x and y are vectors of data points.


def solve_normal_eqn(x, y):
  """Solve the normal equations to produce the value of theta_hat that minimizes
    MSE.
    Args:
    x (ndarray): An array of shape (samples,) that contains the input values.
    y (ndarray): An array of shape (samples,) that contains the corresponding
      measurement values to the inputs.
  Returns:
    float: the value for theta_hat arrived from minimizing MSE
  """

  # Compute theta_hat analytically
  theta_hat = (x.T @ y) / (x.T @ x)

  return theta_hat


theta_hat = solve_normal_eqn(x, y)
y_hat = theta_hat * x

with plt.xkcd():
  plot_observed_vs_predicted(x, y, y_hat, theta_hat)