回归问题学习

梯度下降 Gradient Descent

lr：learning rate 学习率

函数导数的方向指向的是函数变大的方向，函数向左增大，导数>0，反之<0，公式中减去导数即意味着寻找减小的方向，寻找最小值，lr作用防止导数过大导致越过最小值，一般取值很小。

多点图 线性拟合 y=wx+b

loss函数，也称为损失函数，易见loss函数取值越小，拟合程度越好

代码实现

求当前loss平均

def compute_loss_for_line_given_points(b,w,points): #b&w为每次迭代之前的初始值，point100个点[100,2]
    totalLoss = 0
    for i in range(0,len(points)):
        x = points[i,0] #p[i][0]
        y = points[i,1]
        totalLoss += (y - (w*x +b)) **2
    return totalLoss/float(len(points)) #loss函数取平均

对loss函数求w与b偏导

进行一次梯度下降

def step_gradient(b_current, w_current, points, learningRate):
    b_gradient = 0
    w_gradient = 0
    N = float(len(points))
    for i in range(0, len(points)):
        x = points[i, 0]
        y = points[i, 1]
        # grad_b = 2(wx+b-y)
        b_gradient += (2/N) * ((w_current * x + b_current) - y)
        # grad_w = 2(wx+b-y)*x
        w_gradient += (2/N) * x * ((w_current * x + b_current) - y)
    # update w'
    new_b = b_current - (learningRate * b_gradient)
    new_w = w_current - (learningRate * w_gradient)
    return [new_b, new_w]

进行迭代

def gradient_descent_runner(points, starting_b, starting_w, learning_rate, num_iterations):
    b = starting_b
    w = starting_w
    # update for several times
    for i in range(num_iterations):
        b, w = step_gradient(b, w, np.array(points), learning_rate)
        if i<10  or i % 1000 ==0:
            print("After {0} iterations b = {1}, w = {2}, loss = {3}".format(i, b, w,compute_loss_for_line_given_points(b, w, points)))
            #输出前10次迭代和每千次迭代loss值
    return [b, w]

运行

import numpy as np
def run():
    points = np.genfromtxt("data.csv", delimiter=",")
    learning_rate = 0.0001
    initial_b = 0  # initial y-intercept guess
    initial_w = 0  # initial slope guess
    num_iterations = 1000
    print("Starting gradient descent at b = {0}, w = {1}, loss = {2}"
          .format(initial_b, initial_w,
                  compute_loss_for_line_given_points(initial_b, initial_w, points))
          )
    print("Running...")
    [b, w] = gradient_descent_runner(points, initial_b, initial_w, learning_rate, num_iterations)
    print("After {0} iterations b = {1}, w = {2}, loss = {3}".
          format(num_iterations, b, w,
                 compute_loss_for_line_given_points(b, w, points))
          )

if __name__ == '__main__':
    run()

运行结果

可以明显看到loss值的下降和w与b的变化

打赏