ReduceOverfitting

• More data
• Constraint model complexity 减少模型复杂的
  • shallow
  • regularization
• Dropout
• Data argumentation 数据增强
• Early Stoping

Regularization

正则化在loss函数中引入模型复杂度指标，利用给W加权值，减弱表达能力，弱化了训练数据的噪声

• L1-regularization 在原来loss基础上加一范数
• L2-regularization 在原来loss基础上加二范数

l2_model =keras.models.Sequential([
	keras.layers.Dense(16,kernel_regularizer=keras.regularizers.l2(0.001),#正则化的权值
	activation=tf.nn.relu),
	keras.layers.Dense(16,kernel_regularizer=keras.regularizers.l2(0.001),
	activation=tf.nn.relu),
	keras.layers.Dense(16,activation=tf.nn.sigmoid)
])

for step, (x,y) in enumerate(db):
    with tf.GradientTape() as tape:
        loss = tf.losses.categorical_crossentropy(y_onehot, logits, from_logits=True)
        loss_regularization = []
        for p in network.trainable_variables: #根据存储方式只取w
            loss_regularization.append(tf.nn.l2_loss(p))
        loss_regularization = tf.reduce_sum(tf.stack(loss_regularization))
        loss = loss+0.001*loss_regularization
    grads = tape.gradient(loss, network.trainable_variables)
    optimizer.apply_gradients(zip(grads, network.trainable_variables))

Early Stopping

在训练过程中，训练集正确率持续上升，验证集正确率到达最高点后会因为训练过拟合而衰减，我们需要在最高点后让训练停止。

Dropout

network =keras.models.Sequential([
    keras.layers.Dense(256,activation=tf.nn.relu),
    layers.Dropout(0.5) #去掉50%
    keras.layers.Dense(128,activation=tf.nn.relu),
    layers.Dropout(0.5),
    keras.layers.Dense(64,activation=tf.nn.relu),
    keras.layers.Dense(32,activation=tf.nn.relu),
    keras.layers.Dense(10)])

使用dropout时,Train和Test是不同的

#train
out = network.(x,training=true)
#test
out = network.(x,training=false)

打赏