非常感谢分享!
另外,不对偏置项进行权重衰减那里的代码有点小小的问题,应该改成下面这样子 :grin:
bias_list = (param for name, param in model.named_parameters() if name[-4:] == 'bias')
others_list = (param for name, param in model.named_parameters() if name[-4:] != 'bias')
parameters = [{'params': bias_list, 'weight_decay': 0},
{'params': others_list}]
optimizer = torch.optim.SGD(parameters, lr=1e-2, momentum=0.9, weight_decay=1e-4)