Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

-2 votes312 views2 answers
Convert list of list to list python
Python
1 vote327 views1 answer
Python swapcase() method Example
Python
2 votes312 views1 answer
Add two column values of a datframe into one
Python
3 votes319 views1 answer
How to change window size in kivy python
Python
0 vote319 views1 answer
Knn example
Python
3 votes321 views1 answer
Annotation type of collection python
Python
0 vote322 views1 answer
Python unzip list
Python
37 votes319 views6 answers
Lambda function in python
Python
1 vote310 views2 answers
How to have player input in python
Python
2 votes310 views2 answers
Check if boolean is true python
Python