Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

51 votes120 views6 answers
Pickle load
Python
0 vote126 views1 answer
Add one element to tuple python
Python
10 votes105 views6 answers
Range function in python
Python
-1 vote117 views1 answer
What is admin.tabularinline django
Python
1 vote117 views1 answer
Create square matrix python
Python
0 vote107 views1 answer
Python get website chrome network tab
Python
0 vote111 views1 answer
Python greater than dunder
Python
0 vote107 views1 answer
Python how to import library absoluth path
Python
2 votes77 views6 answers
Python discord bot embed
Python
0 vote108 views2 answers
A guide to numpy and pandas
Python