Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

11 votes293 views1 answer
How to make an argument optional in python
Python
1 vote291 views1 answer
Shutil move overwrite
Python
8 votes229 views2 answers
Python "is"
Python
2 votes299 views1 answer
Importying listviewin django
Python Django
0 vote266 views1 answer
Python dt error only use with datetimelike values
Python
0 vote287 views1 answer
Sorting-a-python-list-by-two-fields
Python
5 votes270 views1 answer
Python put quotes in string
Python
1 vote282 views1 answer
How to slice dataframe based on daterange in pandas
Python
17 votes261 views6 answers
Python matplotlib
Python
15 votes269 views3 answers
Save numpy array to csv
Python