Pytorch l2 regularization

# add l2 regularization to optimzer by just adding in a weight_decay 
optimizer = torch.optim.Adam(model.parameters(),lr=1e-4,weight_decay=1e-5)

Regularization pytorch

 loss = mse(pred, target)
 l1 = 0
 for p in net.parameters():
  l1 = l1 + p.abs().sum()
 loss = loss + lambda_l1 * l1
 loss.backward()
 optimizer.step()

Source: discuss.pytorch.org

Similar codes

2 votes228 views2 answers
Django and operator
Python
5 votes230 views3 answers
Sys module in python
Python
0 vote211 views1 answer
How to sort values of pandas dataframe for iqr
Python
0 vote224 views2 answers
Python datetime yyyy-MM-dd'T'HH:mm:ssZ
Python
10 votes219 views2 answers
Browser refresh selenium python
Python
0 vote211 views1 answer
Take space away from strings ion pyhton
Python
0 vote219 views1 answer
Django.db.utils.ProgrammingError: relation "users" does not exist in django 3.0
Python Django
56 votes216 views5 answers
Check python version ubuntu
Python
0 vote223 views1 answer
Mnist
Python
1 vote238 views1 answer
Import bokeh
Python