Myujth

In pytorch, to update the model, should I use optimizer.step() or model.step() ?

Here is a example snippet:

import torch
import torch nn
class SomeNeuralNet(nn.Module):
 def __init__(self,hs,es,dropout):
 SomeNeuralNet(ClaimRecognizer, self).__init__()
 # Some initialization here
 def forward(x):
 # forward propagation here

model = SomeNeuralNet(es,hs,dp)
optimizer = optim.Adam(model.parameters())
loss_function = nn.NLLLoss()
for epoch in N:
 for x in data:
 logp = model(x)
 loss = loss_function(logp,gold_outs)
 loss.backward()
 # Which one I should call ? Optimizer.step() or model.step() or both ?
 optimizer.step()
 model.step()

To make the gradient descent step, you normally use just optimizer.step().
Here is also an example taken from the documentation (same link at bottom), what it looks like in general:

for input, target in dataset:
 optimizer.zero_grad()
 output = model(input)
 loss = loss_fn(output, target)
 loss.backward()
 optimizer.step()

I don't know where you got this model.step()? Does did you try it?

If your model really possesses some kind of step()-functionality, it probably does something different.

But unless you define something extra, your model gets its functions from nn.Module and this does not have step function!

See this example from the the Pytorch Documentation:

import torch.nn as nn
import torch.nn.functional as F

class Model(nn.Module):
 def __init__(self):
 super(Model, self).__init__()
 self.conv1 = nn.Conv2d(1, 20, 5)
 self.conv2 = nn.Conv2d(20, 20, 5)

 def forward(self, x):
 x = F.relu(self.conv1(x))
 return F.relu(self.conv2(x))

model = Model()
model.step()

Trying to call step() result in an AttributeError:

---------------------------------------------------------------------------
AttributeError Traceback (most recent call last)
<ipython-input-41-b032813f7eda> in <module>
 13 
 14 model = Model()
---> 15 model.step()

~/miniconda3/envs/py37/lib/python3.7/site-packages/torch/nn/modules/module.py in __getattr__(self, name)
 530 return modules[name]
 531 raise AttributeError("'' object has no attribute ''".format(
--> 532 type(self).__name__, name))
 533 
 534 def __setattr__(self, name, value):

AttributeError: 'Model' object has no attribute 'step'

To sum it up, normally your model should not have a step() function, optimizer.step() is the way to go if you want to do the optimization step.

See also here:
https://pytorch.org/docs/stable/optim.html#taking-an-optimization-step