Batch Size Consideration: the choice of batch size can also impact the effective learning rate. Smaller batch sizes may benefit from a higher learning rate, while larger batch sizes may require a lower learning rate.

• batch size ≥ 32 impossible to run because of “torch.cuda.OutOfMemoryError: CUDA out of memory”

Configurations

EQUAL FOR EVERYONE: num_epoch = 1 (except for LR=1e-6 put num_epoch = 3), seed = 20

case 1 [gloria]: batch size 1

learning rate:

1e-4 (c1)

1e-5, (c2)

1e-6 (c3)

case 2 [nicolò]: batch size 2

10e-4 (c4):

10e-5, (c5):

10e-6 (c6) - 1 epoch: