If your loss ever looks like this, it might be because you aren't calling optimizer.zero_grad() in PyTorch!!! Learned this the hard way lol.
Berkan Ottlik
@berkott3
Machine learning research✌️😎🏝
Joined January 2021
- Is the human race just running gradient descent on a mass scale?
- Why does the login to ChatGPT suck and take so long? Bard and HuggingChat are so much better in that respect, which is why I sometimes default to them.
- Yet another blog about transformers: berkan.xyz/posts/2023/01/… (except it's written by me and I think it's decent 😊)
