When I started in deep learning, I felt frustrated that I was spending most of my time debugging instead of the "fun" stuff. (Later, I discovered that debugging never goes away, and the best practitioners still spend most of their time on it.)
As I learned more and began helping others train models, I realized that much of my advice consisted of walking people through a mental decision tree for how to improve their model's performance.
This guide is an attempt to codify that decision tree.
If you know the basics of deep learning (e.g., have gone through one course), I hope you will get something out of this guide.
I'm soliciting feedback on the guide to try to make it clearer and more comprehensive. If you see:
I'm giving another talk on this material in March, and I'll share the lecture video and updated slides after.