Troubleshooting Deep Neural Networks

A Field Guide to Fixing Your Model

Why?

When I started in deep learning, I felt frustrated that I was spending most of my time debugging instead of the "fun" stuff. (Later, I discovered that debugging never goes away, and the best practitioners still spend most of their time on it.)

As I learned more and began helping others train models, I realized that much of my advice consisted of walking people through a mental decision tree for how to improve their model's performance.

This guide is an attempt to codify that decision tree.

Who is it for?

If you know the basics of deep learning (e.g., have gone through one course), I hope you will get something out of this guide.

Want to help?

I'm soliciting feedback on the guide to try to make it clearer and more comprehensive. If you see:

  • Things that are not clear
  • Missing debugging tools, tricks, and strategies
  • Other ways to make the guide better
I'd love to know! Feel free to email me or reply on twitter.

I'm giving another talk on this material in March, and I'll share the lecture video and updated slides after.