Mensing.AI

My journey into AI/ML/DL and other areas of interest.

When Gradients Explode (or Vanish) 💥

This is a nice quick read on how to combat exploding or vanishing gradients, a problem that wreak havoc on your deep learning model. 👹

My TL;DR:

  • Exploding gradient? Use gradient clipping. It sets a ceiling on your gradient but keeps the direction. ✂️
  • Vanishing gradient? If you’re using an RNN, use an LSTM. ✔️

Src: Learn.Love.AI.

Leave a Reply

Your email address will not be published. Required fields are marked *