Loss: nan with example WIDER FACES datatset

I cloned your repository and used python 3.6 and pytorch 1.2. Still I get loss nan when I use the model to train on the widerfaces dataset you provide. I set learning rate to 1e-3 and even use the pre trained model as initialization