Train/Val Loss Issues when training GPT-2 from OWT

I'm currently testing a new system of 4x NVIDIA A4500's and had no problem running the Shakespeare training set and the outcome was as intended. My system is currently training the GPT 124M Model on the OWT dataset and everything seemed perfectly fine until about 20 steps in when our loss value increased until reaching a value of ~7.81 and its been hovering within ~0.5 of that value for around 65 steps now. Checking the Loss graph on the readme this seems highly unusual and could be indicating some form of problem. Any ideas what could be happening? Here is the config changes that were made to efficiently utilize the most VRAM to reduce total training time.

batch_size = 16 block_size = 1024 gradient_accumulation_steps = 5 * 4

Gradient steps were multiplied by 4 since our node only has 4 gpu's and the batch size was increased to 16 since the original config at 12 was utilizing only 60% of GPU memory and an increase to 16 changed that to around 94% and sped up the iteration speed. This is the ONLY code that was changed before starting training. I've also included the loss graphs from wandb to help visualize the problem.

If anyone could explain what could possibly be going on that would be much appreciated. Thank you in advance.