The BERT model seems trained from the dataset itself?

I had a general look at the BERT code. If I am not wrong, the BERT model in this folder is simply trained from the data itself, rather than transferred from a huge corpus? Could you help clarify this part?