Pre-Training - What are the benefits of pre-training?

The benefit of pre-training is that it allows for rapid optimization of the model. In addition, compared to starting from scratch, using pre-trained models reduces the amount of data needed. The bulk of pre-trained models found online were actually trained on very big datasets, which is why this is the case.