Start free trial
Take Naologic for a spin today, no credit card needed and no obligations.
Start free trial

Generative Pretrained Transformer - How big is GPT-3 pretraining data?


By a wide margin, GPT-3 outshines all prior language models with its estimated 175 billion parameters. It was trained using around 45 terabytes of text data extracted from many databases.