Timeline milestone
GPT shows the power of generative pre-training
-
Research
GPT shows the power of generative pre-training
OpenAI showed that a Transformer trained broadly on unlabelled text could be adapted to multiple language tasks.
The first Generative Pre-trained Transformer was modest by later standards, but its two-stage recipe became highly influential: learn general language patterns through prediction, then fine-tune for specific tasks. The approach helped shift the field toward reusable foundation models.Sources & references 1 source
