History of LLMs

The Era of Mechanical Translation and How It Crashed — Fascinating birth of AI, first chatbots and the power of US Department of Defense

Why should I know about it?

“To fundamentally push the deep learning research frontier forward, one needs to thoroughly understand what has been attempted in the history and why current models exist in present forms”

Haohan Wang and Bhiksha Raj from On the Origin of Deep Learning

Large Language Models (LLMs) have a fascinating history that dates back to the early 1930s when the first ideas of computational linguistics were born. You may argue that this tracing is excessive and that LLMs have nothing in common with the old-fashioned prehistoric computer systems. You may also argue that LLMs are based on real, hard-core deep learning. However, deep learning itself originated in 1943, when the first ancestor of the artificial neural model was proposed by McCulloch and Pitts. Exactly 60 years ago! What took us so long to get to modern LLMs?

This series isn’t about drowning you in technical details. While we provide an extensive list of references for those who want to delve deeper, our main goal is to captivate your attention and share the influential developments that have…