There Would Have Been No LLMs Without This

Game-changing 1990s: the bloom of NN, success of RNNLM and backpropagation, introduction of GPUs and advent of Transformers

Introduction

Today, we immerse ourselves in the whirlpool of incredible proposals and developments that emerged in Artificial Intelligence (AI) and Machine Learning (ML) from 1990 to the mid-2000s. We explore how previous research, — despite the lack of computational power! — paved the way for modern Language Models (LLMs).

Luckily enough, since the decline of expert systems in the 1990s, there were no more AI winters. It truly became the renaissance of AI. During this time machine learning as we know it today was born. Though it’s impossible to cover everything that happened in ML and AI in an article, we will highlight the most significant events that have shaped the AI field and made the seemingly impossible possible. You might argue with what we chose! Please feel free to add your views to the comment section.

Before embarking on our journey, let’s describe what LLMs are. In a nutshell, LLMs are deep neural networks that utilize specialized architectures and undergo training on vast amounts of data. The first crucial keyword to consider is “neural networks*,” which serves as an…