LLMs explained simply and quickly

LLMs are trained via modeling, imitation learning, and reinforcement learning

Training large language models begins with pretraining the model to predict the next word in a sequence based on finding patterns in massive amounts of text. Patterns are then fine-tuned to model human-written dialogue and to align responses with user preferences.

LLMs are the backbone of AI tools that process and generate natural language

Large language models identify patterns in massive datasets of books, code, and unlabeled text from which to generate coherent responses. GPT-3 was trained on about 45 terabytes of data and uses 175 billion parameters—each parameter being a value the model adjusts as it learns.

Visualize how transformer architecture generates a large language model's outputs by running words and subwords assigned to specific numbers through multiple rounds of mathematical processing.

Transformer architecture can recognize and predict patterns in language

The underlying software powering text generation in AI tools associates each word or subword—called a token—with a set of values corresponding to how often it appears near other tokens. By recognizing these associations in prompts, the LLM can infer meaning.

Anthropic's interpretability team discusses how AI models' thinking mimics biology

Just as humans engage in complex behaviors to survive and reproduce, teams of experts in neuroscience, virology, mathematics, and other disciplines argue that large language models develop complex mechanisms to achieve their goals. These mechanisms can be manipulated to assess how they modify outputs—much like stimulating individual neurons—to uncover how LLMs "think."

How ChatGPT learns to read

The introduction of ChatGPT last fall jumpstarted public interest in the future capabilities of artificial intelligence and what it means for society. Of many somewhat "jargony" terms thrown around, the phrase "Large Language Model" is commonly seen—but also incredibly unclear as to what it means. In the simplest sense, the programs comb through mountains of internet data in response to a question and guess at what should come next (subscription needed).

Large language models are examples of foundation models

Such models are trained on massive amounts of data, which enable them to perform a variety of tasks, rather than being provided task-specific data to complete a narrow function. Such models come with high compute costs and potential trust issues due to the nature of their unorganized training data.

EOS and maximum length tokens tell an LLM when to stop generating a response

Tokens are words or subwords (e.g., prefixes) that a large language model processes to generate a response. During training, the model is taught that if it ever generates a special token called end-of-sequence, its response is complete.

Large language models struggle to play video games, despite coding them easily

As of 2025, given clear tasks, specific feedback, and test results to learn from, LLMs have been successful at coding simple video games that resemble pre-existing ones. However, given the variety of games, the lack of training datasets, and the need for spatial awareness, which LLMs find challenging, some of these models experience greater difficulty navigating and "winning" games.

View a timeline of AI deployments from 2017 to 2025

The development of transformer architecture and the release of GPTs across AI labs led to rapid changes in LLM scale and power. Companies like OpenAI, Google, Meta, Microsoft, and Anthropic have invested billions of dollars to develop reliable, versatile models.

Tokens—numbers assigned to words and subwords—are the basic units of data that AI tools process to generate outputs that mimic language without understanding the text itself.

Generative AI tools excel at pattern recognition, not contextual accuracy

Large language models are trained on vast amounts of unstructured data, from which they develop parameters for grammar and associations between words. These connections can introduce errors due to inapplicable reasoning when used on new data in unfamiliar contexts.

Explore rankings of large language models

The SEAL LLM Leaderboards provide benchmarks for LLMs by assessing them against agentic, frontier, and safety tasks, providing insights into each model's strengths, deficiencies, and failures.

Model context protocol allows large language models to connect to external data

Using a unified interface, MCP simplifies how AI systems securely connect to external data, tools, and services. The protocol enables AI agents to perform tasks without ongoing human refinement or updates to manually integrate new data.

Understanding how Claude Code and other AI coding agents function

Under human oversight, a supervising large language model interprets user tasks and delegates work to subordinate LLMs, which can generate code, fix bugs, and run tests, often most effectively for proofs of concept. Incremental backups and versioning are crucial when using such agents, which can lose details during their work as a result of compressing context history to work around memory limitations.

Transformer architecture was the breakthrough that made AI chatbots possible

It changed how text was translated from a sequential, "one word at a time" method to one where every word in a text is processed in parallel. Advances in positional encoding and self-attention also helped models recognize context and word order better.

Trick a large language model into revealing secret passwords over multiple rounds, each with increasing safeguards.

How Model Context Protocol allows AI systems to communicate with databases

Developed by Anthropic, the open-source software serves as an integration layer that standardizes how large language models connect to real-time information sources or tools. Just as a USB-C port serves as a universal standard for many peripherals, MCP streamlines these integrations and provides context about external sources to LLMs in a way that AI can understand.

'Context rot' explains why AI conversations degrade the longer they run

Research shows that as a large language model's context window fills up, performance declines, and models begin over-indexing on content at the start and end of a conversation while losing track of what's in the middle. The practical fix is simpler than you'd think: start a fresh chat more often.

Machine learning and deep learning are successive subsets of artificial intelligence

ML allows systems to learn from data, while DL uses complex neural networks to analyze large amounts of unstructured data. Neural networks enable capabilities like image and speech recognition through their use of algorithms to identify patterns in data.

Machine learning lets computers learn from data instead of relying on programming

ML is a core part of artificial intelligence and is foundational to applications like chatbots, fraud detection, and self-driving cars. Businesses are also adopting it to improve efficiency and identify patterns in data, though the latter may be inaccurate.

Temperature is a value that controls how predictable or creative an LLM's output is

In large language models, temperature adjusts the randomness in text generation by modifying how often words with lower probabilities are selected next in the sequence. Lower temperatures produce more obvious outputs while higher temperatures produce more creative responses.

Unlike generative AI systems, such as chatbots that do not take follow-up actions after generating their output, agentic AI systems proactively take steps to achieve goals without continuous user input.

AI vs. machine learning vs. deep learning

Machine learning is best understood with visuals. A collection of the best 9 images, videos and flowcharts that explain what it is and how it works.

Just as large language models are trained on text to enable generative AI systems to mimic natural language, automated AI robotic systems are trained in virtual environments to help them 'learn' about the properties of reality.

There’s an artificial intelligence model trained on bankruptcy law

Although it's designed for lawyers, the tool is able to help everyday browsers synthesize important cases and point users to relevant documents about them. One sample using it asks Chatlaw to explain the significance of a recent case and then to define relevant terminology.

Find out how important you are to AI tools when they generate text responses with this tool, which estimates the weights of people's names.

Neural networks are modeled after an idealization of the brain

These networks—including the backbone of ChatGPT-3, which has approximately 175 billion parameters—naturally discover the implicit rules of grammar and syntax within language by identifying patterns. Deviations in word patterns add creativity to the text generation.

AI gained mainstream attention with tools like IBM Watson and Apple’s Siri

With the release of ChatGPT in 2022, which drew over 100 million weekly users in just two months, natural language processing and understanding could be achieved at scale via machine learning. Unlike earlier artificial intelligence that could pull stored knowledge, generative AI produces text, images, or sounds.

DALL-E generates images by turning words into shared picture meanings

DALL-E first uses a model called CLIP to associate pictures with specific words, followed by a model called GLIDE, which slowly removes noise from random dots until a clear image appears that matches the prompt.

Through prompt injection attacks, bad actors provide deceptive prompts to generative AI systems in order to manipulate their outputs and acquire sensitive data.

The AI that creates any picture you want and how it works

Since January 2021, advances in AI research produced a plethora of deep-learning models capable of generating original images from simple text prompts. Researchers at OpenAI, Google, Facebook, and others have developed yet to be released text-to-image tools, and similar models have proliferated online. This video is a primer on how we got here, how the technology works, and its implications.

What are neural networks and why do they matter?

Learn how a neural network works, why it matters, and how it can be trained to recognize patterns, classify data, and forecast future events.

Decoding the hype around AI

2022 was the year AI broke onto the scene, with applications like ChatGPT amazing users with its ability to generate realistic-sounding answers. Technologist Arvind Narayanan presents a clear-eyed view of what the technologies can (and can't) do.

Graphics processing units, which process large amounts of data in parallel, have become the foundational hardware for AI systems that rely on hardware capable of processing millions of text parameters simultaneously.

Teaching a robot to laugh

Researchers have designed a shared-laughter AI system that responds to human laughter in order to build a sense of empathy into dialogue. Learn how they did it with this explainer.

AI scraping for LLM training data has significantly strained Wikipedia's infrastructure

From January 2024 to April 2025, the site's bandwidth increased by 50% as automated bots downloaded terabytes of data for the large language models powering AI tools. The Wikimedia Foundation found that bots accounted for 65% of the highest demand requests (e.g., videos) despite representing just 35% of page views.

Scientists increasingly can't predict what AI will do

Most AI systems are so-called "black box" models, where we only see inputs and outputs while the inner workings remain obscure. This piece, written prior to the launch of ChatGPT, cautions developers to know why and how a system produces its results, and not to sacrifice explainability for accuracy.

Unlike the models made by large AI companies, open-weight models can have their safeguards removed or weakened, a process that has become increasingly easy and may expose users to harmful information.

Because they are built on content produced by humans, who possess biases, generative AI tools inherit prejudices and stereotypes from training data, which later appear in their outputs.

An explanation of Turing machines—the foundations of digital algorithms and computers

The device reads data, overwrites or ignores it, and then moves on to the next piece of data based on a set of programmed rules. By imagining this data as zeros and ones—binary that makes up computer code—Turing machines became the theoretical models for how any algorithm could be run on a computer if the procedure could be coded into a set of digital rules.

Technologies labeled “AI” have historically lost that title after widespread adoption

Generative AI is the latest entry in a recurring cycle where emerging tools start as "AI" until they become common software, like databases or machine learning. Generative AI and large language models may be the next platform shift after smartphones and the Web.

Computer systems create artificial voices by stitching together phonemes—individual sounds that make up speech, such as 't,' 'long a,' and 'k' to say 'take.'

In 2023, OpenAI spent $520 million on ChatGPT

The ongoing development and refinement of new AI models requires larger datasets and processing more parameters between words and subwords to improve the accuracy of generated content. New startups are expected to harness this technology to provide unthought-of products and services.

Google invented the foundation of the AI movement—but ChatGPT was a shock

For years, Google was the cutting-edge of the AI world, including the development of the Transformer—the basis of contemporary AI chatbots. But since OpenAI debuted ChatGPT in 2022, the company was had to play catch-up. This is a four-hour deep dive into Google's AI history.

Stephen Wolfram on AI's rapid progress & the "Post-Knowledge Work" era

AI, as a concept, has been around since at least the 1950s, but it's only in the last year or so that the technology has been able to equal humans at many tasks -- and in many cases surpass humans. In this podcast, mathematician and computer scientist Stephen Wolfram provides an AI 101, explaining how we got here, how highly advanced chatbots like ChatGPT work, and what our world is in store for next.

Language is evolution's great mystery

What we call language is something more specific than communication. Language is about sharing what's in our minds: stories, opinions, questions, the past or future, imagined times or places, ideas. So what does language mean for humanity, and how did we come to have it? Watch this brief video to discover more.

Using generative AI to create video introduces physics-compatibility challenges

Models involved in generating video from text must maintain spatial and temporal coherence while adhering to the laws of reality to produce smooth, logical motion. Ensuring this consistency creates long-term dependencies with high computational times.

AI tools that generate images, audio, video, and 3D models rely on diffusion models, which begin with random data, such as visual noise, and gradually refine it into the requested media.

Vitalik Buterin explains ethereum in 30 minutes

Every year, ethereum creator Vitalik Buterin explains what his crypto platform is in a 30-minute presentation at the Ethereum Foundation’s event, Devcon. Buterin’s explanation goes deeper than a simple 101 explanation. To learn more about the current state of ethereum, what’s changed recently, and what Buterin expects down the road, watch this video.