What Is a Large Language Model (LLM)?

If you've used ChatGPT, Claude, or Gemini, you've used a large language model. But "it's a chatbot" isn't a real answer to what an LLM is — and if you're planning to build anything with one, the difference matters. This is the plain-language explanation we open every Generative AI course cohort with, before any code gets written.

What is an LLM?

A large language model (LLM) is a neural network trained on massive amounts of text to predict what word — or more precisely, what token — comes next in a sequence. That's it. There's no lookup table of facts, no internal database of "truth." An LLM is a statistical model of language, trained until it gets extremely good at continuing text in a way that resembles its training data.

What makes this useful is scale. Models like GPT, Claude, Gemini, Llama, and Mistral are trained on a meaningful fraction of publicly available text, with billions of internal parameters (the numbers the model adjusts during training). At that scale, "predict the next token" turns out to be enough to answer questions, write code, summarize documents, and hold a conversation — because doing all of those things well requires an implicit understanding of language, facts, and reasoning patterns.

How LLMs generate text

Generation happens one token at a time. The model looks at everything so far (your prompt, plus whatever it has generated in this response already) and outputs a probability distribution over every possible next token. It samples one, appends it, and repeats — this is why response text streams in piece by piece rather than appearing all at once.

A setting called temperature controls how that sampling works. At temperature 0, the model almost always picks the single most likely next token — deterministic, focused, sometimes repetitive. At temperature 1 (or higher), it's more willing to pick less-likely tokens — more creative, more varied, and more prone to going off the rails. In our course, students run the exact same prompt at both settings side by side in Session 2 — it's a faster way to build intuition than any explanation.

The architecture behind this prediction is the transformer — specifically, its self-attention mechanism, which lets the model weigh how relevant every other word in the input is to predicting the next one. You don't need the math to use LLMs well, but knowing that attention is what lets a model connect "it" on page 3 back to a noun on page 1 explains a lot about why longer context helps.

Tokens & context windows

Models don't see letters or whole words — they see tokens, chunks of text that are often shorter than a word ("token" might split into "tok" + "en"). Think of tokens as the LEGO bricks of language: the model only ever builds with these fixed pieces, never with raw characters.

Every model has a context window — the maximum number of tokens it can consider at once, including your prompt, any conversation history, and its own response. Think of it as the model's short-term memory: anything outside that window simply isn't visible to the model anymore. This is exactly why Retrieval-Augmented Generation (RAG) exists — it's a way to feed the model only the most relevant chunks of a much larger document, instead of trying to stuff everything into a limited window.

Training vs inference

These are two separate phases, and mixing them up causes a lot of confusion:

• Training is the (extremely expensive, one-time-per-model-version) process of adjusting billions of parameters on huge text datasets. This is what OpenAI, Anthropic, Google, and Meta do before a model is ever released.
• Inference is what happens every time you send a prompt to an already-trained model and get a response back. This is what you're doing when you call an API — no learning happens during inference; the model's weights don't change.

This distinction matters practically: an LLM does not "remember" your conversation from yesterday unless an application is explicitly re-sending that history to it. Memory in AI products is almost always engineered on top of the model, not built into it.

LLM vs Generative AI vs AI

These terms get used interchangeably, and that's a source of real confusion:

• AI is the broadest category — any system that performs tasks we associate with intelligence.
• Machine Learning (ML) is a subset of AI: systems that learn patterns from data rather than following hand-written rules.
• Deep Learning is a subset of ML using multi-layer neural networks.
• Generative AI is deep learning applied to generating new content — text, images, audio, code — rather than just classifying or predicting a number.
• LLMs are the text-focused branch of generative AI, and the foundation most current AI applications are built on.

An "AI application" (like a customer support bot or a study assistant) is a product built using a foundation model — the model itself isn't the application. That distinction is the entire premise of our course's LLM App Development module.

Why LLMs hallucinate

Because an LLM is a next-token predictor, not a fact-checker, it will happily generate a fluent, confident-sounding sentence that is factually wrong — this is called a hallucination. It isn't lying; it's producing the statistically plausible continuation of your prompt, and plausible isn't the same as true.

The two most reliable mitigations are also two of the most important skills in applied GenAI work: better prompt engineering (being explicit, providing context, asking the model to cite its reasoning), and RAG, which grounds the model's answer in retrieved source documents instead of its own memorized (and sometimes wrong) training data.

How to actually learn this

Reading about LLMs gets you the vocabulary. Building with them is what makes it stick. In the Generative AI & AI Application Development course, this exact explanation is Session 1–2 material — and by the end of Session 1, every student has already built a working chatbot, before the theory is even introduced. If you want the structured, project-based version of this article, that's where to start — or see the full curriculum for exactly what comes next.