What is an LLM? (No Math Edition)

You've probably used ChatGPT, Claude, or Gemini by now. Maybe you've asked them to write an email, debug some code, or explain a concept. But have you ever stopped to wonder: How does this actually work?

This series is about what's happening under the hood - without drowning in equations - so you can reason about LLM behavior like an engineer.

The Simplest Useful Explanation

An LLM (Large Language Model) is a system that predicts the next token in a sequence.

Not "truth." Not "facts." Not "understanding."

In other words: given what came before, what token is most likely to come next?

When you type "The capital of Australia is...", the model scores many possible next tokens. In practice, the completion that leads to "Canberra" is usually the most likely, while alternatives like "Sydney" or "Melbourne" are much less likely.

It picks one (or samples based on probabilities), outputs it, appends it to the input, and repeats until done.

The capability comes from chaining hundreds to thousands of token predictions together.

Tokens, Not Words

A token is a piece of text - sometimes a full word, often a subword chunk, sometimes punctuation.

The model doesn't "think" in words. It operates on token sequences.

This matters because:

• Prompts behave oddly when you rephrase slightly (different tokens)
• Some languages "cost more tokens" than others
• Long inputs hit the context window limit faster than you'd expect

Rule of thumb: Think of tokens as "short chunks of characters," not fractions of words.

The 30-Second Pipeline

Here's what happens when you send a message:

Your text → Tokens → Embeddings → Transformer layers → Logits → Decoded token → Output

1. Tokenization: Your text becomes token IDs
2. Embeddings: Token IDs become vectors (numbers the model can process)
3. Transformer layers: The model mixes context using attention
4. Logits: The model outputs a score for every possible next token
5. Decoding: A strategy selects the next token (greedy or sampling)
6. Repeat until a stop token or length limit

If you remember one technical sentence from this post:

LLMs turn text into tokens, transform them through layers, compute logits, then decode tokens back into text.

We'll go deeper on each step in upcoming posts.

What Makes Them "Large"?

"Large" means some combination of:

• Training data: Very large corpora (web text, books, code, forums, etc.)
• Parameters: The internal "knobs" the model tunes during training
• Compute: Massive accelerators (GPUs/TPUs) running for months
• Post-training: Careful tuning to behave like a helpful assistant

But "large" doesn't automatically mean "best."
In practice, you trade off quality against latency, cost, and controllability.

How They Learn (Pretraining)

Modern LLMs learn primarily through next-token prediction.

The Training Loop

Input: The quick brown
Target: fox

Then:

Input: The quick brown fox
Target: jumps

This happens at massive scale across a huge amount of text. The model adjusts its parameters to reduce prediction error.

Key insight: It doesn't store documents like a database. It learns statistical patterns - the compressed structure of language.

Why Chat Models Feel "Helpful" (Post-Training)

A raw pretrained model isn't automatically a polite assistant. It's good at continuing text, not necessarily answering you.

So most chat LLMs go through post-training:

• Instruction tuning: Learning to follow prompts
• Preference optimization: Learning which answers humans prefer (via techniques like RLHF, DPO, etc.)
• Safety alignment: Refusing harmful requests

This is why "base models" and "chat models" behave very differently, even with the same architecture.

Base Model vs Instruct Model

When you use ChatGPT or Claude, you're using a highly tuned instruct model, not a raw language model.

The Transformer (Why Context Works)

The architecture powering modern LLMs is called a Transformer. The key mechanism is attention:

The model can weigh different parts of the input differently when predicting each token.

For example:

"Alex told Sam that they needed to sign the form."

When predicting what "they" refers to, the model attends to earlier words and uses context to resolve the reference.

You don't need the math yet - the core idea is that attention lets the model condition on relevant context.

What They're Great At

Because they've learned broad language patterns, LLMs excel at:

• Drafting and rewriting text
• Summarizing and structuring information
• Translating styles and formats
• Code completion and explanation
• Brainstorming and planning

Where They Fail (Three Buckets)

1. Knowledge Limits

• Training data has a cutoff date
• They don't know your private docs or internal context
• Mitigations: Retrieval (RAG), search/tools, verified sources

2. Reliability Limits

• They produce plausible-sounding wrong answers (hallucinations)
• They'll "fill gaps" instead of admitting uncertainty
• Mitigations: Structured prompts, citation requests, verification pipelines

3. Computation Limits

• Context window: Only limited text can be considered at once
• Latency/cost: Longer prompts and bigger models cost more
• Mitigations: Chunking, summarization, model selection

Why Hallucinations Happen

This deserves a dedicated explanation because it's the most common failure mode.

The mechanism:

1. The model is trained to produce plausible continuations, not to verify truth
2. If the context is missing or ambiguous, it will still complete the pattern
3. Without external checks (retrieval/tools/tests), plausibility can outrun correctness

The result: Confident-sounding nonsense that looks right but isn't.

The fix: Don't rely on the model alone for factual claims. Use retrieval, citations, and verification.

LLM vs Chat App (Common Confusion)

People often confuse the model with the product.

Understanding this distinction helps you debug unexpected behavior.

Engineer Mental Model

When debugging LLM behavior, think in terms of:

Prompt quality + Context quality + Decoding policy → Output behavior

Quick debug checklist:

1. Is the prompt clear and unambiguous?
2. Does the context contain the right information?
3. Are decoding parameters (temperature, top-p) appropriate?
4. Is retrieval grounding working correctly?
5. Do you have evaluation to measure the issue?

We'll cover each of these in depth throughout the series.

Try This Yourself

Want to see next-token prediction in action?

1. Go to ChatGPT or Claude
2. Type: "Complete this sentence: The best way to learn programming is"
3. Regenerate 3-4 times
4. Notice how it gives different but plausible completions

That's the probabilistic nature of LLMs in action.

Key Takeaways

1. LLMs predict the next token, one at a time, then chain predictions
2. Tokens are subword chunks, not words - this affects everything
3. The pipeline: text → tokens → embeddings → transformer → logits → decode
4. Pretraining learns language patterns; post-training makes it helpful
5. Hallucinations happen because the model optimizes for plausibility, not truth
6. Know the limits: knowledge cutoffs, reliability issues, context windows
7. Debug systematically: prompt → context → decoding → retrieval → evaluation

Key Terms

What's Next in This Series

Next, we’ll build up the missing pieces step by step:

1. Tokenization - why small wording changes matter
2. Decoding & Sampling - temperature, top-p, and why "temperature 0" isn't deterministic
3. Embeddings - how text becomes searchable geometry

Once these foundations are clear, we'll move into retrieval (RAG), evaluation, agents, and deployment.

Further Reading

• Transformer architecture (Vaswani et al., 2017): https://arxiv.org/abs/1706.03762