What a large language model (LLM) is
A large language model is a program that breaks text into small pieces called tokens and guesses, over and over, what the next token should be. There is no table of facts inside it; what it has are weights derived from an enormous volume of text, and those decide which continuation is more probable.
- Lesson 2 of 15
- Beginner
- Free, no signup
What enters this calculation, and what does not
Three inputs reach the calculation and a text comes out. The fourth input is not connected at all, and that empty slot explains most of the odd behaviour of these tools.
-
1Your prompt and the whole chat
Re-read every turn, and it has to fit inside the context window. Outside it, nothing exists for the model.
-
2The trained weights
The result of training, fixed afterwards. Your conversation today changes nothing in them.
-
3A little randomness in the pick
The most probable option is not always taken, which is why one question gets two answers.
-
A source of truth
No such input exists unless the model was given search tools. Nothing inside the calculation says this piece is fact and that one is invented.
-
Predicting the next token
One token at a time, then the whole calculation again. The sentence you read is that one step repeated.
-
The text you read
How fluent this text is says nothing about whether it is right; fluency is exactly what this calculation was optimised for.
This drawing is simplified and says nothing about the internal architecture. All it shows is which things affect the answer.
Last checked: Facts and tool names in this lesson are re-checked against their sources on this date.
What a token is, and why everything gets counted in them
A model does not see text word by word. First it breaks the text into pieces called tokens: sometimes a whole word, sometimes half of one, sometimes just a punctuation mark or a space. Nobody wrote that split by hand; it was derived from the same text the model was trained on. That is why frequent words usually become one token and rare words break into several.
Why this matters to you: everything counted in these tools is counted in tokens. The input ceiling, the output ceiling and the bill. When you read that input costs so many dollars per million tokens, that is the unit, not words and not characters.
One point that matters for anyone writing in a less represented language: a language that appeared less often in the training text usually breaks into finer tokens, which means the same meaning is written with more of them and therefore costs more. We have not measured this on this server, because no tokeniser for any of these models is installed here and we do not print a number we have no source for. If it matters to you, drop one paragraph and its translation into the vendor own token counter and put the two figures side by side; it takes ten seconds and the answer is yours.
How big the context window is, and what it is not
The context window is the ceiling on what a model can see in one go: your prompt, the files you attached and the whole conversation so far, all together. Anything left outside that ceiling does not exist for the model. Not forgotten; never seen.
The sizes are large and they differ between vendors. In our own sourced reference, Claude Opus 5 has a window of 1,000,000 tokens, which is what the Anthropic model documentation states. GPT-5.6 Terra sits at 1,050,000 tokens according to OpenAI own model page. And Llama 4 Scout, whose weights are open, is declared at 10,000,000 tokens on Meta model card; that last one is the maker claim, not something we measured.
Now what the context window is not: memory. A model recalls nothing between two conversations, and inside one conversation it re-reads the whole text every single turn. That is why a long conversation gets more expensive and slower. The rule we follow ourselves is simple: in a long session we write the important instruction again near the end rather than relying on having said it at the start.
One warning about these numbers themselves: a bigger window means the model sees more, not that it understands better. No vendor promises that the quality of an answer at the far end of a ten million token window matches the quality at its start.
Training and inference: two completely different jobs
Two things that keep getting mixed up. Training happens once, takes weeks or months, runs on a mass of expensive hardware, and produces a file of weights that then stops changing. Inference is what happens when you type: the weights are fixed, and only your input passes through them to produce an output.
What follows from that split, and what most people do not expect: the model does not learn from your conversation. Correct its mistake today and it will make the same mistake tomorrow in a fresh chat. What some products call memory is a note kept beside the model and pasted at the start of the conversation, not a change in the model itself.
One clause that connects to the next block and must not be confused with the above: the model not learning from your conversation does not mean your text is not stored anywhere. Data retention, and whether it feeds the training of later versions, is the vendor decision and varies by plan and settings.
Training once, inference every time
Training: once, before release
- Takes weeks or months and runs on a mass of expensive hardware
- Its output is a file of weights that does not change afterwards
- The model knowledge cutoff is decided right here
- You play no part in it and have no access to it
Inference: every time you type
- Takes seconds and gets billed by the token
- The weights are fixed and only your input passes through them
- Nothing from this turn stays inside the model
- The only place you genuinely have control, which is through the prompt
Neither is better than the other; they are two different stages in the life of a model. Mixing them up is the source of most wrong expectations about these tools.
Why the word knows is the wrong word
At every step a language model does exactly one thing: given all the tokens so far, it works out which next token is more probable, picks one, and starts again. The sentence you are reading is that single step repeated, not the retrieval of a stored answer from somewhere.
Two consequences follow directly. First, it writes what it does not know just as fluently, because nothing in that calculation flags this part as fact and that part as invented. That is why an article title or a page number can look entirely right and not exist. Anthropic itself calls this hallucination in its own documentation and explains how to reduce it, which means the vendor treats it as a real weakness rather than a rumour.
Second, picking the next token does not always take the most probable option; there is some randomness in it, and that is what makes the same question get two different answers. Through programmatic access this can be controlled; in a chat window, usually not.
A practical test shorter than any philosophical argument: any claim the model gives you that you cannot open and look at in under a minute is still a claim. That sentence runs on any model, this year and next.
What this explanation is worth in daily work
Four things, each straight out of one of the blocks above. One: because the model re-reads the whole conversation every turn, break long sessions up and start each new job on a blank page. Two: because it invents references, accept no source without opening its link; if it gave you no link at all, no search happened.
Three: because the bill is in tokens, asking for shorter output genuinely costs less, and sending a hundred page file to get a two line answer genuinely costs more. Four: because the model does not learn from your conversation, keep the instruction you repeat every time in a short file and paste it at the start of each session. In our own work that single habit has returned more time than anything else, and it needs no more expensive plan.
And one thing this explanation does not give you: which model is better for your work. That answer depends on benchmarks, price and access, and it changes from week to week, so its place is the reference section of this site rather than a concept lesson meant to still be right next year.
The fast path, with AI
Once you know the model re-reads the whole conversation every turn and remembers nothing between sessions, a practical technique falls straight out of it, one that few people who consider themselves power users actually use: instead of stretching a session forever, ask the model at the end to write a handoff sheet, then paste that into a fresh chat. A fast cheap model is enough here, because this is summarising work rather than judgment work.
- The moment the session feels slow or the model starts ignoring something you said at the beginning, stop it there. That moment usually arrives earlier than you expect.
- Run the recipe below inside that same session and save the output to a plain text file, not into the product own memory.
- Read the sheet once yourself and cross out anything you do not remember saying. This step is not optional and the reason is below.
- Start the new session with that sheet. The same file is the base next time too, and it gradually turns into your standing instruction for that work.
Copy-ready recipe
The work in this session is done. Write a handoff sheet I can paste into an empty chat so you can continue from there.
The sheet format, exactly these headings:
Goal: {one sentence}
Decisions made: {short list, each with its reason in half a line}
Current state: {where we are}
Open questions: {what still has no answer}
Do not try again: {what was tried in this session and did not work}
Rules:
1. Write only from what was actually said in this session. If you do not know something, leave it out.
2. No extra explanation and no preamble. Only those five headings.
3. Keep it short. The point is to reduce volume, not to replay the conversation.
4. Where you are unsure, start that line with a question mark so I can see it.Before you trust the output: The sheet is written by the same model that may have drifted during the session, so anything it invented travels into the next chat as well, this time wearing the face of a settled decision. Reading it once before pasting is the entire cost of this method. And if the work involves customer data or anything confidential, the sheet holds that data too; the rules you apply to the conversation apply to this file as well.
AI in this kind of work
To understand this topic, the best thing to do with the tools themselves is make them show you their boundaries. Three questions that get instructive answers: what part of this conversation can you no longer see, which part of your answer are you unsure about, and give me a source I can open. Our position: asking a model about itself, such as its version or the size of its window, produces the least trustworthy answer of all, because that information did not exist yet when its training ended.
Tools that actually help
- Claude Asked which part of its answer it is unsure about, it gives more direct replies and fills less of the gap with manufactured confidence. Iran is on neither of Anthropic two supported-countries lists; we read that on Anthropic own page rather than measuring it.
- Gemini It handles Persian well and suits testing how the context window behaves on a long file. Google own page says the Gemini web app runs in over 230 countries and territories, and Iran is not on that list.
- ChatGPT The most common choice and acceptable for this job. We make no claim about access from Iran: the OpenAI supported-countries page, like the rest of that domain, returns an access error to this server, and we do not write a claim with nothing behind it.
Where it backfires
The specific risk in this topic is that the model is an unreliable narrator about itself. It follows from the training and inference split you read above: a model does not know its own version, the size of its window, or the settings of the product it runs inside unless the prompt told it, and because its output is built for fluency rather than for confidence, it answers confidently anyway. So exactly where you want the most precision, the odds of a manufactured answer are highest. Anthropic calls this hallucination in its own documentation and has written up how to reduce it. The second risk is about data rather than accuracy: depending on plan and settings, the text you send may be used for training, which Anthropic explains on its privacy page, and the handoff sheet you built in the section above falls under the same reckoning. For how access and payment for these tools stands from Iran, see the buying guide.
Sources: Anthropic: reduce hallucinations Anthropic: is my data used for model training Anthropic: supported countries Google: where the Gemini web app is available
Where this advice stops
This lesson explains the shape of the work, not its inside. Why one particular answer came out is a question nobody outside the vendor can settle, and in many cases nobody inside it can either. Second, the three numbers on this page carry a date and were taken from the sourced files of this site AI section; vendors change these figures, and that is exactly what the checked date at the top of this page means. Third, a language model is only one kind of model: image, audio and video models follow a different logic and this page is not about them.
From our own work
For the AI section of this site we maintain 110 model version pages. On the day this lesson was checked, 37 of them carried a context window with a named source: the smallest at 8,000 tokens on a translation model, and the largest the same 10 million from Llama 4 Scout. What surprised us while counting was that 16 of those 37 sit at exactly one million, across different vendors. A figure that sixteen models from competing companies agree on exactly is not a technical ceiling; it is a product decision. If you want to count for yourself, the files under wp-content/plugins/rgb-ai/data/versions/ and the context_tokens key say the same thing.
Real follow-up questions
Is a large language model different from a chatbot?
Yes. The model is the engine and the chatbot is the product built around it: the interface, the chat history, the tools, the limits and the company policies. One model can sit behind several different products, and one product can call several different models depending on what you are doing.
Why does it answer the same question differently each time?
Because picking the next token does not always take the most probable option; there is some randomness in it. That setting is deliberate and it makes the text read more naturally and repeat itself less. If you need identical output, it can be controlled through programmatic access; in a chat window, usually not.
Does a bigger context window mean a better model?
No. A bigger window only means the model sees more text in one go. How much of it actually gets used is a separate question, and no vendor has promised equal quality across the whole window. For ordinary work, the window on today cheapest models is already larger than most needs.