AI glossary

Models and how they are built

These twelve terms are about the model itself: what it is made of, what it is measured by and how it runs.

Large language model: a program trained on a large volume of text whose job is to predict the most likely continuation of a text. Everything you do with it, from translation to writing code, is a form of that one job.

Parameter: a number inside the model that is set during training and determines part of the final behaviour. The parameter count tells you the size of the model, not its quality.

Weights: the trained parameters themselves, as a file you can download and run. Open weights means that file has been published, not that the training code and data have been.

Transformer: the neural network architecture most of today models are built on. Its defining feature is the attention mechanism, which lets the model look at the rest of the text while processing each piece of it.

Token: a piece of text the model counts in, usually smaller than a word. Costs and limits are all measured in tokens rather than words.

Context window: the largest number of tokens a model can see at once, covering both your input and the answer it produces. When a conversation grows long its beginning falls out of the window and the model no longer has it.

Multimodal model: a model that understands more than one kind of input, such as text, images and audio. Being multimodal does not mean being good at all of them.

Reasoning model: a model that produces more intermediate steps before answering, in order to break a problem down. Its answer comes back slower and costs more, and it is not worth it for a simple question.

Open weight model: a model whose weight file has been published so you can run it on your own hardware. Open is a matter of degree and each model licence has to be read separately.

Model card: the document a maker publishes alongside a model, stating its capabilities, limits and terms of use. It is the first thing to read before building on a model.

Quantisation: shrinking a model by reducing the numeric precision of its weights so it fits on weaker hardware. In exchange for less memory, output quality drops a little.

Inference: the stage of using a trained model to get an answer, entirely separate from training. The cost of every request you send belongs to this stage.

Training and data

These twelve terms are about work that was done before the model reached you and finished once and for all.

Training: the process in which a model parameters are set by exposure to a large volume of data. It is expensive, takes months, and once it is over it does not continue through your conversation.

Training data: the collection of text, images and audio a model was trained on. Any bias or error present in that collection finds its way into the behaviour of the model.

Pretraining: the first and largest stage of training, where the model learns general patterns of language from raw data. What comes out of this stage is not an assistant yet, only a text predictor.

Fine tuning: retraining a finished model on smaller, more specialised data so its behaviour moves closer to one task. It is not a substitute for a good prompt and only makes sense when you have many consistent examples.

Reinforcement learning from human feedback (RLHF): a stage in which people rank the model answers and the model learns which shape of answer is preferred. This is the stage that turns a text predictor into something resembling an assistant.

Distillation: building a smaller model by training it on the output of a larger one. The result is cheaper and faster and usually does not reach the same quality on hard work.

Knowledge cutoff: the date up to which the training data of a model was collected. Anything that happened after it is not in the model, however confidently it talks about it.

Overfitting: when a model memorises the training data instead of learning the pattern and performs poorly on new data. In evaluations it shows up as a suspiciously high score.

Bias: a systematic tilt of the model toward one answer or one group, coming from the training data or from the training process itself. Bias differs from random error because it repeats in one direction.

Benchmark: a standard test for comparing models on a defined task. A high benchmark score does not guarantee the model is better at your real work, because the test conditions are not your conditions.

Data labelling: the human work of marking what each data sample is or which answer is better. A large part of model quality rests on this rarely seen work.

Synthetic data: data that is itself the output of one model and is used to train another. It is cheap, and used carelessly it carries the errors of the first model into the second.

Working with a model

The thirteen terms that come up most in daily work, and the ones most often confused.

Prompt: the text you give a model so it can produce an answer. Everything written in the conversation is part of the prompt, not only the last message.

System prompt: the fixed instruction a maker or developer gives the model before your conversation, setting its general behaviour. You usually do not see it, yet it affects every answer you get.

Prompt engineering: the work of writing and revising a prompt until the output comes close to what you want. Most of it is clear writing and very little of it is tricks.

Few shot: putting a few example inputs and outputs inside the prompt so the model picks up the format you want. It is the best route for work that is hard to describe but easy to demonstrate.

Chain of thought: asking the model for intermediate steps before the final answer. It improves results on multi step problems and only makes a simple question longer.

Temperature: the setting that governs how far the model reaches toward less likely options. A low number gives more uniform, more predictable output; a high one gives more variety and less stability.

API: the route by which your program talks to the model directly, with no graphical interface. Data policy and pricing on that route usually differ from the consumer plan.

Conversation memory: what a tool keeps from your earlier conversations and places in front of the model again. It is not the same as the context window: the window is a limit on one request, memory is a product feature.

Retrieval augmented generation (RAG): a method where relevant pieces of your documents are found and added to the prompt before the model answers. It is the usual way to make a model answer from your own data without training it.

Embedding: turning a text into a list of numbers that represents its meaning, so that texts close in meaning can be found. Semantic search and the retrieval stage above both run on this.

Structured output: forcing the model to answer in a defined format such as JSON so a program can consume it directly. Without it the output is prose and differs slightly every time.

Tool calling: the capability that lets a model request a function or a search instead of answering, and then use the result. It is the foundation of what gets called an agent.

Agent: a system that puts a model in a loop so it can work with tools and carry several steps in sequence. What separates it from a chatbot is that it does not only answer, it does something.

Safety, error and rights

The twelve terms that start to matter once the work gets serious, and that are usually missing from marketing copy.

Hallucination: when a model produces something that looks right but is not real, such as a source that does not exist or a number that appears nowhere. It comes from the prediction mechanism rather than from a fault, so the next version does not remove it entirely.

Prompt injection: when text the model reads, such as a web page or an email, contains an instruction aimed at the model, and the model carries it out. For agents with access to the internet or to files this is the central risk.

Jailbreak: an attempt to get around a model behavioural rules through the way a request is phrased. Makers keep closing them and new ones keep appearing.

Watermark: a mark embedded into the output at the moment of generation so it can later be identified as machine made. It only works on the output of models that embed it, and it is read with that maker own tool.

Content provenance (C2PA): a standard for recording the creation and editing history of a file inside the file. It differs from a watermark: this is metadata that can be stripped, while the other is a mark inside the content itself.

Deepfake: an image, video or voice generated by AI to resemble a real person. The problem it raises is not technical, it is consent and deception.

Alignment: the work of making a model behave in accordance with human intent and values. It is an open research problem, not a feature that gets ticked off one day.

Guardrail: a restriction placed outside the model itself to block a particular output or behaviour. Because it sits outside the model it is both configurable and bypassable.

AI detector: a tool that guesses whether a text was written by a machine. Its output is a probability rather than a verdict, and it errs more on non English text and on edited text.

Training opt out: a setting or a right that stops your data being used to train the next model. It does not stop storage or human review, which are two separate things.

Copyright of output: the question of who owns the text or image a model produced and whether it is protected at all. The answer depends on the country and on the terms of the service, and it is changing; we issue no general ruling here.

Sycophancy: a model tendency to agree with the user rather than disagree correctly. It is why asking what do you think of my idea gets a low value answer, and asking what would break this idea gets a better one.

Words whose meaning changes in marketing

The five words below have clear meanings in technical writing and almost none in marketing copy. Telling the difference is the skill a glossary should hand you.

Smart. On its own it says nothing. The right question is which decision this system makes and on what input; if there is no clear answer, ignore the adjective.

It learns. A deployed model usually does not learn from your usage. That is what the split between training and inference means, and what products call learning is almost always conversation memory, that is keeping text and putting it back into the prompt. If it really is being trained, the place for that claim is the product data policy, not its sales page.

No hallucinations. This claim does not fit the mechanism. The chance of error can be lowered by retrieving from a source and by checking the output, but no is a promise that no serious maker writes in its documentation.

Open source. This one is a spectrum. The Open Source Initiative has published its own formal definition of open source AI, and many models called open do not meet it, because only the weights were released and the licence carries commercial restrictions. Read each model licence separately before you build a product on it.

Free. A free plan is a point in a data policy, not only a price. It is the place where the chance of your input being used to improve the product is at its highest, and we set out the detail with a per vendor table in the privacy in AI tools lesson.

The practical test for any AI claim is the ordinary one: if this sentence were false, what would be different? A claim with no way of being false is not a claim, it is decoration.