Running AI on your own computer
Running locally means the model file sits on your own disk and the computation happens on your own processor and graphics card, so the text you type never leaves the machine. What decides whether this is workable for you is your free RAM against the size of the model file.
- Lesson 11 of 12
- Intermediate
- Free, no signup
Model file size against the RAM you have, in gigabytes
The four sizes of the gemma3 family, as published in the Ollama library. The needle sits on the RAM of a real server so the scale means something.
- gemma3:1b815 MB, on almost any current laptop
- gemma3:4b3.3 GB, on an 8 GB laptop with the other programs closed
- gemma3:12b8.1 GB, from 16 GB of RAM upwards
- gemma3:27b17 GB, a dedicated graphics card or a lot of RAM
These are download sizes, not memory requirements. The Ollama documentation says a larger context length increases the memory needed, so the RAM required is always more than these figures.
Last checked: Facts and tool names in this lesson are re-checked against their sources on this date.
What on your own computer means
When you use an AI website, your text goes to that company server, the computation happens there, and the answer comes back. Running locally shortens that chain: a file of several gigabytes sits on your disk, a program opens it in memory, and the same computation happens on your own hardware.
Three things change. No account is required. After the model is downloaded, no internet is required. And the text you write is not recorded anywhere, because it goes nowhere.
There is a trap right at the start and it is worth knowing before anything else. Tools that run local models often have a cloud version too, and they look identical. Ollama own documentation says cloud models are automatically offloaded to Ollama cloud service, and that using them requires an account on ollama.com. So a model name ending in cloud is not local; same window, same command, but your text has left the building.
How much RAM you need
The simplest way to estimate is to look at the download size of the model itself. These numbers are published by the maker on each model page and are not guesses. For the gemma3 family, the Ollama library page gives these four sizes:
| Model | Download size | What machine it makes sense on |
|---|---|---|
| gemma3:1b | 815 MB | almost any current laptop |
| gemma3:4b | 3.3 GB | an 8 GB laptop, with other programs closed |
| gemma3:12b | 8.1 GB | 16 GB of RAM and up |
| gemma3:27b | 17 GB | a dedicated graphics card, or a lot of RAM |
Now the rule that makes the table usable: the download size is a floor, not the requirement. The model has to be opened in memory, and the conversation text has to fit on top of it. The Ollama documentation says plainly that setting a larger context length will increase the amount of memory required to run a model. So if your free RAM is exactly the size of the file, the model either will not run or will crawl badly enough to be useless.
The same documentation gives another number that paints a more realistic picture of an ordinary machine: the default context length is 4k for a graphics card with less than 24 GiB of VRAM, 32k for 24 to 48 GiB, and 256k for 48 GiB or more. Most laptop graphics cards are in the first band. On an ordinary machine, then, a local model is not only smaller; it also has a shorter memory.
From install to the first answer, on your own machine
-
1
Install
The documentation supports macOS, Windows and Linux. No account is asked for.
-
2
Pick a size
Start from a download size that fits comfortably in your free RAM, not the biggest thing the disk can hold.
-
3
Pull and run
One command, and the first time it downloads the model file too. After that, no internet is needed.
-
4
Try one real text
One paragraph of your own work, not a general question. Persian quality only shows up this way.
These steps come from the Ollama getting started guide. We have not run it on this server, because Ollama is not installed here and this machine has another job.
Why this route matters from Iran
The three permanent obstacles between an Iranian user and AI tools are these: the account will not open, the card is not accepted, and the country is not on the list. A local model gets past all three, not by a trick, but because it needs none of them.
The Ollama getting started documentation supports macOS, Windows and Linux, and nowhere in it does running a model on your machine ask for a sign in or a registration. An account becomes necessary only when you reach for the cloud models, and that is no longer local execution.
Let us state one boundary honestly: the model file has to be downloaded once, and we cannot promise that download completes from every connection in Iran. We have not measured it and it is not written anywhere. What can be said is that after the download nothing else is required, and that is what separates this route from the others.
The third benefit is the least discussed and may be the most important: privacy. A client document, a contract, a medical file, a text that has not been published yet; on a local model none of these are recorded anywhere. How that compares with the defaults of cloud services is set out, with sources, in our lesson on privacy in AI tools.
Where it falls short
If someone tells you a local model replaces the big ones, do not believe it. A model that fits in 3.3 GB is not in the same class as one that runs on a cluster of graphics cards, and no setting makes up for that.
Three places show it most. Multi step reasoning, where a small model loses the thread halfway. Precise knowledge, where it invents something with confidence. And long text, which simply does not fit through a short context window.
Let us be honest about Persian too: the quality of small models on Persian is uneven and we have not measured it on this server, because none of these models are installed here. The only valid route for you is to give one real paragraph of your own work to two or three models and put the answers side by side. It takes ten minutes and the answer is yours. If you want to know where these models come from and what open actually means, the open models lesson is that discussion.
There is a hidden cost the advertising does not show: your machine is busy the whole time it runs. The laptop gets hot, the fan works, the battery drains faster, and everything else slows down. For one short request that is nothing; for processing a hundred files it is a decision in itself.
Local model and hosted model, two different places
Hosted: on a company server
- Far larger models and a longer context window
- Your machine stays idle and the answer arrives faster
- But it needs an account and payment, and your country may not be on the list
- And your text may be retained depending on the plan and the settings
Local: on your own machine
- No account, no card, no country list
- The text is recorded nowhere because it goes nowhere
- After the download it works offline too
- But its ability is limited and your machine stays busy
The choice between these two is not made once for good; it is made per task. Most people who take this seriously keep both and know which job goes where.
So what does a local model actually do well
Our position: do not treat a local model as a replacement, treat it as the tool for mechanical and sensitive work. That means jobs with volume but no judgment, or jobs whose text must not leave the machine.
- Stripping names and numbers out of a text before you send it anywhere
- Pulling fixed fields out of dozens of emails or invoices with one steady pattern
- Summarising a short text that is confidential
- Writing a first draft that you then rewrite yourself
- Rough translation for understanding, not for publishing
All of these share a pattern: the local model output is the input to the next stage, not the finished product. The fast path below turns exactly that pattern into a workflow.
The fast path, with AI
Someone who uses a local model seriously does not fill the big model place with it; they put it <em>in front of</em> the big model. The bulky mechanical work happens on your machine, and only the small cleaned result goes to the strong model for judgment. Two things improve at once: the cost drops and what leaves the machine is no longer raw client data. If your question so far has been which model is better, the better question is which stage runs where.
- Cut the job in front of you in two: the mechanical half and the judgment half. Stripping names, pulling fields, making the format uniform, all mechanical. Deciding tone, whether a claim is right, and what to drop, that is judgment.
- Run the mechanical half on the local model. A small model is enough here, because the rule is fixed and no creativity is needed. The recipe below does exactly this stage.
- Read the output <strong>yourself</strong>, once, and make sure nothing was dropped. A small model silently loses rows and that is the most common failure of this method. If the input was a list, the number of input and output rows must match.
- Now hand only that cleaned result to a strong model for the judgment work. The current pick among the strong models is kept, with its date, in <a class="text-link" href="/en/ai/">our AI section</a>.
Copy-ready recipe
# Stage one, on your own machine
ollama pull gemma3:4b
ollama run gemma3:4b
# then give it this, together with your file contents:
Clean the text below and do nothing else.
Rules:
1. Replace every person name with [NAME].
2. Replace every phone number with [PHONE], every email with [EMAIL], every card or account number with [NUMBER].
3. Replace every company or brand name with [COMPANY].
4. Leave the rest of the text alone. Do not rewrite sentences and do not summarise.
5. The number of output lines must exactly match the number of input lines.
6. At the end, on a separate line, say how many of each label you replaced.
Text:
{file contents}
# Stage two, with only this cleaned output, on a strong model
# Ask for the judgment work there: analysis, rewriting, decisionsBefore you trust the output: Rule six is not there for decoration: the count is the only thing that shows you the model missed something. A small model does not find a pattern you wrote no rule for, and national ID numbers, case numbers and addresses slip through exactly there. So before you send the output anywhere else, look through it once yourself for what is still inside. And obviously this method keeps the text on your machine but does not take responsibility for it: if it is client data, you also need permission.
AI in this kind of work
Here the subject of the lesson is itself an AI tool, so the discussion changes: the question is not which assistant helps you, it is which job runs where. Our position is that for anything with sensitive data, or with high volume and low judgment, reach for the local model first and keep the big model for the stage that genuinely needs thought.
Tools that actually help
- Ollama The simplest way to run a model on a personal machine. Its documentation supports macOS, Windows and Linux and asks for no account for local use; an account is needed only for the cloud models, which run on their own servers.
- Claude For the judgment half of the work, after the local model has cleaned the text. Iran is not on Anthropic supported countries list, which we read on their own page.
- Gemini The same role, from the Google family. Google own page says the Gemini web app runs in over 230 countries, and Iran is not on that list.
Where it backfires
There are two specific risks here and both are the same kind: believing something is local when it is not. First, the cloud label. Ollama own documentation says cloud models are automatically offloaded to their cloud service and require an account on ollama.com. So the same window and the same command can send your text out, purely because the model name ends in cloud. Before you type anything sensitive, look at exactly which model came up. Second, where the file came from. The model and the program that runs it are both executables, and if they are downloaded from an unknown mirror you have run an unverified program with full access on your own machine; the rule from safe software installation applies here and the official source is the only source. And let us clear up a common confusion: local means the text goes nowhere, not that the answer is right. A small model invents more than a big one, and there is nobody there to stop it.
Sources: Ollama docs: cloud models Ollama docs: quickstart Anthropic: supported countries Google: where the Gemini web app is available
Where this advice stops
All of this lesson is about text models on a personal machine. Image and video generation have a different memory profile and most of them want a dedicated graphics card, so the numbers on this page do not apply to them. The numbers are also a single day snapshot: model sizes change with every release and we read only four sizes of one family, on the date printed at the bottom of this page. And one thing that should be explicit: we have not run this route on this server; it does another job here and Ollama is not installed on it.
From our own work
The server that hands you this page, running seven sites, has about 7.6 GB of RAM in total, and at the moment we looked while writing this lesson about 2.7 GB of it was free. The gemma3:12b file alone is 8.1 GB. So this machine, which does serious work and is on around the clock, cannot even open that model in memory. We do not write this to discourage you; we write it so you know the right number to compare against is free RAM, not the RAM printed on the spec sheet. Open the task manager or system monitor on your own machine right now and look at the free RAM; that number decides which row of the table above concerns you.
Real follow-up questions
Does it work without a graphics card?
With small models yes, but slower, because the work runs on the processor. We have not measured that speed here and we will not invent a number. The Ollama documentation ties the default context length to graphics memory and gives 4k tokens below 24 GiB, which means on a machine without a card you should not expect long texts.
Does it handle Persian well?
It is uneven and we have not measured it, so an answer worth having will not come from us. Give one real paragraph of your own work to two or three models and put the outputs side by side. It takes ten minutes and the answer is yours, not somebody else claim.
Are local models and open models the same thing?
No. Local says where it runs; open says under what licence the weights were published. Most models that run locally do have published weights, but the two say different things and each model licence is read on its own. The open models lesson unpacks that.