Video formats and codecs
A container and a codec are two different things: MP4 is the name of the box, H.264 is the name of the method the picture was compressed with. That one distinction settles most of the "why will this file not play" problems, because a player has to recognise both the box and the codec inside it.
- Lesson 3 of 10
- Beginner
- Free, no signup
One video file, four layers
From what you see down to the file on disk. Each layer is a separate decision, and the bottom two are the ones people mix up.
- Resolution and frame rateHow many pixels per frame and how many frames per second. This is what the eye sees. 1080p or 4K
- BitrateHow much data is spent per second. A higher resolution without a higher bitrate means a worse picture. fixed or constant quality
- CodecThe compression method. Licensing and hardware support are decided right here. H.264, H.265, VP9, AV1
- ContainerThe file on disk that keeps everything together. Changing it usually needs no re-encode. MP4, MKV, WebM
These four layers explain file size and compatibility, not the quality of the footage. A badly lit scene is not fixed by any combination of them.
Last checked: Facts and tool names in this lesson are re-checked against their sources on this date.
Container or codec: which is which
A codec is the method by which picture and sound are compressed and decompressed again, and a container is the file that keeps those compressed streams together: the video, one or more audio tracks, subtitles, and information such as duration and frame rate. MDN's container guide says exactly this in its first sentence: a container encapsulates one or more media streams along with metadata, so they can be stored and played back together.
So MP4 is the name of a box and H.264 is the name of a codec. Asking someone to "send me an MP4" says almost nothing about the file, because that same box can hold H.264 or AV1, and those two have very different fates on half the devices in the world.
Here is the practical consequence: for a file to play, the player has to know both. A browser that opens the MP4 and does not recognise the codec inside will open it and show you a black frame. The reverse happens too: a codec the browser knows, sitting in a box the browser will not play, is just as useless. Once you know this, diagnosing a broken file becomes two simple questions instead of one guess.
Why playback depends on both
Container
The box: MP4, MKV, WebM
- Keeps video, audio and subtitles together
- Carries duration and frame rate in its metadata
- Each box accepts only a short list of codecs
Codec
The method: H.264, H.265, VP9, AV1
- Compresses the picture and decompresses it again
- Licensing and royalties belong to this layer
- A hardware decoder either knows this or does not
The player has to know both; one is not enough.
The player or browser
Where the real decision is made
- Does not know the box: the file will not open at all
- Does not know the codec: the file opens and the picture is black
- Knows both but has no hardware for it: it plays and stutters
This diagram tells you whether the file plays. It says nothing about whether it looks good.
The three boxes you actually meet
MP4 is the delivery box. Practically every device, browser and platform accepts it, which is why it is what you hand a client and what you upload to social media. MDN says the same thing: if you can only provide one format, provide MP4.
MKV is the working box. It accepts nearly any codec, holds several audio tracks and several subtitle tracks without complaint, and is popular for archives and downloads. Browsers do not play it, and that is not a flaw in MKV; MKV was not built for the web.
WebM is the web box, and deliberately undemanding: MDN's list gives it only AV1, VP8 and VP9 for video and Opus and Vorbis for audio. That restriction is exactly what becomes useful later in this lesson.
There is also MOV, Apple's box, which you meet constantly in editing. For delivery to the outside world it usually gets turned into an MP4 first, and that conversion is almost always just a change of box rather than a re-encode. That distinction is the one we put a number on in the section on our own experience.
The four codecs you see everywhere
The columns below are read from MDN's web video codec guide, not from personal experience, because these are facts and they have a shelf life.
| Codec | Licensing | Browser support | Usual container | Compression claim |
|---|---|---|---|---|
| H.264 (AVC) | Proprietary and patented; commercial use requires a licence | All browsers | MP4, 3GP | The baseline |
| H.265 (HEVC) | Proprietary, multiple patent pools; fees are charged to developers rather than to content producers | Depends on the operating system and a hardware decoder | MP4, MPEG-TS, QuickTime | Theoretically half the size of AVC at comparable quality |
| VP9 | Open and free of royalties | All browsers, no alpha transparency in Safari | WebM, MP4, Ogg | Quality comparable to HEVC at similar bit rates |
| AV1 | Royalty-free open standard | All browsers; in Safari only on devices with a hardware decoder | WebM, MP4, MPEG-TS | Better than VP9 and HEVC, and as much as 50% better than AVC |
One thing about that last column that is easy to misread: these are compression-rate claims, not a promise that your file will halve. The real result depends on the footage itself, on the encoder settings, and on how long you are willing to wait.
And the Safari cell for AV1 is the one that bites on a real project. MDN writes that Safari support is limited to devices that have a hardware decoder, meaning M3 MacBooks and later, the iPhone 15 Pro, and the iPhone 16 and later. "AV1 is in every browser" and "AV1 plays on my client's iPhone" are two different sentences.
How bitrate differs from resolution
Resolution is the number of pixels in each frame, such as 1080p or 4K. Bitrate is how much data the codec is allowed to spend per second. The two are independent, and this is where most bad exports get made.
Raise the resolution while keeping the bitrate fixed and the picture gets worse, not better: the same amount of data now has to be spread across more pixels. A 4K export at a bitrate that was set for 1080p is a bigger file with a softer, blotchier picture. You can see it anywhere, and most obviously in scenes where something moves fast.
In practice there are two ways to set the bitrate. One is to name a fixed number, which is what you do when the destination has a ceiling. The other is constant quality, set with CRF in ffmpeg: you tell the encoder to hold the quality and spend whatever bits that takes. For a file that is going to sit on a website, constant quality is almost always the better choice, because a still scene takes fewer bits and a busy scene takes more, which is exactly what the eye expects.
Why the file will not open on your client's phone
Three things answer this question and all three come back to the codec, not the box.
First, hardware. Most devices decode H.264 in a dedicated chip, which is why playing it stays cool and cheap. Newer codecs want newer chips, and on an older device they fall back to software, meaning the CPU does the work: the battery drains faster and, on a weak device, frames drop. MDN attaches that same hardware caveat to both AV1 and HEVC.
Second, licensing. H.264 and H.265 are patented and AV1 and VP9 are not, and that alone explains why each browser ships what it ships. MDN adds that the Via LA patent pool does not require licence fees for streaming internet video in AVC as long as the video is free to the end user, and that HEVC fees are charged to developers rather than to content producers and distributors. That is why HEVC support in browsers is so patchy.
Third, the box itself. A browser will not play MKV even if it knows the codec inside. And here is a real trap: WebM does not accept an H.264 stream at all, so "convert the MP4 to WebM" is not possible without changing the codec. You can test this, and in the section on our own experience we did.
So the diagnostic order is: find out which codec is inside the file, check whether the destination accepts that codec, and only then look at the container. Renaming the file extension has never been the answer and still is not.
Which combination to pick on export
There are three destinations and each has its own answer. The answers fall out of what you have already read rather than from an arbitrary list.
Delivering to a client or uploading to social media: H.264 with AAC audio in MP4. You do not control the other end here, so compatibility beats file size. MDN names exactly that combination as the safe option, with a reminder that verifying your compliance with the licence terms is your own job.
Playing on your own site: AV1 or VP9 with Opus audio in WebM, plus an MP4 as the fallback. The video tag lets you list several sources and the browser takes the first one it recognises. MDN's own recommendation is those two sources in that order: WebM with AV1 and Opus first, then MP4.
Archive and master: whatever came out of the camera or out of the edit, untouched. Take every new export from the master and never from a previous export. Compress a lossy file again and the discarding step runs a second time, this time on something that has already been discarded from. It is the one rule in this lesson with no exceptions.
And one thing you will not see on this page: a number telling you how much converting to AV1 saves. We ran that test on this server, but our test clip was a synthetic pattern, and a compression ratio measured on a synthetic pattern does not transfer to real footage. Printing no number is better than printing one that will not hold for you.
One question that settles the export
Do you decide where this file gets played, or does someone else?
Compatibility first: H.264 and AAC in MP4
- Client delivery, social upload, an email attachment
- Every device decodes it, usually in hardware
- The file is bigger, and that is the price of certainty
Smaller: AV1 or VP9 and Opus in WebM
- Video on your own site, with a video tag and several sources
- Always add an MP4 as the fallback as well
- Encoding is slower; budget for that in the schedule
This fork is about delivery, not archiving. The master always stays untouched and every new export comes from the master itself.
The fast path, with AI
The usual route is to open the file in an application and export it again. The fast route starts with a different question: does anything need re-encoding at all? Half the jobs called "video conversion" are just a change of box, and changing the box costs neither quality nor time. A language model does two things well here: reading a raw ffprobe dump and writing the ffmpeg command line, which is unfriendly enough that people avoid it. A fast, cheap model of the Flash class is enough for this, and our current pick is listed in this site's AI section.
- First find out what is inside the file, not what its extension says. One line does it: ffprobe -v error -show_entries stream=codec_type,codec_name,profile,width,height,r_frame_rate,bit_rate:format=format_name,duration,size -of default=nw=1 input.mp4
- State the destination and its constraints explicitly. Without them the model will opine about codecs while knowing nothing about the size ceiling or the accepted resolution, and its answer turns generic.
- Fill in the recipe below and run it. The whole point is its second clause: if only the box is wrong, the answer must be one command with c copy and stop there.
- After running it, check two things. If you expected only the box to change, the video stream must be byte for byte the same; ffmpeg -i out -map 0:v -c copy -f md5 - on both files proves it. And if it really was re-encoded, watch the output on the very device that has to play it.
Copy-ready recipe
Role: video engineer. Your job is to decide the container and the codec, not to give general advice. Do not produce any number that is not in the ffprobe output or in the destination spec.
Destination for this file:
{e.g. playing on our own site with a video tag | uploading to Instagram | delivering to a client | archive}
Constraints the destination has published:
{e.g. size ceiling, maximum resolution, accepted codecs. Write "not published" for anything that is not}
ffprobe output:
{paste the output here}
Answer in this order:
1. What is inside this file: container, video codec, audio codec, resolution, frame rate, duration.
2. Does this destination require a re-encode at all? If only the container is wrong, give the c copy remux command and stop there.
3. If the codec must change, give exactly one ffmpeg command and, under it, one line saying what each option does.
4. If the destination has not published something, write "not published" and propose the safe option. Do not guess.
5. Finish by saying what I must check with my own eyes after running it.Before you trust the output: The model cannot see your picture. It can tell you the file is H.264 sitting in an MP4; it cannot tell you the export looks bad. So the fifth clause of the recipe is not a formality: always check quality yourself, on the destination device. And there is one job never to hand it, described in the risk section below.
AI in this kind of work
Choosing a codec is not a task where AI adds anything new; the rules are written down and they are on this page. Where a model genuinely frees up time is the command line: ffmpeg has more than a hundred options and nobody has them memorised. And where it all goes wrong is the "AI enhancement" tools.
Tools that actually help
- Claude For writing the ffmpeg command line and explaining each option, because the explanation can be argued with and checked against the documentation. Iran is not on Anthropic's supported-countries list and there is no official payment route from Iran.
- Gemini For the mechanical pass over the ffprobe output of a whole folder, on its fast cheap model. Google's own page says the Gemini web app runs in over 230 countries and territories, and Iran is not on that list.
- FFmpeg Not an AI tool, and deliberately on this list. The genuinely repetitive part of this lesson is done by one command line and needs no model at all. The model decides and proposes, the tool executes.
- RGB The access and payment layer for Iran, kept separate from the tools themselves.
Where it backfires
Three risks, and the first is the serious one. Video enhancement tools do not restore detail that was never recorded in the file; they invent plausible detail. For a teaser or decorative work that hardly matters, but when the video is meant to be evidence, meaning an interview, a client testimonial, a security camera frame, the "enhanced" version is no longer what was recorded, and presenting it in place of the source is a misrepresentation. Second risk: re-encoding does not remove an AI watermark. Google writes of SynthID that its watermark is designed to stand up to cropping, filters, frame rate changes and lossy compression, and that it runs its own detector. The third risk is smaller but daily: do not ask a model which browser supports which codec today. It will answer, the answer will look confident, and it comes from training data whose date it does not tell you. Read that from MDN or caniuse, not from a model and not from the page you are reading.
Sources: MDN: Web video codec guide MDN: Media container formats Google DeepMind: SynthID
Where this advice stops
The codec is a small lever. A file with four times the resolution of the slot it plays in is the wrong file whichever codec you built it with; duration and resolution beat codec choice almost every time. The browser support column has a shelf life too: everything in this page's table was read from MDN on the date printed at the top, and the right way to check today is to open that guide or caniuse, not to trust this page. And there is one thing no page can tell you: what your client's hardware decoder actually recognises. The only real test is playing that file on that device.
From our own work
We measured the claim that "the container is only a box" on this server, with ffmpeg 6.1.1, on the date printed at the top of this page. We made a ten second 1280x720 H.264 clip and then poured it into two other containers without re-encoding anything. The three files came out at 3776719, 3775504 and 3776666 bytes, meaning all three land within less than a kilobyte of each other, and the whole job took under a tenth of a second. More telling than the size is this: the md5 of the video stream itself is exactly the same in all three files, 93dc8bd0df89a3adff8aa9233b8bbb46. Not one byte of the picture changed; only the packaging did. Then we poured the same stream into WebM with the same command and ffmpeg refused: Could not write header (incorrect codec parameters ?). Those two results together are what this lesson opened with.
Real follow-up questions
Is MP4 better than MKV?
They are two different jobs. MKV holds nearly any codec and any number of audio and subtitle tracks, which is good for archives and downloads; browsers do not play it. MP4 is the delivery format, because it opens everywhere. Converting one to the other is usually just a change of box and costs no quality.
Why did my video get bigger after converting it?
Because an already compressed file was encoded again, this time at a higher bitrate. The encoder does not know how much quality the input had; it only honours the number you gave it. The fix is to export from the master rather than from a previous export.
Should I use AV1 now?
On your own site yes, provided you also put an MP4 there as the fallback. It has two costs worth knowing: encoding is noticeably slower, and in Safari it only works on devices with an AV1 hardware decoder. For a file you hand to a client, H.264 is still the lower friction choice.