4B, 70B, 397B-A17B. What the numbers on open AI models really mean
Anyone procuring open models runs into numbers first. They decide the hardware, the cost, and whether a model can handle the job at all. An explanation without jargon, with the examples worked through.
In short
- The parameter count is a size, not a grade. It says how much room the model needs in graphics memory, not how well it does your job.
- With '397B-A17B', both numbers count. The first sets the memory, the second the speed. A model can be huge and still answer quickly.
- German costs about twice as many tokens as English. Measured: the same administrative sentence takes 32 tokens in German and 15 in English. That feeds straight into price and context length. Worked through in the section on tokens.
In every procurement discussion about open models, sooner or later somebody says something like 'shall we take the 70B, or will the 14B do?'. Behind that sits the assumption that the number is a kind of performance class, the way horsepower is for a car. That is only partly true, and the part that is not true costs the most money in practice.
What a parameter is
A language model consists of a very large number of adjusted values. These values are called parameters. Think of them as dials that were set during training so that the model produces a fitting continuation for a given text.
'B' stands for billion. A 14B model has around 14 billion such dials, a 70B model 70 billion. More parameters mean, to begin with, only one thing: more room for what has been learned. It does not automatically mean the model does your job better.
The parameter count is a size, not a grade. A well-trained 30B model beats a badly trained 100B model on many tasks. What the number does predict reliably is the hardware you need.
What the number means for the hardware
This is where the parameter count gets concrete. Every parameter takes up room in graphics memory, and how much depends on the number format. At the original precision a parameter takes two bytes. Convert the model to a coarser scale, which is what 'quantisation' means, and it gets smaller: one byte at 8-bit, half a byte at 4-bit.
70 billion × 2 bytes. Does not fit on any single graphics card.
Halved. Runs on an 80 GB card, but with no headroom.
Halved again. Runs on a 48 GB card, at some cost in quality.
Quantisation is therefore the lever that makes a model fit on affordable hardware. It is not free: the coarser the scale, the sooner the model slips on tasks that demand precision, arithmetic for instance, or long chains of reasoning. On text summaries the difference is barely noticeable; on code generation it very much is.
On the arithmetic: in operation, another 10 to 20 per cent comes on top of the values above for what is called the context memory, where the model holds the conversation under way. The longer the inputs, the larger that share. The table further down allows for it.
Why '397B-A17B' carries two numbers
Current models often carry a second number after the hyphen, Qwen3.5-397B-A17B for instance, or Nemotron-3-Nano-30B-A3B. The 'A' stands for active.
Models like these are divided internally into specialist areas, 'experts' in the jargon. Only some of them are switched on for any one request. In 397B-A17B there are 397 billion parameters present, but only 17 billion do the work on each request.
From that follows a rule for procurement that is often missed:
The first number sets the memory, the second the speed. All the experts have to be loaded, including the ones not currently computing. A 397B-A17B model needs the memory of a 397B model and answers about as fast as a 17B model.
That is why models like these look cheap at cloud providers and still cost a lot in your own data centre. The provider spreads the memory across many customers; you would have it to yourself.
What each size class can do
The classification below names, for each class, a model that a Swiss provider actually runs today, together with the memory it needs and the typical limit. The assignment is a rule of thumb, not a benchmark.
The size classes at a glance
Classifying, pulling out keywords, finding similarities. Runs on a laptop. Too weak for free-form writing.
Summarising, drafts, standard answers. Covers a large part of administrative work. One card is enough.
Longer lines of argument, specialist language, multilingual documents. From here on the hardware becomes a cost factor.
Complex code work, difficult analyses. Running it yourself takes several cards. Rarely the first choice.
| Model | Total | Active | Memory at 4-bit | Fits on | Typical task |
|---|---|---|---|---|---|
| Qwen3-Embedding-8B | 8B | – | 4 GB | a laptop | search, comparing documents |
| Ministral-3-14B | 14B | – | 7 GB | one 24 GB card | summarising, drafts |
| Nemotron-3-Nano-30B-A3B | 30B | 3B | 15 GB | one 24 GB card | quick answers in dialogue |
| Gemma-4-31B | 31B | – | 16 GB | one 24 GB card | prose, translation |
| Apertus-v1.5-70B | 70B | – | 35 GB | one 48 GB card | specialist texts, multilingual |
| Qwen3.5-122B-A10B | 122B | 10B | 61 GB | one 80 GB card | demanding analyses |
| Qwen3.5-397B-A17B | 397B | 17B | 198 GB | three cards | code, complex tasks |
Memory = parameters × 0.5 bytes at 4-bit quantisation. 'Fits on' allows 15 per cent headroom for the context memory. Model selection: Infomaniak's catalogue, as at 7 August 2026. The 'Typical task' column is an editorial judgement, not a measurement.
The Apertus row is the notable one. Apertus is a model developed at ETH Zurich and EPFL whose training data is disclosed. For offices that need to know not only where a model runs but also what it was trained on, that is a difference no parameter count expresses.
Tokens: the unit you are billed in
Models do not read letters and do not read words, they read tokens. A token is a fragment of text, sometimes a whole word, sometimes a syllable, sometimes a single character. Two things follow from it: the price, and how much text fits in at once.
How many tokens a text comes to can be measured. We put four samples of text through the Qwen2.5 tokeniser:
| Text | Characters | Tokens | Characters per token |
|---|---|---|---|
| 'Die Einwohnerkontrolle prüft die Niederlassungsbewilligung und erteilt eine Aufenthaltsbestätigung.' | 99 | 32 | 3.1 |
| the same content in English | 92 | 15 | 6.1 |
| 'Krankenversicherungsprämienverbilligung, Grundstückgewinnsteuerveranlagung, Arbeitslosenversicherungsbeitrag.' | 109 | 33 | 3.3 |
| one A4 page of administrative German | 1,600 | around 500 | 3.2 |
Measured on 7 August 2026 with the Qwen2.5 tokeniser. Checked against GPT-NeoX and GPT-2, where German comes out slightly worse still at 2.8 to 2.9 characters per token. The three monster compounds in row three are three words and come to 33 tokens.
German costs about twice as many tokens as English. The same content, 32 tokens against 15. Anyone taking prices or context lengths from English-language sources is budgeting systematically too low for German-language operation.
The reason lies in how tokenisers are built: they were developed largely on English text. German compounds such as 'Niederlassungsbewilligung' rarely appear there and are therefore broken into many small pieces, while English words mostly pass through whole.
What that means in practice
Two consequences that are regularly missing from procurement papers. First, on price: an offer at 'CHF 0.20 per million tokens' sounds the same whatever the language. For the same amount of content you pay roughly double in German.
Second, on the context window. When a model states '128k context', that means 128,000 tokens. In German that comes to around 256 A4 pages, in English to almost twice as many. Anyone choosing a model because a particular case file has to fit must work with the German figure.
That lets you convert any offer into pages.
In English it would be almost twice as many.
Applies to price and context alike.
What to watch for
The numbers are useful, but they do not answer the question that counts: can the model handle the job in hand? Only a test with your own material answers that.
What we recommend
Start in the 14B to 31B class. It covers summarising, drafts and standard answers, runs on a single graphics card and costs little in the cloud. Only when a specific case defeats it is the step up worth taking.
For offices with documentation duties, Apertus is worth a closer look in its own right: the model comes from ETH Zurich and EPFL, and the training data is disclosed. That lets you document not only where it runs but also what it learned from. Our comparison matrix sets out where it stands.
And whatever the size: test with ten real files from your own organisation before you decide. That costs half a day and answers more than any table of parameters.
When reading AI model specifications
- With two numbers such as 122B-A10B, always take the first for the memory and the second for the speed. Offers that name only the smaller one are hiding the hardware requirement.
- Ask which number format the stated memory requirement applies to. 'Runs on one card' is often true at 4-bit and almost never at full precision.
- Convert context lengths and prices for German rather than taking them from English data sheets. Rule of thumb: 500 tokens per A4 page.
- Do not read the parameter count as a ranking. Two models of the same size can differ considerably in practice.
- Test with your own documents before deciding, not with the provider's examples.
Open questions
Not sure which class covers your case?
If you are facing a decision like this, or want to try a model on your own material: in an initial conversation we look at your particular case and work the order of magnitude through together. No sales pressure.
Sources
- Our own research. Model catalogue and prices, Infomaniak AI Services, retrieved on 7 August 2026 · infomaniak.com
- Switzerland. Apertus, open language model from ETH Zurich, EPFL and CSCS · huggingface.co/swiss-ai
- Our own measurement. Token counts, evaluated with the tokenisers of Qwen2.5, GPT-NeoX-20B and GPT-2 via the transformers library, 7 August 2026
- Our own calculation. Memory requirement, parameters × bytes per number format, plus 15 per cent for the context memory
- Our own work. digitario, comparison matrix of AI coding tools · digitario.ch/ki-coding-tools-vergleich
Advises public administrations and companies on procuring and introducing AI systems. Maintains the comparison tables on this site.