The frontier that matters is no longer model size
Published on · LINAGORA
The thesis
Ranking models by parameter count no longer has any operational value. What decides a project is the cost per million tokens at a given level of performance, measured on your own use cases. That figure appears in no public leaderboard, because it depends on your data.
Parameter count has stopped being information
For three years, the size of a model served as a proxy for capability. That period is over. Training techniques, corpus quality and partially activated architectures have broken the relationship between parameter count and useful performance.
A compact, well-trained model now beats models ten times its size published eighteen months earlier on specific tasks. Conversely, a very large model remains superior on long reasoning and difficult synthesis. Both statements are true, and that is exactly why the generic leaderboard is of no use to you: it averages situations that have nothing to do with yours.
The only question that decides is this one: what is the smallest model that meets my requirements on my real use cases, and what does it cost to run?
An open model almost nobody can run
A model with 2.8 trillion parameters can be open in the sense of its weights, publicly downloadable, under a permissive licence, and still be unreachable in practice. If it requires 1.7 terabytes of graphics memory, or sixteen latest-generation accelerators to answer at an acceptable latency, then the model is open and its use is not.
This distinction is not a legal detail. It separates declarative openness from effective openness. A digital common that no mid-sized organisation can operate remains a useful research object, but it grounds no independence.
That is why the availability of mid-sized open models, operable on reasonable infrastructure, matters more to European autonomy than the race to the largest published model.
The bottleneck is almost never compute
Common intuition says the limit is compute power. In production practice it rarely is. The limiting factor is the memory available to hold weights and context, the bandwidth to fetch the right bytes at the right moment, and the fast storage feeding all of it.
This has direct consequences for sizing. A machine whose accelerators sit idle waiting on memory costs the same as a well-sized machine, for a fraction of the throughput. Many projects buy compute where they should have bought memory and bandwidth.
Sizing must therefore start from the real profile of your requests: context length, concurrency, latency tolerance. Three parameters that can be measured on your side within days, and that change the bill by an order of magnitude.
The factor of fifty
At comparable performance on real tasks, the output price gap between a very large proprietary model and an optimised compact model routinely reaches a factor of fifty. This is not marginal tuning, it is a difference in kind between two architectural decisions.
A gap of that size cannot be recovered through commercial negotiation. It is recovered upstream, at the moment you decide which model handles which type of request. Most production systems do not need the best available model for the vast majority of their calls: they need it for an identifiable fraction, and that fraction can be measured.
The model mix, orchestrated by the harness, is therefore first and foremost an economic decision. It is also why harness design and economic scoping should be conducted together rather than one after the other.
How to measure it on your own data
The protocol has four steps and does not require large means. Build an evaluation set from your real data, including the hard cases and the ambiguous ones, not only the demonstration cases. Define the success criterion with the business teams concerned, accepting that it will be partly subjective and making it reproducible.
Then measure several models against that set, recording success rate, latency and cost per request. Finally, compute the cost per successful request, which is the only relevant figure: a model that is half the price but fails one time in three costs more than one that succeeds.
This work takes two to three weeks. It prevents multi-year commitments made on the strength of a public leaderboard.
What this means for your organisation
Stop comparing models, compare model mixes on your own data. The result is almost always counter-intuitive, and almost always favours models smaller than the ones initially considered.
Size from memory and bandwidth, not from compute. It is the most frequent and most expensive source of error in scale-up projects.
Compute a three-year total cost of ownership for the three possible scenarios: buy and operate, self-host, hand over as a managed service. The cheapest scenario in year one is rarely the cheapest in year three.
Finally, reassess every six months. A factor of fifty does not stay stable in a market moving at this pace.
The related advisory module
Measure your cost per useful token before committing
Economic scoping measures that cost on your real use cases, compares several model mixes and sizes the matching infrastructure.