An analysis released by Ai2 (Allen Institute for AI) in June 2026 shows, through per-token loss gaps, that a hybrid language model predicts meaning-bearing content words better than a transformer but loses almost all of that edge on tokens that repeat earlier passages verbatim and on closing brackets. Comparing the 7B Olmo 3 (transformer) against Olmo Hybrid side by side, the study concludes that a single overall loss averaged across all tokens is too blunt to tell the two architectures apart.

A Single Average Loss Cannot Compare Two Architectures

Ai2's central claim is that a model's overall average loss, the average error across all tokens, is too blunt to compare a transformer with a hybrid. Even when two models look similar on a headline score, collapsing where each one wins and loses hides which underlying ability actually differs.

The method is to split tokens by type and measure the loss gap. The researchers grouped tokens into categories such as parts of speech, bracket types, and n-gram repetitions, then measured the difference in prediction loss between the two models per category.

Where the Hybrid Wins: Content Words

Olmo Hybrid leads the transformer on meaning-bearing content words, with a reported loss gap of about 0.04. In the same comparison, the gap on function words such as articles and prepositions is about 0.02, so the content-word advantage stands out more than the function-word one.

The direction this points is clear. The hybrid is stronger at predicting the next meaning-bearing word, which lines up with how recurrent layers track the state of the context.

Where the Advantage Disappears: Repetition and Closing Brackets

Olmo Hybrid's edge shrinks toward zero on tokens that repeat an earlier passage verbatim, and on closing braces or brackets the advantage vanishes entirely. The more a token has to copy prior text exactly, the smaller the hybrid's lead becomes.

The cause is that exact copying is a different kind of ability. Pulling a specific distant token forward and reproducing it verbatim is a strength preserved by transformer attention, and it works against a recurrent structure that compresses into a fixed-size state.

Why the Difference Arises: State Tracking vs. Exact Copying

Olmo Hybrid's strengths and weaknesses come from a design that combines the state-tracking ability of recurrent layers with attention. State tracking from recurrent layers helps with content prediction, while transformer attention keeps the upper hand on exact token retrieval and copying.

The same conclusion was checked at the 1B scale. Beyond the 7B Olmo 3 and Olmo Hybrid, the researchers analyzed three 1B models, transformer, hybrid, and pure recurrent, to confirm whether the per-architecture trade-offs hold across scale.

The Bottom Line

The lesson of Ai2's analysis is that architecture comparison should not end at a single score. Olmo Hybrid beats the transformer on content-word prediction but trails on exact copying and closing brackets, and a single average loss makes these opposing trade-offs cancel out of view. A practical implication follows: when choosing between a hybrid and a transformer, first ask whether the task is closer to meaning prediction or to exact copying.

---

Reference: Which tokens does a hybrid model predict better? (Ai2, 2026)