Gander Takes the Floor on Time in All 100 Scenarios and Interrupts Only 8.0%, While Scoring Last on Task Accuracy

Gander is a full-duplex multimodal interaction model released on September 8, 2026 as arXiv 2609.08977 by Tencent's Hunyuan Speech Team with Zhejiang University, Shanghai Jiao Tong University, CUHK, and NTU, and it listens and speaks at the same time rather than taking strict turns. The 9B interaction model is built on MiniCPM-o 4.5, and on all 100 scenarios of Full-Duplex-Bench v3 it began speaking at an appropriate moment every time, cutting in before the user finished in only 8.0% of turns against 13.5% for GPT-Realtime. Its tool-selection F1 of 0.759 is the lowest of the seven systems in the table. ASAP reads this paper at the point where conversational timing and task accuracy trade against each other.

A cerebellum and a brain, bolted together

The core of Gander's design is a division of labor that assigns realtime response and complex reasoning to different components. In what the paper calls the Cerebellum-Brain collaborative framework, the Cerebellum handles realtime interaction while the Brain handles complex reasoning and longer-horizon agentic tasks, and the two communicate continuously through tool calling and an agent orchestration runtime. The Cerebellum is built on a streaming Thinker-Talker architecture in which user inputs and model outputs are flattened into an ordered token stream at the chunk level. Within each chunk the model explicitly predicts whether to listen or speak, learning to control its own interaction state.

The paper's justification for splitting them is a difference in requirements. Casual conversation favors immediate responses and continuous contextual adaptation, while complex workflows involve longer-horizon reasoning, iterative planning, tool use, and sustained execution, and handling both inside one monolithic model creates problems. The orchestration runtime reflects that split in its own vocabulary, managing five entities: Project, Task, Run, WorkerEvent, and Delivery.

One detail of the evaluation setup deserves attention up front. Where the back brain participates, it is instantiated training-free through the Codex worker provider described in Section 3 and driven by GPT-5.6. The 9B model the team releases is the Cerebellum, and the tool-use figures in the table include the performance of an external commercial model wherever the brain took part. The researchers also found an asymmetry in the protocol, since the benchmark's system prompt never reaches the brain as an external agent, and corrected it by placing a contract file carrying the same requirement in the brain's working directory.

Solving the problem of speaking on time without cutting in

On the timing metrics Gander posts the best figures in the table. Full-Duplex-Bench v3 places a spoken assistant in 100 tool-augmented service scenarios containing natural disfluencies and measures the fraction of turns taken at an appropriate moment, the fraction begun before the user has finished, and the fraction prefaced by a conversational placeholder. Gander took the floor at the right moment in all 100 scenarios and began prematurely in 8.0% of turns. GPT-Realtime records 96.0% and 13.5%, Gemini Live 3.1 records 78.0% and 19.2%, and a cascaded Whisper-to-GPT-4o-to-TTS pipeline records 100.0% and 33.0%.

The paper itself explains why the two numbers must be read together. A system can trivially suppress interruptions by waiting longer, at the cost of missing its turn altogether, and both failure modes appear in the table. Gemini Live 3.1 holds interruptions to 19.2% but answers in only 78.0% of scenarios, while the cascaded pipeline answers every time and interrupts 33.0% of the time. Gander is the only row that sits at the top of both columns.

The measurement conditions are not lenient either. The team ran the released scoring scripts unmodified with the GPT-4o judge from the baseline protocol, and evaluated the deployed stack end to end: audio enters the Thinker, the Talker and speech decoder synthesize a reply, and scoring reads an ASR transcript of that audio. Every reported number therefore carries real speech-synthesis and recognition error. That is a reason not to line these figures up beside text-level evaluations, and a reason they sit closer to real deployment conditions.

What it gives up: task accuracy and filler words

The ordering on the accuracy metrics is the exact reverse of the timing table, and Gander sits at the bottom of every one of the four columns. Gander scores 0.759 on tool-selection F1, 0.503 on argument accuracy, 0.490 on response quality, and 0.400 on the strict Pass@1 that requires the tool multiset to be exactly right with every argument correct. GPT-Realtime at the top of the table posts 0.876, 0.680, 0.792, and 0.600, and even Ultravox v0.7, the weakest baseline, posts 0.794, 0.513, 0.510, and 0.410. The paper notes the gap is narrow at the lower end, and the Pass@1 difference is indeed 0.010, but last place across all four columns is still last place.

The filler figure needs separate handling. Gander sits at 51.6%, second only to Ultravox v0.7 at 88.0% and far above Gemini Live 2.5 at 8.9% or GPT-Realtime at 16.9%. The metric is defined over the 91 scenarios in which the system takes the turn without interrupting. More than half of the turns it started on time therefore opened with a conversational placeholder.

Stacked together, the three numbers describe one behavior. Taking the floor on time in 100.0% of scenarios, cutting in only 8.0%, and opening with a placeholder 51.6% of the time is the shape of a familiar human strategy: wait without interrupting, start making sound as soon as the turn arrives, and let the content catch up. It implies that deciding whether the other party has finished and preparing what to say are separate problems, and in that arrangement timing scores and content scores have no reason to rise together. The paper does not claim this causal story, and the reading above is ASAP's, drawn from placing the three metrics side by side.

The table prints what the speech path costs

The most informative row in this paper is its last one. The researchers separately report a back-brain-only condition that bypasses the front cerebellum and the audio path entirely, driving the same agent and tool server from a transcript of the user turn. That condition scores 0.934 on tool-selection F1, the highest in the table, with 0.590 argument accuracy, 0.740 response quality, and 0.520 Pass@1, against 0.759, 0.503, 0.490, and 0.400 for the full stack.

Reading that contrast honestly requires the paper's caveat. The authors mark this row as text-driven and comparable to the cascaded pipeline rather than to the full-duplex rows, and note that the three interaction metrics cannot be measured without an audio timeline. The drop from 0.934 to 0.759 is therefore not the amount the cerebellum degrades the brain; it is the cost imposed by the realtime speech condition as a whole, which mixes together synthesis and recognition error, the constraint of fixing intent from chunk-wise streaming input, and the burden of judging while speaking.

Turned the other way, that row is evidence for the architecture. Dropping GPT-5.6 into the brain slot training-free to reach 0.934 and then running the same configuration through the speech path shows that the brain really does behave as a swappable part. Raise reasoning quality by changing the brain; raise interaction quality by training the cerebellum. The catch is that the upper tier's skill does not arrive intact at the bottom, which is why effective feedback from the Cerebellum to the Brain appears on the authors' own list of open problems.

Going duplex also cost understanding accuracy

Comparison with the base model is where the price of duplex adaptation becomes visible, because Gander scores below MiniCPM-o 4.5 on both understanding benchmarks. MiniCPM-o 4.5, the model Gander is built on, scores 55.70% on WorldSense and 80.20% on Daily-Omni, while Gander scores 49.62% and 78.53%, a drop of 6.08 points on WorldSense. Layering chunk-wise streaming and a listen-or-speak decision on top for the sake of duplex interaction costs some pure multimodal understanding. For reference, the same table lists Gemini 2.5 Flash at 52.60% and 79.30% and Qwen3-Omni at 54.00% and 70.70%.

The value of holding both streams was measured too. Table 6 runs the same question sets under audio-visual, video-only, and audio-only conditions for 13,107 inferences in total. On WorldSense the audio-visual condition reaches 49.62% against 44.61% for video alone and 43.32% for audio alone, a fusion gain of 5.01 points. On Daily-Omni the spread is 78.53% against 59.40% and 57.81%, a gain of 19.13 points, and the overall gain is 8.88 points. A substantial share of these questions cannot be answered from one stream, which is the argument for keeping both inside the interaction model.

Spoken question answering splits by peer group. Among full-duplex streaming models, the 9B Gander records 75.60% on Llama Questions and 59.30% on Web Questions, ahead of Moshi at 62.20% and 26.30% and Audio-Interaction 3B at 67.31% and 54.34%. On VoiceBench, however, its AlpacaEval rating of 3.96 out of 5 trails Audio-Interaction's 4.28 and its SD-QA accuracy of 46.84% trails 52.14%. Widen the comparison to turn-based models and Qwen2.5-Omni 7B leads on Web Questions at 62.80% and SD-QA at 55.71%. Going full-duplex does not by itself improve speech understanding.

What matters for teams evaluating realtime voice agents

The practical differentiator here is what ships. GPT-Realtime and Gemini Live at the top of the table are commercial services reachable only through an API, while the researchers state that they release Gander together with its models, code, and data. For work where realtime voice has to run on owned infrastructure, such as call handling whose audio cannot leave the building, that condition outranks the leaderboard. The 9B size also makes it realistic to self-host the cerebellum while delegating the brain to an external model only when needed.

The second benefit of the split is substitutability. If the brain connects only through tool calls, the choice of reasoning model behind it belongs to whoever runs the system, leaving room to swap it for cost or compliance reasons. Commercial realtime voice APIs bundle conversational control and reasoning into one product, with no option to change only one half. As the 0.934 against 0.759 gap shows, though, plugging in a strong brain does not by itself determine final quality.

Anyone evaluating a non-English deployment should note one gap. The corpus breakdown for the 2.7 million training examples reports category shares, roughly 37% speech interaction, 41% audio-visual interaction, 13% agentic interaction, and 9% robustness and negative data, but it does not report language composition, and only the simultaneous-translation entry is described as bilingual. Whether the ability this paper leads with, judging when a speaker has finished, survives in another language cannot be read off the table and has to be tested directly. Endpoint judgment leans heavily on prosody and sentence-final morphology, so it carries more transfer risk than most other capabilities.

Five limitations from the authors, and one question beyond them

The conclusion of arXiv 2609.08977 lists five limitations and future directions, and the list maps the unfinished parts of this architecture. The first is data and model scaling, with the finding that agent invocation and conversational behavior stay sensitive to the training data distribution, especially in complex multimodal scenarios. The second is stable post-training, since the current model has not been extensively optimized with on-policy distillation or reinforcement learning. The third is further exploration of the collaborative structure, including feedback from the Cerebellum to the Brain, and the fourth is memory and long-context management across long multimodal histories, tool calls, and evolving task states.

The fifth reframes the whole paper. Existing benchmarks evaluate multimodal understanding, duplex interaction, and agentic execution in isolation, leaving the unified duplex interaction agent setting insufficiently evaluated, and, as the authors write, they do not adequately capture Brain-Cerebellum communication and coordination either. The central claim of this work is collaboration between two tiers, and no instrument yet measures that collaboration directly, which is why the last row of Table 3 had to be constructed and reported separately.

One more item belongs on the list. The evaluation covers 100 scenarios. The 100.0% appropriate turn-taking and 8.0% interruption figures come from those 100 scenarios, and the filler metric is defined over 91 of them. Timing in a realtime conversational system swings with a speaker's habits, background noise, and network delay, and how much of a perfect score over 100 scenarios survives in deployment cannot be read from this table. The team states that Gander supports background noise, multi-party interaction, and backchannel communication without placing quantitative results for those conditions in the same table, which points the same way.

Sources: arXiv 2609.08977, "Multimodal Duplex Interaction Agent" (submitted September 8, 2026, v3 September 12, 2026, Tencent Hunyuan Speech Team et al.), drawing on the abstract, Section 5, Tables 2 through 6, and the limitations in the conclusion. Code is published at github.com/Omni-Interaction-Gander/Omni-Interaction-Agent.