Google DeepMind's SL2T translates sign language straight to text: 70 BLEURT zero-shot on FLEURS-ASL, with only coordinates leaving the device

Google DeepMind announced SL2T, a sign-language-to-text translation model, on August 12, 2026, and shipped it into Gboard and Live Transcribe on Pixel 11 at no additional cost. Training data covers over 100,000 hours across more than 50 sign languages, with roughly a quarter of that data in American Sign Language. On the FLEURS-ASL benchmark the model scores 70 BLEURT zero-shot, which DeepMind describes as significantly higher than any previously reported score. The most striking architectural choice is that SL2T never handles raw video: it works from a sequence of pose landmark coordinates extracted on the device. ASAP works from DeepMind's own blog post to lay out what this model changes and how far it has actually come.

Only coordinate sequences leave the device

SL2T sees sign language as a sequence of pose landmark locations rather than a raw camera feed, and DeepMind states this design exists to protect user privacy. What crosses the boundary between phone and server is the structural heart of this announcement.

Processing splits into two stages. On the device, a MediaPipe Holistic model tracks body pose, converting the position of face, hands, and upper body into geometric coordinates while the original video is discarded immediately. Only those coordinates are sent onward, and SL2T turns the coordinate sequence into text.

The shipped features run in two directions. In Gboard, signing becomes a text input method usable for composing messages, running web searches, and drafting documents, and it can also carry queries and tasks to Gemini. In Live Transcribe, it supports the person answering in sign during a conversation. The features are available on Pixel 11 at no additional cost, and the launch language pair runs from American Sign Language to English.

Skipping gloss is the technical fork in this model

SL2T translates coordinate sequences directly into text without the intermediate gloss notation used in prior work, and DeepMind states this is what lets it capture complex aspects of sign languages such as non-manual markers and spatial constructions. A long-standing convention in sign language translation research disappears here.

Gloss is a notation that assigns a written word to each sign, and it has served for years as the intermediate representation in sign-to-text research. The two-stage structure of video to gloss and gloss to sentence carries a real practical advantage: each stage can be trained separately.

The problem is what gloss cannot carry. DeepMind names non-manual markers and spatial constructions as the examples. In sign languages, eyebrow position, eye gaze, and body tilt carry grammatical function, and placing a referent at a point in space and then pointing back to that point is part of the grammar too. A notation that maps one word to one sign has nowhere to put those layers.

Seen this way, removing the intermediate representation repeats a familiar direction. Speech recognition and machine translation both established that a human-designed intermediate representation sets a ceiling on the information that reaches the next stage. Learning end to end from coordinates to sentences preserves signal that gloss discarded. The structure carries a cost as well: without an intermediate artifact, a wrong translation is harder to attribute to a specific stage, and the only place a user can correct the output is at the sentence level.

How to read a zero-shot score of 70 BLEURT on FLEURS-ASL

The FLEURS-ASL result of 70 BLEURT zero-shot is the only performance figure in the announcement, and the two qualifiers attached to it determine how large it actually is. Those qualifiers are zero-shot and BLEURT.

The zero-shot condition comes first. It means the score was measured without tuning to that benchmark's training data, so placing it beside benchmark-specific fine-tuned numbers puts SL2T at a disadvantage. DeepMind's framing that the result is significantly higher than any previously reported score carries different weight once that condition is factored in.

The nature of the metric matters too. BLEURT is a learned model that estimates semantic similarity rather than a count of overlapping words, so a differently phrased output still earns credit when the meaning survives. For a language whose word order and grammatical structure diverge sharply from spoken language, that responds closer to whether meaning actually transferred than overlap-based metrics do. What the number does not do is convert into a felt user experience. On a learned similarity metric, the distance to a perfect score and real-world satisfaction do not track linearly.

The fact that there is only one figure deserves attention as well. What is published covers ASL to English in one direction, and no performance is given for any of the other 50-plus sign languages in the training set. Inverting the statement that roughly a quarter of the data is ASL means the remaining fifty-odd languages divide the other three quarters, so per-language data volume plausibly sits far below ASL. DeepMind notes that multilingual joint training helps the model learn shared structures across languages, dialects, and proficiency levels, but how much of that effect lands in any individual language is not disclosed.

The list of remaining errors is the most honest part of the announcement

DeepMind's August 12, 2026 announcement is explicit about what still fails, naming five error categories: rare signs, rapid fingerspelling, passive constructions, classifier depictions, and tense without context. That list functions as usage guidance rather than marketing copy, and the five items are not the same kind of problem.

Rapid fingerspelling is closer to a data and resolution issue. Fingerspelling spells words through fast handshape changes and clusters exactly where proper nouns and new coinages appear, and in a coordinate-only pipeline, fine differences at the finger joints become the accuracy budget. The privacy design of discarding video and keeping only landmarks plausibly cuts against precision at that specific point.

Classifier depictions and spatial constructions are different in kind. Using a handshape to denote a category of object and then moving it through space to describe relations has no fixed word-level equivalent, and interpretation shifts with context. Tense without context belongs to the same family. Time information in sign languages is often established once early in a conversation and carried forward rather than marked in every sentence, so a system that translates sentence by sentence can miss it as a matter of design.

That distinction matters in practice because the two categories improve at different rates. Errors that more data will fill in and errors that require holding context across sentence boundaries do not shrink on the same schedule. DeepMind's formation of the AI Sign Language Advisory Committee (AISLAC), bringing together global Deaf organizations and subject-matter experts, and its co-authored joint impact report detailing capabilities and limitations, read as a procedure for telling users about that difference in advance.

What Korean sign language users can and cannot confirm today

Whether Korean Sign Language is included in SL2T's training set is not confirmed in the published announcement, which states only that training covered more than 50 sign languages and that the launch pair is ASL to English. Keeping inside that boundary matters when mapping this news onto the Korean market.

What can be confirmed is that the architecture was designed with language expansion in mind. Multilingual joint training across many sign languages, combined with the removal of gloss as a per-language notation system, means adding a new language does not require inventing a notation scheme for it first. DeepMind also lists expansion to additional sign languages in its future plans.

What cannot be confirmed is the order or timing of that expansion. Korean Sign Language is one of Korea's official languages, and any team handling accessibility requirements in a domestic service would be premature in setting an adoption schedule on the basis of this announcement. The judgment available today is directional: sign input has begun entering the operating system as an input method on the same tier as the keyboard. Actual language support awaits a separate announcement.

Open questions: latency, the edge of on-device, and the generation direction

The first undisclosed item is latency, and for a feature used as a text input method that number governs usability as much as any accuracy score does. Pose extraction runs on the device, but translation sends coordinates to a server, which introduces a network round trip. The announcement says nothing about response time or whether any offline mode exists.

The second is where on-device actually ends. Discarding raw video on the phone and transmitting only landmarks is a clear improvement in that the camera feed itself never leaves. Coordinate sequences are still personal body-movement data, however, and they encode the content of what was said. How long those coordinates are retained on the server, and whether they feed model improvement, is not addressed in the published post. Judging the character of a privacy design requires the retention policy, not only the transmission boundary.

The third is the reverse direction. DeepMind lists sign language generation among its future plans, and text-to-sign is not the symmetric twin of sign-to-text. Generating sign requires producing hand, facial, and body movement along a time axis, and everything that pose coordinates alone cannot capture becomes an immediate problem there. What shipped is one direction, and two-way communication between Deaf and hearing people completing itself inside a phone still has an axis missing.

Source: ASAP analysis based on Google DeepMind's official blog post "Putting sign language AI into users' hands" (August 12, 2026)