ASAPAGI Soon As Possible · Deep reads on AI & tech
Article

Why AI-written text is full of commas and dashes: 93.7% of GPT-3's training documents were English

2026-07-26 · 5 min read

AI-written text carries an unusual number of commas and dashes because the models learned to write in English. In the dataset statistics OpenAI published for GPT-3, English accounted for 93.68882% of documents by count, while Korean accounted for 0.01939%, a gap of roughly 4,800 times. English leans on commas and dashes to join clauses, and when a model produces text in a language that uses inflection instead, the punctuation slots survive the translation even though their grammatical job does not. ASAP breaks down the mechanism and the three checks that catch it before publishing.

The training corpus is overwhelmingly English

The dataset statistics OpenAI released in the GPT-3 repository put English at 93.68882% of documents, followed by German in the 1.2% range and Japanese in the 0.25% range. Korean sat at 0.01939%, about one four-thousand-eight-hundredth of the English share.

As of July 2026, the major labs no longer publish this breakdown. Data sources and language distribution became withheld sections in technical reports after GPT-4, and Anthropic and Google follow the same practice. The exact Korean share of any current model is therefore unverifiable. What has not changed is the tilt of the underlying web corpus, and the last large-scale published statistic points in an unambiguous direction: these systems learned sentence construction almost entirely from English.

The consequence shows up in structure rather than vocabulary. Machine translation quality has risen sharply, so word choice rarely sounds wrong. What still sounds off is the joinery, meaning how clauses connect and where a qualifier gets parked. That layer inherits the habits of whatever language the model saw most.

English punctuation slots survive into other languages

In English, commas and dashes carry grammatical weight rather than decoration. The Chicago Manual of Style documents the em dash as the standard mark for setting off a parenthetical, and the convention is settled enough that writers reach for it automatically. The same marks insert asides, signal apposition and splice clauses together.

Korean assigns those jobs elsewhere. An adnominal clause handles the aside, a connective ending handles the dash, and a particle handles apposition. English needs two sentences or a dash for "performance is good but it is expensive"; Korean fuses them with a single verb ending and no punctuation at all.

The failure mode is that a model converts the meaning into the target language while leaving the architecture in English. Place an English source next to its Korean output and the dashes and commas stand in nearly identical positions. In the Korean sentence those marks have lost their function and kept their seat. That residue is what readers register as translationese.

Commas and dashes stay open, and periods do not

A second reason comes from how the text is produced. A period is a commitment to end the sentence and cannot be walked back. A comma or a dash accepts whatever follows, which means the model does not have to have settled on the continuation yet.

Autoregressive models build sentences one token at a time, so that asymmetry is not academic. Leaving the sentence open is always the safer local choice, and the marks accumulate. This is an inference from the generation mechanism rather than a published measurement, and it should be read as such. It does match the observed pattern that the extra marks cluster mid-sentence rather than at boundaries.

Three checks that catch inherited punctuation

The fix is a pre-publication check rather than a matter of taste, and it has to be expressible as a search to survive contact with a real workflow. ASAP runs three before anything ships.

Dashes. Do not use English-style dash constructions in Korean. When continuing after a list, close it with a phrase rather than a mark. Split an appended qualifier into its own sentence.

Commas. Rework any sentence carrying two or more commas. Never substitute a comma when removing a dash. Do not relocate asides or appositions into comma pairs, and set lists with a middle dot or a conjunction instead.

Particles. Match the particle after a proper noun to how the word is pronounced. ASAP ends in a consonant sound, so the consonant-final forms are the correct ones. The error rate climbs noticeably in text dense with English acronyms.

Banning the dash alone just moves the problem to the comma

Of the three checks, the comma rule is the one most often missing, and ASAP itself ran with only the dash rule for a long stretch. The outcome of a dash-only rule is easy to predict. A comma takes the vacated slot, the mark changes, the English clause structure does not, and the output still reads as machine-written.

That is why the dash rule needs an explicit clause forbidding comma substitution. Splitting the guidance into three parts exists for the same reason. Blocking one layer relocates the habit instead of removing it.

The check itself is mechanical. Search for dashes, find sentences with two or more commas, and verify the particles after English acronyms. Three searches change how the piece reads.

What is still unverified as of 2026

The open questions are worth stating plainly. The Korean share in current training runs is not disclosed, and neither is how much Korean data entered post-training. No published comparison shows whether punctuation habits differ from model to model.

The output side is not in doubt. Whatever the language mix looks like now, Korean text coming out of these systems still carries the imprint of English punctuation. Removing it is the writer's rule set, not the model's.

Source: ASAP analysis based on OpenAI's published GPT-3 dataset language statistics (by document count).

ASAP — AGI Soon As Possible

AI & tech,
read in depth

Beyond the headlines — into the context and the structure

AGI Soon As Possible · asapai.co.kr

← All posts