Why does LLMs love the usage of –?

As an AI expert, I can confidently explain why Large Language Models (LLMs) appear to favor the use of dashes (en-dash – and em-dash —). This preference is not due to a conscious "love" but rather a consequence of their training data, their tokenization mechanics, and the functional versatility of the dash.

1. Training Data Bias and Statistical Dominance

LLMs are trained on vast corpora of text, and the statistical distribution of punctuation in these datasets directly influences their output. The dash, particularly the em-dash, is significantly prevalent in many high-quality, contemporary text sources that constitute LLM training data, such as:

• ·Newswire and Press Releases: Used frequently to embed "which-clauses" and avoid breaking flow. Examples like "The company – which had posted losses for three years – finally turned a profit" appear dozens of times per article.
• ·Academic and Technical Writing: Style guides (APA, Chicago) encourage dashes for parenthetical citations or brief asides.
• ·Literary Fiction and Essays: Authors use dashes for dramatic pauses and emphasis, contributing to their over-representation in "high-quality" segments.
• ·Wikipedia and Reference Works: Wikipedia's style guide explicitly prefers em-dashes for parenthetical phrases.
• ·Web-scraped Forums and Chat Logs: Informal writing often substitutes a dash for a spoken pause.

This exposure means LLMs encounter sentences with dashes far more often than those without, leading the models to learn a high conditional probability for the dash token when certain linguistic cues (e.g., a preceding noun, a relative clause starter) are present.

2. Token-Level Economics and "Zero-Cost" Punctuation

Most modern LLMs utilize sub-word tokenizers like Byte-Pair Encoding (BPE) or SentencePiece. In these systems, the dash characters (-, –, —) are typically stand-alone tokens. This has two crucial implications:

1. ·Zero-cost separation: Inserting a dash does not create a new sub-word piece for each adjacent word. The model can insert – between two already-known tokens without inflating the total token count. This makes it a "cheap" way for the model to break a sentence or add punctuation.
2. ·High conditional probability: When predicting the next token, the probability mass is distributed among various punctuation tokens. Because –/— are frequent in the corpus, they often receive a relatively larger share of this probability, making them common choices, especially in low-temperature decoding.

This makes the dash an efficient and high-probability choice in many contexts where punctuation is needed.

3. Functional Versatility and Stylistic Utility

The dash is a remarkably versatile punctuation mark, capable of serving multiple linguistic functions. From a linguistic perspective, it can replace four distinct functions:

Because a single token can satisfy several patterns, the model learns to favor the dash as a "Swiss-army-knife" punctuation mark. It provides expressive power and sentence flexibility, creating pause effects, handling parenthetical content, and signaling interruption or shifts more fluidly than commas, parentheses, or colons. This aligns well with contemporary writing trends where readability and flow are prioritized over strict formalism.

Furthermore, the dash can prevent ambiguity that commas or colons might introduce, and its visual prominence aids readability, especially in digital formats where visual chunking is beneficial.

4. Over-Usage and Control

While functionally useful, LLMs often exhibit an "over-use" of dashes for several reasons:

• ·Longer Generated Passages: The more text an LLM produces, the more likely the frequent statistical patterns (including dashes) will manifest, making their recurrence more noticeable.
• ·Absence of Authorial Voice: LLMs lack human authors' self-imposed stylistic restraints, directly reflecting corpus statistics instead.
• ·Decoding Settings: Lower decoding temperatures (e.g., greedy or beam search) emphasize the most probable tokens, which are often dashes.

To control dash usage, explicit prompt instructions (“avoid em-dashes”), logit bias (applying a negative bias to dash token IDs), or post-processing can be employed. Fine-tuning on a dash-light corpus or increasing the temperature can also adjust the frequency. A balanced approach often involves a modest logit bias (-2) combined with a moderate temperature (0.7–0.8).

In essence, the LLM's "love" for dashes is a reflection of the data it learns from, coupled with the efficiency and versatility of the dash within its tokenization and generation process.