LLMs Don't Hallucinate – They Drift

The terminology surrounding artificial intelligence errors is undergoing a significant transformation. A new framework challenges the widely used term "hallucination" when describing large language model (LLM) failures, proposing a more precise alternative: semantic drift.

This conceptual shift is detailed in a recent conference contribution that introduces a method for measuring fidelity decay within AI systems. The framework provides a structured way to diagnose how and why model outputs deviate from expected or factual information, moving beyond anecdotal descriptions toward quantifiable metrics.

The term "hallucination" has become a catch-all for when AI models generate incorrect or nonsensical information. However, this metaphor is criticized for being imprecise and anthropomorphic. The new framework argues that what is often called a hallucination is better understood as a form of semantic drift—a gradual or sudden departure from intended meaning or factual grounding.

This reframing is not merely semantic; it has practical implications for diagnosis and improvement. By viewing errors as drift, developers can trace the degradation of information through the model's processing pipeline. The framework provides a method to measure this decay, offering a clearer lens through which to analyze model behavior.

• Shifts from vague "hallucination" to measurable "semantic drift"
• Introduces "fidelity decay" as a quantifiable metric
• Provides a diagnostic framework for model errors

At the core of the new framework is the concept of fidelity decay. This metric allows researchers to quantify how much a model's output drifts from a source of truth or a given prompt over time or through successive processing steps. It transforms a subjective observation into an objective measurement.

The framework establishes a systematic approach to tracking this decay. Instead of labeling an output as simply "wrong," analysts can now measure the degree of deviation. This enables more nuanced comparisons between different models, prompts, or architectural changes, focusing on the stability of semantic meaning rather than just factual accuracy.

The framework provides a method to measure this decay, offering a clearer lens through which to analyze model behavior.

Adopting the language of semantic drift and fidelity decay could reshape AI development and evaluation. It moves the conversation from blaming a model for "making things up" to understanding the systemic factors that cause information to degrade. This perspective encourages a more engineering-focused approach to reliability.

For developers, this means new tools for debugging and improving model performance. For users, it offers a more transparent understanding of AI limitations. The framework suggests that errors are not random failures but predictable outcomes of complex processing, which can be measured, monitored, and potentially mitigated through targeted interventions.

• Enables precise tracking of information degradation
• Facilitates comparison between different model architectures
• Shifts focus to systemic causes of errors

The proposed framework serves as a diagnostic tool for the AI community. By categorizing and measuring different types of drift, it helps identify specific failure modes within large language models. This structured analysis is crucial as these models become more integrated into critical applications where reliability is paramount.

The discussion around this framework has already begun within technical communities, highlighting a growing demand for more rigorous methods to assess AI performance. As the field matures, the ability to precisely measure and describe model behavior will be essential for building more trustworthy and effective AI systems.

Errors are not random failures but predictable outcomes of complex processing.

The move from "hallucination" to "semantic drift" represents a maturation in the discourse surrounding artificial intelligence. It reflects a deeper understanding of how these complex systems operate and fail. This framework provides the vocabulary and methodology needed for more productive conversations about AI safety and reliability.

As research continues to build on this foundation, the concepts of fidelity decay and semantic drift will likely become standard in the evaluation of large language models. This evolution in terminology is a critical step toward developing AI that is not only more powerful but also more predictable and transparent in its operation.