Encodes pronoun resolution and anaphoric coreference.
What It Does
Relation.Reference neurons activate on anaphoric and cataphoric references — constructions where one expression refers back or forward to another: pronouns ('he', 'she', 'they', 'it') resolving to antecedents, demonstratives ('this', 'that', 'these'), and ellipsis ('X did so too' referring back to a previously described action). They encode the referential link between two expressions.
How It Behaves
Reference neurons are distributed across middle and late layers, where coreference resolution requires sufficient contextual integration to correctly identify the antecedent. They are one of the most important Relation sub-types for discourse coherence — incorrect Reference neuron resolution produces pronoun errors, coreference confusions, and discourse inconsistencies. Models with larger context windows show richer Reference neuron patterns, reflecting the added computational resources available for long-distance coreference.
Research Example
In OLMo 3 7B, Relation.Reference neurons are strongly active during complex pronoun resolution in passages with multiple same-gender referents — 'Mary told Sarah that she was wrong.' The model must activate Reference neurons to track which 'she' refers to which antecedent, and Reference neuron errors at this point produce the characteristic ambiguous pronoun hallucinations seen in multi-person narratives.