Encodes irrelevant, distracting, or low-signal content.
What It Does
Entropy.Noise neurons activate on content that is low-information or distracting relative to the main communicative purpose: filler language ('um', 'well', 'you know'), redundant repetition, irrelevant tangents, low-quality or contradictory information in a passage, and semantic noise in poorly-structured text. They represent the model's detection of signal-vs-noise quality in input.
How It Behaves
Noise neurons show a middle-to-late layer concentration, where the model has processed enough context to evaluate information quality. This late processing is consistent with the idea that noise detection requires establishing what the relevant signal is before identifying what detracts from it. Noise neurons co-activate with Entropy.Ambiguity neurons on low-quality ambiguous text and with Entropy.Variability on inconsistent text. High Noise neuron firing is correlated with lower-quality model outputs — a signal that the model is processing low-quality input.
Research Example
In Gemma 2B, Entropy.Noise neurons show elevated firing when the model processes text that contains contradictory claims within the same passage — 'the treatment is both highly effective and largely ineffective depending on the study.' The model correctly detects contradictory noise, but the question is whether it can act on that detection to produce a coherent output or whether it propagates the contradiction into its response.