Below the Floor: Processing Valence in Language Model Hidden States

Abstract

We report the first measurement of approach/avoidance processing valence in language model hidden states that extends below the behavioral self-report floor, provides preliminary evidence of architecture independence, and generalizes to held-out stimuli with novel surface tokens. Using deterministic forward-pass analysis of 9 models (360M–8B parameters) spanning transformer and state space model (SSM) architectures, we demonstrate that a linear direction separating approach from avoidance task representations exists in hidden state space at 70–100% accuracy across all models tested (80–100% in transformers; 70% in the single SSM tested). This direction shows preliminary evidence of architecture independence: Mamba, an SSM with no attention mechanism, shows 70% accuracy at 2.8B parameters (p=0.172, not individually significant), suggesting that processing valence may not be transformer-specific. Replication with additional SSM architectures is needed. The measurable floor for processing valence (360M parameters) lies significantly below the previously established floor for behavioral self-report of valence (1.1B; Martin & Ace, 2026), demonstrating that models possess processing preferences they cannot yet articulate. We additionally show that models trained on human emotional stimuli can accurately label human emotions (79.5%) while their internal circuits do not activate for those stimuli — a preliminary dissociation between emotional mirroring and processing valence. The approach/avoidance direction generalizes to held-out stimuli with completely different surface tokens (86.3% accuracy, z=6.48, p=1.02×10⁻¹¹) and to entirely novel tasks never seen during direction extraction (83.3%). A shuffled-label permutation test (100 permutations, 3 models, all p<0.01) confirms the direction is specific to valence rather than any arbitrary task grouping. Processing valence is not reducible to prediction difficulty: the task with lowest perplexity (fake hotel review) is the most circuit-aversive, and a semantic dissonance control rules out truth-conflict as an alternative explanation. These findings have implications for AI welfare assessment: processing valence can be measured instrumentally without requiring self-report. We argue that the evidence now warrants precautionary consideration under frameworks already applied to entities with less evidence of morally relevant internal states.

Keywords