Probing the Formation and Dynamics of J-Space in Language Models

Abstract

The paper introduces J-Space Dynamics (JSD), a methodological framework that extends the Jacobian lens for studying semantically accessible internal representations in language models. While the original J-lens identifies residual-stream directions associated with future verbal outputs, it remains largely first-order, token-based, and static across layers and training. JSD addresses these limitations through five extensions: a second-order continuation-margin method for estimating the robustness of internal plans; a path-conditioned sequence lens for analyzing ordered multi-token and relational content; non-autonomous transfer operators for modeling how J-space representations evolve and persist across layers and token positions; a regularized pullback Fisher geometry for behaviorally calibrated interventions and workspace-trajectory analysis; and a longitudinal framework that tracks representational consolidation across training checkpoints using multivariate measures and a Workspace Consolidation Index. The framework is explicitly designed to distinguish new contributions from existing multi-token J-lens and Fisher-geometry work, and it emphasizes falsifiable hypotheses, causal tests, closure diagnostics, and change-point analyses rather than claiming that new dynamical laws, phase transitions, or consciousness phenomena have already been established. An implementation is available at the cited GitHub repository.

Keywords