Weight Editing and Decoding for Reasoning Loops in Qwen3.5-0.8B

Abstract

We audit a public Astra edit to Qwen3.5-0.8B that targets repetitive reasoning. At a 32,768-token ceiling with default thinking-mode decoding, the edit raises strict final-answer accuracy from 6.6% to 49.0% on MATH-500 and from 11.1% to 68.6% on a 256-question GSM8K sample. A frequency penalty selected on training questions sharply reduces budget exhaustion, but frequently stops generation inside thought and yields 3.5% and 11.3% accuracy on the unedited model. Decoder-only policies can work at shorter budgets. At 3,072 tokens, forcing an end to thought recovers more correct answers than replaying the same prefix with a sham restart. At 8,192 tokens, the unedited non-thinking policy leads the edited thinking policy on both datasets in a single-seed comparison. The ordering reverses at 32,768 tokens, where the edit uses more tokens on average. The audit shows why evaluations of sparse weight edits should track answer completion and compare decoding policies across token budgets.

Keywords