Editing a Frozen Locomotion Policy Under Reduced Actuator Authority

Abstract

Reducing actuator authority can invalidate a learned gait even when the new command limits are known. We test whether direct reward search over the last layer of a frozen PPO actor can recover locomotion. Across four held-out Hopper-v5 checkpoints and three known command boxes, tuning three gains and three offsets raises mean return from 559.6 under projection to 1,414.6 after 312 adaptation episodes per condition. In a follow-up with controlled single-joint caps, the same 0.40 cap is much harder to compensate at the ankle than at the hip. Row reversion shows that the uncapped-joint edits retain most of the recovery under a knee cap. On four independently trained Walker2d-v5 checkpoints, reusing the search recipe raises mean return from 913.1 to 1,598.5 across four cap proffles. Recentering proposals on the incumbent improves returnover ffxed-center search at the same episode budget in both follow-up tasks. Severe limits remain hard, and Hopper edits lose return when reused under healthy actuation.

Keywords