Skip to main content

Locomotion Policy Training

Train, preview, and export a K1 velocity policy from this guide. Run the commands inside the Cyclo Lab container from /workspace/cyclo_lab. Select the physics backend with physics=newton_mjwarp or physics=isaacsim_physx.

Train​

python scripts/reinforcement_learning/rsl_rl/train.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 4096 \
--headless --visualizer none \
physics=newton_mjwarp

To resume training, add --checkpoint logs/rsl_rl/k1_rev1_velocity/<run>/model_<iteration>.pt to the training command. --checkpoint latest selects a checkpoint from the newest compatible run created by the current training script.

Training output is written to:

logs/rsl_rl/k1_rev1_velocity/*/

Each run directory contains:

Play and Export​

Run the same task ID and physics backend as training. By default, play.py loads the latest run and the latest model_*.pt checkpoint for the selected task. It runs the policy in simulation and exports exported/policy.onnx in the training run directory.

python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp

For playback without a window, replace the --visualizer option with --headless --visualizer none.

To play a specific checkpoint instead of the latest one, add --checkpoint:

python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--checkpoint logs/rsl_rl/k1_rev1_velocity/2026-01-01_12-00-00/model_1000.pt \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp

After play.py finishes, check the same training run directory. The exported/ directory is added by play.py:

Prepare for Sim2Real​

Before moving to Sim2Real, make sure these required files are ready:

  • exported/policy.onnx
  • params/sim2real.yaml

Then follow How to Deploy Your Own Cyclo Lab-Trained Policy to place the asset under the Sim2Real asset root and connect it to the Velocity mode.

note

exported/policy.onnx and params/sim2real.yaml are a matched pair from one logs/rsl_rl/.../<run>/ directory. Do not pair policy.onnx from one run with sim2real.yaml from another run.