Locomotion Policy Training
Train, preview, and export a K1 velocity policy from this guide.
Run the commands inside the Cyclo Lab container from /workspace/cyclo_lab.
Select the physics backend with physics=newton_mjwarp or physics=isaacsim_physx.
Train
- Newton (K1 Default)
- PhysX
python scripts/reinforcement_learning/rsl_rl/train.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 4096 \
--headless --visualizer none \
physics=newton_mjwarp
python scripts/reinforcement_learning/rsl_rl/train.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 4096 \
--headless --visualizer none \
physics=isaacsim_physx
To resume training, add --checkpoint logs/rsl_rl/k1_rev1_velocity/<run>/model_<iteration>.pt to the training command.
--checkpoint latest selects a checkpoint from the newest compatible run created by the current training script.
Training output is written to:
logs/rsl_rl/k1_rev1_velocity/*/
Each run directory contains:
logs/rsl_rl/k1_rev1_velocity/*/model_*.pt- params/
agent.yamlenv.yamlsim2real.yaml
Play and Export
Run the same task ID and physics backend as training.
By default, play.py loads the latest run and the latest model_*.pt checkpoint for the selected task.
It runs the policy in simulation and exports exported/policy.onnx in the training run directory.
- Newton (K1 Default)
- PhysX
python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp
python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--num_envs 16 \
--visualizer kit \
--real-time \
physics=isaacsim_physx
For playback without a window, replace the --visualizer option with --headless --visualizer none.
To play a specific checkpoint instead of the latest one, add --checkpoint:
python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Velocity-Flat-K1-Rev1-v0 \
--checkpoint logs/rsl_rl/k1_rev1_velocity/2026-01-01_12-00-00/model_1000.pt \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp
After play.py finishes, check the same training run directory.
The exported/ directory is added by play.py:
logs/rsl_rl/k1_rev1_velocity/*/model_*.pt- exported/
policy.onnxpolicy.pt
- params/
agent.yamlenv.yamlsim2real.yaml
Prepare for Sim2Real
Before moving to Sim2Real, make sure these required files are ready:
exported/policy.onnxparams/sim2real.yaml
Then follow How to Deploy Your Own Cyclo Lab-Trained Policy to place the asset under the Sim2Real asset root and connect it to the Velocity mode.
exported/policy.onnx and params/sim2real.yaml are a matched pair from one logs/rsl_rl/.../<run>/ directory.
Do not pair policy.onnx from one run with sim2real.yaml from another run.