Locomotion Policy Training

Train, preview, and export a K1 velocity policy with Cyclo MJLab. Run the commands inside the Cyclo MJLab container from /workspace/cyclo_mjlab.

Train

Run the velocity task with 4,096 parallel environments:

python scripts/reinforcement_learning/train.py \
  Cyclo-Velocity-Flat-K1-Rev1-v0 \
  --env.scene.num-envs 4096

Training output is written to:

logs/rsl_rl/k1_velocity/<run>/

<run> is the timestamped directory created when training starts. The main files in each run directory are:

logs/rsl_rl/k1_velocity/<run>/

• model_*.pt
• params/
  • agent.yaml
  • env.yaml
  • sim2real.yaml

The training configuration saves a checkpoint every 100 iterations.

Play and Export

Select a checkpoint from the training run and pass its path to play.py.

Native MuJoCo Viewer

python scripts/reinforcement_learning/play.py \
  Cyclo-Velocity-Flat-K1-Rev1-v0 \
  --checkpoint-file logs/rsl_rl/k1_velocity/<run>/model_<iteration>.pt \
  --num-envs 1 \
  --viewer native

Use the native viewer when a desktop display is available.

Viser Viewer

python scripts/reinforcement_learning/play.py \
  Cyclo-Velocity-Flat-K1-Rev1-v0 \
  --checkpoint-file logs/rsl_rl/k1_velocity/<run>/model_<iteration>.pt \
  --num-envs 1 \
  --viewer viser

Open the URL printed by play.py in a web browser.

Replace <run> with the timestamped run directory and <iteration> with the checkpoint iteration to use.

If --viewer is omitted, the default auto mode uses the native MuJoCo viewer when a desktop display is available and otherwise uses the Viser viewer. Before opening the selected viewer, play.py exports the policy to the same training run directory.

To export the policy without opening a viewer, add --export-only:

python scripts/reinforcement_learning/play.py \
  Cyclo-Velocity-Flat-K1-Rev1-v0 \
  --checkpoint-file logs/rsl_rl/k1_velocity/<run>/model_<iteration>.pt \
  --num-envs 1 \
  --export-only

After play.py exports the policy, the run directory contains:

logs/rsl_rl/k1_velocity/<run>/

• model_*.pt
• exported/
  • policy.onnx
  • policy.pt
• params/
  • agent.yaml
  • env.yaml
  • sim2real.yaml

Prepare for Sim2Real

Cyclo MJLab uses the same Sim2Real deployment process as Cyclo Lab. Before deploying the trained velocity policy, prepare these files from the same training run:

• exported/policy.onnx
• params/sim2real.yaml

Then follow How to Deploy Your Own Cyclo Lab-Trained Policy to place the files under the Sim2Real asset root and connect the policy to the Velocity mode.

note

exported/policy.onnx and params/sim2real.yaml are a matched pair from one logs/rsl_rl/k1_velocity/<run>/ directory. Do not combine policy.onnx from one run with sim2real.yaml from another run.