Skip to main content

Mimic Policy Training

Train, preview, and export a K1 dance mimic policy from this guide. Run the commands inside the Cyclo Lab container from /workspace/cyclo_lab. Select the physics backend with physics=newton_mjwarp or physics=isaacsim_physx.

Train​

python scripts/reinforcement_learning/rsl_rl/train.py \
--task Cyclo-Mimic-K1-Rev1-Dance1 \
--num_envs 4096 \
--headless --visualizer none \
physics=newton_mjwarp

To resume training, add --checkpoint logs/rsl_rl/cyclo_mimic_k1_rev1_dance1/<run>/model_<iteration>.pt to the training command. --checkpoint latest selects a checkpoint from the newest compatible run created by the current training script.

Training output is written to:

logs/rsl_rl/cyclo_mimic_k1_rev1_dance1/*/

Each run directory contains:

Play and Export​

Run the same task ID and physics backend as training. By default, play.py loads the latest run and the latest model_*.pt checkpoint for the selected task. It runs the policy in simulation and exports exported/policy.onnx in the training run directory.

python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Mimic-K1-Rev1-Dance1 \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp

For playback without a window, replace the --visualizer option with --headless --visualizer none.

To play a specific checkpoint instead of the latest one, add --checkpoint:

python scripts/reinforcement_learning/rsl_rl/play.py \
--task Cyclo-Mimic-K1-Rev1-Dance1 \
--checkpoint logs/rsl_rl/cyclo_mimic_k1_rev1_dance1/2026-01-01_12-00-00/model_1000.pt \
--num_envs 16 \
--visualizer newton_gl \
--real-time \
physics=newton_mjwarp

After play.py finishes, check the same training run directory. The exported/ directory is added by play.py:

Prepare for Sim2Real​

Before moving to Sim2Real, make sure these required files are ready:

  • exported/policy.onnx
  • params/sim2real.yaml
  • The motion reference CSV for the same motion used during training.

Then follow How to Deploy Your Own Cyclo Lab-Trained Policy to place the asset under the Sim2Real asset root and register the new mimic mode.

note

exported/policy.onnx and params/sim2real.yaml are a matched pair from one logs/rsl_rl/.../<run>/ directory. Do not pair policy.onnx from one run with sim2real.yaml from another run.

How to Add Your Own Mimic Task​

Use this flow when you want to add a new mimic task for your own motion.

note

This guide assumes you already have a Soma-retargeter output CSV from the Kimodo > Soma-retargeter pipeline.

The examples below use my_motion as the new motion name. Use your own motion name in the same places.

  1. Place and convert the motion file.

    First, place the Soma-retargeter output CSV under the K1 Rev.1 motion data directory:

    my_motion_soma.csv is the Soma-retargeter output CSV.

    Convert it because Cyclo Lab does not train directly from the Soma-retargeter output CSV.

    python3 scripts/tools/motion/soma_retargeter_csv_converter.py \
    -f source/cyclo_lab/data/motions/K1_rev1/my_motion/my_motion_soma.csv \
    --output_name source/cyclo_lab/data/motions/K1_rev1/my_motion/my_motion.npz \
    --viz none

    This creates the Cyclo Lab motion files used after this step:

    • my_motion.csv for Sim2Real deployment
    • my_motion.npz for training

    After conversion, my_motion_soma.csv is not used by training or Sim2Real deployment, so it can be removed from this folder.

  2. Create a new mimic task config.

    Create my_motion_env_cfg.py by duplicating the existing dance1_env_cfg.py. In the new file, set TRAJECTORY_FILE to the new NPZ path.

    TRAJECTORY_FILE = f"{CYCLO_LAB_ASSETS_DATA_DIR}/motions/K1_rev1/my_motion/my_motion.npz"
  3. Register the new task.

    The files you edit are here:

    Add an entry to source/cyclo_lab/cyclo_lab/manager_based/mimic/config/k1_rev1/__init__.py:

    gym.register(    id="Cyclo-Mimic-K1-Rev1-MyMotion",    entry_point="isaaclab.envs:ManagerBasedRLEnv",    disable_env_checker=True,    kwargs={        "env_cfg_entry_point": f"{__name__}.my_motion_env_cfg:K1Rev1EnvCfg",        "rsl_rl_cfg_entry_point": f"{agents.__name__}.rsl_rl_ppo_cfg:K1Rev1MimicPPORunnerCfg",    },)

    Check that the task ID is registered:

    python3 scripts/tools/list_envs.py | grep MyMotion

    Check that the output includes Cyclo-Mimic-K1-Rev1-MyMotion.

  4. Train and export.

    Run train.py with the new task ID. Then run play.py to preview the policy and export exported/policy.onnx.