Skip to main content
GRID supports the training and evaluation of reinforcement learning agents in Isaac Sim for the supported quadruped, bipeds, arms, and humanoid robots. RL Training

​
Training

GRID supports training reinforcement learning agents using the RSL-RL training methodology. Agents can be trained by modifying the agent.yaml file as follows: 
- rsl_rl_agent:
    type: "rsl_rl"
    mode: "play"
    config: 
      resume: false
      max_iterations: 1000
      num_steps_per_env: 24
      video: false
      video_length: 200
      video_interval: 2000
      empirical_normalization: false
      experiment_name: "my_experiment_name"
      load_run: .*
      load_checkpoint: model.*
      policy: 
        class_name: "ActorCritic"
        init_noise_std: 1.0
        actor_hidden_dims: [512, 256, 128]
        critic_hidden_dims: [512, 256, 128]
        activation: "elu"
      algorithm: 
        class_name: "PPO"
        value_loss_coef: 1.0
        use_clipped_value_loss: True
        clip_param: 0.2
        entropy_coef: 0.008
        num_learning_epochs: 5
        num_mini_batches: 4
        learning_rate: 1.0e-3
        schedule: "adaptive"
        gamma: 0.99
        lam: 0.95
        desired_kl: 0.01
        max_grad_norm: 1.0
To run the RL training headless, use the following configuration in workflow.yaml, change the variable headless to True.
The video parameter in the agent.yaml can be used to save inference videos of the policy during training.
The checkpoints are saved in the same directory as the configuration files with the appropriate date and time-stamp.