> ## Documentation Index
> Fetch the complete documentation index at: https://docs.generalrobotics.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Training Workflow

GRID supports the training and evaluation of reinforcement learning agents in Isaac Sim for the supported quadruped, bipeds, arms, and humanoid robots.

<img src="https://mintcdn.com/scaledfoundations/bbcuyC2Oq40Jh0Ty/assets/images/rl-training.gif?s=a5a3bd5e5eeb78749cd08085ed3dca5a" alt="RL Training" width="800" height="454" data-path="assets/images/rl-training.gif" />

## Training

GRID supports training reinforcement learning agents using the [RSL-RL](https://github.com/leggedrobotics/rsl_rl) training methodology.
Agents can be trained by modifying the `agent.yaml` file as follows:

```yaml theme={null}
- rsl_rl_agent:
    type: "rsl_rl"
    mode: "play"
    config: 
      resume: false
      max_iterations: 1000
      num_steps_per_env: 24
      video: false
      video_length: 200
      video_interval: 2000
      empirical_normalization: false
      experiment_name: "my_experiment_name"
      load_run: .*
      load_checkpoint: model.*
      policy: 
        class_name: "ActorCritic"
        init_noise_std: 1.0
        actor_hidden_dims: [512, 256, 128]
        critic_hidden_dims: [512, 256, 128]
        activation: "elu"
      algorithm: 
        class_name: "PPO"
        value_loss_coef: 1.0
        use_clipped_value_loss: True
        clip_param: 0.2
        entropy_coef: 0.008
        num_learning_epochs: 5
        num_mini_batches: 4
        learning_rate: 1.0e-3
        schedule: "adaptive"
        gamma: 0.99
        lam: 0.95
        desired_kl: 0.01
        max_grad_norm: 1.0
```

<Note>
  To run the RL training headless, use the following configuration in `workflow.yaml`, change the variable `headless` to `True`.
</Note>

<Note>
  The `video` parameter in the `agent.yaml` can be used to save inference videos of the policy during training.
</Note>

<Note>
  The checkpoints are saved in the same directory as the configuration files with the appropriate date and time-stamp.
</Note>