mdp_cfg.yaml
is necessary for defining the MDP component settings. This includes parameters related to the high level commands given to the agent (like base velocity commands or end-effector commands), lower level actions provided to each of the DOFs (like relative change pose and binary gripper control), the set of observations fed to the policy network, termination conditions for each episode, reward design, and curriculum utilized in the training methodology.
The following example depicts the MDP settings for training a Unitree Go2 that receives velocities commands and executes them based on a locomotion policy.
actions:
- joint_actions:
type: JointPosition
config:
asset_name: robot
joint_names: [".*"]
scale: 0.5
use_default_offset: True
commands:
- base_velocity:
type: VelocityCommand
config:
asset_name: robot
rel_standing_envs: 0.02
rel_heading_envs: 1.0
heading_control_stiffness: 0.5
debug_vis: True
heading_command: True
resampling_time_range: [10.0, 10.0]
ranges:
lin_vel_x: [-1., 1.]
lin_vel_y: [-1., 1.]
ang_vel_z: [-1., 1.]
heading: [-3.14, 3.14]
observations:
- policy:
- base_lin_vel:
type: base_lin_vel
- base_ang_vel:
type: base_ang_vel
- projected_gravity:
type: projected_gravity
- velocity_commands:
type: velocity_commands
- joint_pos_rel:
type: joint_pos_rel
- joint_vel_rel:
type: joint_vel_rel
- actions:
type: actions
config:
params:
action_name: joint_actions
- height_scan:
type: height_scan
terminations:
- time_out:
type: time_out
- body_contact:
type: body_contact
config:
params:
threshold: 1.0
- terrain_out_of_bounds:
type: terrain_out_of_bounds
config:
params:
distance_buffer: 3.0
events:
- add_base_mass:
type: add_base_mass
config:
params:
mass_distribution_params: [-2.5, 2.5]
operation: add
- base_external_force_torque:
type: base_external_force_torque
config:
params:
force_range: [0.0, 0.0]
torque_range: [-0.0, 0.0]
- reset_base:
type: reset_base
- reset_robot_joints:
type: reset_robot_joints
rewards:
- air_time:
type: air_time
config:
weight: 5.0
params:
mode_time: 0.3
velocity_threshold: 0.5
- base_angular_velocity:
type: base_angular_velocity
config:
weight: 5.0
params:
std: 2.0
- base_linear_velocity:
type: base_linear_velocity
config:
weight: 5.0
params:
std: 1.0
ramp_rate: 0.5
ramp_at_vel: 1.0
- foot_clearance:
type: foot_clearance
config:
weight: 0.5
params:
std: 0.05
tanh_mult: 2.0
- action_smoothness:
type: action_smoothness
config:
weight: -1.0
- air_time_variance:
type: air_time_variance
config:
weight: -1.0
- base_motion:
type: base_motion
config:
weight: -2.0
- base_orientation:
type: base_orientation
config:
weight: -3.0
- foot_slip:
type: foot_slip
config:
weight: -0.5
- joint_pos:
type: joint_pos
- joint_torques:
type: joint_torques
config:
weight: 1.0
If you want to take a look at the possible set of options for all MDP components available at your disposal, refer to this page.