> ## Documentation Index
> Fetch the complete documentation index at: https://docs.generalrobotics.dev/llms.txt
> Use this file to discover all available pages before exploring further.

# Sensors

GRID currently supports the following sensors with Isaac Sim.

## Camera

This is a camera sensor that can be of type RGB or Depth.

### Configuration

The configuration for the camera sensor can be set up in the the configuration file. The following parameters are available:

| Parameter               | Description                                                             |
| ----------------------- | ----------------------------------------------------------------------- |
| **Height**              | The height of the captured image                                        |
| **Width**               | The width of the captured image                                         |
| **Offset**              | The offset of the camera with respect to the robot's frame of reference |
| **Focal Length**        | The focal length of the camera                                          |
| **Focal Distance**      | The focal distance of the camera                                        |
| **Horizontal Aperture** | The horizontal aperture of the camera                                   |
| **Clipping Range**      | The clipping range of the camera                                        |

These parameters can be configured by adding the sensor in the scene entity inside `env.yaml` in the following manner:

```yaml theme={null}
- camera_sensor_name:
    type: Sensor
    name: CameraRGB
    config:
    prim_path: "/World/envs/env_.*/tabletop_rgb_camera"
    height: 256
    width: 256
    offset: 
        pos: [1.5, 0.0, 0.7]
        rot: [0.0, -0.3, 0.0, 1.0]
        convention: "world"
    spawn:
        focal_length: 24.0
        focus_distance: 400.0
        horizontal_aperture: 20.955
        clipping_range: [0.1, 100000]
```

## Contact Sensor

This is a simple contact sensor that works as a ray caster simulating surface pressure by detecting contact points with precision.

### Configuration

The configuration for the height sensor can be set up in the the configuration file. The following parameters are available:

| Parameter          | Description                                                     |
| ------------------ | --------------------------------------------------------------- |
| **History length** | The length of history to consider while computing sensor values |

These parameters can be configured by adding the sensor in the scene entity inside `env.yaml` in the following manner:

```yaml theme={null}
- contact_sensor_name:
    type: Sensor
    name: ContactSensor
    config: 
    prim_path: "/World/envs/env_.*/robot/.*"
    history_length: 3
```

Note that you need to make sure `activate_contact_sensors` flag is set to true for your robot.

## Height Sensor

This is a simple height sensor that works as a ray caster to obtain the height values of points in a specified pattern.

### Configuration

The configuration for the height sensor can be set up in the the configuration file. The following parameters are available:

| Parameter              | Description                                                                |
| ---------------------- | -------------------------------------------------------------------------- |
| **Offset**             | The offset of the height sensor with respect to robot's frame of reference |
| **Pattern Resolution** | The resolution of the grid pattern around the sensor                       |
| **Pattern Size**       | The start and end of the grid pattern                                      |

These parameters can be configured by adding the sensor in the scene entity inside `env.yaml` in the following manner:

```yaml theme={null}
- height_sensor_name:
    type: Sensor
    name: HeightSensor
    config:
    prim_path: "/World/envs/env_.*/robot/body"
    offset: 
        pos: [0.3, 0.0, 0.22]
        rot: [0.0, 0.0, 0.0, 1.0]
    pattern_cfg:
        resolution: 0.1 
        size: [1.6, 1.0]
```

## LiDAR

This is a LiDAR sensor that works as a ray caster to obtain a point cloud around the sensor.

### Configuration

The configuration for the height sensor can be set up in the the configuration file. The following parameters are available:

| Parameter                 | Description                                                                    |
| ------------------------- | ------------------------------------------------------------------------------ |
| **Offset**                | The offset of the height sensor with respect to the robot's frame of reference |
| **Pattern Size**          | The start and end of the grid pattern                                          |
| **Max Distance**          | The maximum sensing distance of the height sensor                              |
| **Vertical FOV**          | The vertical field of view of the sensor                                       |
| **Horizontal FOV**        | The horizontal field of view of the sensor                                     |
| **Horizontal Resolution** | The horizontal resolution of the sensor                                        |
| **Channels**              | The number of channels used by the sensor                                      |

These parameters can be configured by adding the sensor in the scene entity inside `env.yaml` in the following manner:

```yaml theme={null}
- lidar_sensor_name:
    type: Sensor
    name: LiDAR
    config:
        offset: 
            pos: [0.0, 0.0, 0.0] # add init (x, y, z) position  
            rot: [0.0, 0.0, 0.0, 0.0] # add init (x, y, z, w) quaternion
        pattern_cfg:
            channels: 16 
            vertical_fov_range: [-15.0, 15.0]
            horizontal_fov_range: [-180.0, 180.0]
            horizontal_res: 0.2
        max_distance: 100
```

## IMU

The Inertial Measurement Unit (IMU) provides orientation and acceleration data by measuring the robot’s linear accelerations and angular velocities in its own body frame.

### Configuration

The configuration for the IMU sensor can be set up in the the configuration file. The following parameters are available:

| Parameter        | Description                                                                                             |
| ---------------- | ------------------------------------------------------------------------------------------------------- |
| **Offset**       | Pose of the IMU relative to the robot’s frame of reference                                              |
| **Gravity Bias** | The constant gravity vector to subtract from raw acceleration readings, as a 3-tuple `(x, y, z)` (m/s²) |

These parameters can be configured by adding the sensor in the scene entity inside `env.yaml` in the following manner:

```yaml theme={null}
- imu_sensor_name:
    type: Sensor
    name: IMU
    config:
        prim_path: "/World/envs/env_.*/robot/body"
        offset: 
            pos: [0.0, 0.0, 0.0] # add init (x, y, z) position  
            rot: [0.0, 0.0, 0.0, 0.0] # add init (x, y, z, w) quaternion
        gravity_bias: [0.0, 0.0, 9.81]
```

## Rectified Camera

This is a rectified camera sensor that applies intrinsic and extrinsic calibration to produce undistorted images (e.g., for stereo rigs or calibrated monocular setups).

### Configuration

The following parameters are available on the `RectifiedCameraCfg`:

| Parameter           | Description                                                                        |
| ------------------- | ---------------------------------------------------------------------------------- |
| **Width**           | The width of the captured image                                                    |
| **Height**          | The height of the captured image                                                   |
| **Offset**          | The offset of the camera with respect to the robot's frame of reference            |
| **D**               | Distortion coefficients `[k1, k2, p1, p2, (k3)]`.                                  |
| **K**               | Intrinsic matrix, flattened `[fx, 0, cx, 0, fy, cy, 0, 0, 1]`.                     |
| **R**               | Rectification (rotation) matrix, flattened 3×3.                                    |
| **P**               | Projection matrix, flattened 3×4 `[fx′, 0, cx′, Tx, 0, fy′, cy′, Ty, 0, 0, 1, 0]`. |
| **Clipping Range**  | The clipping range of the camera                                                   |
| **Focal Length**    | The focal length of the camera                                                     |
| **Focal Distance**  | The focal distance of the camera                                                   |
| **F Stop**          | Aperture setting for spawn config (`0.0` means wide open).                         |
| **Projection Type** | Type of image projection -- one of "pinhole"`or`"fisheye".                         |
| **Lock Camera**     | Whether to lock camera's transform or not.                                         |

### Example

Add a rectified camera to your `env.yaml` like so:

```yaml theme={null}
- rectified_camera:
    type: Sensor
    name: RectifiedCamera
    config:
      prim_path: "/World/envs/env_*/stereo_camera"
      width: 640
      height: 480
      offset:
        pos: [0.0, 0.0, 1.2]
        rot: [0.0, 0.0, 0.0, 1.0]
      D: [0.1, -0.05, 0.001, 0.0005, 0.0]
      K: [600.0, 0.0, 320.0,
           0.0, 600.0, 240.0,
           0.0,   0.0,   1.0]
      R: [1.0, 0.0, 0.0,
          0.0, 1.0, 0.0,
          0.0, 0.0, 1.0]
      P: [600.0, 0.0, 320.0, 0.0,
          0.0, 600.0, 240.0, 0.0,
          0.0,   0.0,   1.0, 0.0]
      clipping_range: [0.1, 100000]
      focal_length: 24.0
      focus_distance: 500.0
      f_stop: 2.8
      projection_type: "pinhole"
      lock_camera: true
```
