GRID Docs | General Robotics

Run PI05 policy inference on robot observations.

Parameters

ImageInput

required

Base camera RGB image as file path, URL, PIL Image, or numpy array.

ImageInput

required

Wrist camera RGB image as file path, URL, PIL Image, or numpy array.

Union[np.ndarray, list]

required

Joint positions as numpy array or list.

Union[np.ndarray, list]

required

Gripper position as numpy array or list.

Union[np.ndarray, list]

required

Robot state as numpy array or list (concatenation of joints and gripper).

Optional[str]

Optional task prompt string.

Optional[Dict[str, Any]]

Optional real-time-chunking params (see :py:meth:preprocess). When set, the server runs the RTC path and returns a prev_chunk field to feed back on the next call.

Optional[Union[np.ndarray, list]]

Previous predicted chunk fed back as RTC guidance (None on the first call).

float | None

Optional timeout in seconds for the HTTP request.

Returns

Action dictionary containing:

“actions”: numpy array of shape (horizon, action_dim) with predicted actions
“prev_chunk”: (RTC only) chunk to feed back on the next call
Other fields from policy.infer() (e.g., “state”, “policy_timing”, etc.)

Example

from grid_cortex_client import CortexClient, ModelType
import numpy as np
client = CortexClient()
base_img = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
wrist_img = np.random.randint(0, 255, (224, 224, 3), dtype=np.uint8)
joints = np.array([0.1, 0.2, 0.3, 0.4, 0.5, 0.6])
gripper = np.array([0.5])
state = np.concatenate([joints, gripper])
action = client.run(
    ModelType.PI05,
    base_rgb=base_img,
    wrist_rgb=wrist_img,
    joints=joints,
    gripper=gripper,
    state=state,
    prompt="pick up the cup"
)
print(action["actions"].shape)

Overview

Cortex Models

pi05

Parameters

Returns

Example

​Parameters

​Returns

​Example

Parameters

Returns

Example