GRID Docs | General Robotics

Extract DINOv2-base patch tokens and the [CLS] token for one RGB image.

Parameters

ImageInput

required

RGB image as file path, URL, PIL Image, or numpy array.

float | None

Optional timeout in seconds for the HTTP request.

Returns

Dict[str, np.ndarray]: Dict with patch_tokens of shape (16, 16, 768) and cls of shape (768,), both float32 and L2-normalized per vector. Pool patch_tokens over a foreground mask and re-normalize to compare crops by cosine similarity without their background.

Example

from grid_cortex_client import CortexClient, ModelType
import numpy as np
from PIL import Image
client = CortexClient()
crop = np.array(Image.open("lock_crop_224.png"))
mask = np.array(Image.open("lock_mask_224.png")) > 0      # (224, 224) bool
out = client.run(ModelType.DINOV2_BASE, image_input=crop)
out["patch_tokens"].shape, out["cls"].shape               # (16, 16, 768), (768,)
fg = mask.reshape(16, 14, 16, 14).mean((1, 3)) > 0.5      # (16, 16) patch mask
v = out["patch_tokens"][fg].mean(0)
v /= np.linalg.norm(v)                                    # cosine == v @ w

Overview

Cortex Models

dinov2-base

Parameters

Returns

Example

​Parameters

​Returns

​Example

Parameters

Returns

Example