Big datasets. How much contact?
Episode counts tell only part of the story. A closer look at open robot datasets through force, touch, and timing.
INTERACT ROBOTICS · Insights · Reviewed September 7, 2026
The question after scale.
Open datasets have broadened robot learning. Diverse objects, environments, and robots matter for contact intelligence too. But more demonstrations do not necessarily mean richer contact observations. A camera scene, a commanded motion, and a measured collision force are different kinds of information.
After “How many hours?” comes “Which signals, under which conditions?” A robot having a force sensor, a schema including a force field, and a downloaded episode containing valid force measurements are three separate facts.
What the public documentation describes
| Dataset | What is documented | What contact learning needs to check |
|---|---|---|
| Open X-Embodiment | A repository combining datasets from multiple institutions and robots | Sensors and schemas vary across constituent datasets. A single tactile-data label cannot describe the entire collection. |
| DROID | 76,000 demonstrations and 350 hours of interaction in diverse real settings | Check the schema for the contact channels you need, alongside its rich vision, robot-state, and action data. |
| RH20T | Includes force and torque, with fingertip tactile data in some sequences | Check each sensor’s acquisition rate and the sensor configuration of the specific sequence. |
| RoboMIND 2.0 | Describes 12K tactile-enhanced episodes within more than 310K trajectories | Distinguish the full collection from its tactile subset. |
| AgiBot World Alpha | The Alpha data card lists effort and wrench as currently unavailable | Do not generalize this release’s status to every AgiBot release. |
Official project and repository documentation reviewed September 7, 2026. This table describes observation scope, not a ranking of datasets.
References
A shared timeline is not a shared sampling rate.
Reducing video and force to a low common frame rate can erase brief contact changes. Preserving original rates and timestamps gives learning methods more choices in how to construct time windows. Frequency-aware work such as ManipForce explores the value of high-frequency force.
Force needs units, coordinate frames, sensor zeroing, and tool-weight compensation. Touch needs mounting locations, contact areas, and calibration conditions. Missing data, saturation, retries, and definitions of success matter before equal numbers across demonstrations can be compared.
The data we want to add.
Saying there is no contact data would overlook existing research. Our challenge is to record varied real-world contact tasks across meaningful condition changes, connected to the operator’s response.
Dream Set is designed to add operator force and stiffness adjustment, actual contact force, fingertip touch, and attention context to video and action. We distinguish measurements from estimates and annotations, defining acquisition scope and quality criteria for each process.
References
- ManipForce / Frequency-aware RGB–force learning ↗
An example of learning across sensors with different temporal resolutions.