Data pipeline
From capture to
training data
Connect task capture, action annotation, quality review and export in one workflow. Records from each stage stay with the data, making training datasets traceable from capture to delivery.
Production workflow
From recording to a reviewed dataset
Each recording is linked to its task, capture device and review history. Reviewers can trace an issue back to the affected segment.
Capture platform
- Before recording, operators select the task and check the main and auxiliary camera feeds and depth images.
- During teleoperation, they monitor control latency, ROS nodes and recording topics.
- Operators check the recording. The platform aligns the streams by timestamp and uploads them together.
CaptureCamera preview and recording Choosing streams to record Annotation and review
- Annotators label each action and mark its start and end times.
- A second reviewer plays back the camera views together to check the action and its start and end times.
- If a label or boundary is wrong, the annotator corrects it and the reviewer checks the segment again.
AnnotationMarking the start and end of an action ReviewChecking an action across camera views Data management platform
- The platform groups recordings by project and task and keeps the review history with each batch.
- Reviewers spot-check annotated batches. A batch that fails goes back for correction and another spot check.
- The team exports each approved batch in the agreed format, with a file manifest and a quality report.
Daily recording volume and capture time ExportChoosing an export format
What we check before delivery
- Raw recordings
- Match the recorded streams to the task requirements and check that their timestamps line up.
- Labeled batches
- Check the spot-check results in the quality report. If a batch needed corrections, review the results of the follow-up check.
- Acceptance
- Check the files and formats against the delivery manifest. When the project calls for them, the batch report also lists discarded recording time and P95 latency for teleoperation.
What’s in a delivery
A/B/C describe the depth of data content. ZIP and LeRobot describe how that content is packaged and exported. Agree the required fields and export format for each project.
Raw recordings
Timestamped camera streams with inertial readings for human demonstrations, or robot states and control commands for teleoperation.
Annotated segments
Each segment has an action label and start and end times. A second reviewer checks the annotations.
3D reconstructions
For selected tasks, annotated segments can include estimates of body pose, hand meshes, object trajectories or hand–object contact. The fields are agreed for each project.
File formats
Raw data (ZIP)
Preserves the full recording in its original file structure, including the time outside labeled segments.
Annotated clips (ZIP)
Exports the intervals marked by annotators. Each clip contains synchronized data for its time range.
LeRobotDataset v3.0
Packages annotated intervals as Parquet files, video and metadata. The export distinguishes trainable actions from observation-only data.
data_packages/ └── [package ID]/ └── original recording files manifest.json task-level file manifest