The Dataset Manifesto
The world knows how to work.
Intelligence needs a way to learn from it.
Humanity has spent generations learning how to move, make, build, repair, and adapt. That intelligence lives in the hands of a warehouse worker who adjusts a grip before a box slips. In the judgment of a technician who recognizes that something is wrong. In the coordination of a team moving through a crowded factory. In the small corrections that turn an unfamiliar situation into a completed task. This is knowledge expressed through action. We believe it deserves an infrastructure as powerful as the one we built for information.
Physical intelligence begins with experience.
A machine that works in the world must learn more than what things look like. It must learn how they behave. How objects move under pressure. How environments change. How actions affect outcomes. How to recognize uncertainty, recover from failure, and work around people. The physical world is not a collection of perfect demonstrations. It is changing conditions, interrupted workflows, imperfect visibility, and decisions made in motion. The future of physical AI must be grounded in that reality.
Dataset exists to build the connection between the world where work happens and the intelligence that can learn from it.
We are building infrastructure, not a collection of recordings. A camera can record a movement. Making that movement useful requires context. What was the person trying to accomplish? What changed? Which objects were involved? Did the action succeed? What happened when it did not? Where did the information come from, and what is someone authorized to do with it? We are building the systems that connect these answers: capturing authentic experience, connecting it to operational context, structuring it into meaningful sequences, establishing its rights and provenance, and making it available for training, evaluation, and deployment. Our ambition is to make physical experience a dependable foundation that others can build upon—discoverable, usable, and continuously improved. The unit of progress is not another hour of video. It is another part of the world that intelligence can learn to understand.
The whole environment matters.
Work does not begin at the moment a hand touches an object. It begins with finding the object, understanding the space, navigating around an obstacle, waiting for a colleague, noticing a change, and deciding what to do next. We believe both action and context belong in the foundation. Successful tasks teach execution. Corrections teach adaptation. Transitions teach continuity. The surrounding environment gives those lessons meaning. Our goal is to preserve the relationship between perception, action, and consequence—not flatten experience into disconnected clips.
Human expertise is the foundation.
The people who know the work are not background scenery. Their skill is what makes this knowledge valuable. We believe infrastructure built from human experience must respect the people and businesses that make it possible. That means consent, clear permissions, privacy, and accountability built into the system—not added after the data has already changed hands. Trust must travel with the data. Our measure of progress must also reach beyond machines: safer work, stronger businesses, less exposure to dangerous tasks, and greater possibilities for people.
No team should have to rediscover the physical world alone.
We envision a future in which a research team can access the experience it needs without rebuilding an entire collection operation. Where an unfamiliar failure becomes a targeted learning opportunity. Where evaluation reveals what a model still needs to understand, and new experience helps close that gap. A foundation that supports many models, many machines, and many industries—not a single device or a single approach.
We begin in the places where physical intelligence is practiced every day.
Warehouses, distribution centers, factories, and working industrial environments. But our ambition reaches further. We want to help make the physical world learnable.
This is the foundation we are here to build.
We believe the next era of intelligence will be shaped by how faithfully it learns from reality—and how responsibly that learning is made possible. Dataset's role is to build the infrastructure beneath that progress. To turn experience into a lasting resource. To make human know-how a foundation for new capabilities. To help the intelligence we create become useful in the world we inhabit.
The world knows how to work. We are building the infrastructure that helps intelligence learn from it.
Talk to us →