Physical AI Glossary: 30 Terms Your Team Needs Before Building a Robotics Data Pipeline

Author: Jesal Thakkar · Reviewed by: Praveen Kumar · Published: 07/09/26 · Last updated: 10/09/26 · Review cycle: quarterly

Most robotics data projects do not fail on the modelling. They failed because two teams used the same word to mean different things, and nobody noticed until delivery. A vendor says "trajectory," a client hears "episode," and the mismatch surfaces only when the delivered dataset cannot answer the question it was collected for.

A physical AI glossary defines the vocabulary used across robot perception, action, model architecture, data collection and compliance. The 30 terms below are the ones that actually appear in a robotics data specification and change what gets collected, so getting one wrong costs a re-collection rather than an edit. That cost is compounding right now, because the pace at which teams are standing up physical AI program has outrun the pace at which the field has agreed on what its own words mean.

Terms are grouped below by where they show up in a pipeline, moving from the umbrella terms that scope a program through to the compliance vocabulary that governs consent. Contested definitions are marked as contested rather than smoothed over, because pretending consensus exists is how the mismatches start in the first place.

The Three Umbrella Terms

Physical AI is an industry category, popularized by NVIDIA from 2024, covering machines that perceive, reason about and act in the physical world. It is a boundary drawn around a market rather than a technical claim, which means it cannot be right or wrong, only useful or not. Treat it as the word for scoping a program or a budget, not for specifying a dataset.

Embodied AI is a research hypothesis, traceable to Rodney Brooks in 1991, holding that intelligence requires a body and emerges from sensorimotor interaction with an environment rather than from internal representation. Unlike physical AI, this is a falsifiable claim with a literature behind it. This one is contested: some sources treat it as a strict subset of physical AI, others as a synonym. It is neither. The two words describe different kinds of claims, which is why comparison tables built to distinguish them keep disagreeing with each other.

Robotics AI is not a term of art. No academic, standards-body or industry definition exists for it. It appears in search queries and vendor copy, but no source actually defines it. The established phrasings are "AI in robotics" or "robot learning." If this phrase shows up in a specification handed to your team, ask what the writer meant, because they have not told you, and the ambiguity is theirs to resolve before work starts.

What the Robot Senses

Proprioception is the robot's sense of its own body state: joint angles, joint velocities, torques and end effector pose, reported by internal sensors rather than observed from outside. This is the channel most often missing from datasets sold as robot data. Third-person video captures none of it, and it cannot be reconstructed after the fact once collection is over.

Sensor fusion is the combination of multiple sensor streams into a single coherent estimate of the environment or the robot's state. The hard part is not the combining, it is time synchronization. Streams recorded at different rates with unaligned clocks fuse into an estimate that looks confident and is wrong, which is a failure mode that surfaces late and is expensive to trace back to its source.

Egocentric video is video recorded from the first-person viewpoint of the acting agent, rather than from a fixed observer position. For embodied learning the viewpoint is the point: it encodes what the agent could see at the moment it decided to act. Third-person footage of the same task is a different data type, not a lower-quality version of the same one, and the two are not interchangeable in a training pipeline.

Point cloud describes a set of points in three-dimensional space, each with coordinates and often color or intensity, typically produced by LiDAR or a depth camera. Point clouds are unordered and variable in density, which is why annotation cost per frame runs well above what a comparable 2D bounding box task costs.

What the Robot Does

Degrees of freedom, or DoF, is the number of independent parameters defining a robot's configuration, usually one per actuated joint. A 7-DoF arm can reach a given pose in more than one configuration; a 6-DoF arm generally cannot. This matters for data collection because demonstrations recorded on one DoF count do not transfer cleanly to a robot built with a different one.

End effector refers to the tool at the end of a robot arm that interacts with the world: a gripper, suction cup, welding tip or hand. Two robots with identical arms and different end effectors are, for training purposes, different embodiments, and datasets collected on one should not be assumed to generalize to the other.

Action space is the set of all actions a policy can output, together with their representation, for example joint velocities, end effector deltas, or discretized tokens. Two datasets can describe the same physical motion in incompatible action spaces. Confirm the action space before assuming any two datasets can be combined for training.

Trajectory is the time-ordered sequence of states and actions produced as a robot performs a task. This is the atomic data unit in most robot-learning datasets, and scale in this field is typically reported in trajectory counts; Open X-Embodiment, for instance, is measured this way.

Episode is one complete recorded attempt at a task, from initial state to termination, containing a trajectory plus its metadata and outcome. It is near-synonymous with trajectory in practice, and the literature uses both terms loosely. The useful distinction, where anyone bothers to draw one, is that a trajectory is the motion and an episode is the recorded attempt including whether it succeeded. Failed episodes still contain trajectories and are often the most informative records in a set, since they show a model exactly where things went wrong.

How the Model Is Built

A vision-language-action model, or VLA, is a model that maps visual observation and a language instruction directly onto robot actions. The action head is what makes it a VLA rather than a vision-language model, and that head requires action-labelled data, which is the expensive kind to collect at scale.

A world foundation model is trained to predict how an environment changes in response to an action, learned from large-scale video rather than from authored physics. This is distinct from a simulator: a simulator has its physics written into it by an engineer, while a world model infers those physics by watching. That difference is why a simulator tends to be precise within its authored rules while a world model generalizes further beyond them.

Imitation learning is training a policy to reproduce behavior demonstrated by an expert, rather than discovering behavior through trial and reward. It is the dominant approach in robot manipulation today, and it is the reason demonstration data, not compute, is the bottleneck most teams actually hit.

Behavioral cloning is the simplest form of imitation learning: supervised learning that maps observations directly to the demonstrator's actions. Its known weakness is compounding error. Small deviations take the robot into states the demonstrator never visited, where the policy has no guidance left to draw on. Coverage of recovery behavior in the training data is the practical mitigation most teams reach for.

Diffusion policy is a policy that generates actions by iterative denoising, the same generative mechanism used in image diffusion models, applied instead to action sequences. It handles multimodal behavior well, meaning cases where several different actions are all equally valid responses to the same state. Both Cosmos and GR00T use diffusion transformers in their action paths for this reason.

Where the Data Comes From

Teleoperation is a human operator remotely controlling a robot while its sensors and actions are recorded, producing demonstrations directly on the target hardware. The defining constraint here is arithmetic: a single robot yields at most twenty-four hours of collection per day, and adding robots and operators costs linearly rather than getting cheaper at scale. There is no scraping equivalent for this kind of data.

Kinesthetic teaching is recording a demonstration by physically guiding the robot's limbs through a task by hand, rather than driving it through a controller. It is cheaper and faster to set up than a teleoperation rig, but it produces no operator-viewpoint imagery and can distort force readings, since the human is bearing part of the load during the demonstration.

Open X-Embodiment is an open collaborative dataset of over one million real robot trajectories spanning 22 robot embodiments, assembled from 60 existing datasets contributed by 21 institutions. It is the largest open real-robot dataset currently available, and a realistic anchor for scale expectations: one million trajectories is enormous by robotics standards and negligible next to the clip counts used to pre-train world models, which is a gap teams frequently underestimate when planning a collection budget.

Simulation and the Reality Gap

The sim-to-real gap is the drop in performance when a policy trained in simulation is deployed on physical hardware. It is caused by mismatch in physics, sensor noise, lighting, materials and timing between the simulated and real environments. Closing it is a data design problem more than a rendering problem, which is a distinction that changes where a team should actually spend its engineering time.

Domain randomization deliberately varies simulation parameters across training so the policy cannot overfit to any single configuration. The evidence indicates that spatial factors such as surface height dominate appearance factors such as texture, and that per-frame variation outperforms per-episode variation. Every randomization range is effectively a claim about the deployment environment, so those ranges should be measured against real conditions rather than guessed at.

A digital twin is a synchronized virtual replica of a specific physical asset or environment, kept current with real-world state. It is distinct from a generic simulation: a twin is tied to one particular deployment, not a general-purpose environment. Vendor usage of the term is loose, so it is worth confirming whether a live data link back to the physical asset is actually implied or whether the term is being used as a synonym for "simulation."

Synthetic data is training data generated rather than captured, produced from simulation, rendering or generative models. It multiplies coverage from a given set of real demonstrations, extending what a limited collection budget can support. It does not remove the need to know your deployment conditions in detail, since those conditions are exactly what determine what should be generated in the first place.

Training Stages

Post-training is adapting a pre-trained foundation model to a specific embodiment, task set and environment, using targeted data rather than web-scale corpora. This is the stage that consumes your own collected data, and it reopens every time a team adds a site, a task or a hardware revision. Pre-training is a fixed cost that somebody else already paid; post-training is a variable one that scales directly with how much of the deployment surface a team is trying to cover.

A conditioning schema is the structured set of variables a model is conditioned on at training and inference: embodiment, camera pose, instruction phrasing, environment state, and outcome. Fixing this schema before collection begins is what makes a dataset filterable and satisfiable afterward. Datasets collected without one are frequently unusable for post-training regardless of episode count, because the conditioning information simply cannot be reconstructed later once collection is finished.

Consent and Compliance

Informed consent is agreement to participate given freely and specifically, by someone who understands what is being recorded, why, how it will be used and for how long. In physical AI collection this reaches beyond the operator alone. Bystanders, household members and anyone appearing in an egocentric frame are in scope. Batch-level attestation does not meet this standard; consent has to be tracked per contributor, not per collection run.

PII in video data refers to personally identifiable information captured incidentally in footage: faces, voices, licence plates, screens, documents, and identifying features of a home or workplace. Robotics footage is unusually exposed here because the camera moves through real environments rather than framing a single controlled scene. Redaction needs to be built in as a pipeline stage, not treated as a post-hoc favor done after the fact.

Data subject rights are the rights an individual holds over personal data about them, typically including access, correction, erasure and grievance redressal. The operational consequence is architectural: honoring an erasure request means being able to locate every episode a given contributor appears in, and that is only possible if provenance was recorded per episode at the time of collection.

Data provenance is the recorded origin and chain of custody of a dataset: who contributed it, under what consent, on what hardware, when, and what has been done to it since. Provenance logged per episode rather than per batch is what makes a dataset auditable in the first place. It also functions as a quality mechanism, because it is what lets a team trace an anomalous model behavior back to the collection conditions that actually produced it. This is where ethical sourcing stops being a compliance checkbox and becomes a quality argument: contributors who gave specific, informed consent tend to produce more intentional, consistent recordings than a batch collected under loose or bundled terms, and that consistency is visible in the data itself.

How to Use This Glossary

Copy the four lines that matter into your next data brief rather than trusting vocabulary alone to carry them: viewpoint, embodiment, sensor channels, and whether the demonstrator's body is part of the label. That is the practical test of whether a term has actually been understood between two teams, not just used by both.

A brief that specifies those four survives any drift in what the industry decides to call this field next, and this field renames itself roughly every two years. Everything above is a reading aid for other people's documents, useful when a vendor proposal, a paper or a job description puts a term in front of you and the meaning is load-bearing. It is not meant to be copied wholesale into your own brief, because a specification built entirely out of glossary terms inherits every ambiguity the glossary just finished describing.

If you are scoping a collection against a specific embodiment and deployment environment, multimodal data collection covers the kind of coordinated capture these terms describe, across sensor channels, viewpoints and embodiments in a single specification.

Frequently Asked Questions

Q. Are "trajectory" and "episode" the same thing in robot datasets?
A. They are used near-interchangeably and most datasets pick one. Where the distinction is drawn, a trajectory is the time-ordered sequence of states and actions, while an episode is one complete recorded attempt including its metadata and outcome. The practical consequence shows up at counting time: a dataset reporting episodes may include failures, while one reporting trajectories may have filtered them out. Ask which is being used before comparing two datasets by size.

Q. How many terms does a data brief realistically need to define?
A. Four, not thirty: viewpoint, embodiment, sensor channels, and whether the demonstrator's body is part of the label. A glossary is useful for reading other people's documents, but a brief that specifies those four is unambiguous even to a vendor using different vocabulary. Most delivery disputes in robotics data trace back to one of those four being assumed rather than stated outright.

Q. Which of these terms are genuinely contested?
A. Three of them. "Embodied AI" versus "physical AI" has no industry consensus, and sources actively contradict each other on how the two relate. "Robotics AI" has no definition at all. "Digital twin" is used loosely, sometimes meaning a live-synchronized replica and sometimes just a detailed simulation. The rest are stable enough that two engineers will agree on them without argument. Treat the contested three as requiring clarification whenever they appear in a contract or specification.

Q. Why does viewpoint matter so much in robotics data?
A. Because it determines what the data can be used for, and it cannot be changed after the fact. Egocentric footage records what the agent could see at the moment it acted, which is the information a policy actually needs to learn from. Third-person footage records what happened, which is a related but different thing. A well-rigged egocentric capture can often be used to derive third-person coverage, but the reverse is not true.

Where This Leaves Your Data Pipeline

Vocabulary is the cheapest thing to get right in a robotics data program, and the most expensive to get wrong, because a misunderstood term does not surface as an error message. It surfaces months later as a dataset that cannot answer the question it was built to answer, and by then the fix is a re-collection, not an edit. If your team is scoping a robotics data pipeline and wants the four load-bearing terms fixed in a specification before collection starts, talk to the FutureBeeAI team to map the right approach for your embodiment and deployment environment. If you are still at the research stage and want to see what a properly specified dataset looks like, browse FutureBeeAI's library of ethically sourced datasets across multimodal categories.