Robot learning turns demonstrations and feedback into policies that must be tested beyond familiar training conditions. Photo: Natalia Dziubek / Unsplash
Learn how robots turn human demonstrations into actions, why imitation can fail, and how recovery data and real-world tests improve learned behavior.
A person moves a robot arm toward a block, closes the gripper and places the block in a bin. After enough examples, a trained model attempts the job itself. The first successful pickup is an important milestone, but it leaves the harder question unanswered: what exactly did the robot learn?
Robot learning uses data and experience to improve how a robot chooses actions. One important approach, imitation learning, teaches a model from demonstrations of a task. Instead of writing a separate movement rule for every situation, developers train a policy to connect observations with suitable actions. The quality of that connection determines whether the robot can adapt when the object moves or the first grasp misses.
This guide follows an illustrative block-sorting project through demonstration collection, training and testing. It explains the methods behind research systems such as ALOHA and Open X-Embodiment, while keeping a practical distinction in view: learning to reproduce a familiar task is different from becoming a reliable autonomous worker.
What Does a Robot Actually Learn?
A policy is a function or model that selects actions from the information available to it. That information might include camera images, joint positions and a task instruction. The output could specify where the gripper should move, how the arm’s joints should be positioned or a short sequence of future actions.
Learning changes the model’s parameters so its predictions better meet a training objective. In a basic approach called behavioral cloning, that objective is to reproduce the actions demonstrated by an operator in corresponding situations. The model is not necessarily learning the operator’s full reasoning, and matching demonstrations does not by itself prove that it understands every physical consequence of an action.
Training and execution are also different phases. During training, software updates the policy using recorded examples or other learning signals. During execution, often called inference, the trained policy selects actions from new observations. A deployed robot does not automatically retrain itself whenever it makes a mistake; collecting that experience and using it to update the model requires a learning process.
The policy remains one part of the machine. Sensors, controllers and mechanisms still determine what it can observe and physically achieve. Our physical AI guide explains how those components work together from camera input through motion and feedback.
A Demonstration Is More Than a Video
In our sorting example, an operator repeatedly picks up a block and places it in a designated bin. One collection method is teleoperation: a person controls the robot through a device such as a leader arm or joystick. Hugging Face’s LeRobot tutorial describes a workflow that records demonstrations, trains a policy and evaluates the resulting behavior on real hardware.
A useful recording links what the robot observed with the actions taken at that time. Depending on the setup, this can include camera views, robot state, action commands and task information. LeRobot’s dataset format organizes multimodal time-series data and camera video, rather than treating each image as an unrelated example. A complete attempt is commonly called an episode; its sequence of observations and actions forms a trajectory.
Imagine that a recording pairs an image captured before the gripper reaches the block with a command issued after the grasp. The example now teaches the wrong relationship between observation and action. For the sorting project, checking timing, missing frames and recording consistency would be as important as checking whether the final block reached the bin.
Ordinary footage of a person sorting objects can provide useful visual information, but it does not automatically contain the commands a particular robot needs. The UMI project, presented at RSS 2024, addresses that gap with handheld grippers and cameras designed to collect demonstrations that can train robot policies. Its interface includes action representations and timing considerations for transferring the behavior to hardware. This is a designed teaching system, rather than simply asking a robot to copy an arbitrary online video.
Start With a Task You Can Measure
“Sort these objects” leaves too much unspecified for a useful first experiment. For our hypothetical project, a clearer task would be to move one red block from a defined tabletop area into an empty bin, without dropping it outside the bin. That definition establishes what counts as success and limits the conditions the first policy is expected to handle.
Now consider the demonstrations. If the block always starts in the same place and orientation, the recordings may not reveal how behavior should change when it moves. If the bin never changes position, a model might rely on that fixed arrangement. A large collection of nearly identical attempts can still leave important parts of the intended task uncovered.
Useful variation should answer a specific question. Moving the block across its permitted starting area tests position coverage; turning it tests orientation coverage; adding a similar-looking object tests whether the target remains identifiable. These are examples for designing the sorting project, not a universal checklist that every robot must follow unchanged.
There is also a tradeoff between coverage and clarity. Adding many objects, viewpoints and task variations at once can make it difficult to identify why an early policy fails. A manageable approach is to establish a repeatable baseline, expand the task deliberately and record what changed. The resulting dataset should have a clear relationship to the behavior the robot will be asked to perform.
From Recorded Examples to a Working Policy
Training repeatedly presents recorded observations and target actions to a model, measures prediction error and adjusts its parameters. Developers normally keep some data separate for validation so they can compare model versions without judging them only on the examples used for fitting. An improving training score is useful, but it does not show that a robot can complete the task on hardware.
Robot actions are sequences, which creates design choices beyond predicting one movement at a time. ALOHA research introduced Action Chunking with Transformers, or ACT, which predicts chunks of actions from observations. The original work paired a teleoperation system with imitation learning for fine manipulation. Action chunking is one way to address the difficulty of learning coordinated behavior over time; it is not evidence that a robot can execute indefinitely without fresh observations.
LeRobot’s ACT implementation takes inputs including camera images and robot joint positions and produces future action sequences. The important point for a beginner is the relationship between the inputs and the intended behavior. A model cannot be expected to use information that the recording and execution setup never supplies.
For our project, a policy trained with one camera arrangement would need to be evaluated carefully if the camera moved. A policy trained to command one arm would need compatible action handling before use with another. Model size alone does not resolve those mismatches. The dataset, software interface and physical robot have to fit together.
Why a Robot That Learned the Task Can Still Fail
Suppose the sorting robot approaches the block slightly off center and nudges it sideways. Its next camera image now shows a situation absent from the successful demonstrations. A second imperfect action may move it further from familiar conditions. This is a central difficulty in imitation learning: the policy’s own choices affect the observations it will receive next.
The DAgger paper formalized an iterative response to this problem. Dataset Aggregation collects states visited while a policy runs, obtains expert actions for those states and trains on the accumulated data. The aim is to teach behavior for situations the learner actually encounters, rather than relying entirely on the cleaner situations an expert visits.
For the block example, the missing lesson might be how to reposition after a poor approach, not how to repeat another perfect pickup. Adding corrective examples targets that gap. Simply storing a video of the failure does not necessarily provide the desired action labels; the training process needs to know what useful behavior should replace the mistake.
Human supervision can also change what a demonstration appears to prove. A run completed after an operator rescues the grasp is evidence about an assisted system. It should not be counted as an uninterrupted autonomous success. That distinction remains important even when the intervention data is valuable for future training.
Teaching Recovery With Human Feedback
Hugging Face’s human-in-the-loop workflow runs a trained policy while an operator watches, takes over when needed and records recovery or correction. Those segments can be combined with earlier demonstrations for another round of training. This creates a targeted cycle: deploy under supervision, collect useful corrections, update the policy and evaluate again.
In the sorting project, an operator might correct the arm after it bumps the block, then return control once the task is in a recoverable state. Recording who controlled each segment helps distinguish autonomous behavior from human actions. The goal is not merely to finish today’s attempt, but to capture a lesson that can improve a later model.
This process does not imply that every correction transfers successfully or that human intervention can immediately be removed. A revised policy may improve at one failure mode while becoming worse elsewhere. Evaluation should therefore include earlier capabilities as well as the newly targeted problem, so progress is measured across the supported task.
Where Simulation and Shared Robot Data Help
Demonstrations are not the only route to learning. Reinforcement learning optimizes behavior using rewards from interaction, whereas imitation learning uses demonstrated behavior as a teaching signal. The two can be combined. A developer might start with demonstrations and then refine behavior using another objective, provided the resulting policy is evaluated against the actual task requirements.
Simulation can make large numbers of experiments practical without performing every attempt on physical hardware. Nvidia’s Isaac Lab supports both imitation and reinforcement learning in simulated environments. For our example, a virtual task could vary block positions and other conditions. Its usefulness still depends on whether the simulated observations and interactions represent the aspects that matter on the real table.
Shared datasets offer a different form of scale. Open X-Embodiment assembled more than a million real-robot trajectories spanning 22 robot embodiments. The collaboration explored whether experience gathered on different platforms could improve learned policies. It provides a concrete example of pooling robot experience rather than collecting every training example anew for one machine.
Neither approach eliminates the need to match data with the intended deployment. A large shared dataset may contain valuable patterns while leaving a particular gripper, object or task poorly covered. A successful simulated policy may still fail when real contact or sensing differs. Scale gives developers more material to learn from; real-world testing establishes whether it helped.
How to Test Whether the Learning Worked
Begin with the success definition established before training. For the block sorter, reaching toward the object is progress; closing the gripper is an attempted grasp; placing the correct block in the bin is the desired outcome. The evaluation should distinguish these events rather than letting a visually convincing movement stand in for task completion.
Keeping entire attempts separate is more informative than randomly mixing nearby frames from the same recordings into training and testing. Adjacent images can be nearly identical. For a stronger test of our example, reserve complete demonstrations and also run new physical trials with starting conditions chosen independently of the training recordings.
Then test the kinds of change the system is supposed to handle. The LIBERO-plus benchmark illustrates this approach in simulation by perturbing dimensions such as object layouts, camera viewpoints, lighting, language instructions and sensor noise. Its results concern its specified benchmark conditions; they do not replace testing the actual deployed robot.
A useful evaluation record for the sorting project would include:
Completed attempts: Report successes alongside the total number of trials and the conditions tested.
Human assistance: Record interventions separately from uninterrupted autonomous completions.
Failure and recovery: Explain whether a missed grasp led to a retry, a stop or an incorrect success claim.
Time and side effects: Measure task duration, dropped objects and other outcomes relevant to the job.
Model version: Identify the policy and setup used so comparisons remain meaningful after updates.
Success rate alone can conceal important differences. A robot that completes a task slowly but consistently may be useful in one setting, while frequent supervision makes it unsuitable in another. Nor does a good task score establish that the complete machine is safe to operate around people. Learning performance and deployment safety need to be assessed separately.
What to Look for in the Next Robot Demo
When a developer says a robot learned a skill from a small number of demonstrations, ask what varied between those demonstrations, what was held out for testing and how much human assistance remained. Also ask whether the system learned a new policy, adapted a pretrained one or simply executed an existing skill. Those are different achievements, even if their videos look similar.
There is no universal demonstration count that makes a task solved. Requirements depend on the starting model, hardware, task complexity and range of conditions. A narrow but clearly measured capability can be valuable. Claims become less informative when the task boundary disappears and a selected success is presented as general competence.
Good robot learning is an iterative engineering process: choose a measurable task, collect relevant experience, train, test and use failures to decide what to improve. For our block sorter, the decisive result is not that it once copied a person’s motion. It is that the system can repeat the intended job under stated conditions, recognize when it has failed and provide evidence for the autonomy being claimed.
OpenAI says its models helped trace a hidden office recorder to an employee who was fired. The company says an FBI investigation is ongoing after an arrest.
