Moving a plastic tote from a shelf to a conveyor belt is the kind of manual warehouse task no one really thinks is difficult. But it’s still hard to teach a robot to do it.
Recently, Agility Robotics’ Digit humanoid robot performed this task more than 100,000 times at a GXO Logistics warehouse in Flowery Branch, Georgia. You might assume this simply involved programming a robot to follow instructions, but in reality, it required the robot to learn from real-world experience.
Agility Robotics’ Digit Humanoid Moving Totes (Source)
Unlike a controlled demonstration, those 100,000 movements took place in a live warehouse where inventory changes, object positions shift, and conditions vary throughout the day. In that type of environment, simply following pre-programmed instructions isn’t enough. Every task comes with small but important differences, requiring robots to adapt to changing conditions much like human workers do, almost unconsciously.
That learning often starts with teleoperation, where a human operator remotely steps in and guides a robot through a task it can’t confidently complete on its own. The robot records what it sees, how the operator responds, and which actions lead to a successful outcome. Those demonstrations can then become training data, helping it handle similar situations more independently over time.
Today, this human-guided robot learning is becoming a central part of how warehouse robots improve. Rather than using teleoperation to just keep operations running, companies are also using it to generate the real-world data robots need to become more capable over time.
Let’s take a closer look at these warehouse robots, how teleoperation can be used to teach them various tasks, and why that training is important for real-world warehouses.
Most warehouse robots are built to handle one of two main types of tasks. Autonomous mobile robots (AMRs) transport inventory between storage locations, while manipulation robots handle pick-and-place work, sorting items and positioning them where they need to go. In many warehouses, these systems work together to carry out picking and other fulfillment operations.
This type of work may seem repetitive, but the environment in which it is happening is constantly changing. Products come in different shapes, sizes, and packaging, and their position can change every time they are picked up, stored, or transported. Warehouse robots need to recognize these differences and respond correctly before they can complete a task.
For example, Amazon’s Vulcan robot uses cameras to inspect fabric storage pods and software to identify the items it sees before deciding how to pick them up with suction cups. This ability to see and understand its surroundings is enabled by computer vision. Using a combination of camera systems and computer vision, the robot can successfully handle many of the items it encounters.
A Glimpse of Vulcan Performing Tasks on Tightly Packed Bins (Source)
However, tightly packed bins, awkward item positions, and unfamiliar object arrangements can still make picking much more difficult. These situations aren’t edge cases. They are part of everyday warehouse automation use cases, and they are exactly where full autonomy still has limits.
Next, let’s take a closer look at some common warehouse operations these robots are used for and where they may encounter challenges:
Warehouse robots come in different forms, shapes, and sizes, with each type designed to handle specific tasks. Some move goods around the facility, while others pick products, transport pallets, or bring inventory to workers.
Here are five of the most common types of robots used in warehouses:
An Example of an Autonomous Forklift Moving Beverage Pallets (Source)
While these robots are built to work on their own, they still need human help. Warehouses are busy, constantly changing environments where robots may encounter situations they aren’t trained to handle. In such cases, a human operator can step in and sort things out, which is often driven by a process known as teleoperation.
The combination of autonomous robots and remote human support is becoming an important part of smart warehouse operations. Using teleoperation, experts can guide a robot through complex issues and keep work moving without interruption.
But how does it work? Such a system works by using live camera feeds and sensor data. With the feeds and data, an operator can help a robot complete the task, then return control once the issue is resolved.
In this way, teleoperation works alongside autonomy without causing issues. It reduces downtime, keeps warehouse operations running smoothly, and allows a single operator to support multiple robots when needed.
Teleoperated data that is collected during this process also has many advantages. A good example of this is Figure AI’s work developing Helix for logistics package sorting. The company used teleoperators to guide robots through sorting tasks, generating data that captured both successful actions and the corrections made when errors occurred. By learning from these demonstrations, robots can improve their performance in similar situations and operate more independently over time.
Figure AI’s Helix Robots Sorting Packages at a Logistics Facility (Source)
As you explore warehouse robots, you may be wondering: what type of data is collected using teleoperation? The data collected includes what the robot sees, its movements, the operator’s actions, and the sequence of decisions made throughout the task. This creates a structured example (or demonstration) showing how an experienced person responds when the robot encounters a situation it can’t handle on its own.
By using these demonstrations as training data for the AI models in the robots, the robots can develop more reliable ways to perform tasks in difficult environments. They let robots learn what action to take and how to adapt when the situation changes.
Mobile ALOHA is another example of a teleoperated system that does this. Human professionals guide the robots to perform tasks such as storing heavy pots in a cabinet, while data from the process is recorded by cameras and sensors for training. The data allowed the robot to adapt and recover from errors rather than repeat a fixed sequence of actions.
Mobile ALOHA Learning a Multi-Step Storage Task from Human Demonstrations (Source)
We’ve learned that robots’ cameras and sensors can record human demonstrations, but what makes these demonstrations usable as training data? A raw recording of a warehouse robot completing a task contains crucial information, but it isn’t immediately ready for machine learning.
Before an AI model can learn from it, the data has to be reviewed, organized, and validated to ensure it accurately reflects what happened during the task. Here is what that process typically involves:
A Robot Uses Multiple Cameras to Capture Task Demonstrations (Source)
Building reliable training data for robots requires a structured process for data collection, annotation, validation, and quality control. Since maintaining this pipeline in-house can be challenging, many robotics teams work with experienced data partners like Objectways.
At Objectways, we collect and annotate the data that warehouse robots learn from, including teleoperation demonstrations, manipulation recordings, egocentric video, and multi-sensor data.
Every dataset follows a structured pipeline. The dataset moves through collection, filtering, annotation, and validation.
Instead of treating teleoperation as a one-time recording session, we review each stage carefully, fixing issues before the data moves forward. We also remove idle or low-quality recordings, break demonstrations into steps such as approach, grasp, lift, and place.
Our expert review process helps to maintain consistent annotations. Before delivery, every dataset is validated by training an AI model and testing it on a real robot. This confirms that the data supports reliable robot learning and performs well in real-world warehouse environments. The result is clean, training-ready data that makes it possible for teams to build more accurate and dependable warehouse robots.
Warehouse robots are becoming more capable as they take on a wider range of robotic picking, handling, and logistics tasks. Even as warehouse automation continues to advance, robots still rely on human guidance to navigate unfamiliar situations and edge cases.
Teleoperation data plays an important role in powering robots to learn from those experiences. Each validated demonstration gives models more examples of how tasks are performed in real warehouse environments, helping them handle variability more effectively over time.
As warehouse robotics continues to evolve, the quality of that data becomes increasingly important. Teams that invest in structured data collection and strong quality standards are better positioned to build robots that perform reliably in real-world conditions.
Working on a warehouse robotics project? We can support your teleoperation data collection, annotation, and validation needs. Reach out to our team at Objectways to learn more.
A robotic warehouse is a facility where robots are used to automate tasks such as picking, packing, sorting, and moving inventory. These systems work alongside warehouse software and, in some cases, human workers to improve efficiency and handle repetitive tasks. Robotic warehouses help businesses process orders faster and manage inventory more accurately.
Learn what a mocap suit is, how wearable motion capture works, and why suit-captured movement has become crucial training data for robots
See how reinforcement learning works in robotics, how teleoperation data feeds and stabilizes it, and what it takes to turn that data into reliable robot...
