Encord Tests Brain-Wave Tagging to Expand Robot Training Data in San Leandro
Encord trials brain-wave and muscle-sensor tagging to create richer robot training data for manipulation tasks, aiming to close a major gap in physical-AI learning.
Encord, a data tooling company for AI, has begun trials in a San Leandro warehouse to manufacture robot training data that includes brain-wave and muscle-sensor signals. The initiative pairs human operators with leader-follower robotic rigs while recording egocentric video, sensor feeds and electroencephalography (EEG) signals to tag moments of error, intent and surprise. The company says the goal is to determine whether these multimodal labels measurably improve manipulation models before deciding to scale production.
Brain-wave headsets used to annotate operator intent
A headset built by Zander Labs is being worn by pilots as they perform dexterous tasks, with EEG sensors capturing neural activity in real time. Encord’s trial uses those brain signals to produce annotations keyed to cognitive states such as confusion or attention spikes. Researchers hope those tags will tell modelers when to deploy higher-capacity policies and which moments in an interaction carry the most training value.
Leader-follower rigs and egocentric video for manipulation data
In the facility, human operators control one robotic arm while a mirrored follower records the motion for training, generating egocentric video paired with actuator trajectories. These setups produce fine-grained examples of tasks like pouring, stacking and cable manipulation that are difficult to source at scale. Encord says combining multiple camera angles with first-person footage yields richer contextual views for model learning.
Muscle sensors and 3D hand inference to fill visual gaps
Beyond video and EEG, Encord is testing forearm electromyography sensors to infer hand pose when cameras do not capture full dexterity. The muscle-signal approach aims to reconstruct a 3D depiction of hand position and force during object interactions. Company engineers believe that combining inferred hand kinematics with dense annotations will reduce ambiguity in training samples for grasping and precision tasks.
Dense, semantic labels to multiply dataset value
Encord annotates its recordings with action-level descriptions — for example, “right hand tightens bolt” — so language models can translate perceptual inputs into task semantics. The company’s engineers estimate densely labeled recordings are worth an order of magnitude more to specialized robot learners than unstructured ego-video. Although dense annotation increases costs, Encord positions the trade-off as cheaper per-unit value compared with producing massive but noisy corpora.
Economic constraints and the scale problem for physical AI
Unlike text or images scraped from the web, physical training data must be manufactured, which drives up expense and operational complexity. Encord’s leadership estimates that breakthroughs in general-purpose manipulation will require datasets vastly larger than current collections, a scale that is costly to produce with human pilots and lab rigs. The company frames its role as not only annotating but also engineering repeatable data-production pipelines to bring costs down over time.
Industry visibility and cross-customer insights
Encord engineers say the company’s position working with multiple robotics developers allows it to spot which data modalities and collection techniques are gaining traction across the sector. That vantage helps prioritize which trials to expand and which niche skills warrant focused datasets. Encord is running the brain-wave trial as an initial experiment, intending to run tagged data through customer models and evaluate performance improvements before committing to wider rollout.
Pilots at the San Leandro site come from prior roles in annotation and automation, and they perform a variety of household and industrial manipulation tasks in controlled testbeds. Storage racks are stocked with mock household items and wiring harnesses to create repeatable interaction scenarios that stress grip, alignment and force control. Operators also practice delicate server-cable insertion and small-part handling to generate data targeted at real-world applications.
The initiative reflects a broader industry shift: as end-to-end learning arrives in robotics, companies increasingly find that model architecture alone cannot overcome a shortage of high-fidelity physical-world training examples. Encord’s multimodal approach — pairing egocentric video, muscle signals and brain-wave tagging with dense semantic labels — is an attempt to supply the specialized datasets that next-generation manipulation models require.
