Skild AI Introduces a New Way for Robots to Learn
Skild Robots are becoming increasingly capable, but teaching them to adapt to changing environments remains one of the biggest challenges in industrial automation. Manufacturing floors, warehouses, and production lines rarely remain unchanged. Products evolve, layouts are modified, and new tasks are introduced regularly. Traditionally, each change can require extensive programming, new training data, and lengthy validation before a robot is ready to work.
Skild AI is taking a different approach. Its new S1 robotic foundation model is designed to help robots learn unfamiliar, long-horizon tasks from a single video demonstration. Instead of requiring engineers to retrain the model whenever a new task is introduced, S1 uses the video as an instruction and applies what it sees directly to the robot.
The model, launched recently, uses a technique known as in-context learning. This allows S1 to interpret a video demonstration, understand the intended task, and translate that understanding into actions without changing the model’s weights or undergoing task-specific post-training.
Skild AI developed S1 using NVIDIA AI infrastructure, with the collaboration extending beyond model training. NVIDIA technologies support multiple stages of the robotic development process, including synthetic data generation, simulation, reinforcement learning, optimization, and deployment. Together, Skild AI and NVIDIA are working to bring more adaptable physical AI systems from research environments into factories and other dynamic real-world settings.
“Learning by experience, and not preprogramming, is the step change that has happened in robotics,” said Deepak Pathak, cofounder and CEO of Skild AI. According to Pathak, NVIDIA Isaac Lab and NVIDIA Cosmos technologies help Skild create the broad and diverse experiences robots need to operate across different situations and robotic platforms.
The progress comes as Skild AI reports reaching a $100 million annual revenue run rate just 10 months after its first commercial deployment. During that period, the company has established more than 60 deployment partnerships across industries and applications including manufacturing, logistics, inspection, security, and food preparation.
Teaching Robots New Tasks With a Single Video
Most industrial robots are designed around predefined workflows. While this approach works well for repetitive operations, it becomes less practical when factories introduce new products, alter their layouts, or change production processes.
Traditionally, teaching a robot a new task can involve collecting large amounts of demonstration data, retraining models, and validating the resulting behavior before the robot can safely operate in the real world.
S1 aims to reduce that burden. An operator can record a video showing how a task should be performed and provide the recording to the model as a prompt. S1 analyzes the demonstrated objects, actions, sequence, and intent before translating that information into movements for the robot operating in front of it.
Importantly, the robot does not need to be retrained for each new task. This means S1 can potentially handle tasks that were not explicitly included in its original training data.
The model is designed to perform unfamiliar tasks lasting as long as 10 minutes. These can involve dozens of individual manipulation steps that must be completed in the correct sequence. Demonstrated examples include planting and potting tasks, pancake preparation, pour-over coffee brewing, and kit assembly.
One plant-potting demonstration highlights the speed of the approach. According to Skild, the team moved from recording the demonstration to autonomous execution on physical hardware in approximately 11 minutes.
S1 can also adapt when conditions change. If an object moves, an action fails, or something unexpected happens during the workflow, the system can adjust and recover rather than simply stopping because the environment no longer matches a predefined script.
Skild’s testing suggests a substantial improvement on new, multistep tasks. In its evaluations, S1 achieved approximately a 66% success rate at each step, compared with 9% for a similar AI system. That represents more than a sevenfold improvement.
The company also estimates that a single short video demonstration can provide learning value comparable to roughly 380 hands-on training examples. Collecting that number of physical examples manually could require approximately 50 to 100 hours of human effort.
Moving From Research to Real-World Manufacturing
The ability to teach robots new tasks without repeatedly retraining them could be particularly valuable in manufacturing environments, where processes and products can change frequently.
Rather than creating a new dataset and training cycle every time a production requirement changes, operators could potentially demonstrate the new workflow directly. The robot can then interpret the demonstration and apply its existing knowledge to the new situation.
Skild’s robotic intelligence is already being deployed in real manufacturing environments. Skild AI, NVIDIA, and Foxconn are working together to deploy the Skild Brain on dual-arm robotic systems for high-precision assembly of NVIDIA Blackwell systems.
One demonstrated workflow involves installing a busbar and limit block, fastening 16 screws, and responding to disturbances throughout the multistep operation.
Such tasks require much more than simple movement. Robots must maintain precise positioning, understand physical contact, track the sequence of operations, and recover when the environment differs from the expected plan. These capabilities are critical for moving physical AI from controlled demonstrations into demanding production environments.
NVIDIA Technology Supports the Full Development Pipeline
NVIDIA’s technology plays a role across Skild AI’s robotic development cycle, from generating training data to deploying optimized models on physical machines.
NVIDIA accelerated computing provides the infrastructure required to train Skild’s shared robotic intelligence at scale. The company can combine simulation, human video, teleoperation data, and, where permitted, information from commercial deployments to build broader robotic capabilities.
NVIDIA Cosmos open-world foundation models help diversify training data and can transform video into structured descriptions. Cosmos Curator helps organize, filter, and annotate large datasets, making it easier for researchers to manage the information needed to train physical AI systems.
Simulation is another important part of the development process. Skild extensively uses NVIDIA’s robotics simulation technologies to train and validate robot behaviors before testing them on physical hardware.
NVIDIA Omniverse libraries and NVIDIA Isaac Sim provide physically based virtual environments where engineers can generate synthetic training data, test unusual scenarios, and evaluate robot behaviors. Simulation allows developers to expose robotic systems to a wide variety of situations without the time, expense, or safety risks associated with reproducing every scenario in a physical facility.
Skild also uses reinforcement learning through NVIDIA Isaac Lab, an open and modular framework for robot learning. Powered by the Newton physics engine, Isaac Lab allows engineers to model physical factors such as forces, collisions, contact, and pressure.
Accurately modeling these interactions can help reduce the gap between simulated environments and real-world robot behavior, often referred to as the simulation-to-reality gap.
Skild and NVIDIA are also working together on new GPU-accelerated simulation solvers designed to model how robots physically interact with solid objects, including gripping, touching, and manipulating them. These technologies are expected to become available to developers through Newton.
Optimizing Robots for Real-World Performance
The development process does not end once a model has been trained. Robots must also be able to execute AI models quickly and reliably in real-world environments.
NVIDIA Nsight tools help engineers identify performance bottlenecks during development and training, while NVIDIA TensorRT is used to optimize inference performance. Faster inference can allow robots to respond more quickly to changes in their surroundings, which is essential for physical systems operating around people, equipment, and moving objects.
Together, these technologies create an integrated pipeline connecting data generation, simulation, training, optimization, and deployment.
Skild AI’s S1 represents a broader shift in robotics: moving away from robots that simply follow fixed instructions toward systems capable of learning from experience and adapting to new situations.
If the approach continues to scale, teaching a robot a new task could become less about writing new programs and collecting massive datasets and more about simply showing the robot what needs to be done. Combined with NVIDIA’s simulation, AI infrastructure, and deployment technologies, that vision could help make adaptable robotic intelligence a practical part of factories, warehouses, and other rapidly changing environments.
Source Link: https://blogs.nvidia.com/