NVIDIA Unleashes Open-Source Agent Tools and Foundation Models for Physical AI Development

NVIDIA's latest announcement marks a pivotal moment in the evolution of physical AI, focusing on making its entire AI stack 'agent-ready.' This means that core NVIDIA libraries, models, and frameworks are being optimized to be directly callable by AI agents, streamlining the development process for robotics, autonomous vehicles (AVs), vision AI, and industrial digital twins. The NVIDIA Agent Toolkit is central to this effort, introducing new physical AI skills that translate intricate development workflows into repeatable, optimized, and agent-executable instructions. These skills dictate which tools an agent should invoke, what outputs to generate, and how developers can validate the results, effectively creating a blueprint for autonomous AI development.

This comprehensive approach covers a broad spectrum of NVIDIA's offerings. It includes NVIDIA Cosmos™ world foundation models for physical world reasoning and generation, NVIDIA Omniverse™ libraries for simulation and digital twins, NVIDIA Isaac™ for robotics simulation and robot learning, NVIDIA Metropolis for vision AI, NVIDIA Alpamayo for autonomous driving, and the NVIDIA Jetson™ platform for edge AI development. The goal is to speed up every stage of the pipeline, from synthetic data generation and simulation to model training, evaluation, and deployment. Industry leaders such as Agile Robots, Cadence, Dassault Systèmes, Delta Electronics, Foxconn, Pegatron, PTC, Siemens, Synopsys, and TSMC are already leveraging these tools to accelerate their physical AI development, highlighting the immediate impact and broad applicability of NVIDIA's open-source strategy.

Furthermore, NVIDIA is providing developers with the NVIDIA NemoClaw™ blueprint and the NVIDIA OpenShell™ runtime for safely building and deploying autonomous agents. These components offer policy-based security and privacy governance, allowing for secure operations on both local and cloud hardware. The availability of these tools and skills on GitHub and skills.sh ensures widespread access and encourages community-driven innovation in physical AI.

At the heart of NVIDIA's open-source push for physical AI is Cosmos 3, hailed as the frontier foundation model for physical AI. This advanced platform is purpose-built for physical AI, featuring state-of-the-art generative world foundation models (WFMs), robust guardrails, and an accelerated data processing and curation pipeline. Cosmos 3 is designed to enable physical AI systems to deeply understand the real world before acting within it, allowing robots, autonomous vehicles, and smart spaces to comprehend current events, predict future occurrences, and generate appropriate actions for specific environments and tasks.

Cosmos 3 unifies physical reasoning, world generation, and action generation within a single open model. NVIDIA is not only open-sourcing the Cosmos 3 models but also their training scripts, deployment tools, and associated datasets, making physical AI development more transparent and reproducible. These datasets, available on Hugging Face, cover a diverse range of scenarios including embodied robot scenes, physical interaction, spatial reasoning, human motion, autonomous driving, and warehouse operations.

The Cosmos 3 family includes two primary models: Cosmos 3 Nano, an 8B parameter model optimized for efficient inference on workstation-grade compute like the NVIDIA RTX PRO 6000 GPU, and Cosmos 3 Super, a 32B parameter model designed for maximum quality and capability, targeting datacenter deployment on NVIDIA Hopper and Blackwell GPUs. Cosmos 3 demonstrates leading performance on benchmarks such as PAI-Bench, R-Bench Physics-IQ, and RoboLab, solidifying its position as a state-of-the-art open-source solution for physical AI.

NVIDIA's commitment to robotics is further exemplified by the advancements in the Isaac GR00T platform and Isaac Lab. Isaac GR00T, standing for Generalist Robot 00 Technology, is a foundation model platform specifically designed for humanoid robot research and development. The flagship model, GR00T N1, first introduced in March 2025, is an open foundation model for generalized humanoid reasoning and skills. It supports multimodal inputs, processing language commands, visual information, and proprioceptive data simultaneously. The latest iteration, GR00T N1.7, released in early access as of April 2026, introduces a new Vision-Language Model backbone and improved performance, with models hosted on Hugging Face under the OpenMDW-1.1 license.

Complementing GR00T is NVIDIA Isaac Lab, an open-source, GPU-accelerated, agent-ready simulation framework crucial for training robot policies at scale. Built on NVIDIA Isaac Sim, Isaac Lab offers flexible integration across various physics engines (like Newton, NVIDIA PhysX, NVIDIA Warp, and MuJoCo) and learning algorithms, accelerating vision and perception training for real-world robot applications. Isaac Lab's modular architecture and GPU-based parallelization make it ideal for developing policies for a wide range of robotic embodiments, from humanoid robots to manipulators and autonomous mobile robots (AMRs). It supports both imitation and reinforcement learning methods and provides a comprehensive framework for robot learning, from environment setup to policy training.

NVIDIA also announced the Isaac GR00T Reference Humanoid Robot, an open reference design built on NVIDIA Jetson Thor™ and the Isaac GR00T open development platform. This reference design, featuring a Unitree H2 humanoid chassis with Sharpa Wave tactile five-finger hands, aims to democratize frontier humanoid robotics research by providing access to advanced hardware and a unified, open software stack. This integrated platform will significantly accelerate loco-manipulation research and the development of more capable real-world systems.

The impact of NVIDIA's open-source agent tools extends beyond individual robots to encompass broader applications in simulation, digital twins, and widespread industry adoption. NVIDIA Omniverse, a platform for building and operating 3D industrial metaverse applications and digital twins, plays a critical role in this ecosystem. Companies like SK hynix are implementing semiconductor fab digital twins using NVIDIA Omniverse, and collaborating with NVIDIA and SK Telecom to validate the NVIDIA Agent Toolkit for manufacturing-specific physical AI. PTC, MetAI, and Lightwheel are also utilizing the Isaac Sim framework and OpenUSD-based workflows to transform CAD data into simulation-ready assets and environments.

The ability to generate synthetic data through tools like NVIDIA Replicator within Isaac Sim is paramount for training robust AI models, especially in scenarios where real-world data collection is expensive, dangerous, or impractical. This synthetic data generation, combined with physically accurate simulation capabilities, allows developers to design, test, and optimize systems before physical deployment, thereby reducing risks and accelerating development cycles.

NVIDIA's strategy is attracting significant industry collaboration. Leading technology companies are adopting and building on NVIDIA's open model technologies across various domains. The expansion of the NVIDIA AI Cloud ecosystem further supports this, providing the infrastructure for training, inference, and physical AI development at scale. Partners are leveraging NVIDIA AI Clouds to meet the growing demand for agentic AI applications, bringing AI factories closer to where data, developers, and industries reside. This collaborative environment, coupled with the open-source release of powerful tools, is poised to unlock a multi-trillion-dollar economic opportunity as humanoid robots and other physical AI systems transform major industries.