AI World Models: How GenAI Is Moving From Language to Physical Understanding

An AI world model is a learned system that represents the state of an environment and predicts how that state will change, including how it changes when an agent acts. Because a world model can answer “what happens if I do this?”, an agent can use it to plan, or to practise in simulation before acting in the real world.

Large language models learned about the world mostly from what people wrote down. Since January 2025, Google DeepMind, NVIDIA and Meta have each released systems they call world models. Yann LeCun is among the founders of AMI Labs (Advanced Machine Intelligence), a company that says it is “developing world models that learn abstract representations of real-world sensor data.”

The phrase gets used loosely. In June 2026 World Labs, Fei-Fei Li’s company, called “world model” one of “the most important and most overloaded terms in AI today”: computer vision, robotics, reinforcement learning and generative AI each mean something different by it. So every system named here comes with its release date and its actual status, whether that’s available, open, in preview or research.

What is a world model in AI?

A world model in AI is a learned model of an environment that predicts its next state, usually from the current state plus an action, so an agent can try out choices before committing to them. Whatever the architecture, it involves three pieces.

1. It represents state. The current situation gets compressed into something the model can compute on: a latent vector squeezed from video frames, a 3D scene, or a set of object positions.
2. It predicts transitions. Given the current state, and usually an action, the model predicts the next state. The action input is what lets it answer counterfactual questions such as “what if the robot pushes left instead?”
3. Something uses the predictions. A planner searches imagined futures for the best action. A training loop lets a policy practise inside the model. A test harness replays scenarios that are too rare or too dangerous to stage for real.

The idea predates the current wave. In their 2018 paper World Models, David Ha and Jürgen Schmidhuber built an agent from three pieces. A variational autoencoder compressed each game frame, a recurrent network with a mixture density output (an MDN-RNN) predicted the next compressed frame, and a small linear controller picked actions. Their headline result was that the agent could be trained entirely inside its own “dream” generated by the world model, and the policy then transferred back to the real environment.

DreamerV3, published in Nature in April 2025 by Danijar Hafner and colleagues, took the learn-a-model-then-imagine approach much further. One configuration beat specialised methods across more than 150 tasks, and the authors describe it as, to their knowledge, the first algorithm to collect diamonds in Minecraft from scratch without human data or curricula.

Yann LeCun argued for a different design in his 2022 position paper, A Path Towards Autonomous Machine Intelligence. His Joint Embedding Predictive Architecture (JEPA) predicts in an abstract representation space instead of predicting every pixel. Meta’s explanation of the reasoning is that a model made to fill in every missing detail wastes effort, “even though the world is inherently unpredictable.”

Meta’s JEPA research led to V-JEPA 2, which Meta released on 11 June 2025. It has 1.2 billion parameters and was trained on more than a million hours of video, then adapted for robot control using 62 hours of robot data from the open DROID dataset. Meta reports 65% to 80% success on pick-and-place tasks with unfamiliar objects in new environments, without training for those specific settings.

Meta researchers followed up with V-JEPA 2.1, first posted to arXiv on 15 March 2026. It focuses on denser, more spatially detailed video features, and its authors report a 20-point improvement in real-robot grasping success rate over V-JEPA 2-AC, the action-conditioned version of V-JEPA 2.

Three things people mean by “world model”

World Labs’ June 2026 taxonomy sorts world models by what they output. We’ve found it the most useful filter for reading launch announcements.

Applied to a vendor demo, the test is simple. A world model that produces beautiful video is a renderer until someone shows that its predictions still hold when an agent acts inside it.

How are world models different from large language models like GPT or Gemini?

A large language model such as GPT or Gemini is trained to predict the next token in a sequence, and it learns about the world from text and other recorded media. A world model is trained to predict the next state of an environment, usually given an action, so you can ask it what will happen if an agent does something specific.

Tap to enlarge

Whether a text-trained LLM builds any internal model of the world is still argued over by researchers. The engineering difference doesn’t depend on settling that. An LLM is judged on the text it produces. A world model is judged on whether its imagined futures match the real one.

The boundary between LLMs and world models is blurring from both sides. NVIDIA’s Cosmos Reason, announced in March 2025, uses chain-of-thought reasoning over video to predict the outcomes of interactions and answers in natural language. Cosmos 3, released on 31 May 2026, combines vision reasoning, world generation and action prediction in a single mixture-of-transformers model. Genie 3 builds a world from a text prompt. The better question to ask of any system is what it was trained to predict, and whether you can condition it on actions.

AI agents sit on top of both. An agent is the loop that decides, acts and checks the result, and that loop can wrap either kind of model. The loop itself is covered in agentic AI vs generative AI.

What are world models being built for?

AI world models are being built for four areas: autonomous driving, robotics and manufacturing, biomedical research (so far mainly surgical robotics), and interactive 3D worlds. Much of that work is about producing data and test scenarios that would be too rare, costly or dangerous to collect for real. Maturity varies widely: the examples below include an in-house simulator, open models published for download, and previews with limited access.

Autonomous driving

Waymo announced the Waymo World Model on 6 February 2026. It’s built on Google DeepMind’s Genie 3 and adapted to output both camera and lidar data. Waymo describes three ways to steer it. Driving-action control replays a scene with a different decision. Scene-layout control changes roads, signals and other road users. Language control changes the time of day or weather, or generates a fully synthetic scene.

The appeal of a driving world model is the long tail. Events that almost never happen on real roads can be generated on demand, then rerun with a different driving choice.

Robotics and manufacturing

NVIDIA launched Cosmos on 6 January 2025, a platform of world foundation models released under an open model license. The first adopters it named included 1X, Agile Robots, Agility, Figure AI, Foretellix, Uber, Waabi and XPENG, while Wayve said it was evaluating Cosmos to find edge-case driving scenarios. NVIDIA’s March 2025 release added Cosmos Transfer, which turns structured inputs such as segmentation maps, depth maps or lidar scans into photoreal video for synthetic data. In the same release, Agility Robotics’ chief technology officer Pras Velagapudi put the appeal plainly: Cosmos offered a way “to scale our photorealistic training data beyond what we can feasibly collect in the real world.”

For Cosmos 3, NVIDIA lists industrial AI, warehouse safety and defect-image generation among the uses, and names Doosan Robotics, LG Electronics and Samsung Electronics among the adopters. By 20 July 2026 all three Cosmos 3 sizes were available on Hugging Face: Edge (4 billion parameters), Nano (16 billion) and Super (64 billion). Defect images suit synthetic data well, because a well-run production line is supposed to produce very few real ones.

World Labs reported in July 2026 that manipulation policies for tasks such as bimanual box packing and elastic cable insertion were “trained entirely in simulation, with zero real-world training data” and still worked on physical robots. The results come from World Labs’ own testing on specific manipulation tasks, so read them as a company report.

Biomedical research

The clearest biomedical use of world models we could verify is surgical robotics. NVIDIA’s Cosmos-H-Surgical-Simulator, published on Hugging Face in March 2026, is built on Cosmos-Predict2.5-2B. It takes a surgical scene plus a sequence of robot kinematic commands and generates video of what happens next. Its model card limits the intended use to evaluating surgical-robot policies in simulation and generating synthetic data before deployment on a physical system.

NVIDIA’s own July 2026 write-up states the limitation that applies to every generative simulator: learned predictions “can be visually realistic without providing the explicit physical guarantees of a numerical solver.” Claims about world models in drug discovery also circulate. We couldn’t tie any of them to a primary release, so they’re left out.

Interactive worlds and agent training

Google DeepMind announced Genie 3 on 5 August 2025. It generates navigable worlds in real time at 24 frames per second and 720p, stays consistent for a few minutes, and supports “promptable world events” that change a world through text. DeepMind also ran its SIMA agent, which it describes as a generalist agent for 3D virtual settings, inside Genie 3 worlds. DeepMind’s announcement lists Genie 3’s limits too: a constrained action space, trouble modelling several agents interacting, imperfect geographic accuracy, poor text rendering, and interactions that top out at a few minutes.

Project Genie, the public prototype built on Genie 3, started rolling out in the US on 29 January 2026, launching with 60-second sessions and without promptable events. On 19 May 2026 Google added a Street View feature and began gradually rolling Project Genie out to eligible Google AI Ultra subscribers (18+) globally.

On the 3D side, World Labs made Marble available to everyone on 12 November 2025. It turns text, images, video or coarse 3D layouts into worlds exported as Gaussian splats, meshes or video. A newer World Labs model, Atlas, which works natively across text, images, video and 3D, entered early access with select partners on 1 September 2026.

What are SynthID and C2PA, and why are AI companies adding provenance to generated media?

SynthID is Google DeepMind’s invisible watermark, embedded into AI-generated images, video, audio and text so detection tools can recognise it later. C2PA is an open technical standard from the Coalition for Content Provenance and Authenticity for attaching a signed record of a file’s origin and edit history, shown to people as Content Credentials. Used together, each covers a weakness in the other.

Content provenance belongs in a discussion of world models for a practical reason. A model that renders a convincing street or face on demand makes “was this captured or generated?” harder to answer by looking.

SynthID puts the signal inside the content. Google says the image and video watermark is designed to survive cropping, filters, frame-rate changes and lossy compression, and has open-sourced SynthID’s text watermarking. In May 2026 Google said it had watermarked over 100 billion images and videos and 60,000 years of audio, and named OpenAI, Kakao, ElevenLabs and NVIDIA (for Cosmos) among the companies bringing SynthID to their platforms. SynthID’s weakness is on the negative side. Google’s Gemini help page says that when no SynthID watermark is found, the content wasn’t made or edited by Google AI but “could have been created by other AI systems.”

C2PA puts the signal next to the content. A C2PA manifest holds assertions about an asset (where and when it was created, what edited it, whether AI was used), hashed and digitally signed with X.509 certificates, and usually embedded in the file. The latest published specification is version 2.4, dated April 2026. Metadata can be lost when files pass through tools that drop it, so the standard describes “durable Content Credentials” that pair the cryptographic hash with a watermark or fingerprint. The C2PA explainer is also explicit that Content Credentials “do not provide value judgments” about whether provenance data is true.

Tap to enlarge

Google’s May 2026 announcement argues for industry-wide adoption on the grounds that “digital media travels across many platforms.” A watermark only one company can read, or a manifest only one app displays, doesn’t help a file that has been downloaded, edited and reposted somewhere else.

India’s rule on synthetically generated information

India has turned provenance into a compliance question. The Ministry of Electronics and Information Technology (MeitY) notified the Information Technology (Intermediary Guidelines and Digital Media Ethics Code) Amendment Rules, 2026 through G.S.R. 120(E) on 10 February 2026, and they came into force on 20 February 2026. MeitY publishes the consolidated rules as amended. According to MeitY’s FAQ on the rules:

• Scope. “Synthetically generated information” (SGI) means audio, visual or audio-visual content artificially created or altered so that it appears real and could be taken for a real person or event. Pure text outputs are not SGI by themselves.
• Who is covered. Rule 3(3) applies to intermediaries offering tools or platforms that create, modify or spread SGI, including AI image and video generators and voice-cloning tools.
• What they must do. Label lawful SGI prominently (a visible label for visual content, a prefixed disclosure for audio) and embed permanent metadata or technical provenance mechanisms, including a unique identifier, “to the extent technically feasible.” They must not let users remove those labels or identifiers; MeitY’s example is a “remove watermark” or “export without metadata” feature.
• Large platforms. Significant social media intermediaries must collect a user declaration on whether content is SGI, verify it with technical measures, and label confirmed SGI before publication.

The rules don’t name SynthID or C2PA. If your product lets users in India generate or share realistic synthetic media and counts as an intermediary under these rules, provenance is already a design requirement with a date attached. This is a summary of MeitY’s published FAQ, not legal advice.

Do IT professionals need to learn world models?

Most software engineers, IT professionals and product managers don’t need to learn to build world models yet. Outside robotics, autonomous vehicles, simulation and 3D graphics, the work is still concentrated in research labs and specialist teams. What pays off now is knowing what world models can and can’t do, and building the deep learning foundations they share with the rest of GenAI.

A few roles should look closer today:

• Robotics and embodied-AI engineers, where synthetic data and simulated evaluation are becoming part of the workflow.
• Simulation and validation engineers on autonomous-vehicle and driver-assistance programmes.
• Computer vision engineers who already work with video.
• Game and 3D developers, since tools like Marble export Gaussian splats and meshes.
• ML platform teams asked to host open physical-AI models such as Cosmos 3, which NVIDIA distributes through Hugging Face and its NIM microservices.

For everyone else, two things are worth knowing now. If you’re a product manager evaluating a “world model” pitch, ask what it predicts, whether it can be conditioned on actions, and how its predictions were checked against reality. If your product generates realistic images, audio or video for users in India, check whether the labelling and provenance duties in MeitY’s 2026 amendment apply to you.

The foundations behind world models overlap heavily with the rest of GenAI. Deep learning for vision covers how models encode images and video. Transformers show up throughout: Cosmos 3 is a mixture-of-transformers model, and World Labs describes Atlas as a multimodal autoregressive diffusion transformer. Generative models matter too (Ha and Schmidhuber’s agent started with a variational autoencoder), and reinforcement learning basics explain how Dreamer-style agents learn inside imagined futures.

We’d put evaluation discipline ahead of any single architecture. A simulator’s output can look right and still be wrong, and so can a chatbot’s. For the broader case on which AI skills repay the time, read AI skills for software engineers.

If you want hands-on exposure without a career bet, read the model card for V-JEPA 2 or Cosmos 3, check the hardware requirements, and try the smallest open checkpoint you can run.

Skillfyme’s GenAI with ML program doesn’t teach world models, and it would be misleading to suggest otherwise. It does teach several of the foundations above: deep learning with convolutional neural networks, autoencoders and GANs, transformers and attention, a speech and multimodal NLP class, and a full Agentic AI & LLMOps module covering LangGraph, CrewAI and Autogen.

Frequently asked questions