Pretrained to Imagine, Fine-Tuned to Act: The Rise of World-Action Models

Introduction

Last year, my Scholar Inbox digest was dominated almost every day by new VLA papers. This changed in the last months, and a different keyword is coming up almost daily now too: WAM, short for World-Action Model. In October 2025, I wrote in my State of VLA post that WAMs were a small subfield within VLA research and far less popular than VLAs initialized from VLMs [60]. That has changed fast, and my wish to see more work in this direction has already become reality.

So what changed, and why now? Maybe it is because WAMs are the shiny new thing everyone wants to work on, or VLA authors ran out of new names for their own VLAs, since basically all “-VLA” names like “X-VLA” and “Ego-VLA” are already used. So now we can recycle them for the WAM area. But more likely it has something to do with VLM-based VLAs getting stuck. Modern VLAs benefited from massive vision-language pretraining, but they still hit a language-to-action grounding wall. The problem of mapping language and pixels into behavior still has to be learned from robot data. WAMs offer a different starting point. They use pretrained video or world-model backbones that already model how scene dynamics change under language conditioning. If that prior transfers to behavior generation, the remaining video-to-action gap may be smaller than learning language-to-action grounding directly.

But the ideas behind WAMs are not new. Early WAMs like UniPi [10] proposed essentially this approach back in 2023. So why did it take several years for the paradigm to enter the robot foundation model mainstream, and where does it actually stand today? This post takes a closer look at the modern WAM landscape to answer the central question:

Central question: Is this a real paradigm shift in research and industry, or just a short hype cycle? And if the recipe works so well, why did it take several years after early papers like UniPi for WAMs to become so popular?

My take: WAMs will become the second major recipe for robot foundation models, alongside VLM-based VLAs. The open questions are which formulation of them wins, and which parts of the model architecture and pipeline actually matter. It is likely that the winner is neither pure VLA nor pure WAM, but a hybrid of both.

This is my map of the modern WAM space: how to categorize and understand WAMs, what changed since the early models, and how current results compare to VLAs. For a broader survey, see the recent NTU survey “World Model for Robot Learning: A Comprehensive Survey” [57], which maps world models for robot learning across simulation, evaluation, navigation, and autonomous driving.

Table of contents

The two representation bets for generalist policies

The field currently has two major representation bets for robot foundation models in both research and industry. Many teams are building on the traditional VLA recipe established by Pi-0 [2] and later refined by Pi-0.5 [4], using VLM backbones as the starting point for policy learning. This VLM-backbone recipe appears in public work from teams including NVIDIA GR00T [5], Xiaomi Robotics [27], Being-H0.5 [28], and others.

More recently, a different paradigm has emerged: using pretrained video backbones as an alternative path toward generalist manipulation. Public examples now span NVIDIA’s DreamZero [8] and Cosmos Policy [13], Ant Group’s LingBot-VA [9], Rhoda AI’s DVA [40], Sereact’s Cortex 2.0 [45], and Mimic Robotics with mimic-video [14]. At the same time, many university labs and open research groups are also pushing the frontier with new ideas, including Video Prediction Policy [24], Unified Video Action Model [39], and Fast-WAM [23]. We discuss these in more detail below.

The choice of backbone impacts the full training and evaluation pipeline, from training recipe and data mixture to inference optimizations. Given the cost of running these models at scale, most teams will likely have to prioritize one direction (VLA or WAM) first rather than fully pursuing both in parallel. Which path proves out, or whether the two converge, is still open. Which one would you bet on today? In the following sections, we dive deeper into both sides of this decision.

Why World-Action Models? Our hypotheses

Before we dive deeper into current models, let’s first review why WAMs are attractive as an alternative to VLM-based VLAs. It also helps to first place WAMs inside the broader landscape of world models in robotics.

The grounding gap

To understand why WAMs are attractive, it helps to understand the core challenge of “classical” VLAs built on VLM backbones. The motivation for the first VLAs was to leverage the internet-scale knowledge of VLMs for robotics. VLMs are trained on massive amounts of vision-text data and show notable zero-shot performance on many vision tasks. The VLA recipe then adapts these pretrained representations for action generation.

However, there is a major domain gap between VLM pretraining and embodied manipulation. Several VLA papers either observe degradation of pretrained VLM capabilities or design around it, particularly when the action-learning objective diverges sharply from the original VLM objective. VLM2VLA frames this directly as catastrophic forgetting during the VLM-to-VLA transition [55]. Knowledge Insulation reports similar findings and makes the concern architectural: it isolates the gradients of the flow-matching action expert from the VLM backbone to preserve pretrained language/vision knowledge, improving training convergence, task performance, and language following [20]. Recent solutions like VLM co-training and discrete action tokenizers have helped, but the core challenge remains: grounding language into physical action from limited robot data. We cover these solutions in the modern VLA baseline section below.

This naturally raises the question: what if we started from a backbone that already represents how language maps to visual change in the world?

Core hypotheses for WAMs as policy representations

The core idea is simple: instead of using a VLM backbone to jump-start imitation learning, use a pretrained video backbone. Current video models are trained on large video corpora and learn spatiotemporal representations of how visual scenes evolve. Crucially, current video models are often text-conditioned: they are trained to generate videos from precise language descriptions, sometimes with a reference frame and sometimes from text alone. Many of these videos contain intentional behavior: hands reaching, tools moving, objects being manipulated, and scenes changing because someone or something acted. That makes video backbones attractive as a model prior for generalist manipulation. Before seeing any robot actions, the backbone already encodes useful links between language, visual change, and plausible object interactions. The Veo 3.1 demonstration below is a quick illustration.

I would treat the next three points as hypotheses, not conclusions. They are recurring claims across papers, discussions with peers, and my own read of the field, supported by qualitative intuition, simulation evidence, and a few early real-world signals, but not by clean matched comparisons yet:

1. Predicting future world changes correlates with generating the necessary actions. Inverse dynamics prediction is often easier than pure action generation [26]. If the desired outcome is known, inferring the action that produced it is usually simpler than predicting the action directly from the instruction and current observation. Pi-0.7’s visual-subgoal results point in the same direction: when the policy is given a desired future image, action prediction becomes more direct and training converges faster [43].
2. Video pretraining provides grounding between language and physical change. Video models learn to map text descriptions to visual outcomes. If this transfers to robotics, it could reduce the amount of grounding that has to be learned from robot demonstrations alone.
3. Video data regularizes robot policies. Robot datasets are small relative to web-scale video. Either through pretraining on video first or through co-training on video alongside robot data, the broader visual prior can reduce overfitting; the benefit depends on the dataset, objective, and architecture. DreamZero [8] and Fast-WAM [23] both show that, during robot fine-tuning, WAMs perform best when action learning is co-trained with a video-prediction objective.

A quick experiment: how much does a frontier video model already “understand” about robot manipulation?

How much do modern video models already capture before any robotics-specific action head is added? We ran a simple experiment with Google’s Veo 3.1, a frontier video generation model. Given a single context frame from an original RoboArena rollout of a toaster task in the DROID setup, we prompted Veo to push the toaster lever (the reference task, matching the original DROID demonstration) and then pick up an orange sitting to the left (the composed extension, beyond the demonstration). This video is very unlikely to be part of Veo’s pretraining data, but we cannot verify the training set directly; treat this as a qualitative check of the prior, not a controlled probe of training-set membership. One-shot attempt, no prompt optimization.

The prompt used was:

“Given this initial frame, generate a video of the robot arm pushing the toaster lever. After finishing that task, the robot should pick up the orange on the left side of the toaster and stop after it has picked it up.”

Context frame and ground-truth rollout:

Veo 3.1 generated rollouts (zero-shot, no robotics fine-tuning):

The generated rollout is surprisingly good for a model that was not explicitly trained as a robot policy. The generated motions are smooth, the background remains stable and consistent, and the robot follows a plausible trajectory toward both target objects. Even the sequencing is respected: finish the lever, then move to the orange.

The limitations are equally visible: The model does not fully push the toaster lever down and at points appears to attempt the opposite motion (pulling it up). More visibly, the pinch gripper from the original DROID setup morphs into a four-fingered hand. The fixed-base robot arm is reimagined, almost instantly after the context frame, as a different robot with fewer degrees of freedom. These artifacts are consistent with the model using broad visual priors rather than faithfully modeling the specific hardware.

Still, the result illustrates why video backbones are attractive for robotics: the model has a useful prior for what robot-object interaction should look like, even though it is not yet reliable enough for control. WAM fine-tuning is the attempt to turn that zero-shot imagination into reliable control.

Understanding modern WAMs: Core formulations

After establishing the core motivation, we can now focus on the current WAM research. In contrast to VLM-based VLAs, where the training recipe has largely converged around VLM co-training with a flow transformer for action generation, WAMs are still splitting into several active formulations. This is exactly what makes the area interesting right now: the field does not yet know which combination of design choices will win, or whether the best systems will merge parts of several.

To make the design space readable, we organize WAMs along three axes (which are not fully independent):

1. Paradigm: what does the model predict, and how is the predicted video used to generate actions? (inverse dynamics vs joint prediction vs representation-only)
2. Action integration: how do actions actually enter the model? (default action tokens vs action-as-image vs latent actions/plans)
3. Architecture: how are the components composed? (Mixture-of-Transformers vs monolithic vs hierarchical)

The axes are not fully independent, and some WAMs do not fit well into a single category. I would not treat this as a perfect taxonomy. It should be more a practical map for reading the current papers without getting lost in naming choices. For each axis, I present the idea with an older paper and then a modern scaled-up version of the same rough recipe.

Paradigm: What the model predicts

The first axis is the policy formulation: what the model predicts, and how the predicted video is used to generate actions. Across modern WAMs, we see three directions that differ at the inference boundary: inverse dynamics, joint prediction, and representation-only.

Inverse dynamics: Predict the future, then infer the action

The inverse-dynamics setup is the easiest WAM recipe to understand: first imagine the future, then predict the most likely action from the video. This shifts the hard language-grounding problem into the video stage: translate the command into a plausible visual change. The bet is that video pretraining has already learned a useful part of this language-to-visual-change mapping, so the action head does not have to learn everything from robot demos and can focus on the inverse-dynamics problem instead.

UniPi [10] is a pioneering paper for this direction. It was probably the first modern execution of this recipe that was clearly aware of the potential of video diffusion for robotics: use video as a high-level plan, then use inverse dynamics to recover the low-level controls. In hindsight. A lot of recent WAM work looks like an improved version of this.

UniPi also shows why it took several more years until WAMs hit the mainstream. it used a CNN-based video diffusion stack from the Imagen Video era [56], and the video generator had to be pretrained from scratch. Our rough estimate in Footnote 2 puts that pretraining around 167 ZFLOPs, well outside the budget of most robotics labs. While this recipe existed before, it was not really reproducible for the average lab Modern inverse-dynamics WAMs can now sidestep this by starting from open DiT-based video backbones and fine-tuning them².

A modern version of this direction is LingBot-VA [9]. It turns Wan 2.2-5B into a robot video-action model through 16k hours of cross-embodiment pretraining. The important difference from UniPi is not just scale.  LingBot-VA is causal and trained on long visual histories for closed-loop rollouts rather than open-loop video generation. It also uses a Mixture-of-Transformers (MoT) architecture: separate experts for video and action, each with its own weights, coupled through shared self-attention in each layer.

There are several variants around the same theme. Video Prediction Policy [24], DiT4DiT [37], and mimic-video [14] do not necessarily need the final RGB video; they use intermediate video-model features as the predictive plan for an action decoder. DVA [42] and LingBot-VA lean more directly on generated or predicted future rollouts. The difficult part is that most papers change the video backbone, use different amounts of large-scale pretraining, tune different hyperparameters, and evaluate on different setups.

Joint prediction: Learn video and actions together

The second formulation is joint prediction. Instead of generating a future video first and then decoding actions, the model predicts video and actions together. This is the more coupled version of the WAM idea: the model is forced to learn what should happen and how to make it happen in the same prediction step.

GR-1 [11] is an early foundation paper for this direction. It pretrained on large-scale video and then fine-tuned on local robot datasets with both video and action supervision. It used a GPT-2-style transformer policy, pretrained it on internet video prediction with readout tokens, and then fine-tuned it on robotics data with a joint video-action objective. Earlier works like R3M [15] and Voltron [16] had already shown that video and language can help representation learning for robotics, but GR-1 made a simple and important shift: it used video to learn better policy representations, not only image-level visual representations.

At the time, the CALVIN results were useful simulation evidence. On the harder ABC→D split, prior methods in the GR-1 table stayed below an average sequence length of 1.0, while GR-1 reached 3.06/5. This result is useful here because it makes the generalization signal easiest to read. By 2026 this number is outdated, but I still think the result matters historically. It showed that predicting future visual states could shape a better policy representation, not just a better visual encoder.

DreamZero [8] is the modern scaled version of this idea. Instead of training a smaller transformer-style policy around a video-prediction head, it starts from Wan 2.1-I2V-14B-480P and turns the video diffusion backbone into a joint world-action model. The model denoises video and action tokens together inside one monolithic DiT. There is no separate inverse-dynamics module: action is another generated modality inside the same denoising process.

DreamZero’s reported RoboArena score is an important real-world signal for WAMs. While most papers still focus on popular benchmarks like LIBERO and other simulation benchmarks, RoboArena is one of the few public real-world, open-ended evaluations, which is what makes the snapshot below worth pausing on.

In the April 2026 snapshot above, DreamZero reaches 1750 compared to 1622 for Pi-0.5, which is a meaningful signal for the potential of WAMs. This is not proof that WAMs are the better default, but it is a positive signal for their potential. It is interesting because DreamZero was trained only on DROID without an additional large-scale cross-embodiment robot training stage.

GR-1 showed the policy-level version of joint video-action prediction, while DreamZero combines the idea with a modern video-foundation-model and flow-matching setup. The core joint-prediction idea is the same as GR-1, but DreamZero changes nearly everything else around it, so this is far from a clean comparison.

GR-2 [12], Seer [29], PAD [57], UWM [58], UVA [41], and DreamVLA [40] sit around this broader joint-prediction wave. PAD was another early attempt for joint future image prediction and robot action generation inside one joint denoising process. UWM uses independent noise for video and action to support more flexible inference modes inside a joint transformer.

Representation-only: Skip video generation at inference

The third option is to use the video backbone purely as a representation and skip video generation at inference entirely. Fast-WAM is a nice example of this idea.

Fast-WAM [23] uses a similar Wan/MoT-style setup to LingBot-VA and closely matches its performance on simulated benchmarks even without 16k hours of large-scale robot pretraining. In addition, skipping video generation at test time makes its inference several times faster. However, Fast-WAM is one of the few public pieces of evidence for the representation-only hypothesis, and current simulation evidence is not enough to really convince me of this idea. But I am happy to be convinced in future work.

Most WAMs today keep some form of video generation at inference and are very slow. Faster WAMs like Fast-WAM will become a much bigger research area in the future.

Action integration: How actions enter the model

After discussing how to combine video and action prediction, let’s focus on how actions are represented inside the model. The choice is important because the pretrained backbone knows how to denoise visual tokens, not continuous robot actions, so there is a real modality mismatch. I see three variants in the current papers.

Default action tokens

The simplest default is to add action tokens, continuous or discrete, and an action head, where actions are treated as another modality alongside video. UniPi, GR-1, DreamZero, LingBot-VA, VPP, mimic-video, and Fast-WAM all use some version of this. The risk is the modality mismatch: action chunks are different from the visual tokens the backbone was pretrained on, so the model has to adapt its representation during action fine-tuning.

Action as image

Another option is to turn the actions into something the video model already knows. Instead of new action tokens or a separate action head, encode actions as visual targets inside the same generation interface, so the pretrained video representation is not disrupted.

The closest early ancestor here is GENIMA [31]. GENIMA fine-tunes Stable Diffusion to draw joint-action targets on RGB images, then uses a controller to map those visual targets into joint-position actions. The interesting part is the interface choice: actions are expressed as something the generative image model can draw.

A modern version of this direction is Cosmos Policy [13], which treats actions as synthetic latent video frames. Instead of adding a separate action decoder, it encodes action, proprioception, and value targets as fake frames inside the video model’s own denoising interface, and at inference time decodes the predicted action image back into an action vector by averaging spatial dimensions. This setup keeps the pretrained video backbone close to its native video denoising space while still producing robot actions.

Latent actions and plans

A different option is to compress behavior into latent plans or latent actions and condition the policy on those. This is attractive because full video prediction is expensive, and most pixels are not actually needed for control. Latent plans and latent actions are not exactly the same thing, but for this discussion I group them together: both are compact behavior abstractions learned from trajectories or video.  The main differences are granularity and supervision. Plans usually cover a multi-step window and often need paired robot data; latent actions in the Genie/LAPA style can be learned from unlabeled video.

Play-LMP [32] pioneered this idea in 2019. This is worth remembering because the basic idea is older than the current foundation-model wave. Play-LMP compressed sub-tasks into a small latent space as an intermediate abstraction for conditioning the low-level policy, long before access to today’s larger robot datasets and pretrained models. Concretely, a posterior network compressed short trajectory windows into a latent plan, a prior learned to predict that latent plan from the current observation and goal image, and a low-level policy decoded the sampled plan into actions.

The modern latent-action wave changes the scale and the data source. Genie [19] showed that latent action tokens can be learned from unlabeled internet video and used to drive an action-conditioned world model. Genie itself does not decode those latents into real robot motor commands, so it is not a robot policy. But it made the idea much more scalable: learn an action-like abstraction from video without having ground-truth robot actions. LAPA [33] then pushed this kind of latent-action pretraining toward VLA-style robot learning.

Being-H0.7 [42] is a modern WAM version of the original Play-LMP idea. It keeps the prior/posterior latent-plan logic, but executes it at foundation-model scale with several major changes. Instead of a small hierarchical latent-plan policy, it uses a larger Mixture-of-Transformers backbone. Similar to Play-LMP, the model has a posterior branch and a prior branch. The posterior branch gets access to future observations, encodes them with a frozen V-JEPA2.1 [64] visual encoder and Perceiver resampler, and compresses them into K future embeddings. The prior branch uses learnable latent queries and learns to match those future-informed latent states from the available context. At test time, the posterior branch is removed, so the policy gets a fast latent interface instead of forcing the model to regenerate full video sequences. The action generation part is still a flow-matching action policy. Being-H0.7 is trained on 200,000 hours of egocentric human video together with 15,000 hours of robot demonstrations.

The key difference is not the latent variable itself. Play-LMP already had the core prior/posterior latent-plan idea. Being-H0.7 shows how that interface can be scaled inside a modern WAM/VLA hybrid.

Latent actions have also become popular as an abstraction for action-conditioned world models. A recent example is DreamDojo [44], which learns continuous latent actions from large-scale egocentric human video for a controllable world model. The important distinction from inverse dynamics is the supervision path. Inverse-dynamics WAMs usually need paired video and action data to learn how visual transitions map to motor commands. Latent-action methods try to first learn a behavior abstraction from video itself, then connect that abstraction to robot action later.

Architecture: Hierarchical, Monolithic, or MoT?

The third axis is architecture: how the components are structurally composed. This is mostly orthogonal to the first two axes. Inverse dynamics can be hierarchical or MoT-style, joint prediction can be monolithic or expert-based, and latent-action methods can sit inside several different wrappers.

Hierarchical is the most flexible design because the action head is fully modular. It can be anything from a simple CNN regressor (UniPi) up to a full VLA stack (Pi-0.7’s BAGEL subgoals plus full VLA-based action expert), with VPP [24] and mimic-video [14] sitting in between by passing intermediate video-model features instead of full RGB rollouts. The downside is weak coupling between the video and action stages. Information flows one way, so this style is less natural when video and action should strongly influence each other.

Monolithic transformers like DreamZero [8] put video and action denoising in the same stack, which gives them strong coupling between the two streams. They are also the natural fit for action-as-image setups such as Cosmos Policy [13], where actions and video already live in the same latent space. The risk is dual optimization: the same model weights have to handle dense visual tokens and much sparser action targets.

Mixture-of-Transformers (MoT) is the current default, including in modern VLAs (Pi-0, Pi-0.5) and in recent WAMs like LingBot-VA [9] and Fast-WAM [23]. Modality-specific parameters keep the representations separate while shared attention still lets video and action exchange information. My guess is that MoT-style designs will become the dominant WAM architecture as well, mostly because they are a practical compromise between modularity and coupling.

Why WAMs took off now

My short answer for why WAMs took off now: while the idea was not new, the required tools like pretrained video models finally caught up. The early formulations (UniPi for inverse dynamics, GR-1 for joint prediction, Play-LMP for latent abstractions) had the right ideas but limited tools: smaller backbones, weaker video data, no openly available video foundation models, and per-step action heads that did not work well compared to modern action chunk policies. Their modern counterparts (LingBot-VA, DreamZero, Being-H0.7) use infrastructure and large-scale robotic datasets that did not exist a few years ago.

First, video backbones got much stronger. DiT-based models like Wan [21] and Cosmos [22] replaced earlier CNN-based stacks, with better temporal compression, flow-matching objectives, and well-curated web-scale video data. Second, those backbones became openly available. Researchers can now fine-tune a strong pretrained video model instead of paying the full pretraining cost themselves.Third, the action side caught up: modern systems predict action chunks with transformer or flow-matching heads instead of small per-step MLP heads. That is why WAMs now look like a real recipe, not just an old idea with better branding.

WAM comparison

The table below summarizes the models we covered before by categorizing them across different design decisions: what does the model predict, how do actions enter, what backbone does it use, and what architecture is used? The WAM space is moving fast, so this is only a selected subset of papers. For a broader survey of world-model and WAM-related robot-learning papers, see the NTU survey “World Model for Robot Learning” [57].

Practical considerations

We have seen some promising WAM models above and some promising results. However, there are also several core issues:

• High training cost. Video backbones process many more tokens than image-conditioned action policies, and full video pretraining is expensive.
• Slow inference. Policies that generate or denoise future video latents are much slower than simple VLAs.
• Memory and systems complexity. Long video-token sequences push GPU memory, communication, and data loading. Good luck trying to run 10B+ WAM models on your local GPU without additional engineering.

The cost of the video prior

A strong video prior can reduce robotics-data requirements in some settings [24, 25, 42] and still deliver strong zero-shot performance when using modern video models like Wan [21]. In practice, this often trades robot-data efficiency for compute cost. Let’s look at a very rough lower-bound comparison for this.

Training cost is very hard to compare between models, but we can make a rough lower-bound estimate from the details available in papers and GitHub repos. Therefore, we use a simple dense-transformer lower-bound estimate, C ≈ 6NT, where N is the number of trainable dense parameters and T is the number of tokens¹.

VLM-based VLAs are cheaper in both stages of training because their sequences are smaller: they encode one to a few images plus text, then predict text or a short action-token sequence. WAMs train to predict a sequence of video latents with additional action tokens. The video token sequence is often around 10x longer compared to VLA sequences. This makes training on the same dataset more expensive than default VLA training.

Figure 24 gives an overview of different VLA/WAM training-cost estimates. DreamZero-style action tuning is roughly 9 ZFLOPs, which is large compared to the lightweight VLA training rows. A modern VLA like MolmoAct2 reports the full cost from Molmo2-ER to a DROID checkpoint at about 9.8 ZFLOP-equivalent. This assumes a strong inherited Molmo2-ER backbone and does not count Qwen3 or SigLIP2 pretraining cost. Summer-22B is a modern public video-pretraining token/data reference for understanding the cost required to train a competitive video foundation model at scale: with a 22B-parameter model and the paper’s ~500B-token training scale, it gives a ~66 ZFLOP video-pretraining estimate. If we reduce this to match the DreamZero Wan size of 14B, we can estimate 51 ZFLOPs for training the video model and WAM stage together. Compared to the efficient VLA Foundry recipe at 6.9 ZFLOPs, this results in a ~7.4x gap. These numbers show the challenges of WAM training at scale.

Beyond total FLOPs, there is a hardware and engineering barrier. A 14B-parameter model with roughly 8k-token action-tuning sequences needs substantial GPU memory and typically multi-node setups with high-end interconnects. Successful video-model training also depends on robust data filtering, captioning, video decoding, latent preprocessing, distributed I/O, and long-sequence DiT infrastructure.

There is also a data-quality version of the same argument. DreamZero argues that stronger video generation translates into stronger policy performance [8], so WAMs are not only compute-hungry but also video-data-quality hungry: filtering, captioning, latent representation, and generative pretraining all become part of the policy recipe. VLM-based VLAs do not show the same clean link. VLM4VLA [51] finds VLM initialization helps versus training from scratch, but generic VLM capabilities are poor predictors of downstream VLA performance. For WAMs, video-generation quality is a requirement for good policies; for VLAs, spatial objectives are much more important than other vision capabilities.

For row-by-row caveats and the derivations behind each estimate, see the reference table below.

Inference speed

Overall, VLM-based VLAs are not always fast, but default WAM setups with test-time video generation can be even slower. Exact numbers depend on hardware, implementation, diffusion steps, and action-chunk length, but representative values from Fast-WAM [23] give a useful reference: two common WAM inference modes (joint prediction and inverse dynamics with full video generation) take between 590ms and 800ms per action chunk, compared to roughly 190ms for Pi-0.5. That is a 3–4x slowdown at inference time, which matters a lot for real-time control. There are ways to speed this up, as shown in the DreamZero [8] paper and the Fast-WAM approach of skipping video generation entirely, but without access to large GPUs, running these models locally is still challenging.

Why the modern VLA baseline still matters

Modern VLM-based VLAs have improved quickly, and the strongest baseline now combines four ideas: discrete action tokenization, VLM-preserving co-training, isolated action heads, and much broader data mixtures. Any claim that video backbones are the better default has to beat the current SOTA recipe.

The architecture of VLAs has converged to one default setup: the Mixture-of-Transformers recipe, initially introduced in vision by Transfusion [30] and made popular in robotics by Pi-0 [2]. What changed is mostly the training recipe. Early flow-based action heads caused a strong disruption from discrete next-token VLM pretraining to continuous action denoising. Newer recipes try to reduce that disruption.

First, many modern VLAs use discrete tokenizers like FAST [3] or BEAST [34] to represent actions as a new type of language the VLM can learn. This is motivated by the optimization tension: VLMs are pretrained for discrete next-token prediction with a cross-entropy loss, while robot actions live in a continuous space typically modeled with flow matching. Naively fine-tuning a VLM with flow-matching objectives leads to catastrophic forgetting of the pretrained language and vision capabilities [51]. Co-training with discrete action tokenization, often in combination with isolated gradients from the flow-matching head, sidesteps this issue. The VLM can remain closer to its preferred discrete space and learn useful representations for embodied control, while the flow-matching head conditions on those features for its own action prediction. At test time, systems with a separate action head can drop the slow autoregressive action-token prediction path and let the action head do its job.

To get some intuition on the impact of this catastrophic forgetting problem, let’s look again at the RoboArena [1] snapshot. Pi-FAST uses the same backbone as Pi-0-DROID, but without the flow component and with discrete FAST tokens for action generation. Both are fine-tuned on DROID. Pi-FAST reaches a score of 1592, while Pi-0 only reaches 1475, which is a fairly large gap. This supports the view that the discrete-action recipe can preserve more useful pretrained capability than the original Pi-0 flow-based setup.

Second, Pi-0.5-style systems co-train on VLM data and robotics data, often while isolating gradients between the VLM and flow/action components for faster and more stable convergence [4,20]. This lets the VLM continue practicing language and visual understanding while the action side specializes for manipulation. The same pattern appears across recent VLAs such as Pi-0.5, Xiaomi-robotics-0 [27], and Being-H0.5 [28]. Pi-0.5 surpasses both Pi-FAST and Pi-0 on RoboArena by a good margin (1622 vs 1592 for Pi-FAST and 1475 for Pi-0). These results are consistent with the importance of these training design decisions for policy performance.

Even with these recipe improvements, VLAs still hit the grounding wall. Language is an underspecified way to express goals for behavior. A text instruction in a cluttered scene rarely pins down the relevant object instance or the desired physical state. So policies can overfit to spurious correlations like background objects or other dataset biases. Pi-0.7’s reported gap between language-only prompting and goal-image conditioning supports this view: visual subgoals improve language following and make training converge faster [41]. DreamZero’s 1750 elo-score on the same RoboArena snapshot is another argument that a video/image goal prior can help with this kind of problem.

So currently there is no real winner between WAMs and VLAs, and it is questionable if there ever will be one. First comparisons like Zhang et al. [25] benchmark LingBot-VA, Cosmos Policy, and Pi-0.5 on LIBERO-Plus and RoboTwin 2.0-Plus under matched perturbations. Their results show that WAMs can reach strong robustness without the broader training-data mixture used by VLA baselines. However, the comparison is limited to simulation environments and does not cover real-world generalization.

Are the two representation roads actually one?

The open question is whether the two paths even remain distinct in the long run. Some recent VLAs already use world-model-style components for better goal following (see Pi-0.7), and many recent WAMs borrow from the VLA MoT recipe for the action expert. The future of robot foundation models appears to be a mix of both.

There are already first signs of this direction in recent works like Motus [17] and BagelVLA [18]. Instead of deciding whether language or video should be the main representation for robotics, train one model that does it all. Figure 26 shows a simplified version: an understanding/VLM component, a video-generation component, and an action expert. Each tower has specialized weights while exchanging information through shared self-attention, often with non-symmetric patterns so each tower can expose different information to the others. Dense transformers or MoE-style routing could implement the same high-level idea.

A hierarchical version of this hybrid also appears in Physical Intelligence’s recent Pi-0.7 [41], a steerable VLA whose action expert is conditioned on visual subgoals generated at test time by a BAGEL– based world model [50]. A high-level policy emits subtask instructions, the world model turns those instructions into subgoal images, and the action expert executes conditioned on the current observation plus that subgoal.

The reported ablations support the language-following argument: adding world-model subgoals improves instruction following on complex referential tasks and is reported as necessary for some dataset-bias-breaking tasks where no-subgoal variants fail. The authors also report that subgoal images make training significantly faster, because action prediction becomes closer to an inverse-dynamics problem between the current frame and a desired future frame. On the evidence ladder, this is a real-world signal that visual subgoals can close part of the language-grounding gap, even inside a VLA-style stack. It does not require the stronger claim that every strong VLA needs a full video-generation head.

Sereact’s Cortex 2.0 [47] is another startup example pointing in this hybrid direction. Cortex 2.0 adds a world model that generates candidate future trajectories in visual latent space, scores them for expected progress, risk, and efficiency, and conditions execution on the best-scored rollout. This makes it an industry signal for WAM-style foresight becoming a planning layer inside deployed manipulation systems.

Being-H0.7 [42] is the best example of a foundation-model hybrid: it is a latent-plan-style WAM/VLA built on the pretrained VLA Being-H0.5, with InternVL3.5 as the understanding expert, Qwen3 as the action expert, and V-JEPA2.1 [64] visual encoders. It manages to combine VLA-style pretrained components, V-JEPA2.1 [64] future-observation embeddings, a Play-LMP-style prior/posterior latent interface, and a flow-matching action policy.

Compute cost is the main reason we have seen only a few “one-model-does-it-all” systems so far. Training a strong VLM is already expensive; layering large-scale video modeling on top compounds the cost. The split between VLA-style and WAM-style training therefore remains useful in the near term, both because of compute limits and because we still do not know which ingredients matter most for robotics. Do you think these two roads genuinely merge, or does one of them win outright?

A fourth path: Robotics-first foundation models

A fourth possibility is a robotics-first foundation model (RFFM).  Basically, this would be a large transformer architecture designed around robotics challenges: embodiment, action, contact-rich interaction, and embodied memory. A clean version of this path would not simply start from a web VLM or video generator and then attach actions later. It would make interaction and action central to pretraining from the beginning.

The cleanest example I am aware of is GEN-1 from Generalist AI, which introduced a large robotic behavior model pretrained on 500k hours of UMI-style wearable data. The core issue for this direction is access: outside of well-funded startups and larger companies, almost no one has access to this kind of large-scale human or robot data. So this research path is currently blocked for the community until we get more open-source robotics data at scale.

Closing thoughts

WAMs will become a core research subfield of robot foundation models. Whereas VLAs have converged on a roughly shared recipe (VLM backbone, gradient-isolated action expert with flow matching, and co-training on a broad web and robotics mix), WAMs are still in an exploration phase. Papers vary widely in video backbone, policy formulation, training recipe, and evaluation setup. That research diversity is healthy for a young area, and many new ideas are being published. However, no one really knows what works best yet.

To summarize the conclusions from this blog:

• The gap from instruction to motion is still open. Even modern VLAs, with discrete action tokenization, VLM-preserving co-training, and broad data mixtures, do not fully close it. WAMs promise to attack this gap from the video side, but current results do not show that they have solved it yet.
• Robot benchmarking is still a core issue. I have to repeat a finding from my last blog post: modern VLA and WAM benchmarking is not solved. We need more benchmarks like RoboLab [62] or MolmoSpaces [61] that make benchmaxxing harder and require proper policy generalization to get good scores.
• The likely next generation of robot foundation models will be WAM+VLA hybrids. Pi-0.7’s BAGEL subgoals, Cortex 2.0’s planning-by-foresight, Being-H0.7’s latent prior/posterior bridge, and Motus / BagelVLA-style hybrids already merge VLA and WAM thinking. The first robot foundation models trained from scratch are another likely bet, especially once we get access to more and better open-source robotics data.

That’s my current read on where WAMs stand. If you see it differently, or have a strong argument for one of these paths over the others, reach out and let me know — I’d be very happy to hear it.

Footnotes

Acknowledgements

The author thanks Alexander Schwarz, Ankit Goyal, Elie Aljalbout, Fabio Ramos, Seonghyeon Ye, Shenyuan Gao, Xuning Yang, Yashraj Narang, and Yixuan Wang (listed alphabetically by first name) for insightful feedback and discussions that improved this post.

Sources

1. Atreya, Pranav, et al. “RoboArena: Distributed Real-World Evaluation of Generalist Robot Policies.” CoRL 2025. paper
2. Black, Kevin, et al. “Pi-0: A Vision-Language-Action Flow Model for General Robot Control.” arXiv 2024. paper
3. Pertsch, Karl, et al. “FAST: Efficient Action Tokenization for Vision-Language-Action Models.” arXiv 2025. paper
4. Physical Intelligence, et al. “Pi-0.5: A Vision-Language-Action Model with Open-World Generalization.” arXiv 2025. paper
5. Bjorck, Johan, et al. “GR00T N1: An Open Foundation Model for Generalist Humanoid Robots.” arXiv 2025. paper
6. Liu, Bo, et al. “LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning.” NeurIPS 2023. paper
7. Mees, Oier, et al. “CALVIN: A Benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks.” RA-L 2022. paper
8. Ye, Seonghyeon, et al. “World Action Models Are Zero-shot Policies.” arXiv 2026. paper
9. Li, Lin, et al. “Causal World Modeling for Robot Control.” arXiv 2026. paper
10. Du, Yilun, et al. “Learning Universal Policies via Text-Guided Video Generation.” NeurIPS 2023. paper
11. Wu, Hongtao, et al. “Unleashing Large-Scale Video Generative Pre-training for Visual Robot Manipulation.” ICLR 2024. paper
12. Cheang, Chi-Lam, et al. “GR-2: A Generative Video-Language-Action Model with Web-Scale Knowledge for Robot Manipulation.” arXiv 2024. paper
13. Kim, Moo Jin, et al. “Cosmos Policy: Fine-Tuning Video Models for Visuomotor Control and Planning.” arXiv 2026. paper
14. Pai, Jonas, et al. “mimic-video: Video-Action Models for Generalizable Robot Control Beyond VLAs.” arXiv 2025. paper
15. Nair, Suraj, et al. “R3M: A Universal Visual Representation for Robot Manipulation.” arXiv 2022. paper
16. Karamcheti, Siddharth, et al. “Language-Driven Representation Learning for Robotics.” arXiv 2023. paper
17. Bi, Hongzhe, et al. “Motus: A Unified Latent Action World Model.” arXiv 2025. paper
18. Hu, Yucheng, et al. “BagelVLA: Enhancing Long-Horizon Manipulation via Interleaved Vision-Language-Action Generation.” arXiv 2026. paper
19. Bruce, Jake, et al. “Genie: Generative Interactive Environments.” ICML 2024. paper
20. Driess, Danny, et al. “Knowledge Insulating Vision-Language-Action Models: Train Fast, Run Fast, Generalize Better.” NeurIPS 2025. paper
21. Wan Team, et al. “Wan: Open and Advanced Large-Scale Video Generative Models.” arXiv 2025. paper
22. Agarwal, Niket, et al. “Cosmos World Foundation Model Platform for Physical AI.” arXiv 2025. paper
23. Yuan, Tianyuan, et al. “Fast-WAM: Do World Action Models Need Test-time Future Imagination?” arXiv 2026. paper
24. Hu, Yucheng, et al. “Video Prediction Policy: A Generalist Robot Policy with Predictive Visual Representations.” ICML 2025. paper
25. Zhang, Zhanguang, et al. “Do World Action Models Generalize Better than VLAs? A Robustness Study.” arXiv 2026. paper
26. Schäfer, Lukas, et al. “When does predictive inverse dynamics outperform behavior cloning?.” arXiv preprint arXiv:2601.21718 (2026). paper
27. Cai, Rui, et al. “Xiaomi-robotics-0: An open-sourced vision-language-action model with real-time execution.” arXiv 2026. paper
28. Luo, Hao, et al. “Being-H0.5: Scaling Human-Centric Robot Learning for Cross-Embodiment Generalization.” arXiv 2026. paper
29. Tian, Yang, et al. “Predictive Inverse Dynamics Models are Scalable Learners for Robotic Manipulation.” ICLR 2025. paper
30. Zhou, Chunting, et al. “Transfusion: Predict the Next Token and Diffuse Images with One Multi-Modal Model.” ICLR 2025. paper
31. Shridhar, Mohit, Yat Long Lo, and Stephen James. “Generative Image as Action Models.” CoRL 2024. paper
32. Lynch, Corey, et al. “Learning Latent Plans from Play.” CoRL 2020. paper
33. Ye, Seonghyeon, et al. “Latent Action Pretraining from Videos.” ICLR 2025. paper
34. Zhou, Hongyi, et al. “BEAST: Efficient Tokenization of B-Splines Encoded Action Sequences for Imitation Learning.” NeurIPS 2025. paper
35. Chi, Cheng, et al. “Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots.” RSS 2024. paper
36. Chi, Cheng, et al. “Diffusion Policy: Visuomotor Policy Learning via Action Diffusion.” IJRR 2025. paper
37. Ma, Teli, et al. “DiT4DiT: Jointly Modeling Video Dynamics and Actions for Generalizable Robot Control.” arXiv 2026. paper
38. Zhang, Wenyao, et al. “DreamVLA: A Vision-Language-Action Model Dreamed with Comprehensive World Knowledge.” NeurIPS 2025. paper
39. Li, Shuang, et al. “Unified Video Action Model.” arXiv 2025. paper
40. Rhoda AI Team. “Causal Video Models Are Data-Efficient Robot Policy Learners.” Rhoda AI Blog, 2026. blog
41. Physical Intelligence. “Pi-0.7: a Steerable Model with Emergent Capabilities.” Physical Intelligence Blog/Paper, April 2026. blog, paper
42. Luo, Hao, et al. “Being-H0.7: A Latent World-Action Model from Egocentric Videos.” arXiv 2026. paper, project page
43. Assran, Mido, et al. “V-JEPA 2: Self-Supervised Video Models Enable Understanding, Prediction and Planning.” arXiv 2025. paper
44. Gao, Shenyuan, et al. “DreamDojo: A Generalist Robot World Model from Large-Scale Human Videos.” arXiv 2026. paper
45. Aida, Adriana, et al. “Cortex 2.0: Grounding World Models in Real-World Industrial Deployment.” arXiv 2026. paper, project page
46. Mercat, Jean, et al. “VLA Foundry: A Unified Framework for Training Vision-Language-Action Models.” arXiv 2026. paper, model collection
47. Ryu, Simo, and Chunghwan Han. “Summer-22B: A Systematic Approach to Dataset Engineering and Training at Scale for Video Foundation Model.” arXiv 2026. paper
48. Physical Intelligence. “openpi.” GitHub repository, 2025-2026. code
49. Zhang, Jianke, et al. “VLM4VLA: Revisiting Vision-Language-Models in Vision-Language-Action Models.” arXiv 2026. paper
50. Deng, Chaorui, et al. “Emerging Properties in Unified Multimodal Pretraining.” arXiv 2025. paper
51. Hancock, Asher J., et al. “Actions as Language: Fine-Tuning VLMs into VLAs Without Catastrophic Forgetting.” ICLR 2026. paper, project page
52. Ho, Jonathan, et al. “Imagen Video: High Definition Video Generation with Diffusion Models.” arXiv 2022. paper
53. Wang, Lirui, et al. “Prediction with Action: Visual Policy Learning via Joint Denoising Process.” arXiv 2024. paper
54. Wen, Junjie, et al. “Unified World Models: Coupling Video and Action Diffusion for Pretraining on Large Robotic Datasets.” arXiv 2025. paper
55. ByteDance Seed. “GR-1.” GitHub repository, 2024. released config
56. Wan-Video Team. “Wan2.2.” GitHub repository, 2025. release
57. Hou, Bohan, et al. “World Model for Robot Learning: A Comprehensive Survey.” 2026. project page, repo
58. Fang, Haoquan, et al. “MolmoAct2: Action Reasoning Models for Real-World Deployment.” arXiv 2026. paper, model
59. Clark, Christopher, et al. “Molmo2: Open Weights and Data for Vision-Language Models with Video Understanding and Grounding.” arXiv 2026. paper, model
60. Reuss, Moritz. “State of VLA Research at ICLR 2026.” Blog post, October 2025. blog
61. Kim, Yejin, et al. “MolmoSpaces: A Large-Scale Open Ecosystem for Robot Navigation and Manipulation.” arXiv 2026. paper
62. Yang, Xuning, et al. “RoboLab: A High-Fidelity Simulation Benchmark for Analysis of Task Generalist Policies.” arXiv 2026. paper
63. Sun, Jingwen, et al. “VLA-JEPA: Enhancing Vision-Language-Action Model with Latent World Model.” arXiv 2026. paper
64. Mur-Labadia, Lorenzo, et al. “V-JEPA 2.1: Unlocking Dense Features in Video Self-Supervised Learning.” arXiv 2026.

Cite this post

If you’d like to cite this article, use the BibTeX below: