Technical Report · 2026

Astronex‑World

Real-Time Interactive World Model Foundation
for Autonomous Driving and Embodied Intelligence

Xin Zhou1 · Cong Miao2

1Astronex Robotics  ·  2Nanjing University of Information Science and Technology

Technical Report (PDF) Code Weights Website arXiv
720p A rain-soaked megacity pursuit, generated from a single reference frame and a forward camera trajectory. All clips on this page are project outputs.
  • 5Bcontrollable video DiT
  • 832×480native output at 24 fps
  • 8-stepUniPC causal sampling
  • 73.5WBench Navi 158
  • 1×L2048 GB inference

Overview

A shared 5B backbone with bidirectional and causal releases

Abstract

We present Astronex-World, a 5B interactive video world-model foundation for autonomous driving and embodied intelligence. Given text or an initial observation, it consumes frame-aligned camera trajectories, continuous actions, and an embodiment identifier to predict future visual states under control. The family provides a bidirectional model for full-context generation and supervision and a causal model with block-causal attention and cross-block KV caching for persistent generation. Both inherit the Wan2.2-TI2V-5B visual prior. PRoPE injects camera intrinsics and extrinsics, while a 64-dimensional action stream modulates every Transformer layer. Training combines bidirectional control adaptation, autoregressive teacher forcing, online UniPC trajectory distillation, mixed-domain supervised fine-tuning, and DMD/DMD2 distribution matching. The causal model generates 480×832 video at 24 fps using eight-step UniPC, eight latent frames per block, a 20-frame local context, and four persistent sink frames. It scores 73.5 on WBench Navi 158 and 70.0 on Full 289, demonstrating strong temporal, background, and geometric consistency. Training uses two NVIDIA L20 48 GB GPUs, and inference runs on one L20 48 GB.

  • Two model forms, one interface

    A bidirectional checkpoint for full-context quality and teacher supervision, and a causal checkpoint that reuses historical KV states for few-step, cacheable, persistent interaction.

  • Camera and action control

    Complete intrinsics and extrinsics enter all 30 Transformer layers through PRoPE; a 64-D continuous action stream and 32 embodiment IDs modulate every visual layer.

  • Documented training lineage

    From bidirectional control adaptation and AR teacher forcing to online UniPC distillation, mixed-domain SFT and asymmetric DMD — with the data, checkpoints, update counts and batch sizes that were used.

  • Open post-training path

    A reserved action_output head can be post-trained on robot trajectories as an inverse-dynamics or action-sequence decoder, connecting the shared trunk to driving and embodied control.

Montage of distinct frames sampled from Astronex-World generated videos
Frames sampled from project-generated videos across interactive and text-to-video settings. The montage contains no repeated frames and no synthetic placeholders.

Demos

Generated video, straight from the released checkpoints

Every clip below plays from the files in the project video folder. Cinematic clips are image-to-video generations with an explicit camera trajectory; the action-controlled group is a causal rollout where the model keeps consuming control input. Click any tile to open it at full size.

Cinematic generations camera-trajectory conditioned

Rain-soaked megacity pursuit

1280×7045.2 sforward

An unbranded electric hypercar accelerates through a near-future city at sunrise; wet asphalt, spray and reflections stay physically coherent.

Storm lighthouse at dusk

1280×7045.2 sforward

A slow push toward a solitary stone lighthouse while ocean waves break against black cliffs and the beam sweeps through rain and spray.

Portrait in a gentle breeze

832×4805.2 slateral

A cinematic close portrait: the subject holds affectionate eye contact while a continuous breeze moves long hair in layered strands without breaking facial identity.

Alpine lake at sunset

832×4805.2 sforward

The camera glides over wildflowers toward a turquoise lake; ripples, swaying grass and water reflections stay geometrically coherent as it advances.

Ocean cliff at golden hour

832×4805.2 slateral

A slow slide along a colossal ocean cliff: Atlantic waves explode against dark sea arches while the horizon and rock geometry remain locked.

Open-world coastal pursuit

832×4805.2 sforward + lateral

A red supercar holds its lane between two lane markings while three police cars and a helicopter keep their identities and spacing across the shot.

Wormhole approach

832×4805.2 sapproach

A spacecraft closes on a luminous traversable wormhole: the ring stays centered and round while blue-gold lensing rotates slowly clockwise.

Supermassive black hole

832×4805.2 sapproach

A steady approach toward a supermassive black hole; the event horizon stays circular as the accretion disk brightens and gravitational lensing intensifies.

Action-controlled rollouts causal model, key inputs shown in-frame

Coastal highway

832×4804.8 skeys W, A

Real-road coastal drive under continuous steering input; lane markings, barriers and daylight stay consistent through the turn.

Suburban road

832×4804.8 skey W

A forward-only rollout on a country road, showing how the world model extends a scene while the ego motion is commanded.

Night road

832×4804.8 skeys W, D

Low-light driving with headlight pools, lane markings and distant buildings held across the clip — a stress test for photometric stability.

Mountain road

832×4804.8 skeys W, D

Vehicle-view rollouts on a mountain road, where geometry and parallax have to stay coherent under combined forward and lateral control.

Snow ridge path

832×4804.8 skeys W, A

A bright, low-contrast environment where structure and terrain detail are easily repainted; the causal model keeps the path readable.

Open-world game

832×4804.8 skeys W, D

First-person game-style footage driven by CrossFPS-style stick channels, the same control space used for WBench keyboard commands.

Character close-up

832×4804.8 swith audio

Starting from a single state, the model keeps identity, lighting and scene consistent while expression and speech motion evolve. Open the tile to hear it.

33-second continuous rollout

832×48033.3 ssingle take

One uninterrupted causal rollout that keeps accepting actions and extending the future without clip stitching — the long-horizon behaviour that persistent sinks and the local window are built for.

Method

World-model controls on top of a video DiT

Astronex-World keeps the Wan2.2-TI2V-5B latent representation and its 30-layer DiT backbone rather than repeating large-scale video pretraining, and introduces world-model controls through low-rank adaptation plus newly initialized modules.

Astronex-World model family and real-time-oriented streaming inference pipeline
Model family and streaming pipeline. A shared 5B controllable video DiT yields bidirectional and causal checkpoints. The streaming path aligns the first frame, text, camera and action controls to latent frames, denoises each block with eight-step UniPC, and returns generated KV states to the next block.

Block-causal generation

The bidirectional model uses unmasked spatiotemporal self-attention over a complete clip. The causal model replaces temporal attention with a block lower-triangular mask: positions inside the current latent block interact bidirectionally, cross-block attention sees generated history only, and completed blocks are cached as KV instead of being recomputed. Training uses four latent frames per block; inference uses eight to reduce boundaries and improve throughput.

Camera and action conditioning

At each latent time the camera branch receives an intrinsic matrix and a world-to-camera transform; PRoPE turns the projection matrices into a frustum-related positional transform applied to attention queries, keys, values and outputs in all 30 layers. A two-layer MLP maps the 64-D action vector, the embodiment embedding is added, and six modulation groups per frame (shift, scale, residual gate for attention and FFN) are summed into the Wan time modulation. The final action projection is zero-initialized so insertion preserves base behaviour.

Long-horizon context

The model keeps 20 latent frames of local attention context. The earliest four frames form a persistent attention sink and are never removed as the window slides; the rest stores recent history and the current block. A camera-frustum-overlap retrieval option can substitute high-overlap historical frames when the camera revisits a region, and the final WBench configuration enables it only for revisit patterns such as round trips, loops and zigzags.

Released configuration

ModelAstronex-World-5B
FormsBidirectional control model and causal few-step model
Base modelWan2.2-TI2V-5B
Parameters5,351,000,000 (bfloat16)
DiT depth30 layers · hidden width 3072 / 24 heads · FFN width 14,336
Latent geometry48 channels · VAE compression 4× temporal, 16×16 spatial
Native conditionsText, first frame, camera intrinsics/extrinsics, 64-D actions
Camera injectionPRoPE in all 30 layers
Action space64-D continuous action, 32 embodiment IDs
Action outputOpen post-training head, 64-D per latent frame, enabled when training action-sequence decoding
Causal context20 latent frames plus four persistent sink frames
Default samplerUniPC, eight steps, CFG 3.0
Inference blockEight latent frames
Output480×832 at 24 fps

Training

From bidirectional control adaptation to a causal few-step model

Every retained stage uses two NVIDIA L20 48 GB GPUs, bfloat16 mixed precision, FSDP, gradient checkpointing, 1,000 training time indices and internal timestep shift 5.0. Training tensors have shape [1, 20, 48, 30, 52] and contain four latent frames per block; all LoRA stages use rank 64 and alpha 128.

Four-stage path

StageFunctionAstronex-World realization
Blockwise AR initializationConvert full temporal attention into causal history-only attentionTeacher-forced AR diffusion, 500 updates
Online causal transferMatch a frozen teacher while conditioning on student historyCF++-style 25-step teacher to 12-step student UniPC distillation, 20 updates
Physical / action adaptationRecover subject motion and mixed-domain controlMixed-data causal SFT, 540 retained updates
Few-step distribution matchingImprove the few-step sample distribution with a bidirectional score teacherDMD / DMD2-based asymmetric matching, 45 updates

Retained lineage

StageData & initializationCore configurationUpdates / eff. batchLR and trainable modules
Bidirectional cameraControl2V 2,000; Wan2.2 baseFull attention, CFG 8, PRoPE0–2400; 322e-5; LoRA + camera/PRoPE, ~444.4M
Bidirectional actionDROID 1,472; camera-240064-D action, 32 embodiments; camera frozen0–2000; 322e-5; LoRA + action, ~227.6M
Camera recoveryControl2V 2,000; action-2000Camera/action present; fixed control graft2000–2200; 322e-5; LoRA, ~161.2M
Direction enhancementControl2V_combined 6,000; restore-2200Bidirectional flow matching, CFG 80–100; 81e-5; LoRA + camera
CF / CF++ Stage 1Control2V_combined; restore2200_sft000100_fullCausal teacher forcing; block 4, window 200–500; 8Trunk/camera 2e-5; ~444.4M
CF++ Stage 2Control2V 2,000; Stage-1 checkpoint25-step teacher to 12-step student; window 7, sink 10–20; 2Trunk/camera 2e-6; ~161.2M
Physical/action SFTMixed 7,784; CF++ base + overlaysWindow 20, sink 4, reference probability 0.7, rolling forcing 0.8Labels 960–1500; 540 updates; 64Trunk 2e-6, camera 1e-6, action 2e-5
DMD / DMD2Mixed 7,784; SFT1500; bidirectional teacherNodes [1000, 750, 500, 250], 1–5 blocks, grad 4 frames, rolling forcing 0.51500–1515–1545; 45 updates; 2Generator 2e-6 → 1e-6; fake-score 1e-6

The mixed index contains Control2V (2,000), CrossFPS (3,700), DrivingDojo (44) and NVIDIA PhysicalAI synthetic embodied robot scenes (2,040) — 7,784 records in total. The bidirectional action branch is first trained on 1,472 locally preprocessed DROID records, which are not counted in that mixture.

Flow-matching corruption and training objectives for bidirectional and causal training
Corruption and objectives. The first row visualizes interpolation from a clean sample toward Gaussian noise through RGB projections while optimization stays in latent space. Bidirectional training shares one timestep across the clip; causal training samples timesteps per four-frame latent block and uses clean-history teacher forcing. During DMD, a causal rollout is re-noised and the score difference between a frozen bidirectional real-score model and a trainable fake-score model updates the generator.
Training path from the bidirectional model to the released causal model
Bidirectional-to-causal path. PRoPE and action modulation are learned under full temporal attention, block-causal teacher forcing follows, and a frozen 25-step teacher then supervises a 12-step student from the same noise with gradients retained through one randomly selected transition. Mixed-data SFT and asymmetric DMD with a bidirectional score teacher produce the released weights.

Results

WBench and VBench

Benchmarks are reported with the official evaluation code and the default VLM judge. Inference is real time on a single NVIDIA L20 48 GB.

WBench five-dimensional scores

SplitAverageQualitySettingInteractionConsistencyPhysical
Navi 15873.578.273.563.483.668.6
Full 28970.078.373.847.682.468.1

The official script macro-averages six Quality metrics, two Setting metrics, eight Consistency metrics and two Physical metrics. Navi Interaction contains navigation trajectory only; Full Interaction also averages event edit, subject action and perspective switch. Average weights the five high-level dimensions equally.

Navi 158 · leaderboard comparison

ModelAvg.QualitySettingInteractionConsistencyPhysical
Astronex-World73.578.273.563.483.668.6
LongCat-Video73.275.472.362.187.168.9

Full 289 · leaderboard comparison

ModelAvg.QualitySettingInteractionConsistencyPhysical
Astronex-World70.078.373.847.682.468.1
LongCat-Video69.977.272.345.186.668.4
Helios (distilled)69.773.375.341.682.276.1

Peer scores are taken from the WBench live leaderboard snapshot dated September 13, 2026. Astronex-World achieves the highest Average among the displayed entries on both splits; LongCat-Video remains stronger in Consistency and Helios in Physical.

WBench submetrics

CategorySubmetricScore
QualityAesthetic / Imaging61.61 / 61.93
QualityTemporal Flicker / Motion Smoothness96.55 / 98.45
QualityDynamic Degree / HPSv388.24 / 62.98
SettingScene / Subject Adherence56.39 / 91.26
InteractionNavigation Trajectory63.38
InteractionEvent Edit / Subject Action51.60 / 54.25
InteractionPerspective Switch20.97
ConsistencyBackground / Geometric / Photometric90.89 / 89.64 / 82.01
PhysicalVisual Plausibility / Causal Fidelity55.74 / 80.48
WBench temporal sequences, nine complete causal rollouts
WBench temporal sequences. Each row is one complete causal rollout of a four-round case: 381 frames, or 15.875 seconds. The first column is the benchmark input and the following five frames are sampled in temporal order, covering orbital Earth, aerial farmland, a lavender field, an autumn park, a toy workshop interior, Venetian canals, a coastal boardwalk, an open plaza and a mushroom village.
Nine additional WBench causal rollouts
Additional WBench rollouts. Vertical control along a mountain lake, a mythological hall, first-person view switching between two paddlers, car-to-car target switching, a third-to-first person cut, lateral-only bazaar control, a staged demolition chain, a fishing subject action, and a jump followed by an incoming helicopter. Together with the previous figure these are 18 independent WBench rollouts.
Detailed WBench consistency evidence with tracked regions
Consistency detail. Red boxes track the principal subject in the astronaut and balloon sequences, blue boxes track canal geometry and facade appearance; the enlarged regions expose identity, texture, colour and structural retention across the complete rollout.

VBench 1.0

SettingVideos scoredFrames / durationResolutionConditionRetrieval
Text-to-video (official 944-prompt suite)240 of 6,220 generated125 / 5.208 s832×480, 24 fpstext onlyoff
Image-to-video (official I2V split)118 of 5,590 generated125 / 5.208 s832×480, 24 fpsfirst frame (+ camera instruction)off
SettingDimensionScoreVideos
Text-to-videoImaging Quality0.715 [0.704, 0.726]240
Text-to-videoAesthetic Quality0.508 [0.497, 0.519]240
Text-to-videoDynamic Degree0.254 [0.199, 0.309]240
Text-to-videoMotion Smoothness0.990 [0.988, 0.991]240
Text-to-videoTemporal Flickering0.987 [0.985, 0.989]240
Text-to-videoOverall Consistency0.220 [0.212, 0.227]240
Image-to-videoCamera Motion0.485103
Image-to-videoI2V Background0.99715
Image-to-videoBackground Consistency0.98515
Image-to-videoImaging Quality0.73815
Image-to-videoAesthetic Quality0.47315
Image-to-videoTemporal Flickering0.99515

Brackets are 95% confidence intervals from bootstrapping the 240 per-video text-to-video scores with 4,000 resamples; Dynamic Degree additionally matches a binomial interval because it is a per-video binary decision. Image-to-video dimensions are point estimates with their exact video counts.

VBench text-to-video sequences, nine independent videos
VBench text-to-video sequences. Each row is an independent 125-frame, 5.208-second text-only video shown as six chronological frames with no input image: library, sea cliff, courtyard, palace, coastal tower, restaurant counter, garden bench, ceramic bowl and cup.
Nine additional VBench text-to-video sequences
Additional VBench sequences. Bed, grapes, Stonehenge, dining table, red barn, lakeside cabin, Petra, apple and bedroom — 18 independent VBench videos in total, spanning single objects, multi-object arrangements, interiors, architecture, monuments and outdoor landscapes.
Detailed VBench consistency evidence
VBench consistency detail. Red boxes enlarge a ceramic bowl to compare shape and material over time, blue boxes expose the persistent library layout, and purple boxes track a garden bench with its surrounding flowers. Each column comes from a different time in the same generated video.

What the numbers say

  • Long-horizon consistency is the strongest capability. On Full 289, Consistency is 82.4, with Background Consistency 90.89 and Geometric Consistency 89.64; Temporal Flicker and Motion Smoothness reach 96.55 and 98.45.
  • Controllable generation leads the displayed comparison. On Navi, Astronex-World exceeds LongCat-Video by 0.3 Average points, 2.8 Quality points and 1.3 Interaction points; on Full it exceeds both LongCat-Video and Helios in Average, Quality and Interaction.
  • Complex interaction is the weak spot. Navi Interaction is 63.4 while Full falls to 47.6, because the Full definition also averages three weaker metrics: Perspective Switch 20.97, Event Edit 51.60 and Subject Action 54.25.
  • Physical fidelity is moderate. Physical is 68.1 with Causal Fidelity 80.48 but Visual Plausibility 55.74, and Scene Adherence (56.39) trails Subject Adherence (91.26) — temporal continuity should not be read as accurate contact or rigid-body dynamics.

Real-time streaming

The causal model streams block by block with cross-block KV caching and few-step UniPC sampling, so frames leave the sampler as the rollout is produced instead of after the whole clip is denoised. It runs in real time on a single NVIDIA L20 48 GB, at 480p and at the 1280×704 preset.

Motion diagnostic

Model stateResidual amplitudevs. bidirectional teacher
Bidirectional88.8 px100%
SFT150036.1 px41%
DMD151521.2 px24%
DMD154516.6 px19%

A single development basketball sequence, not a public benchmark and not comparable across models. It shows that final DMD improves few-step quality and consistency while strongly suppressing subject motion.

Limitations & roadmap

What 1.0 does not solve

Representative failure cases under strong turns and low light
Representative failures. In Case 137 the abstract tunnel drifts in colour and geometry under a W → right → left sequence; in Case 165 a warehouse gradually darkens and loses local detail. Attention sinks delay but do not eliminate long-term colour and structural drift, and a return turn does not guarantee reversible scene evolution.
  • Weak complex interaction. Training emphasizes continuous camera and action streams. Event editing, subject action and perspective switching rely on a limited text-switch interface that cannot express multiple independent timed events.
  • Remaining long-horizon drift. Four sink frames and local history delay forgetting, but strong turns, loops and low-light scenes still show colour shift, darkening, structural repainting and detail loss.
  • Distillation suppresses motion. DMD can prefer stable but static solutions, and internal diagnostics stay far below bidirectional motion amplitude.
  • Limited physical fidelity. The data contain driving and robot simulation, but the model has no explicit physical state, depth, collision constraint or 3D scene representation.
  • Action transfer across embodiments. The open [F, 64] action-output path needs embodiment-specific adapters and post-training data for new robot morphologies or vehicle control spaces.

Astronex-World 2.0

2.0 will include 14B and 5B scales. The 14B model targets greater visual capacity, complex dynamics and long-horizon scene modelling; the 5B model keeps the low-deployment-cost, real-time-oriented path. Both scales will use unified text, image, camera, continuous-action and embodiment interfaces and will retain bidirectional and causal forms.

A primary objective is long-term colour drift, structural repainting, loop inconsistency and motion decay. Future training will combine full-sequence bidirectional representations with causal self-rollout distributions, longer sequences, cross-block consistency supervision, geometric and photometric constraints, and systematic memory training. "Combination" here means one model family, control interface and objective set, not two models running in parallel at inference.

Autonomous driving and embodied intelligence are the main action extensions: ego trajectory, steering, speed, throttle/brake and route intent with driving-state and action-sequence output heads; embodiment-specific adapters for joints, end effectors, grippers and mobile bases. Evaluation will expand from video quality to action adherence, future prediction, planning candidates, collision and closed-loop driving metrics, simulation, and real-robot task success.

  • 14B and 5B scales, bidirectional + causal
  • Unified camera, action and embodiment interfaces
  • Driving and embodied action heads, open for extension

Citation

BibTeX

@techreport{zhou2026astronexworld,
  title       = {Astronex-World: Real-Time Interactive World Model Foundation
                 for Autonomous Driving and Embodied Intelligence},
  author      = {Zhou, Xin and Miao, Cong},
  institution = {Astronex Robotics},
  year        = {2026},
  type        = {Technical Report}
  % TODO: add archivePrefix = {arXiv}, eprint = {XXXX.XXXXX} once the preprint is posted.
}

Benchmark protocols: WBench with the official evaluation code and default VLM judge, and VBench 1.0 with the official evaluation code in custom-video mode using locally cached checkpoints.

Resources