PPMem.
PPMem

Persistent Planar Memory
for Video World Models

1Tsinghua University 2Ant Group 3HKUST

*Corresponding authors.

NeurIPS 2026

Camera-controlled video generation
with persistent 3D memory.

Paper Code Hugging FaceModel Watch video

VIDEO

Full demonstration

From persistent plane memory to repeated visits, rotations, and handheld camera motion.

Camera-controlled generationVIDEO & GEOMETRY
Generated views of a corridor.
Corridor
Generated views of an amusement park.
Amusement park
Generated views of a courtyard.
Courtyard
Generated views of a rooftop.
Rooftop
0.0 / 16.7 s

Four forward–backward visits to the same scene.

Choose Normal or Depth to compare geometry with RGB.

THE IDEA

Persistent memory, explicit geometry.

PPMem generates video from a single image and a camera trajectory, while building a persistent 3D memory from plane primitives.

Walls, floors, facades, and other surfaces become a compact set of oriented planes. As generation proceeds, newly predicted geometry is merged into this memory and rendered from upcoming camera viewpoints to guide subsequent video segments.

Read the abstract

Video world models that generate and explore environments at scale need a persistent 3D memory of what they have produced. Plane primitives offer a natural candidate: a compact set of oriented planes captures the dominant structural elements (e.g., walls, floors, facades) of indoor and outdoor scenes, while requiring only 7 parameters per primitive.

We present PPMem, a video world model that adopts plane primitives as its persistent 3D memory. PPMem augments a video diffusion transformer with a planar prediction head and a Depth Token Fusion (DTF) module: each autoregressive chunk jointly produces RGB video and planar geometry, which merges into a persistent plane memory that conditions subsequent chunks via rendered depth. The resulting memory is three orders of magnitude more compact than per-pixel point clouds (~100K primitives, ~5 MB for a full scene), geometrically explicit (metric depth, normals, exportable surfaces), cross-view consistent (jointly predicted by the shared backbone), and adaptive (refined by the merging procedure as generation proceeds). On DL3DV and RealEstate10K, PPMem outperforms prior 3D-grounded world models in video quality while jointly producing metric-scale depth, surface normals, and exportable planar geometry, sustaining coherence across hundreds of frames under large viewpoint changes.

Indoor and outdoor generated video sequences, with insets showing the accumulated plane memory at successive camera views.
The scene grows in video and in memory. Insets show the accumulated planes as the camera explores.

METHOD

Method overview

Planar geometry connects one video segment to the next.

PPMem architecture: a video diffusion transformer, plane prediction head, Depth Token Fusion module, and persistent plane pool with rendered conditioning.
01

Predict planes

A plane prediction head extracts structured geometry from the video model’s features. Each plane is described by seven parameters.

02

Update memory

New planes are merged with earlier geometry, removing redundant or inconsistent surfaces as the scene expands.

03

Condition new views

The accumulated memory is rendered at the next camera poses, providing geometric conditioning for subsequent generation.

LONG-CONTEXT GENERATION

Long-context comparisons

Long-context video comparison showing ground truth, PPMem, and baseline methods at the frame indices labeled in the paper.
Rows follow time; columns compare methods. Select the figure to inspect scene structure and appearance.

BibTeX

@article{he2026ppmem,
  title   = {Persistent Planar Memory for Video World Models},
  author  = {He, Yuze and Tan, Bin and Gao, Zelin and Shen, Yujun
             and Liu, Yong-Jin and Xue, Nan and Xu, Yinghao},
  journal = {arXiv preprint arXiv:2610.23795},
  year    = {2026}
}