← Back to blog
AiAbout 6 min read

4LC Caches the Scene So a Video Model Stops Redrawing It

Published Nov 14, 2026
4LC Caches the Scene So a Video Model Stops Redrawing It

A video world model generates a scene, turns the camera, comes back, and the building is different. Not dramatically different. The windows are in slightly wrong places, the roof pitch has shifted, the tree that was on the left is now nearer the middle.

That drift is the specific failure the 4LC paper attacks. The name stands for 4D Latent Cache, and the premise is almost embarrassingly simple: the model already computed what that scene looks like, so why compute it again.

What gets cached

4LC binds static and dynamic scene content, extracted by several visual encoders, to shared geometry and motion information. Later generation requests, at a different time or from a different viewpoint, read from that cache instead of re-rendering and re-encoding from scratch.

The mechanism that makes it work is a motion mask. Every time the model generates a new segment, it appends the result as a new cache entry. Subsequent generation reuses only the parts of earlier entries the mask marked as static, and doesn't replay the motion captured in them.

That single decision solves the obvious objection to caching in a video model. If you cache a frame and reuse it, the scene freezes. Nothing moves, nothing changes, and you've built a panorama instead of a simulation. By storing motion separately from structure, the geometry stays pinned while the state keeps evolving.

The comparison table is the paper

The evaluation runs on MBench-A, which scores long-horizon memory on two axes. Environment measures whether the spatial structure of the scene stays stable. Causal measures whether the scene's state evolves sensibly.

4LC-Video scores 85.56 on environment and 89.06 on causal, the highest on both among the methods compared. That double win is unusual, and the reason is visible in what the other methods score.

HY-WorldPlay and Infinite-World hold spatial consistency well but score low on state evolution. HY-WorldPlay's state score is 49.50, which describes a scene that looks permanently frozen. It remembers the room and nothing happens in it.

Yume-1.5 and LingBot-World go the other way. Their state scores are 97.90 and 96.00, so things genuinely change. LingBot-World's reprojection score is 7.57, which means the structure drifts badly enough that returning to a viewpoint gives you a different room.

Every method in that table is optimising one axis at the expense of the other. 4LC's contribution is showing that the trade-off comes from how memory is organised, not from the task.

Training without multi-view data

The training setup is the part practitioners will care about. 4LC-Video used 200,000 monocular video clips drawn from SpatialVID, RealCamVid and OpenVid, with no paired multi-view supervision. Ordinary single-lens footage, in other words, not expensive multi-camera rigs.

The augmentation strategy is worth reading on its own. The researchers deliberately manufacture incomplete conditions so the model learns to work when information is missing. They randomly drop some source frames. They simulate a virtual camera panning, rotating and zooming to work out which points would be visible. They perturb the thresholds used for motion decisions. And ten percent of the time they build the conditioning from the first frame alone, simulating the extreme case where you have a single image and no video.

Real conditions are never clean. Reconstruction models have error, motion estimation is approximate, some angles were never observed. A model trained only on perfect conditioning has no idea what to do when handed a gap.

The detail I keep coming back to

The paper validates whether features are read correctly using relative representational similarity rather than absolute similarity. Comparing two feature vectors numerically is easy to mislead yourself with, because coordinate systems and normalisation differ, and two vectors describing the same content can look far apart.

A thick glass pane standing upright, a beam entering one edge and leaving the other as a cyan line.

Relative similarity asks a better question: did the relationship between this feature and the anchor features survive. That's closer to what "information was preserved" actually means.

There's a related observation in the paper that the authors don't develop. V-JEPA 2.1, a video self-supervised model, showed almost no consistency loss when read directly, and actually reduced rotation and translation error. The suggested reason is that it learned from video, so temporal coherence is baked into the representation itself, and that coherence survives being read, projected and propagated.

If that holds, features trained on the time axis are better suited to cross-time reuse than features trained on single frames. That's a hint about architecture choice that nobody has followed up on yet in this paper.

What it doesn't solve

The paper is explicit about the gap. 4LC supports caching both static and moving content, but 4LC-Video, in the long-horizon generation task specifically, only reuses the static portion of history.

Dynamic content, the part that actually moves, isn't reused across segments yet. The authors leave that to future work.

Which means the current system is very good at keeping a world stable and still relying on the generator to keep the moving parts coherent. For a flythrough of a fixed environment that's the right split. For a scene with people walking through it repeatedly, the hard half remains open.

Why the framing matters for anyone building with video models

The practical lesson is that memory organisation is a design surface, and it's cheaper to work on than the generator.

Anyone building long-form video or interactive simulation hits the same wall: the model can't afford a full history of key-value cache, and a compressed recurrent state forgets. 4LC's answer is to keep the states independently addressable and retrieve them by geometry, which is a retrieval problem wearing a generation costume.

The code and checkpoints for a system like this are usually the least interesting part. The idea that you separate what's static from what's moving, and then only replay the static part, is the transferable piece.

It also reframes the cost problem. A full key-value cache grows without bound, which is why long rollouts get expensive. A compressed recurrent state stays flat and forgets, which is why long rollouts drift. Caching structure separately from motion gives you a third option: a store that grows slowly, keeps what matters, and is retrievable by where the camera is looking rather than by how long ago it was seen.

That last point is the design choice a reader should steal. Retrieval keyed on geometry means the cost of finding a memory scales with the scene rather than with the length of the rollout. The same idea made vector indexes useful for text.

Where this lands in production is likely less dramatic than the paper implies. Video models that generate thirty-second clips don't hit the memory wall that a ten-minute interactive scene does. The methods that matter first are the ones building walkable environments, and those are also the ones with the least tolerance for a building changing shape between visits.

Related articles