← Back to blog
AiAbout 6 min read

Video Models That Predict the Chunk After Next

Published Nov 13, 2026
Video Models That Predict the Chunk After Next

Autoregressive video models are trained the way language models are: show the model a prefix, ask it to predict the next token or the next chunk, compare, repeat. The method works. It also has a specific pathology that a paper from Tencent ARC, Nanjing University and Zhejiang University names directly.

Temporal redundancy encourages local shortcuts.

Translated: when consecutive frames are nearly identical, a model can score well by copying the frame it just saw and nudging it. That is a legal move under the training objective. It is also the opposite of learning dynamics, because dynamics is exactly the part where the next frame is not a copy.

The fix is a second prediction

The paper introduces Future Chunk Prediction, or FCP. During training, auxiliary branches take intermediate features from the current chunk and try to predict a temporally distant chunk, several steps ahead in the sequence.

The important detail is where those branches read from. Not the final output, which already has to be right. The middle of the network, at the point where the model is still forming its internal representation of the current clip. If a feature at block 10 is going to be used to predict what happens three chunks later, that feature has to encode something about motion, contact and consequence rather than surface texture.

The supervision is video pixels only. No optical flow, no action labels, no depth, no physical simulator in the loop. The auxiliary branches predict image content and are scored on image content, and everything else is left for the model to figure out as a side effect.

That is a meaningful constraint. A lot of physics-aware video work imports a signal from somewhere else, which makes the result harder to reproduce and harder to apply to in-the-wild footage. FCP stays inside the modality.

What the authors claim

Two outcomes, stated plainly.

Physical consistency improves in generated videos. Objects stop teleporting, contacts resolve instead of sliding, and the model's guesses about what happens next follow from the current state rather than from a prior about what usually appears next to what.

The internal representations become more semantically structured. That second claim is the more interesting one for anyone outside video generation, because it suggests the auxiliary loss does something to the model's understanding, and to more than its output. A representation that is forced to encode "what happens next" is a representation that has to encode objects, agents and consequences.

The claim is measured how, exactly, is the question to ask before believing it. The paper reports results on its own evaluations, and its weights ship for two model sizes, 1.3B and 5B, as text-and-image-to-video models. There is a demo dataset on Hugging Face and a project page with samples. What is not in the repository is an independent third-party comparison, which is the normal state of affairs for a two-week-old paper and worth remembering when the next summary of it says "solves physics."

Why teacher forcing is the real target

The deeper argument is about a training signal that everyone uses and nobody loves. Teacher forcing feeds the model ground-truth context at every step. It makes training stable and parallelizable. It also means the model never practices recovering from its own mistakes, and never has to maintain a coherent internal state across a long horizon, because the ground truth keeps resetting the state for it.

Combine that with redundant frames and you get a model that is very good at the next 200 milliseconds and untethered at three seconds. Anyone who has generated a long clip has seen the failure: a subject that holds together for two seconds and then becomes a different subject with the same color palette.

FCP attacks this from the representation side rather than the rollout side. Instead of making the model practice longer rollouts, which is expensive and unstable, it makes the middle of the network prove it knows where things are going. That is cheaper and it composes with whatever rollout scheme you already have.

A row of standing dominoes on a polished surface with the first four fallen.

What it would take to prove

The claim that matters is not "better videos." It is that a model trained with FCP has a better world model, meaning its internal state tracks objects and consequences rather than appearances.

That claim is testable in ways the paper does not attempt, and the tests are worth naming because they are cheap. Freeze the model after training with and without the auxiliary loss, then probe the intermediate features for object permanence: does the representation of a scene where a ball rolls behind a chair still contain the ball? Ask for continuations that violate physics in a specific way, a dropped glass that does not break, and measure whether the FCP model resists the temptation longer. Run both on the same seeds and the same prompts.

None of that needs a new benchmark suite. It needs someone with the weights, which are published, and a weekend.

What it takes to run

The repository is straightforward for a research release. Linux, Python 3.12, PyTorch 2.10 with CUDA 12.8, FFmpeg 7.1 if you go the conda route. The recommended install is uv sync --python 3.12 after installing FFmpeg, or a conda environment followed by a pip install of torch and the requirements file. Nothing exotic, no custom CUDA kernels that need to compile against a specific driver, which is more than can be said for a lot of video work.

Two model sizes, 1.3B and 5B, both autoregressive text-and-image-to-video. The team says more sizes and variants are coming. The demo dataset is published alongside the weights.

The honest reading

This is a training-method paper with an unusually clean idea, and clean ideas in video generation tend to get adopted faster than architectures because they do not require retraining a team.

The open question is scale. An auxiliary loss that helps at 5B may wash out at 50B, or it may become essential, and nobody knows which until someone runs it. The other open question is whether "more semantically structured representations" survives contact with a benchmark that was not designed by the authors.

For a practitioner, the practical takeaway is smaller and more portable: if your model is good at the next step and bad at the next three seconds, the problem may not be capacity. It may be that your loss never asked the middle of the network a question about the future. FCP is one way to ask it, and it costs a second set of prediction heads rather than another ten thousand GPU hours.

The repository is at 274 stars, which is what a paper with no product attached looks like. The idea is likely to outlive the star count.

Related articles