← Back to blog
GeneralAbout 6 min read

Kling's Technical Report Says the Bottleneck Is Not the Renderer

Published Nov 10, 2026
Kling's Technical Report Says the Bottleneck Is Not the Renderer

The Kling team published a technical report on November 3 that reads less like a model announcement and more like an argument about where the work actually is. Its title is blunt: "Behind Video Generation." Its claim is blunter. Generation quality, the authors write, is the projection of multimodal understanding through the entire chain of data, conditioning, alignment and evaluation.

Translated out of research dialect: drawing is no longer the hard part. Deciding what to draw, in enough detail, and then being able to tell why a shot failed — that is the hard part.

The report carries close to fifty authors and is currently trending on Hugging Face's daily papers list, which for a document with no weights attached is its own signal. Nothing is being released. No code, no checkpoints. What is being released is a description of the stack.

Frame, clip, shot, sequence

The paper opens with a four-year timeline. 2023 was aspirational. 2024 produced demonstrations that impressed and could not be relied on. 2025 got to shots that a production could actually use. 2026 is where systems generate coherent short sequences — multiple shots, a subject that persists, performance, camera language, dialogue, music, effects.

The unit of progress moved with it: frame, then clip, then shot, then sequence. This is not a cosmetic reframing. Sequence-level quality is compositional. Constraints interact, errors accumulate, and a system has to infer a latent creative intent from messy inputs and then express that intent as conditions a generator can execute. A model that has only ever been judged on single frames has no way to learn that.

A color grading suite with a large reference monitor and a timeline below it

One backbone, many specialists

Underneath sits Kling-MLLM: an autoregressive multimodal model that takes images, video, audio, synchronized audio-video and interleaved input. Three sizes — a dense 27B, an MoE at 35B-A3B, an MoE at 397B-A17B — with language and vision backbones initialized from Qwen3.5 through Qwen3.8 checkpoints.

The audio tower changes with scale. Smaller variants follow the Qwen3-Omni design and encode 16 kHz mono at 13 tokens per second. Larger variants fuse Mel and waveform-VAE features in a dual-branch tower at 25 tokens per second. Training runs a four-stage curriculum: acoustic initialization, joint multimodal training, temporal-capacity expansion, long-context consolidation.

Then the base gets post-trained into specialists — captioner, prompt enhancer, text encoder, reward model, evaluator. The point of sharing one backbone is that visual, temporal and acoustic knowledge is learned once and reused, while each downstream task picks its own trade-off between accuracy and latency.

One detail worth stealing even if you never touch video: averaging the weights of the first and last curriculum stages beat either checkpoint alone. That is a cheap trick with an unusually clean result.

The caption is a contract

The middle of the report is about captions, and it is the part I would hand to anyone who writes prompts for a living.

A diffusion transformer sees far fewer training samples than a language model, so every sample has to carry as much information as possible. That information arrives as a caption written by another model. The authors therefore treat a caption as a semantic contract, and they specify six information families it has to cover: subject and action; space and composition; cinematography and light; time and shots; visible and audible text; aesthetics.

Each family comes with a named failure. Skip the cinematography fields and you get what the paper calls generic camera behavior — appealing but irreproducible style. Skip subject-action detail and the generator produces static posing, role swaps, unresolved actions. That list is the most useful thing in the document, because it converts "my prompt isn't working" into a diagnosis.

They extend the contract to reference-grounded captioning with three properties: scoped binding, residual sufficiency, authority consistency. And they describe a data-scaling move that is almost cheeky — any two videos with enough visual or acoustic similarity can be pressed into service as reference and target, with the caption describing what they share and what changes. The pair does not have to come from an actual edit.

Prompts people actually write

Real users do not write captions. They write three lines and hit enter. So the stack includes prompt enhancement in two layers: a basic version that translates intent into the caption distribution, and a final stage that runs reinforcement learning with the frozen text encoder and DiT in the loop, updating on video-level rewards for visual quality, motion quality and text alignment. There is also an agentic layer that runs before all of it, refining the story and retrieving examples, because a lot of production prompts are themselves AI-written — long, fluent, internally inconsistent.

The text encoder section is the one I did not expect. The claim is that a good conditioning encoder comes from perceptual training, not from stronger language ability; that general QA benchmarks measure autoregressive decoding, not the hidden states a DiT actually consumes; and that candidate encoders can be screened from text alone before anyone pays for end-to-end training. Adapting the backbone and stabilizing the mixing weights raised a human preference ratio from 1.10 to 1.27 against a Qwen baseline.

Rewards split in two. A general reward model built on the 27B writes explainable critiques with defect tags, temporal intervals, per-frame boxes and rationales. A decision reward model scores many dimensions in a single prefill. Together they serve five jobs: DiT RL, prompt-enhancement RL, machine evaluation, checkpoint selection, data filtering.

The numbers, and the caveat

Reported scores for the largest model include 72.48 on LongVideoBench, 77.88 on MVBench, 89.52 on MMBench-CN and a word error rate of 1.18 on LibriSpeech clean. Kling-Captioner scores 21.84 accuracy and 94.81 coverage on VidCapBench with the 35B-A3B, against 20.23 and 94.15 for Doubao2.1-Lite in thinking mode. The decision reward model reaches 77.86 macro accuracy on the team's internal pairwise test, against 73.54 for Gemini-4-Argon.

Two caveats. These are the team's own numbers, and at least one close reading has argued the paper overstates its reward model's public benchmark wins. The comparison against Doubao is also mixed — on several other benchmarks the authors say a competing model leads. That kind of admission is more credible than a clean sweep, but it does not make the headline numbers independent.

The deeper point stands regardless. If quality is compositional, then a single aggregate score cannot diagnose anything. The same bad clip might come from a caption that omitted supervision, conditioning that threw away temporal structure, a reward model that prefers smooth but static motion, or an evaluator that never tested the event the prompt asked for. The paper's recommendation is decomposed, task-specific evaluation. Anyone who has shipped a video feature knows how often "the model is bad" turns out to be "the test was blind."

Why it matters outside Kling

For the last year the visible competition has been about resolution, duration and price per second. Those are real, and they are also the easiest things to compare in a table. This report argues that the table is measuring the wrong layer, and it does so with enough internal detail that competitors can check the reasoning.

If the authors are right, the next round of wins will not come from a better renderer. They will come from whoever builds the cleanest pipeline from "what the client meant" to "what the generator executes" — and from whoever can tell, quickly and specifically, which link in that chain broke.

Related articles