← Back to blog
AiAbout 6 min read

They Rebuilt a Proprietary Vision Model With Public Data and Beat It on One Benchmark

Published Nov 14, 2026
They Rebuilt a Proprietary Vision Model With Public Data and Beat It on One Benchmark

A research group has reproduced a proprietary vision model using only public models and public data, and on at least one benchmark it comes out ahead.

The score is 73.9 mIoU zero-shot on Cityscapes for a 9B model, against 69.9 reported for Vision Banana. That comparison is the headline, and the details underneath it are more useful than the comparison.

What was built

Open Vision Banana comes from the Fundamental AI Lab at the University of Technology Nuremberg, with collaborators at DLR, AIST, Tübingen, TUM and valeo.ai. The approach was to fine-tune an existing open-weight generator, FLUX.2-Klein, at both 4B and 9B scale, on somewhere between 2.1 and 3.8 million training pairs assembled from public datasets including COCO-Stuff, ADE20K, BDD100K and NYUv2.

The framing matters. This is a generator being repurposed for perception: depth, segmentation, referring and reasoning tasks, while retaining the image-editing behaviour it started with. The base model's ability to generate survives the fine-tune, which is why the results are interesting beyond the segmentation score.

The 73.9 needs a footnote

The zero-shot Cityscapes number depends on a technique the authors introduce and name legend ensembling.

A single forward pass gets 65.4 mIoU. The gap to 73.9 comes from running the model several times with different colour-to-class legends and fusing the outputs by a per-pixel majority vote. That adds 8.7 points on Cityscapes. The same method adds 15.0 points on COCO-Object.

So the honest reading of the headline is that the model needs multiple passes to reach it, and the method's benefit scales with how much the task benefits from voting. 65.4 is the number you'd get in a latency-sensitive pipeline where one pass is the budget.

That's not a criticism of the paper. The authors report both numbers, which is more than a lot of benchmark tables do. It's a reminder about how to read a mIoU claim that arrives without a cost column.

Where the gap widens

The segmentation win doesn't carry across the board, and the paper says so.

On referring and reasoning segmentation, the non-zero-shot variants come within five points of Vision Banana. On instance segmentation it trails noticeably: 37.4 cgF1 against 47.5 on SA-Co/Gold.

On depth estimation, the results are described as rivaling state-of-the-art affine-invariant depth estimators, which is a careful way of claiming parity in a crowded field.

So the picture is a model that matches or beats the proprietary baseline on the task the authors optimised for, comes close on two neighbouring tasks, and falls behind on instance-level separation. That's what a good reproduction looks like. A reproduction that wins everywhere would be suspicious.

The release is the real contribution

Academic reproductions usually ship a paper and nothing else. This one ships the data pipeline, the training and evaluation code, and both final and intermediate checkpoints.

That combination is what makes a result checkable. With intermediate checkpoints, you can see whether the model learned depth before segmentation or the other way round, and you can fine-tune from an earlier point instead of paying for the full run.

The compute cost is stated too, which is rare. Roughly 73 hours on 16 GH200 GPUs for the 4B model, and about 79 hours on 32 GH200s for the 9B.

Those numbers define who can replicate it. Sixteen GH200s is a university cluster or a well-funded lab, not a laptop. Thirty-two is a serious allocation. The recipe being public doesn't make it cheap, and the 4B variant mattering almost as much as the 9B is the detail that makes it reachable: two GH200 nodes for three days is a grant application, not a data centre.

Why reproducing a proprietary model matters

Vision Banana's original results were interesting precisely because nobody outside the company could check them. A reproduction that lands within a few points on most tasks and ahead on one changes the status of those results from "a claim" to "a target".

There's a secondary effect that tends to get ignored. When a closed model's evaluation is reproduced on open weights, the comparison becomes a comparison of training data and method rather than of scale. The 9B model matching a proprietary system suggests the proprietary advantage sat in the data mixture and the task formulation, which is exactly the kind of thing that's replicable.

The dataset construction is where most of the effort sits. COCO-Stuff, ADE20K, BDD100K and NYUv2 are all public, all annotated for different purposes, and none of them was built to train a unified model that handles depth, segmentation and editing at once. Assembling 2.1 to 3.8 million pairs from them means reconciling label spaces that don't agree, which is unglamorous work and probably where most of the effort went.

If the recipe is portable, the next question is whether the same trick works for other proprietary perception models. The pattern is general: take a strong open generator, fine-tune it on a large assembled supervision set, evaluate across several tasks so the gaps are visible. That pattern requires no novel architecture, which is what makes it threatening to anyone whose moat was an unpublished training mixture.

And a third effect: the base model is a generator. If the same weights handle depth, segmentation and editing, then a production pipeline can drop three specialised models and one editing model for a single checkpoint, at the cost of some accuracy on the hardest instance-level work. Measure that trade on your own data rather than on Cityscapes.

What to take from it

The transferable lesson isn't the 73.9. It's the structure of the claim: reproduce, report the per-task gap honestly, ship the intermediates, state the compute.

Anyone evaluating a "beats the proprietary model" headline can apply the same three questions. Does the number include a test-time trick, and what does it cost. Does the win hold across the tasks the model wasn't tuned for. Can someone with a smaller cluster get within a few points using the published recipe.

Open Vision Banana answers all three in the text, which is why the paper earns a read past the abstract.

There's one question it leaves open, and it's the one a buyer would ask. The paper shows that public data and a public base model can reproduce a proprietary system's benchmark numbers. It doesn't show that the reproduction is useful on the messy inputs a product actually receives: badly lit photos, screenshots of screenshots, images with text baked in, crops smaller than the training distribution. Benchmarks like Cityscapes are tidy by construction.

That gap is where the next round of comparisons will happen, and it's where an open checkpoint has an advantage over a closed one. Anyone can test it on their own data tomorrow.

Related articles