← Back to blog
AiAbout 7 min read

A 4B Perception Model That Says No, and Mattes Without SAM

Published Nov 1, 2026
A 4B Perception Model That Says No, and Mattes Without SAM

There's a number in the WeVisOne-VL README that most people will skim past, and it's the only one I'd put on a slide: 96.4%.

That's the model's rejection rate on WeRef-Bench, a benchmark where the query sometimes refers to an object that isn't in the image. The baseline it's compared against rejects 73.0% of the time. Which means the baseline — a strong open vision-language model — draws a confident box around something that does not exist roughly one time in four.

For a demo, that's a curiosity. For a pipeline, it's a category of failure that makes the whole thing unusable. If your product crops a person out of a photo and hands it to a print vendor, a 27% chance of cropping the wrong thing, or nothing, is not a quality issue. It's a refund issue.

WeVisOne-VL is a 4B perception model from Tencent's WeChat Vision team, released October 22 with a technical report, weights at 2B and 4B, an inference package, cookbooks, and an evaluation pipeline. Apache-2.0 for Tencent-authored code. It does seven things: detection, referring localization, counting, pointing, OCR, referring segmentation, and promptable alpha matting.

The interesting part isn't the list. It's the architecture that makes the list one model instead of seven.

Coordinates as tokens

WeVisOne-VL is built on Qwen3-VL — the config confirms it: 36 text layers, hidden size 2560, a 24-layer ViT with patch size 16, a 152,680-token vocabulary. That last number is the tell. A standard Qwen3-VL vocabulary is smaller. The extra 1,000-odd tokens are coordinates: <0> through <1000>, representing normalized boxes and points.

So localization isn't a separate head bolted onto a detection model. It's next-token prediction. The model says "the person is at <312><184><498><702>" the way it would say "the person is on the left." Dense tasks also emit an instance token — <OBJ>, ID 152678 — after each predicted box, which the decoder uses to bind a specific object to a specific mask.

Then there's a 9.6M-parameter pixel decoder, shipped as mask_decoder.pt. It reads the emitted positions plus multi-level ViT features and produces masks and stride-1 alpha mattes. No SAM in the loop.

That's the design decision worth sitting with. The 2025-era answer to "I need a mask" was to chain a detector to SAM to a matting model — three models, three failure modes, three sets of weights, and a coordinate-handoff problem in the middle where boxes get rounded and masks get misaligned. WeVisOne replaces the chain with one decoder that receives the box as part of its query.

9.6M parameters is small enough to be unremarkable, which is the point. The expensive part of segmentation was never the decoder. It was the grounding.

A photo of a cluttered desk with thin precise bounding boxes drawn over the objects

The benchmarks, with the caveat attached

All of these are Tencent's own numbers from the README, against the strongest open baselines per benchmark:

LVIS detection F1: 55.3 versus 50.7 for LocateAnything-3B. Dense200 detection F1: 71.3 versus 58.7. RefCOCOg referring localization: 81.8 versus 77.6. PixMo-Count counting accuracy: 89.6 versus 82.8. TotalText OCR: 52.0 versus 43.3. SROIE OCR: 68.1 versus 40.3.

On matting, measured as MAD ×10³ where lower is better: 9.8 on WeRefMat, against 22.4 for a Qwen3.5-9B plus SAM2-Matting pipeline. On ReasonSeg, 71.2 gIoU against 65.0 for a fine-tuned LISA-13B.

A 4B model beating a 9B-plus-SAM chain on matting is the headline, and it's plausible for the structural reason above: the chain loses information at the handoff, and the unified model doesn't.

The caveat is the usual one. Vendor benchmarks measure vendor-chosen tasks. The rejection number is the one I'd actually trust as a differentiator, because rejecting is hard to game — you can't get to 96.4% by being generous with boxes.

Running it

Two paths, and they are not equivalent.

The inference package. Install from the GitHub repo, then:

from PIL import Image
from wevisone_vl import WeVisOneVL

model = WeVisOneVL("tencent/WeVisOne-VL-4B", device="auto", dtype="bfloat16")
image = Image.open("example.jpg").convert("RGB")
result = model.predict(image, task="matting", query="person", temperature=0.0)
print(result.response, len(result.masks))

That's the full path: masks, mattes, and the pixel decoder.

The quantized path. There's a Q4_K_M GGUF that runs on an 8GB GPU through llama.cpp, and a one-line Ollama pull. It handles the object-level tasks — boxes, points, counts, OCR — at a fraction of the memory.

What it cannot do is produce masks. The GGUF build drops the pixel decoder, so matting and segmentation are gone. This is stated openly in the community write-ups rather than buried, and it's the sort of thing that costs someone a weekend if they don't read first. If you need mattes, you need the full inference path and the VRAM that comes with it.

What it won't do

Single image, single turn. The pixel decoder rejects video grids, so no temporal tracking, no batch of frames treated as a sequence. The model also answers without chain-of-thought — deliberate, for latency — which means you can't ask it to reason its way to an answer it didn't produce directly.

And the matting quality has stated sensitivities: image resolution, severe occlusion, complex transparency, and prompt phrasing. That last one is the honest one. "Person" and "the person in the red jacket" will produce different alpha boundaries, and neither is wrong.

Why this matters beyond matting

The category of model this belongs to has been underbuilt. Everyone shipped chat-about-an-image models, because that's what the benchmarks rewarded and what the demos looked like. Perception — counting, pointing, locating, masking — is less fun to demo and far more useful in production, because it's the layer between a language model's intent and an actual pixel edit.

If you're building an image editing product, this is the missing component. The language model decides to remove the lamp from the photo. Something has to find the lamp, produce a soft matte of it including the shadow it casts, and hand that back. Until recently that "something" was a three-model chain with a 27% chance of inventing a lamp that isn't there.

Ninety-six percent isn't perfect. It's the first number in this space that's high enough to build on without a human in the loop for every call, and that's a different kind of milestone than a benchmark win.

The integration nobody writes about

There's a second-order effect that matters if you ship an editing tool. Once grounding and matting live in the same model, the loop gets shorter in a way that's hard to appreciate until you've built the long version.

Old loop: user asks to remove a lamp. Language model turns that into a query. Detector returns boxes. You pick one. You crop, run SAM, run a matting model, composite. Four network calls, four latency budgets, and a coordinate space that has to be reconciled at every hop. When the result is wrong, you don't know which hop broke.

New loop: one call, task equals matting, query equals lamp, back comes an alpha matte. If it's wrong, it's wrong for one reason. Debuggable systems are systems people keep using, and that's usually a bigger determinant of adoption than a two-point F1 gain.

The 2B sibling matters here too. If the 4B is what you validate with, the 2B is what you might actually deploy at the edge of a pipeline where you're processing a batch of product photos overnight. Tencent shipped both on day one, which is the behaviour of a team that expects people to put this in production rather than in a demo.

Related articles