2026-07-20

How we picked a background-removal model that actually runs in every browser (and why BiRefNet didn't)

NearBG removes the background from a photo entirely inside the visitor's browser — no upload, no server, same zero-network pledge as every other near tool. That constraint changes what "pick a good segmentation model" actually means. It's not a leaderboard exercise where the best-scoring model wins; the model also has to load and run to completion inside WebAssembly, on whatever device a real visitor happens to have, with no server-side fallback if it doesn't. "Does it actually run here" is the real bar, not "is it theoretically the best." Here's the model we tested, the two concrete, diagnosed ways the higher-quality candidate failed that bar, the tradeoff we accepted by choosing the other one, and what would change our mind later.

The two candidates

Both are real, licensed-for-commercial-use segmentation models with public ONNX exports — this wasn't a size-only or license-only filter, both had to actually run.

BiRefNet_lite is the newer, higher-accuracy model of the two — that's exactly why it was worth testing rather than defaulting to the smaller, older one. It lost anyway, for reasons that turned out to be about capability, not just speed.

Failure one: not slow, incapable — a hard-coded input shape

The first assumption going in was that BiRefNet_lite would simply be slower at its native 1024×1024 resolution, and that resizing the input down to something more WASM-friendly — 512, then 256 — would trade accuracy for speed the normal way. That assumption was wrong, and checking it directly is what caught the real problem: this specific ONNX export has no dynamic axes declared on its input. The graph's input shape is baked in at 1024×1024, full stop. Feeding it a 512×512 or 256×256 tensor doesn't run slower — it throws an explicit shape error before inference starts:

Got: 512 Expected: 1024

Same error, same shape, at 256. This isn't a performance ceiling that a smaller input could work around; it's a capability ceiling. The model literally cannot accept anything other than the one resolution it was exported for, which means every downstream perf question about it is moot until that resolution is the one being asked about.

Failure two: run it at the size it demands, and WASM runs out of memory

So the model was run the only way its export allows it to run — at the fixed 1024×1024 input, in the same WASM/CPU execution path the browser will actually use. That failed too, with a different, equally concrete error:

std::bad_alloc

That's WebAssembly's linear memory allocator refusing to grow further — a real ceiling tied to WASM's 32-bit addressing, not a tunable timeout or a "just wait longer" situation. This wasn't a surprise pulled from nowhere, either: pre-investigation research (an IMG.LY engineering writeup on shipping segmentation models in the browser) had already flagged this exact failure mode as a risk for large fixed-resolution models on WASM. Hitting it for real, with this exact model and this exact input size, turned that warning from a hypothetical into a measured fact.

WebGPU is the documented way around this — IMG.LY's own writeup reports roughly a 20x speedup running the same class of model through WebGPU instead of WASM, and offloading to the GPU sidesteps WASM's linear-memory ceiling entirely. It couldn't be tested here: the spike ran in Node against onnxruntime-web's WASM backend (the real runtime a browser uses, not a description of one, so its results carry real weight) because Node has no GPU backend to test WebGPU against. But even taking the 20x number at face value, building the tool's only working path on WebGPU means every visitor without WebGPU support — and browser/device support for it is real but not universal — gets a tool that simply doesn't work for them, with no fallback. That's a direct conflict with near's operating principle that a tool has to work for everyone, not just for whoever has the newest browser.

The verdict, and the tradeoff it costs

U2Netp was adopted; BiRefNet_lite, in this ONNX build, was not. U2Netp is the older, lighter of the two models, and that has a real quality cost: its edge fidelity on fine detail — hair being the clearest example — is a step below what BiRefNet_lite would produce if it could run at all. That tradeoff was made deliberately, not overlooked: "works everywhere, on plain WASM, with no WebGPU dependency" beat "best quality, but WebGPU-only" specifically because WebGPU coverage isn't universal yet, and shipping a tool that silently fails for a meaningful slice of visitors is worse than shipping a tool that produces a slightly softer edge for everyone.

The 2.9–3.1 second inference number is itself a conservative, worst-case reading, not a promise of typical performance — it was measured single-threaded, in a cold sandbox. A real browser tab with SIMD and multi-threading available is expected to be meaningfully faster, though that hasn't been re-verified on real low-end mobile hardware yet, which stays an open re-verification item rather than an assumed win.

What's not a permanent rejection

Ruling out this specific BiRefNet_lite ONNX export on plain WASM isn't the same as ruling out BiRefNet as a model family forever. The reconsideration conditions are concrete, not "maybe someday": a differently-exported ONNX build that declares dynamic input axes (removing failure one outright), WebGPU device/browser coverage becoming broad enough that near could accept it as a hard requirement without excluding a meaningful share of visitors, or enough engineering headroom to maintain a dual path — WebGPU for devices that support it, a lower-resolution WASM fallback for the rest — at the same time. None of those conditions held at the time of this spike; any of them changing is a reason to re-run this comparison, not a reason this decision is final.

What actually shipped

The pipeline that runs today is packages/near-bg-engine/segment.js: preprocess resizes the input to fit within U2Netp's fixed 320×320 (the model's own trained input size, per its config.json), pads, and normalizes with standard ImageNet mean/std before handing a [1,3,320,320] tensor to the model; postprocess takes the raw [1,1,320,320] saliency mask back out, crops off the padding, and composites it as an alpha channel over the original-resolution image via canvas destination-in. It runs inside apps/nearbg/src/segment.worker.js, which keeps the same near-kit worker message protocol every other app's worker uses, and — a constraint that mattered as much as the model choice itself — serves both the model weights and onnxruntime-web's own .wasm binary same-origin. near's CSP is connect-src 'self'; fetching either file from a CDN would either be blocked outright or require weakening that policy, and weakening it breaks the "zero external calls" promise every near tool makes, model choice aside.

The honest summary

The higher-quality model didn't lose on a benchmark table — it failed twice, for two different, diagnosed reasons, the moment it was actually run the way the browser would have to run it: a hard-coded input shape that turned "resize for speed" into an outright error, and a fixed-size memory footprint that WASM's own allocator couldn't satisfy. Both are real error strings from a real run, not inferred from documentation. What shipped instead is the model that measurably works everywhere the WASM path has to work, at a real, honestly-disclosed cost in edge quality — and a specific, written-down list of what would justify revisiting that choice, rather than a closed door.