Browser AI or server inference? Design the fallback before choosing

Key points

• Choose the inference location from user coverage, data flow, and failure behavior—not from a blanket claim that “local” or “cloud” is always better.
• Treat Chrome’s LanguageModel.availability() as a four-state capability check, then keep a server fallback for unsupported devices and browsers.
• Test the same acceptance cases through both paths; a hybrid design is only useful when users receive comparable outcomes.

Two execution paths, two different contracts

Chrome’s Prompt API puts a foundation model inside the browser. Thomas Steiner and Alexandra Klepper published the official page on May 20, 2025 and updated it on August 26, 2026. The page says the model is downloaded separately the first time an origin uses the API. After that download, later use does not require a network connection, and the documented on-device path sends no data to Google or another third party.

That privacy and offline story is attractive, but it comes with a device contract. Chrome currently documents the foundation-model APIs for Windows 10 or 11, macOS 13+, Linux, and Chromebook Plus—not Android, iOS, or ordinary Chromebooks. It asks for at least 22 GB of free space and either a GPU with strictly more than 4 GB of VRAM or a CPU setup with at least 16 GB of RAM and four cores. Audio input needs a GPU. These are vendor requirements, not our measurements.

A server path has the opposite shape. Cloudflare’s Workers AI binding documentation, updated April 21, 2026, shows a Worker calling a named model with env.AI.run(). The account supplies the binding, so application code does not carry an API token. The request still leaves the user’s device, however, and availability depends on the network and provider. Cloudflare’s limits page, checked October 13, 2026, currently lists 300 requests per minute for default text-generation models; models requiring the Workers Paid plan use separate per-account, per-model limits of 20 requests per minute with standard billing or 50 with prepaid AI Gateway credits. Limits can change, so production code must handle rejection instead of copying those numbers into assumptions.

The practical choice is therefore not “local versus cloud.” It is whether the feature can be optional. A private page summarizer can degrade gracefully when local AI is absent. A checkout, medical intake, or mandatory support workflow usually cannot exclude a large set of devices, so a controlled server path—or no generative feature—is safer.

A reusable capability router

Chrome warns that availability() must receive the same modality and language options used for prompting. A router should preserve that contract and expose the intermediate download states to the interface.

```js const requirements = { expectedInputs: [{ type: "text", languages: ["en", "ja"] }], expectedOutputs: [{ type: "text", languages: ["en"] }] };

export async function chooseInference({ allowServer, online }) { if ("LanguageModel" in globalThis) { const state = await LanguageModel.availability(requirements); if (state === "available") return { path: "local", state }; if (state === "downloadable" || state === "downloading") { return { path: "wait-for-user", state }; } } if (allowServer && online) return { path: "server", state: "fallback" }; return { path: "none", state: "unsupported" }; } ```

This example is a design template, not a benchmark or executed test. “wait-for-user” matters because Chrome requires meaningful user activation before starting a needed download. Do not silently start a large download on page load. On the server branch, add a timeout, an abort signal, bounded retries for transient failures, and a clear response for rate limiting. Never retry an unsafe action automatically.

Product analytics should record the selected path, capability state, latency bucket, and whether the user abandoned the task—but not raw private prompts by default. Compare completion rate by path. If local users receive a weaker result, the architecture has saved inference calls by shifting cost into inconsistency.

Common mistake: a boolean feature check

The common mistake is if (LanguageModel) useLocal(); else useServer();. Presence of an API name does not mean the model is ready, the requested languages are supported, or the device has enough resources. You can recognise this bug when eligible users see a frozen first interaction, unsupported users get an exception, or model downloads begin without an explanation.

Use all four availability states. Show download progress for downloading; ask for a deliberate click before downloadable; route unavailable to the fallback. Also watch session context usage: Chrome documents that older prompt-response pairs may be removed when the context fills, while an oversized request can raise QuotaExceededError.

When this comparison does not apply

Do not use client-side generative AI merely to avoid building deterministic code. Classification with a fixed rule, search with an index, or a calculator may be cheaper to test and easier to explain.

Client-first is also a poor default when every user needs identical behavior, the model version must be pinned, mobile support is essential, or compliance requires centralized logs and policy enforcement. Server-first is the wrong default when raw content must remain on the device, offline operation is a hard requirement, or the feature is optional and supported hardware is already common among the audience. For high-stakes medical, legal, financial, or safety decisions, neither path makes unreviewed generation reliable.

Practical exercise: test the routing contract

Suggested exercise: replace LanguageModel.availability() with a stub and run eight cases—the four Chrome states, each with the server reachable and unreachable. The checkable result is a table with exactly one selected path per case and no uncaught error. Then add two assertions: downloadable never calls the server before the user chooses, and unavailable + online + allowServer selects the server. Finally, run the same three harmless prompts through both real paths where available and compare whether each output meets one shared acceptance rule, such as valid JSON with two required keys. Record differences; do not claim equivalence from one attractive response.

Our verdict

We recommend a hybrid router with conditions. Use on-device inference for optional, privacy-sensitive assistance, but design the server path as a real product path rather than an emergency afterthought. If you cannot test one acceptance contract across both, choose server-only for consistency or remove the generative dependency. The location of inference is an implementation detail; predictable task completion is the product requirement.

Sources: Chrome for Developers, “The Prompt API,” published May 20, 2025 and updated August 26, 2026 ↗; Chrome for Developers, “Get started with built-in AI” ↗; Cloudflare, “Workers Bindings,” updated April 21, 2026 ↗; Cloudflare, “Limits,” checked October 13, 2026 ↗.