Web & Cross-Platform

This content was generated with the assistance of AI and should be verified against the official documentation of each tool and framework before being relied on in production.

This section’s bibliography lists the reference material consulted while preparing these pages.

This page was written on 2026-09-30 against Transformers.js (@huggingface/transformers) 4.3.0, onnxruntime-web 1.30.0, kokoro-js 1.2.1, react-native-executorch 0.10.4, flutter_gemma 1.11.3, llama_cpp_dart 0.2.2 and the onnxruntime Flutter plugin 1.4.1 (last published in March 2024), as checked that day. llama.rn was not pinned: the latest dist-tag on npm pointed to a 0.13.0 release candidate on that date (the newest stable release was 0.12.9), so check its releases page before choosing a version. Web and cross-platform runtimes are where one code base has to run on very different hardware, so the page is organised around three questions: which runtime executes the model (WebGPU, WASM, ExecuTorch, llama.cpp, ONNX Runtime), which format it needs (see Model Formats), and how the weights reach the device (a browser cache, an app download). It continues the DocsAssistant goes offline scenario of this sub-section — transcribe a spoken question, answer it with a small local model, speak the answer and classify a screenshot — in JavaScript, then shows short React Native and Flutter variants. Producing ONNX files is in Conversion & Quantization; the model catalogues are Speech Models: STT & TTS, Vision Models and Small LLMs & Embeddings; the pipeline() concept itself is in Transformers Pipelines. The same scenario on servers is in Python Backends and Java & Spring Boot; the Apple, Android and Windows variants are in Apple Platforms, Android and Windows & .NET. No benchmark numbers or prices are quoted.

Choosing a runtime

The Hub stores most models as PyTorch safetensors; a browser or phone cannot run those directly. Each platform therefore relies on a converted artefact and a runtime that can execute it.

flowchart LR Hub[("Hugging Face Hub
onnx-community, GGUF, .pte")] --> W["Browser
Transformers.js in a Web Worker"] Hub --> RN["React Native
llama.rn, react-native-executorch"] Hub --> FL["Flutter
flutter_gemma, llama_cpp_dart, onnxruntime"] W -->|"device: webgpu, else wasm"| ORT["ONNX Runtime Web"] RN --> ET["ExecuTorch .pte"] RN --> LC["llama.cpp GGUF"] FL --> LR["LiteRT-LM .litertlm"] FL --> LC ORT --> Cache[("Cache API
browser storage")] ET --> FS[("App file system")] LC --> FS LR --> FS

Figure 1. DocsAssistant on web and cross-platform runtimes

Table 1. Which option fits which DocsAssistant task
Task	Browser (JavaScript)	React Native	Flutter
Transcribe a spoken question	Transformers.js `automatic-speech-recognition` with a Whisper or Moonshine ONNX model	`react-native-executorch` `useSpeechToText`	The opt-in `flutter_gemma_speech` package of `flutter_gemma` (its README lists Moonshine, Whisper and Parakeet; a 0.x package) or `onnxruntime`
Answer it	Transformers.js `text-generation` with a small ONNX LLM	`llama.rn` (GGUF) or `react-native-executorch` `useLLMChatSession`	`flutter_gemma` (`.litertlm`, `.task`) or `llama_cpp_dart` (GGUF)
Speak the answer	`kokoro-js` (Kokoro ONNX, WebGPU or WASM)	A native TTS module or ExecuTorch, depending on the model	`flutter_gemma_speech` text-to-speech (its README lists Matcha, Qwen3-TTS and Inflect-Nano-v2), a platform TTS plugin, or an ONNX model through `onnxruntime`
Classify or describe a screenshot	`zero-shot-image-classification` (SigLIP 2) or a small vision-language model	An ExecuTorch vision model or `llama.rn` with a multimodal GGUF	`flutter_gemma` with a multimodal Gemma model, or `onnxruntime`

The table is a starting point, not a guarantee: each cell depends on a model that a package maintainer has exported and tested, so check the package’s supported-model list before designing around it.

Transformers.js in the browser

Transformers.js is the JavaScript counterpart of the Python transformers library: the same task names, run by ONNX Runtime Web in the browser (and ONNX Runtime for Node.js). It is published as @huggingface/transformers; version 4.x is current. Models come from the Hub as ONNX files, mostly under the onnx-community organisation, and are downloaded on first use.

Terminal: install Transformers.js

npm install @huggingface/transformers

JavaScript: a pipeline on WebGPU with 4-bit weights

import { pipeline } from '@huggingface/transformers';

const pipe = await pipeline('sentiment-analysis', 'Xenova/distilbert-base-uncased-finetuned-sst-2-english', {
  device: 'webgpu',
  dtype: 'q4',
});
const result = await pipe('I love running models offline!');

Official reference: Transformers.js documentation.

WebGPU versus WASM

The device option picks the ONNX Runtime Web execution provider. 'webgpu' runs the graph on the GPU and is what makes an LLM or a large speech model usable; the documentation still calls WebGPU experimental in some browsers, so a real app must detect support and fall back. The default backend is WASM on the CPU, which runs everywhere but is slow for large models.

JavaScript: choosing the device by feature detection

const hasWebGPU = 'gpu' in navigator && (await navigator.gpu.requestAdapter()) !== null;

const options = hasWebGPU
  ? { device: 'webgpu', dtype: 'q4f16' }  // needs shader-f16 support on the adapter
  : { device: 'wasm', dtype: 'q8' };      // the WASM default

The q4f16 choice above assumes the GPU adapter supports 16-bit floats; if it does not, use q4 or fp32. The pipeline options are documented in the Transformers.js pipelines reference; the browser API is specified by WebGPU (W3C).

dtype options

Every ONNX repository published for Transformers.js ships several precisions in its onnx/ folder (for example model.onnx, model_fp16.onnx, model_q4.onnx, model_q4f16.onnx, model_quantized.onnx). The dtype option selects which file is fetched.

Table 2. dtype values documented for Transformers.js
dtype	Meaning
`fp32`	Full precision; the default on WebGPU.
`fp16`	Half precision; halves the download of `fp32` but needs GPU support for 16-bit floats.
`q8`	8-bit quantisation; the default on WASM.
`q4`	4-bit quantisation; the smallest download of the documented values, with some quality loss.
`q4f16`	4-bit weights with 16-bit activations; used in the `onnx-community` LLM model cards for WebGPU. Only files that exist in the repository can be selected.

Because download size is what the user waits for, estimate it before picking a model. With parameters stored at bits:

A 0.6-billion-parameter model is therefore roughly 2.4 GB at 32 bits and roughly 0.3 GB at 4 bits, before tokenizer files, embeddings that stay in higher precision and runtime memory (activations and the KV cache of a language model, see Hardware Sizing). The repository’s file list is the authority: add up the files that your dtype selects. Quantisation itself is covered in Quantization for Inference and, for producing the files, in Conversion & Quantization.

Caching and offline use

By default, Transformers.js stores downloaded model files with the browser’s Cache API (the env.useBrowserCache setting, enabled when available), and ONNX Runtime’s WASM binaries are cached too (env.useWasmCache). The first load is therefore a network download and later loads come from the cache, which is what makes "goes offline" possible after one online visit. The env object also lets you serve models from your own origin, which avoids the Hub entirely.

JavaScript: serve models from your own server only

import { env } from '@huggingface/transformers';

env.allowRemoteModels = false;        // never contact the Hub
env.allowLocalModels = true;          // look under env.localModelPath
env.localModelPath = '/models/';      // a path on your own origin

Browser storage can be evicted and quotas differ per browser, so a production app should ask the browser for persistent storage and show download progress. Official reference: Transformers.js — env and the Cache API (MDN).

Choosing and verifying models

Model identifiers must exist on the Hub and their licences must fit the product. The onnx-community repositories are conversions of a base model; their cards mostly carry no licence field of their own, so read the licence of the base model.

Table 3. Models used in this page (verified on the Hub on 2026-09-30)
Model	Used for	Licence (base model)	Notes
onnx-community/whisper-tiny.en	Transcribe (English)	Apache-2.0 (openai/whisper-tiny.en)	Not gated. `q4` encoder and decoder files exist. Whisper sizes and Moonshine are compared in Speech Models: STT & TTS.
onnx-community/moonshine-base-ONNX	Transcribe (alternative)	MIT (per its card)	Not gated; designed for short live audio.
onnx-community/Qwen3-0.6B-ONNX	Answer	Apache-2.0 (Qwen/Qwen3-0.6B)	Not gated. `q4` and `q4f16` files exist. See Small LLMs & Embeddings for other sizes and families.
onnx-community/Kokoro-82M-v1.0-ONNX	Speak	Apache-2.0	Not gated. Voices are separate `voices/*.bin` files.
onnx-community/siglip2-base-patch16-224-ONNX	Classify a screenshot	Apache-2.0 (google/siglip2-base-patch16-224)	Not gated. The card has no usage snippet, so use the generic pipeline form and test it.
HuggingFaceTB/SmolVLM-256M-Instruct	Describe a screenshot (alternative)	Apache-2.0	Not gated; the official repository ships an `onnx/` folder including `q4` decoder files. Its card documents lower-level model classes rather than a one-line pipeline.

Check any model id with the Hub API before you ship it, for example curl https://huggingface.co/api/models/onnx-community/Qwen3-0.6B-ONNX; the response lists the files, tags and the gated flag. Choosing models in general is covered in Choosing Models & Licences.

DocsAssistant goes offline in JavaScript

Model loading and inference block the thread that runs them for seconds, so the four tasks run in a Web Worker: the page posts a message, the worker loads the pipeline once, runs it and posts the result back. The worker below loads each pipeline lazily and keeps it in a module-level cache, so a second question reuses the loaded models.

worker.js: lazy, cached pipelines in a Web Worker

import { pipeline } from '@huggingface/transformers';

const device = 'gpu' in navigator ? 'webgpu' : 'wasm';
const loaded = new Map();

function get(task, model, options = {}) {
  const key = `${task}:${model}`;
  if (!loaded.has(key)) {
    loaded.set(key, pipeline(task, model, {
      device,
      progress_callback: (p) => self.postMessage({ type: 'progress', task, p }),
      ...options,
    }));
  }
  return loaded.get(key);
}

self.onmessage = async ({ data }) => {
  const { id, step, payload } = data;
  try {
    self.postMessage({ id, result: await steps[step](payload) });
  } catch (error) {
    self.postMessage({ id, error: String(error) });
  }
};

const steps = {};  // filled in below

Official reference: Transformers.js — pipeline API and Web Workers API (MDN).

Transcribe

The automatic-speech-recognition pipeline takes mono audio at the model’s sampling rate (16 kHz for Whisper), either a URL or a Float32Array. The microphone recording is decoded and resampled in the page, then transferred to the worker.

worker.js: transcribe a spoken question

steps.transcribe = async ({ audio }) => {   // Float32Array, 16 kHz, mono
  const asr = await get('automatic-speech-recognition', 'onnx-community/whisper-tiny.en', {
    dtype: 'q4',
  });
  const { text } = await asr(audio);
  return text;
};

Official reference: AutomaticSpeechRecognitionPipeline. Swapping in Moonshine (onnx-community/moonshine-base-ONNX) only changes the model id.

Answer

The text-generation pipeline accepts a chat message list. A TextStreamer forwards tokens as they are produced, so the page can render the answer progressively. The dtype and device below follow the model card.

worker.js: answer with a small LLM and stream tokens

import { TextStreamer } from '@huggingface/transformers';

steps.answer = async ({ question, context }) => {
  const generator = await get('text-generation', 'onnx-community/Qwen3-0.6B-ONNX', {
    dtype: 'q4f16',
  });
  const messages = [
    { role: 'system', content: 'Answer only from the provided documentation excerpts.' },
    { role: 'user', content: `Excerpts:\n${context}\n\nQuestion: ${question}` },
  ];
  const output = await generator(messages, {
    max_new_tokens: 256,
    do_sample: false,
    streamer: new TextStreamer(generator.tokenizer, {
      skip_prompt: true,
      skip_special_tokens: true,
      callback_function: (token) => self.postMessage({ type: 'token', token }),
    }),
  });
  return output[0].generated_text.at(-1).content;
};

Official reference: TextGenerationPipeline and the model card onnx-community/Qwen3-0.6B-ONNX. The context excerpts come from the retrieval step described in Generating Embeddings in Practice. Qwen3 models can emit a reasoning block before the answer; strip it or disable thinking mode as the model card describes.

Speak

kokoro-js wraps the Kokoro ONNX model and its voice packs for the browser (WebGPU or WASM) and Node.js. It is a separate package that uses Transformers.js internally.

Terminal: install kokoro-js

npm install kokoro-js

worker.js: speak the answer with Kokoro

import { KokoroTTS } from 'kokoro-js';

let tts;

steps.speak = async ({ text }) => {
  tts ??= await KokoroTTS.from_pretrained('onnx-community/Kokoro-82M-v1.0-ONNX', {
    dtype: 'q8',      // documented options: fp32, fp16, q8, q4, q4f16
    device: 'wasm',   // "wasm" or "webgpu" in the browser; the README advises fp32 with webgpu
  });
  const audio = await tts.generate(text, { voice: 'af_heart' });
  return { samples: audio.audio, sampleRate: audio.sampling_rate };
};

Official reference: kokoro-js README. The page plays the returned samples with the Web Audio API (AudioContext and an AudioBuffer). For long answers tts.stream() with a TextSplitterStream produces audio sentence by sentence. Kokoro and Piper are compared in Speech Models: STT & TTS.

Classify a screenshot

SigLIP 2 scores an image against labels supplied at call time, so the set of screen types can change without retraining.

worker.js: zero-shot classification of a screenshot

steps.classify = async ({ imageUrl }) => {
  const classifier = await get(
    'zero-shot-image-classification',
    'onnx-community/siglip2-base-patch16-224-ONNX',
    { dtype: 'q4' },
  );
  return classifier(imageUrl, ['a settings page', 'an error dialog', 'a code editor', 'a login form']);
};

Official reference: ZeroShotImageClassificationPipeline. The model card ships no snippet, so confirm that your installed version loads this repository and that the q4 files exist, and fall back to fp32 if not. To describe a screenshot in free text instead, use a small vision-language model such as SmolVLM, loaded with the model classes shown on the SmolVLM-256M-Instruct card; see Vision Models for the catalogue.

Calling the worker from the page

main.js: one promise-based call per step

const worker = new Worker(new URL('./worker.js', import.meta.url), { type: 'module' });
const pending = new Map();
let next = 0;

worker.onmessage = ({ data }) => {
  if (data.type) return;                       // progress or token events: update the UI
  const { resolve, reject } = pending.get(data.id);
  pending.delete(data.id);
  data.error ? reject(new Error(data.error)) : resolve(data.result);
};

export function run(step, payload) {
  return new Promise((resolve, reject) => {
    const id = next++;
    pending.set(id, { resolve, reject });
    worker.postMessage({ id, step, payload });
  });
}

const text = await run('transcribe', { audio });
const answer = await run('answer', { question: text, context });
const speech = await run('speak', { text: answer });

Official reference: Worker.postMessage (MDN).

ONNX Runtime Web

ONNX Runtime Web (onnxruntime-web, 1.30.0) is the engine under Transformers.js. Use it directly when you have a single ONNX graph, such as a classifier you exported yourself, and do not need tokenizers or pipelines. Its execution providers are WebGPU, WebNN, WebGL (limited operator support) and WASM, which supports every ONNX operator. The WebGPU provider is reached through a dedicated bundle import.

JavaScript: an InferenceSession on WebGPU with a WASM fallback

import * as ort from 'onnxruntime-web/webgpu';

const session = await ort.InferenceSession.create('/models/classifier.onnx', {
  executionProviders: ['webgpu', 'wasm'],   // tried in order
});

const input = new ort.Tensor('float32', new Float32Array(1 * 3 * 224 * 224), [1, 3, 224, 224]);
const outputs = await session.run({ [session.inputNames[0]]: input });
const scores = outputs[session.outputNames[0]].data;

Official reference: ONNX Runtime Web — Using WebGPU and ONNX Runtime JavaScript API. The list of execution providers with a fallback order is an ONNX Runtime feature that also applies on servers; see the ONNX Runtime section of Python Backends. Preprocessing (resizing, normalisation) is your responsibility and must match the model card exactly.

ExecuTorch across platforms

ExecuTorch is PyTorch’s on-device runtime for iOS, Android, desktop and embedded targets. A model is exported once from PyTorch with torch.export, lowered to a hardware backend and saved as a .pte file; the same file format is loaded by the iOS, Android and C++ runtimes, with backend choice (for example XNNPACK for CPU) selected at export time. This is the route taken by react-native-executorch below. Android specifics are in Android; the format’s place among others is in Model Formats.

Python: export a model to a .pte file with the XNNPACK backend

import torch
from executorch.backends.xnnpack.partition.xnnpack_partitioner import XnnpackPartitioner
from executorch.exir import to_edge_transform_and_lower

model = MyModel().eval()                       # any torch.nn.Module
example_inputs = (torch.randn(1, 3, 224, 224),)

exported = torch.export.export(model, example_inputs)
program = to_edge_transform_and_lower(exported, partitioner=[XnnpackPartitioner()]).to_executorch()

with open("model.pte", "wb") as f:
    f.write(program.buffer)

Official reference: ExecuTorch — Getting Started. Verify the import paths against the release you install, since the export API has changed between minor versions. Only models whose operators export cleanly can be converted; for large language models, prefer a ready-made .pte from the maintainers' Hub collections over exporting your own.

React Native

The JavaScript bundle of a React Native app cannot run these models; the work is done by native libraries exposed through hooks or native modules (see Native Modules and Components and React Native). Two libraries cover most DocsAssistant needs.

llama.rn

llama.rn binds llama.cpp, so it runs GGUF models (see llama.cpp). Install it with npm install llama.rn. The model file must already be on the device’s file system; download it with a file-system library, following the patterns in Storage and Offline.

TypeScript: load a GGUF model and stream a completion

import { initLlama } from 'llama.rn';

const context = await initLlama({
  model: modelPath,        // a file:// path to a .gguf file on the device
  use_mlock: true,
  n_ctx: 2048,
  n_gpu_layers: 99,        // offload layers where a GPU backend is available
});

const result = await context.completion(
  {
    messages: [
      { role: 'system', content: 'Answer from the documentation excerpts only.' },
      { role: 'user', content: question },
    ],
    n_predict: 256,
    stop: ['</s>', '<|end|>', '<|eot_id|>'],
  },
  (data) => appendToken(data.token),   // partial-completion callback
);
console.log(result.text);

Official reference: llama.rn README. Option names follow the README at the time of writing; because the latest npm tag was a release candidate on 2026-09-30, compare them with the README of the version you install. Release the context when the screen is closed to free native memory.

react-native-executorch

react-native-executorch (0.10.4) wraps ExecuTorch in task-specific hooks. Its documentation lists requirements that matter for project planning: the New Architecture enabled, a recent React Native or Expo SDK with a development build (Expo Go cannot load native code), and recent iOS and Android minimum versions. Install it with its peer dependencies as the official getting-started page lists them (that page’s command is unpinned, while the command below follows the package’s declared peer ranges).

Terminal: install react-native-executorch and its peer dependencies

npm install react-native-executorch react-native-worklets@^0.10.0 react-native-blob-util@^0.24.0

The hooks download the .pte model on first use and expose readiness and progress. The two snippets below use the useLLMChatSession and useSpeechToText hooks with entries of the models registry, as the 0.10 documentation shows. Hook and constant names were reorganised between releases (the 0.8.x documentation used useLLM with constants such as LLAMA3_2_1B), so copy the snippet from the documentation of the version you install. The documented requirements for 0.10 are the New Architecture, React Native 0.83 or newer (or Expo SDK 55 or newer with a development build), iOS 17 or newer, Android 13 or newer and a react-native-worklets release in the range the package declares (>=0.10.0 <0.13.0), so the command above pins it, and the package also declares react-native-blob-util ^0.24.0 as a peer, while the newest releases of both on npm on 2026-09-30 were outside those ranges.

TypeScript: answer with useLLMChatSession

import { useState } from 'react';
import { models, useLLMChatSession } from 'react-native-executorch';

function Answer() {
  const [answer, setAnswer] = useState('');
  const session = useLLMChatSession(models.llm.LFM2_5_1_2B.DEFAULT, {
    initialMessages: [{ role: 'system', content: 'Answer from the documentation excerpts only.' }],
    generationConfig: { temperature: 0.2, maxNewTokens: 256 },
  });

  const ask = async (question: string) => {
    if (!session.isReady || !session.sendMessage) return;
    setAnswer('');
    await session.sendMessage(question, (token) => setAnswer((prev) => prev + token));
  };

  // render session.downloadProgress while loading, then the streamed answer
}

TypeScript: transcribe with useSpeechToText

import { models, useSpeechToText } from 'react-native-executorch';

const stt = useSpeechToText(models.speechToText.WHISPER.EN.TINY.DEFAULT);

// audioData: Float32Array of mono audio sampled at 16 kHz
const text = await stt.transcribe(audioData, { language: 'en' });

Official reference: LLM Chat & Text Generation and Speech-to-Text. The model entries come from the Software Mansion Hub collections, so check the card of the one you ship for its licence. Other hooks cover vision and embeddings; check the catalogue for the screenshot task. Model size, memory and thermal behaviour are discussed in Performance, and microphone and camera access in Device Capabilities.

onnxruntime-react-native

For a single ONNX graph, onnxruntime-react-native (1.24.3 on 2026-09-30) is the React Native counterpart of ONNX Runtime Web and uses the same InferenceSession API on top of the native runtime. Use it when the model is not available as a .pte or GGUF file. Official reference: ONNX Runtime — Mobile tutorials.

Flutter

Flutter apps call native runtimes through FFI or platform channels. Three packages are relevant; their maturity differs considerably.

Table 4. Flutter packages for Hub models (checked 2026-09-30)
Package	Version	Runs	Caveat
flutter_gemma	1.11.3	Gemma family in `.litertlm` (mobile and desktop), `.task` and `.bin` (mobile and web) through separate engine packages	Model files come from the Hub; some Gemma repositories are gated (check the card), in which case the download needs an access token supplied at runtime. Speech (`flutter_gemma_speech`) and ONNX (`flutter_gemma_onnx`) support are separate opt-in packages, and the README describes both as still maturing.
llama_cpp_dart	0.2.2	GGUF models through llama.cpp	You compile and bundle the llama.cpp shared library for each platform yourself.
onnxruntime	1.4.1	Any ONNX graph through FFI	Last published in March 2024 by an unverified uploader; it may trail the ONNX Runtime releases, so evaluate it before relying on it.

Dart: install and chat with a Gemma model using flutter_gemma

import 'package:flutter_gemma/flutter_gemma.dart';

await FlutterGemma.initialize(
  inferenceEngines: const [LiteRtLmEngine(), MediaPipeEngine()],
);

await FlutterGemma.installModel(
  modelType: ModelType.gemmaIt,
  fileType: ModelFileType.litertlm,
).fromNetwork(modelUrl, token: hubToken).install();   // hubToken read from secure storage, never hard-coded

final model = await FlutterGemma.getActiveModel(maxTokens: 2048);
final chat = await model.createChat();
await chat.addQueryChunk(Message.text(text: question, isUser: true));
final response = await chat.generateChatResponse();
await model.close();

Official reference: flutter_gemma on pub.dev. The README requires declaring the file type explicitly and adding the engine packages (flutter_gemma_litertlm, flutter_gemma_mediapipe) that match your model format. The MediaPipe LLM Inference task behind .task files is maintenance-only in favour of LiteRT-LM; see Android.

Dart: generate tokens with llama_cpp_dart

import 'package:llama_cpp_dart/llama_cpp_dart.dart';

Llama.libraryPath = 'libllama.dylib';          // the shared library you built for the platform
final llama = Llama('path/to/model.gguf');

llama.setPrompt(question);
while (true) {
  final (token, done) = llama.getNext();
  stdout.write(token);
  if (done) break;
}
llama.dispose();

Official reference: llama_cpp_dart on pub.dev. The API of this package has changed between releases; treat the snippet as the README’s minimal example and read the README of the version you install.

Dart: run an ONNX graph with the onnxruntime plugin

import 'package:onnxruntime/onnxruntime.dart';

OrtEnv.instance.init();
final options = OrtSessionOptions();
final bytes = (await rootBundle.load('assets/models/classifier.onnx')).buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, options);

final input = OrtValueTensor.createTensorWithDataList(data, [1, 3, 224, 224]);
final outputs = await session.runAsync(OrtRunOptions(), {'input': input});

input.release();
OrtEnv.instance.release();

Official reference: onnxruntime on pub.dev. A bundled model is part of the app download; larger models are better fetched on first launch into the app’s documents directory.

Shipping and operating

Download size and first run. Show progress, allow cancelling, and keep the weights out of the app binary unless the store’s size limits allow it; the formula above gives a first estimate.
Memory. Browsers cap per-tab memory and mobile operating systems terminate apps that exceed theirs, so prefer the smallest model that passes your own evaluation and unload models you no longer use.
Supply chain. Pin model revisions, prefer safetensors or ONNX over pickle formats, never enable remote code from a repository you have not reviewed, and never ship a Hub access token in a client app; gated downloads should go through a server you control or a user-supplied token.
Fallback. Keep a cloud path for devices that cannot run the model; the cloud counterparts are covered in Inference Providers & Spaces.
Desktop runners. For desktop apps, a local runner may be simpler than embedding a runtime; see Desktop Runners.