Multilingual voice-to-text via faster-whisper — lets a worker speak instead of typing annotation text, in both the image and video editors. 99 languages incl. Hindi/regional + English. Chosen over Parakeet/NeMo because it has no heavy dependencies (no NeMo, no torch), so it builds cleanly.

Contract

Request → POST /runsync:

{ "input": { "audio": "<base64 audio>", "language": "en", "timestamps": false } }

audio: base64 wav/mp3/m4a (any sample rate — resampled internally)
language: optional (e.g. "hi", "en"); omit to auto-detect
timestamps: optional — per-segment start/end

Response → { "output": { "transcript": "...", "language": "en" } } (with segments: [{text, start_sec, end_sec}] if timestamps: true), or { "output": { "error": "..." } }.

Deploy (new endpoint — one per model)

RunPod → Serverless → New Endpoint → Deploy from a GitHub repository → Sumit-Pluto/photo_gallery.
Dockerfile path: /workers/voice-to-text/Dockerfile
Config:
- GPU: light — 16 GB (T4 / A4000) is plenty (large-v3 ≈ 5 GB VRAM).
- Network volume: attach your weights volume (caches the model).
- Workers: Max 1, Active 0, FlashBoot on. Execution timeout ~300s (first-run download).

Warm up once (downloads the model, ~1.5 GB):

curl -X POST https://api.runpod.ai/v2/<NEW_ID>/runsync \
  -H "Content-Type: application/json" -H "Authorization: Bearer <API_KEY>" \
  -d '{"input":{"audio":"<base64-wav>","language":"en"}}'

In Vercel, set RUNPOD_STT_URL = https://api.runpod.ai/v2/<NEW_ID>/runsync (server-only).

Tuning (env vars)

Var	Default	Purpose
`WHISPER_MODEL`	`large-v3`	accuracy vs speed — `medium` / `small` are faster/lighter
`WHISPER_COMPUTE`	`float16`	`int8_float16` uses less VRAM

App integration (next step, not yet built)

Mic-record button in the editor's text/annotation tool → /api/ai/transcribe (proxies here) → inserts the transcript. Client rpTranscribe() already exists in apps/web/src/lib/runpod/endpoints.ts.