* Build the native Android dictation keyboard Add a three-page Compose setup and an InputMethodService with QWERTY editing, system IME switching, tap dictation and hold-to-talk. Capture inside the keyboard and decode a checksum-pinned, MIT-licensed Whisper tiny model so a fresh install works offline without an app switch, playback focus request or audio files. Verified on the Studio API 35 ARM64 emulator: clean assembleDebug, three debug/release unit cases, system permission and keyboard setup, actual key editing, two WAVs through the guest microphone, insertion into a separate editor, cancellation and continued media output. The merged debug manifest has no Internet permission. * Document Android setup and the recognizer decision Record the supported toolchain, model and runtime licences, offline-first trade-off, and phase 1 limits. Document the headless emulator microphone path used for verification so the next phase can repeat it. Checked the Studio build/run commands and both recognizers on the API 35 image with airplane mode enabled and media playing. * Measure on-device Android dictation across four languages Replace the phase-one single-phrase evidence with a fixed 40-clip multilingual, partly music-mixed corpus. Record per-clip errors, load, warm inference, memory, two-core sensitivity and delivered platform-service results separately. Keep an independent foreground editor and media track for mic-to-insertion verification. Verified on an API 35 ARM64 Studio emulator: Whisper tiny/base/small, Parakeet TDT v3 int8 and strictly on-device Android recognition. Small in its short window made 15/285 word errors; the platform service returned 33/40. Both native runners also decoded the notebook clip after tightening the PCM input checks. * Use verified Whisper small for styled, offline Android dictation Replace tiny-only English recognition with a pinned small model selected by the multilingual benchmark. Keep bundled tiny for first-run offline use and download failures. Snapshot the style, cleanup, language and translation task per recording so a paste override affects only the intended editor and dictation. Replace the recording status text and old meter with the iOS-style dot waveform, round cancel/finish controls and in-place spinner. Explain the sole network permission on welcome and bundle verified Inter v4.1 and its OFL notice for Compose and the keyboard. Verified assembleDebug and tests on the Studio. The Android formatter matched 41 applicable iOS examples. Exercised tiny before download, observed progress and verified small and Inter hashes. The real IME inserted all four styles and both Spanish paste choices in airplane mode, with music continuing in an external editor; checked recording and transcribing in both themes. * Add offline spatial typing assistance to the Android keyboard Replace literal-only English and Spanish key handling with a native Kotlin Gaussian tap decoder, context-weighted word pairs, missed-space readings, and confidence margins. Map the pinned, uncompressed iOS-compatible Google Books v3 assets; credit the CC BY counts, SCOWL and MPL word filters. Use hashed local rejection counts so a second undo stops that correction without saving typed text. Give the idle strip three frequency-ranked slots and add independent default-on key previews and system-aware haptics, plus slide-to-select accents. Commit space on the same key-up as letters: the previous posted click could reorder rapid input. Make a confident known spelling harder to replace than an unknown one. Verified on Studio API 35: assembleDebug and tests, byte-identical English and Spanish models, the exact centre-tapped typo sentence in both themes, one-backspace undo and suppression after two undos across an IME restart, ranked suggestions, accent insertion, independent switches and the system vibration gate. * Benchmark Android typing against iOS on synthetic key taps Replay the 2,000 held-out English sentences through the installed Kotlin decoder and suggestion path, keeping only public synthetic rows from the mixed iOS experiment. Preserve aggregate scores, the unmodified Google Books model builder, and the two model digests; no labelled personal typing, offsets, or raw logs are committed. Verified 1,461 errors over 13,870 words (10.53% Android; 8.39% iOS without offsets using the byte-identical model; 33.81% raw). On the Studio emulator the real editor received letters in 1.41 ms median and spaces in 3.16 ms across five exact sentence replays. Document the confidence margins, timing boundaries, feedback controls, licences and reproducible aggregate scorer. * Align Android typing with iOS and add German and Bokmål layouts and models Apply R1's English typo and contraction rules before spatial search and match its language weight, shortlist and valid-spelling margins. A mapped SCOWL spelling list supplies the validity check missing on Android. Scale vertical spread and the missing-space cost to real Android row geometry so "seehkw" still splits into "see how". Keep the existing en/es model bytes. Build the new German and Bokmål HXTM files with the shared iOS writer. German uses all Google Books v3 shards; Bokmål uses NB book counts filtered by the complete 2022 Ordbank snapshot. Pin the new assets, credit their sources and licences, and ship their real layouts, accents and suggestions. Replace the post-setup list with the iOS-style Home, local Dictate bar and four settings pages. Follow the system language by default; make the spoken language switchable on space, and optionally pin an independent layout. Verified on the Studio API 35 ARM64 emulator: 1,027 English and 1,374 Spanish word errors vs R1's 1,163 and 1,387 on the same 2,000 synthetic sentences per language. Ran all five unit tests in debug and release, inspected the packaged assets, and checked external-editor suggestions, corrections, undo and offline microphone insertion, plus Home and settings in both themes. The Swift reader round-tripped both new models; the identical iOS files are in the separate local commit 230a437e. * Build a signed, shrunk Android 16 upload bundle Replace the API 35 release defaults with API 36, an external upload key, R8 and resource shrinking. Omit the debug probes, benchmark receiver and unused permission, silence native speech diagnostics, and guard password-field context at the input connection. Give the download a generic user agent and ship the Android privacy policy offline. Verified on the Studio: jarsigner, bundletool and byte-identical installed splits on API 31 and API 36; all ELF load segments and APK native offsets meet 16 KB alignment. Fresh model downloads and offline insertion passed on both. Binder traces showed zero reads in four password variants versus ten in an ordinary editor. Five tests passed in each build variant. * Prepare the Android Play listing and device handoff Add the en-US listing, Data safety and IARC drafts, four release screenshots, launcher-derived store art and a physical-phone checklist. Replace the obsolete API 35 build instructions with the signed API 36 workflow. Flag that the public privacy URL still needs the Android text and that suggestions can include strong language. Verified title and description limits, native screenshot and artwork formats, model downloads and offline microphone insertion in an external editor on 2.5 GiB API 31 and 16 KB API 36 emulators. Recorded bundletool download size and sampled memory from the same signed release.
12 KiB
On-device speech benchmark · Android 15
Default: multilingual Whisper small Q5_1. It made 15 word errors on 285 reference words in Hex’s short-dictation configuration, against 50 for the bundled tiny fallback. Small was also the most accurate in the equal full-context comparison. Base was lighter but made twice as many errors as the default. Parakeet was the fastest batch decoder, but used about four times the native memory. Android’s streaming service was excellent when it returned text, but had incomplete coverage.
What ran
On 20 Nov 2026, an API 35 Google ARM64 emulator ran on an Apple M5 Ultra Studio, with four guest CPUs and 4 GiB guest RAM (emu64a:15/AE3A.240806.043/12960925). Corpus: 24 English, 6 Spanish, 5 German and 5 Norwegian utterances, nine macOS say voices, 157–196 words/minute. Audio manifest: 40 mono 16 kHz PCM16 WAVs, 2.20–3.72 s including padding (median 2.94 s); 24 clean, 16 mixed with a synthesized music bed at 8 or 14 dB SNR. These are short synthetic utterances, not human conversations. The two notebook clips use different voices/rates; the comparison across engines always uses identical bytes for a given clip.
Word error rate (WER) is total word edits divided by total reference words, not a mean of clip percentages. NFC, case folding and word tokenization ignore punctuation and preserve accents/internal apostrophes. A clip without a result counts as deleting every reference word. English supplies 186/285 words; Spanish 37, German 33 and Norwegian 29. These small language samples do not establish general language accuracy.
| Engine | Returned | WER | Clean | Music | EN | ES | DE | NO |
|---|---|---|---|---|---|---|---|---|
| Whisper tiny Q5_1 | 40/40 | 17.54% | 11.66% | 25.41% | 2.15% | 21.62% | 27.27% | 100.00% |
| Whisper base Q5_1 | 40/40 | 10.53% | 7.36% | 14.75% | 0.54% | 2.70% | 24.24% | 68.97% |
| Whisper small Q5_1, full context | 40/40 | 5.96% | 3.07% | 9.84% | 0.00% | 2.70% | 15.15% | 37.93% |
| Whisper small Q5_1, Hex’s short window | 40/40 | 5.26% | 3.07% | 8.20% | 0.00% | 2.70% | 9.09% | 37.93% |
| Parakeet TDT 0.6B v3 int8 | 40/40 | 11.58% | 8.59% | 15.57% | 0.54% | 0.00% | 21.21% | 86.21% |
| Android strictly on-device | 33/40 | 17.54% | 12.27% | 24.59% | 9.68% | 0.00% | 9.09% | 100.00% |
For native engines, one process per model loaded weights once, decoded sequentially using four CPU threads, and recorded monotonic wall clock, process CPU time and process high-water RSS. File read and conversion to floats precede timing. Whisper used greedy decoding, no context from previous clips, no timestamps, and the known input language; Parakeet had no language hint. Tiny, base and small first ran with their standard full encoder window. A second small run used audio_ctx=512; the app uses that only for captures ≤8 s. Longer dictations keep the full encoder window. The window was evaluated on this same corpus, not a held-out dataset.
| Engine | Weights¹ | Load, once | Warm p50 / p95 | Median process CPU | Peak native RSS |
|---|---|---|---|---|---|
| Tiny | 32.2 MB | 43 ms | 390 / 493 ms | 1.39 s | 157 MiB |
| Base | 59.7 MB | 47 ms | 683 / 875 ms | 2.56 s | 224 MiB |
| Small, full | 190.1 MB | 93 ms | 2,570 / 3,039 ms | 9.86 s | 382 MiB |
| Small, short | 190.1 MB | 115 ms | 742 / 897 ms | 2.81 s | 344 MiB |
| Parakeet | 487.2 MB archive | 2,451 ms | 113 / 143 ms | 0.43 s | 1,387 MiB |
| Android provider | OS language packs | opaque | 42 / 65 ms after audio | opaque | shared provider, not attributed |
¹ MB is decimal file size. Parakeet’s three int8 ONNX files and token table total 670.6 MB unpacked. RSS is binary MiB from getrusage. Timings are from this guest, not a phone. Peak native RSS excludes Android’s app UI and other processes. Across two real keyboard sessions, whole-app VmHWM reached 577 MiB and 639 MiB; the app loads/releases the native context per dictation instead of retaining the model while idle.
Platform provider. A disposable app used createOnDeviceSpeechRecognizer, not the generic recognizer with an offline preference. The image initially reported zero installed languages. After confirming the OS downloads, checkRecognitionSupport reported installed en-US, es-ES and de-DE; it never advertised Norwegian. With airplane mode on and Wi-Fi off, there were two music-mixed English no-matches (09/22, error 7) and five unsupported Norwegian clips (error 12). WER on the 33 returned results alone was 2.89%. The device processes audio as it arrives: 3.68 s median from ready to result, including playback; 42 ms median from the end of the injected WAV to result (includes adb stop delivery and 10 ms clock granularity). Where an explicit stop came first, median device stopListening-to-callback was 6.5 ms (32 results). One endpoint completed on its own. These are not batch inference times, nor measurements of natural endpointing without a stop. An earlier paced real-time injection run returned 32/40: 20.00% all-clip WER, 2.15% completed-only WER, and 33 ms median after audio. It is kept separately from the primary results. The primary run paced 20 ms packets with queue back-pressure.
Estimate for a mid-range phone
Budget roughly 3–7 s of CPU inference for a typical short clip on a Galaxy A23 5G / Snapdragon 695-class 4–6 GB phone. This is an estimate, not a handset measurement. Eight fixed clips (01, 02, 12, 19, 25, 26, 32, 37) reran at two threads pinned to two guest vCPUs. Raw measurements. Small’s short window took 1.36 s median on those cores. Assume two effective performance cores at 20–40% of the Studio guest’s per-core throughput: 1.36 / 0.4 to 1.36 / 0.2 = 3.4–6.8 s. For scale only, individual Geekbench 6 single-core scores were 903 on a Galaxy A23 5G and 4,263 on an M5; Geekbench is neither Whisper nor this M5 Ultra VM. Scheduling, memory bandwidth, storage, thermals and other apps can move a phone outside the range. Loading and editor insertion are additional.
For comparison the two-thread guest medians were tiny 0.56 s, base 1.26 s, full-window small 5.42 s, Parakeet 0.16 s. Full-window small is appreciably slower, so the short-clip estimate must not be applied to a minute of dictation.
Keyboard check
The native benchmark consumes files; the debug APK check fed real WAVs through the authenticated emulator microphone, pressed the actual Hex key, and read what another app’s EditText received. Airplane mode was on and Wi-Fi off. The original phase-1 notebook WAV (SHA-256 9e95daa12e469102c75b892820ad7e901e8a2131a3ac7f3e00d312bdb9322828) produced “Please bring the green notebook to the meeting.” in both the practice field and the external editor. A music-mixed keyboard take also got it right; a later take heard “meter.” All native file runs except tiny got the music-mixed sentence exactly; tiny heard “green note.” The clean file alone did not distinguish the engines.
The external editor received the notebook sentence 1.91 s after mic stop and “The coffee is still hot.” 2.13 s after mic stop. These two timings include model initialization, inference and insertion. Its media track remained playing and advanced 481,622 frames over 30.10 s at 16 kHz; no audio-focus loss was observed. Tapping ✕ during inference left its editor empty after native completion.
Same WAV, same style, only cleanup changed: Polished inserted “You know, the coffee is still warm.” with cleanup off and “The coffee is still warm.” with it on. Clip 13 inserted “Meet me on Tuesday, sorry, Wednesday.” with cleanup off and “Meet me on Wednesday.” with it on. Verbatim preserved the spoken hesitation and correction; Casual removed the final period; Notes produced two separate bullet lines. The compiled Android cleanup/formatter also matched all 41 applicable Spanish, German and Norwegian input/output examples drawn from iOS’s SpokenCleanupRulesTests. This checks those rules, not parity with iOS’s learned English tagger.
Model setup: before downloading, bundled tiny inserted “The coffee is still warm.” offline. Retrying online showed download progress, then ready; the file’s verified device SHA-256 matched the small pin below. Airplane mode with Wi-Fi disabled was sufficient for both paste languages afterwards.
Sources and licences
- OpenAI’s multilingual Whisper weights, MIT; Q5_1 files from ggerganov/whisper.cpp at 5359861…. SHA-256: tiny
818710568da3ca15689e31a743197b520007872ff9576237bda97bd1b469c3d7; base422f1ae452ade6f30a004d7e5c6a43195e4433bc370bf23fac9cc591f01a8898; smallae85e4a935d7a567bd102fe55afc16bb595bdb618e11b2fc7591bc08120411bb. whisper.cpp/ggml v1.9.5 MIT. - NVIDIA’s Parakeet TDT 0.6B v3 CC BY 4.0; its 25-language list does not include Norwegian. Int8 conversion: SHA-256
1495e91b59d7fc53ddca91f3431758ec99f8fa253fdb0e1e851be0f72b14b0f9. sherpa-onnx 2.0.2 Apache 2.0, ONNX Runtime MIT. Android runtime archive SHA-25608d22f7eb5c49bd161d248633eeb66e84735725706e21202b684800e5081a029. Benchmark only; not distributed with Hex. - Android’s on-device API is a platform service; Google supplied the proprietary language packs on this image. Hex does not redistribute or require them.
Reproducing the measurements
- On macOS, run
python3 benchmark/generate.py "$WORK/wavs" > "$WORK/audio-sha.tsv"withsay, the named voices, and ffmpeg installed. Compare sample counts and digests with audio.tsv; voice assets can change with macOS. Generate the input list withawk -F '\t' 'NR>1{print $1,$2}' benchmark/corpus.tsv. - Build the Android app as in the Android readme. Compile native.cpp for arm64 with the Android NDK (
aarch64-linux-android31-clang++ -O2 -std=c++17 -static-libstdc++). For Whisper use the app’s whisper/ggml headers and linklibwhisper.aandlibggml*.ain a linker group plus-landroid -llog. For Parakeet definePARAKEET, use the matching 2.0.2 C API header and linklibsherpa-onnx-c-api.sofrom the Android runtime’sjniLibs/arm64-v8a, alongside itslibonnxruntime.so. - Push the executables, weights,
wavs/and two-column list to the same guest scratch directory. From there run./whisperbench models/small.bin 4 list, and repeat for base/tiny. RunHEX_SHORT_WINDOW=1 ./whisperbench models/small.bin 4 listfor the shipping setting;LD_LIBRARY_PATH=$PWD ./parabench parakeet 4 listfor Parakeet. Capture stdout asresults/{tiny,base,small,small-short,parakeet}.tsvand stderr separately. - Build SystemProbe.java with its manifest using JDK 21 and SDK 35 (javac, d8, aapt, zipalign, apksigner). Install the disposable APK with
adb install -g. Startlocal.hex.measure/.SystemProbewith--es op download --es id pack --es lang <locale>for each pack and confirm the OS dialogs. Force-stop the probe and start it fresh with--es op support --es id packs --es lang en-USto check installed locales and reconnect the service. Enable airplane mode and disable Wi-Fi. From the WAV parent, with emulator gRPC Python bindings onPYTHONPATH, runpython3 <benchmark>/system_runs.py > results/android.tsv. python3 benchmark/score.py results > per-clip.tsvemits each reference-word count, edit count, outcome, original hypothesis and timing, plus summaries on stderr. Recorded per-clip results. Only synthetic benchmark transcripts appear here.