Files
hex/ios/HEXShared/UserFacingError.swift
Ryan Vogel 178c16fdc5 iOS: speech model fallbacks for iPhone 11 and unavailable languages; no raw errors in the keyboard (#2048)
* iOS: fall back to DictationTranscriber when a speech model won't install, and never show Apple's error text

A TestFlight tester (iPhone 15, iOS 26.2, build 25, Russian) saw the full
keyboard's strip read "transcription.ru asset not found after attempted
download." Apple Speech lists some languages whose asset never arrives
(Apple's forum shows the same for Arabic), and Hex both stopped there and
showed Apple's sentence: UserFacingError (#2035) kept any description without
an error code in it, and this one has none.

- SpeechModelInstall (HEXShared, used by the app and in-keyboard dictation):
  install the chosen module's model, then the same language on
  DictationTranscriber when SpeechTranscriber's won't install, and go straight
  to DictationTranscriber for three days. When neither installs while online,
  Automatic sets the language aside for three days and moves to the next one
  ("Russian isn't available on this iPhone yet. Using English."); a language
  picked by hand says so and offers that one ("Try English", with Use English
  in the app). Offline or stalled, it says the download failed and keeps the
  language. Every message fits the full keyboard's strip.
- StallWatch: a download whose progress hasn't moved for 60 s is cancelled
  and treated as failed, instead of holding the progress bar forever.
- AppleSpeechAssets.reserve makes room under maximumReservedLocales by
  releasing other reservations first, and a reservation Hex can't get no
  longer fails preparation: the keyboard reserved every language it used and
  never released one, so the app's reserve could throw for good.
- UserFacingError is now an allowlist: only Hex's own errors
  (HexDescribedError) show their text, a lost connection while getting the
  model ready says so, and anything else gets the context's sentence. This
  replaces the marker filter, which let Apple's readable sentences through.
- DictationController re-prepares in the language Automatic moved to
  (at most twice), keeps a notice, and offers a language after an explicit
  choice fails. The dictation bar shows a failure on two smaller lines.
- Snapshot scenes: full-keyboard-language-unavailable[-dark],
  keyboard-language-unavailable, language-unavailable, language-switched.

Verified: xcodebuild build-for-testing on an iPhone 15 / iOS 27.2 simulator,
and the 44 DictationController, picker, SpeechModelInstall and StallWatch
tests pass (fallback, set-aside, offline, unsupported, stall, explicit offer,
Automatic switch, error text). Apple Speech itself can't run in the
simulator, so the install steps are fakes there; which module fails for
Russian on iOS 26.2 is unverified on hardware (no iOS 26.x simulator runtime).

* iOS: dictate English with Apple on iPhone 11, and stop the Parakeet relaunch loop

A TestFlight tester (iPhone 11, iOS 27.3, build 16) reported the model
download sticking at 50% and the app then crashing at every launch, with no
crash log. iPhone 11 (A13, 4 GB) has no SpeechTranscriber
(SpeechTranscriber.isAvailable is false on every A13 iPhone), so Hex used
Parakeet V2 for English. FluidAudio 0.15.5 reports downloads as 0-50% and
Core ML's compile as 50-100%, so 50% is MLModel(contentsOf:) compiling the
encoder, which can't be cancelled; iOS ends the app there. Once the files are
on disk, init saw hasCachedModel and loaded Parakeet again at once, so every
launch ended the same way. iOS ending an app for memory leaves no crash
report, which fits the empty crash list.

- AppleSpeechTranscriber.shouldUse: English without SpeechTranscriber uses
  DictationTranscriber where Parakeet doesn't fit (under 5 GiB of RAM: every
  A13 iPhone) on iOS 26+. Parakeet stays where it fits and before iOS 26.
- ParakeetLoadGuard: a load still marked as running at launch belonged to a
  process that ended, so the next launches don't load Parakeet by themselves
  until a load works; Hex says it closed while loading the model and waits
  for Set up. If Automatic now picks Apple Speech, it prepares that instead.
- Parakeet's download half runs under StallWatch (90 s without progress ends
  it); the compile half isn't watched since it can't be stopped. FluidAudio's
  download errors (Hugging Face paths and HTML in their text) become "Parakeet
  V2 didn't download. Check your connection and try again."

Verified: xcodebuild test on an iPhone 15 / iOS 27.2 simulator, 247 tests,
0 failures, 1 skipped (AppleSpeechClipTests, needs clips), including new
tests for shouldUse on a 4 GB iPhone and the load guard across launches.
Parakeet's memory use, the compile at 50% and the ended launches need a real
iPhone 11; the simulator neither limits memory nor runs SpeechTranscriber.
2026-11-20 15:04:43 +00:00

88 lines
3.6 KiB
Swift

import AVFAudio
import Foundation
/// An error Hex wrote for people: its description is shown as it is. Every other error's
/// description (Apple's, Foundation's, a package's) is for logs, however readable it looks:
/// Speech's "transcription.ru asset not found after attempted download." reads as a sentence.
protocol HexDescribedError: LocalizedError {}
/// Short messages for the errors Hex shows people, in the app and the keyboard. Never a
/// raw OSStatus, error domain, asset name, or Swift type name: those belong in logs.
nonisolated enum UserFacingError {
enum Context {
case microphone
case transcription
case model
var fallback: String {
switch self {
case .microphone: "Hex couldn't start the microphone. Try again."
case .transcription: "Hex couldn't transcribe that. Try again."
case .model: "Hex couldn't get speech recognition ready. Try again."
}
}
}
static let offline = "Hex couldn't download the speech model. Check your connection and try again."
/// Audio session errors get their own message, Hex's own errors (`HexDescribedError`)
/// their description, a lost connection while getting the model ready says so, and
/// anything else gets the context's fallback. An allowlist, not a filter: #2035 kept any
/// description without an error code in it, which let Apple Speech's asset errors through
/// to the keyboard. Microphone errors never show a description.
static func message(for error: Error, context: Context) -> String {
if let code = audioSessionCode(of: error) {
return message(for: code) ?? context.fallback
}
guard context != .microphone else { return context.fallback }
if error is HexDescribedError,
let text = (error as? LocalizedError)?.errorDescription?.trimmingCharacters(in: .whitespacesAndNewlines),
!text.isEmpty {
return text
}
if context == .model, isConnectionError(error) {
return offline
}
return context.fallback
}
/// A URL loading error, itself or underneath another.
static func isConnectionError(_ error: Error) -> Bool {
var current: NSError? = error as NSError
for _ in 0..<4 {
guard let ns = current else { return false }
if ns.domain == NSURLErrorDomain { return true }
current = ns.userInfo[NSUnderlyingErrorKey] as? NSError
}
return false
}
/// The `AVAudioSession` error an error carries (they arrive as `NSOSStatusErrorDomain`).
static func audioSessionCode(of error: Error) -> AVAudioSession.ErrorCode? {
let ns = error as NSError
guard ns.domain == NSOSStatusErrorDomain else { return nil }
return AVAudioSession.ErrorCode(rawValue: ns.code)
}
static func message(for code: AVAudioSession.ErrorCode) -> String? {
switch code {
case .cannotInterruptOthers:
"Another app is using audio. Try again."
case .insufficientPriority:
"Another app is using the microphone. Try again."
case .siriIsRecording:
"Siri is using the microphone. Try again."
case .isBusy:
"The microphone is busy. Try again."
case .cannotStartRecording, .cannotStartPlaying:
"Open Hex, then try again."
case .mediaServicesFailed:
"Audio restarted on this iPhone. Try again."
case .resourceNotAvailable:
"No microphone is available right now."
default:
nil
}
}
}