9 Commits
Author SHA1 Message Date
Ryan Vogel be49c35f2b iOS: privacy and support links in the app, Continue before the mic prompt, typing logs consent (#2063)
* iOS Privacy: Privacy policy and Support links, for App Review 5.1.1(i)

App Review requires a privacy policy link inside the app. The Privacy page
now has a card with two rows between Learn how I type and Advanced:
- Privacy policy opens https://dev.anoma.ly/hex/privacy.
- Support opens https://dev.anoma.ly/hex/iphone.

Both open in Safari inside Hex (SFSafariViewController, SafariSheet).

The URLs come from one base in HexLinks.swift (HexLinks.site). Moving to
https://anoma.ly/hex is a one-line change once the site's production deploy
serves those pages; https://anoma.ly/hex/privacy returns 404 today, while
both dev.anoma.ly pages return 200. README's Privacy page entry says so.

Verified: the build passes on an iPhone 18 Pro simulator (iOS 27.2). I looked
at the settings-privacy render. A temporary, uncommitted hook opened the
Privacy policy row's Safari sheet, which loaded the live dev.anoma.ly page.

* iOS setup sheet: Continue, not Allow, before the microphone prompt

App Review 5.1.1(iv) rejects a custom button that says "Allow" right before
iOS's own permission prompt. The setup sheet's Microphone row had one. It now
matches onboarding's Set up Hex page (#2058): the row says why ("For
dictation", replacing "Required") and its button says Continue. Tapping
Continue still calls requestMicrophone, so iOS asks right after. Allowed and
Off are unchanged, and Off still shows Open Settings.

New scene: --snap-state setup-sheet-microphone, the setup sheet before iOS
has asked for the microphone. README's Setup and scene notes are updated.

Verified: the build passes on an iPhone 18 Pro simulator (iOS 27.2). I looked
at the setup-sheet-microphone render (For dictation, Continue) and the
setup-sheet render (Allowed, no button).

* iOS Privacy: Record typing logs says what it records and asks before turning on

App Review 2.5.14 requires explicit consent before recording what people
type.

- The description now says what the feature records: "Records taps and the
  text you type for 14 days, on this iPhone, to export. Off by default." It
  replaces "Off by default. Stays on this iPhone to export" and fits on two
  lines.
- Turning it on opens an alert before anything changes. Title: "Record
  typing logs?". Message: "Hex's keyboard will record each tap and the text
  you type or dictate with it, including the text around the cursor, for 14
  days. The logs stay on this iPhone for you to export or delete. Password,
  code and card fields are never recorded." Turn On switches it on; Cancel
  leaves it off. Turning it off needs no dialog. The switch still writes only
  TypingLogSettings.isEnabled.
- Every claim matches docs/typing-logs.md: touches on Hex's keys, the text
  each key produced, the text around the cursor, dictations, 14 days in the
  App Group, export and delete from Typing logs, and a "skipped" line only for
  password, one-time code and card fields.

New scene: --snap-state settings-privacy-confirm opens the dialog
(SettingsSnapshotFixture.confirmsTypingLogs). README and docs/typing-logs.md
now describe the consent and the new path (Privacy › Advanced, no longer
Experiments).

Verified: the build passes on an iPhone 18 Pro simulator (iOS 27.2). I looked
at the settings-privacy, settings-privacy-confirm and settings-privacy-on
renders.

* iOS home: Privacy policy and Support join Acknowledgements

#2059 added an Acknowledgements row at the bottom of home's utility card. The
privacy policy and support links read better beside it than on the Privacy
page. Home's last card now has Privacy policy (hand.raised, ↗), Support
(questionmark.circle, ↗) and Acknowledgements (doc.text, ›). Keyboard settings
and Run setup again keep their own card above it.

- Moved, not duplicated: the Privacy page's links card is removed. That page
  is back to "Everything stays on this iPhone", Learn how I type, and
  Advanced.
- Links still come from HexLinks (dev.anoma.ly until production serves
  anoma.ly/hex/privacy) and open in Safari inside Hex.
- README's Settings section now lists the links on home.

Verified: the build passes on an iPhone 18 Pro simulator (iOS 27.2). I looked
at the settings-home render, which still fits one screen without scrolling,
and the settings-privacy render.
2026-11-20 16:23:37 +00:00
Ryan Vogel c14df5a4f0 iOS: spatial autocorrect experiment with Google Books models, plus accent and two-letter fixes (#2059)
* iOS keyboard: Spatial autocorrect experiment (tap + bigram decoder), off by default

What: a noisy-channel word decoder (TypingDecoder) behind Settings ›
Experiments › Spatial autocorrect (experiments.spatialAutocorrect, off
by default; it needs the full keyboard with Autocorrect on). When on, for a layout whose
model is bundled (English, Spanish), it replaces UITextChecker's one-edit
guesses on space and in the strip:
- each tap is scored against every letter with a 2D Gaussian around the
  key (sigma 0.29 x 0.26 of the key pitch), moved by Learn how I type's
  average tap offsets for letters with 20+ taps;
- words come from a beam search over a memory-mapped, sorted word list
  with substitutions, one insertion, deletion or transposition, free
  apostrophes ("dont" -> "don't") and accented letters read from their
  base key (é from e, +1 nat);
- a bigram language model on the word before (stupid backoff) weighs
  them, and a missed space is tried by splitting the taps in two (or
  treating a bottom-row tap as the space);
- what was typed is replaced only when the best reading beats it by 2
  nats, or 7 when it is itself a word the model, UITextChecker, learned
  words or the shorthand list know.
Text replacements, learned fixes, the English contraction/typo table and
words you kept still come first. FullKeyboardEngine now records each
letter's tap point for the word being typed (cleared on anything else).
scripts/build-typing-model.py writes the model format (v2: per-language
alphabet, sorted words, u8 costs, sorted bigram keys); models are kept
out of git (.gitignore). The English prototype model is built from
SymSpell's frequency lists (MIT; Google Books Ngram CC BY 3.0 and SCOWL),
the Spanish one from Tatoeba sentence counts (CC BY 2.0 FR). Typing logs
now record the setting.

Why: Record typing logs show short words and non-words left uncorrected
("tk" for "to"), missed spaces with a typo inside, and o/i taps landing
low on k/j. The UITextChecker path skips 1-2 letter words, never uses
the word before, and can't split a word that also has a substitution.

Verified (anomastudio, iPhone 18 Pro simulator, iOS 27.2): benchmark
harness (kept outside the repo) typing every tap through
FullKeyboardEngine. Synthetic held-out English, 13,870 words: output
word error rate 15.0% today vs 6.7% (6.3% with tap offsets); typos fixed
58% vs 83%; correct words wrongly changed 89 vs 97 of 9,203. Labelled
real typing logs, 172 words: 14.5% vs 4.7%. Spanish held-out, 13,301
words: 26.8% vs 8.1% (Spanish model and test from the same corpus, so
optimistic). Per letter p50 1.0 ms, p95 2.5 ms; mapping the model adds
no measurable phys_footprint. Full HEXMobileTests suite: 221 tests,
0 failures (1 skipped, as before); Release generic-iOS build with
CODE_SIGNING_ALLOWED=NO succeeds with both models bundled. No new tests.

* iOS keyboard: Spatial autocorrect models from Google Books Ngram v3

What: the English and Spanish models (HEXKeyboard/TypingModel/en.hextm,
es.hextm, 3.6 MB each) are now built from Google Books Ngram v3 (CC BY
3.0) word and word-pair counts summed over 1980-2019, keeping only
words SCOWL (sizes up to 70, lowercase, ASCII) or RLA-ES (LibreOffice's
es_ES and es_MX dictionaries, MPL 1.1) accept, and are committed.
- English: 86,133 words, 417,480 word pairs; Spanish: 110,000 words,
  326,366 pairs. Pairs are the ones that tell most over the single-word
  model (count x log of P(b|a) over 0.4 P(b)), as many as fit.
- Format v3: pairs grouped by first word (u32 row starts, u32 second
  word, u8 cost), 5 bytes a pair instead of 9, so ~70% more pairs fit
  in the same size. TypingDecoderModel reads v3 only.
- scripts/build-typing-model.py downloads, counts (each ngram file
  deleted once counted, every stage cached for a resumable rerun) and
  writes a model: python build-typing-model.py <en|es> <workdir> <out>.
- Tuning for the new models (dev sentences): English leans more on the
  word before (lmWeight 1.5; Parameters.tuned(for:)); an accent-only
  change ("esta" -> "está") needs a 2-nat margin instead of 7 (ñ for n
  isn't one: "ano"/"año"); an accented letter also reads from its base
  key when it has its own (ñ from n), whichever is cheaper.
- Settings › Acknowledgements (new row and sheet, AcknowledgementsView)
  and ios/README.md credit Google Books Ngram (CC BY 3.0), SCOWL with its
  and WordNet's notices, and RLA-ES (MPL 1.1).
- AutocorrectEngineTests: decoder ranking cases on the bundled models
  ("need t^o" -> "need to", a missed space with a slip -> "see how",
  correct words kept, Spanish "aqui manana" -> "aquí mañana", "esta"
  kept).

Why: Ryan's licence decision: CC BY with credit, no share-alike, no
non-commercial or GPL/LGPL data, and not SymSpell's word-pair list
(unverified provenance). The Spanish model was built from Tatoeba,
the same corpus as its test set.

Removed: the SymSpell- and Tatoeba-built prototype models and the v2
format (pairs as sorted u64 keys); models are no longer gitignored.

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2; benchmark
harness outside the repo; word error rate on words that aren't names):
English held-out test, 13,870 words: today's engine 13.0%, decoder
3.6% (3.4% with tap offsets), correct words broken 4 -> 2 of 7,716;
labelled real typing logs, 172 words: 14.5% -> 4.1%; the shared
example sentence decodes exactly; Spanish held-out Tatoeba test (no
source shared with the model), 13,301 words: 25.7% -> 8.0%, broken
16 -> 20 of 7,246. Per letter p50 1.0 ms, p95 2.6 ms.
AutocorrectEngineTests 19 passed.

* iOS keyboard: Spatial autocorrect knows your own words

What: the decoder also searches TypingDecoder.UserWords, a small sorted
in-memory list of your words, beside the model: Contacts names and
other UILexicon terms, one-word text replacement outputs, words you
kept twice (LearnedWordsStore) and typo fixes Learn how I type learned.
Each scores as a moderately common word (userWordCost 11 nats) and
comes back as you spell it ("Zorbix", not "zorbix"); typed exactly,
one counts as a word, so it needs the larger margin to change.
AutocorrectEngine rebuilds the list only when a source changes
(revision bumped by loadLexiconEntries, applyLearning, suppressAndKeep)
or the model changes. The search is now generic over TypingDecoderWords
(the model or your words); the model's own narrow moved to it.

Why: a name, brand or word you use that the model lacks could only be
left alone, never reached from a typo.

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2):
AutocorrectEngineTests 20 passed, including a new case: with "Zorbix"
in the lexicon, a slip "zorbiz" becomes "Zorbix" and "zorbix" typed
right stays. Benchmark unchanged (it has no user words).

* iOS keyboard: an accent or tilde is the cheapest autocorrect fix

What: AutocorrectEngine.singleEditKeyboardCost now scores a one-letter
substitution that only adds or changes an accent, diaeresis or tilde
(e/é, u/ü, n/ñ) at 0.15, below every other edit, instead of 1.15 (the
cost of two far-apart keys). New helper sameLetterIgnoringAccents.
Valid words are still never rewritten, so "esta" stays "esta".

Why: Spanish typed without accents was corrected into the wrong word,
because the accented guess cost more than any insertion, deletion or
neighbouring key: "aqui" -> "caqui", "podia" -> "odia", "manana" ->
"manaba". Now they become "aquí", "podía", "mañana".

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, the
benchmark harness kept outside the repo, today's engine with Spatial
autocorrect off): Spanish held-out Tatoeba test, 13,301 words, output
word error rate 26.8% -> 24.7%, words changed into another wrong word
1,015 -> 743, correct words broken 16 -> 16 of 7,246 (names excluded).
English test (13,870 words) unchanged at 15.0%.

* iOS keyboard: fix two-letter typos toward very common two-letter words

What: cautiousReplacement fixes a two-letter non-word (UITextChecker
doesn't know it, not ALL CAPS, no apostrophe) when one neighbouring key
or one swap turns it into a very common two-letter word of the layout's
language (KeyboardLanguage.commonTwoLetterWords, commonest first: "tk"
-> "to", "ot" -> "to", Spanish "nl" -> "no", "ee" -> "de"). English and
Spanish have lists; Norwegian and German have none yet and keep
leaving two-letter words alone. The step numbering in the comments
moves by one.

Why: autocorrect skipped every word under three letters, and the
commonest words are two letters: typing logs show "tk" for "to" left
uncorrected again and again (the o key's taps land low, on k).

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, benchmark
harness outside the repo, today's engine): English held-out test,
13,870 words, output word error rate 15.0% -> 13.2%; labelled real
typing logs, 172 words, 14.5% -> 10.5%; Spanish test 24.7% -> 22.9%;
correct words broken unchanged (4 of 7,716 English, 16 of 7,246
Spanish, names excluded).
2026-11-20 16:18:07 +00:00
Ryan Vogel 42e6e8e12d iOS: remove the return-to-app feature; swipe back after a handoff (#2042)
Ryan decided Hex Mobile won't open other apps' URL schemes after the
keyboard hands off to the Hex app: no return-to-app tricks, since they
are an App Review risk and only worked for apps with a known scheme.

Removed:
- HostApp / HostAppCatalog (the curated scheme list) and the bridge's
  returnAppID / quickReturnApps settings. The stored App Group keys
  (settings.returnAppID, settings.quickReturnAppIDs) are just ignored;
  no migration.
- KeyboardLaunchRequest's returnAppID and the URL's return= item. An
  older keyboard that still sends return= parses fine and it is ignored.
- DictationController's auto-return: AutoReturnState, the URL opener,
  setReturnAppID, setQuickReturnApps, returnToHostApp,
  maybeAutoReturnAfterActivation, triggerAutoReturn, the ~1 s fallback
  task, noteSceneDidEnterBackground, and the attemptAutoReturn flags.
- The app's Settings "Return to" row, its "Edit apps…" sheet
  (ReturnAppsSheet.swift), and the handoff screen's quick-jump app chips.
- The dictation bar's return chip and its app tray, and the full
  keyboard Hex key menu's "Return to" row and page (with its
  return_app typing-log event).
- The handoff-returning snapshot scene.

Replaced by: the handoff screen's existing quiet "Swipe bar to return"
cue over the home indicator, now shown whenever it records or a warm
session is on (it used to hide while an auto-return was attempted).
The rest of the handoff and warm-session flow is unchanged.

Docs: ios/README.md (cold start, Hex key menu, proof limits) and
ios/docs/typing-logs.md.

Verified: xcodebuild test on an iPhone 18 Pro simulator (iOS 27.2):
220 tests, 1 skipped, 0 failures. The handoff test now checks the 10 s
window alone, and the launch URL test checks that a legacy return= is
ignored. Rendered --snap-state setup, warm, handoff-listening,
handoff-no-home-bar, keyboard-cold and full-keyboard-style and checked
them: no Return to row, chips or return trigger; the swipe cue shows on
the handoff screen.
2026-11-20 13:58:31 +00:00
Ryan Vogel 9d26cbdb87 iOS: fix six TestFlight tester issues (audio error, leaked error text, Live Activity, haptics, capitals, push to talk) (#2035)
* iOS: retry '!int' audio activation mixable and show audio errors as sentences

TestFlight testers saw "The operation couldn't be completed. (OSStatus error
560557684.)" in the keyboard strip and the app's setup card. 560557684 is
'!int' (AVAudioSession cannotInterruptOthers): a non-mixable session (the
Bluetooth-microphone .record category) going active from the background while
another app plays or records.

- AudioBackend: when setActive(true) fails with '!int', set .playAndRecord
  with .mixWithOthers (keeping HFP when Bluetooth is allowed) and activate
  once more, so the warm keepalive and captures start alongside the other
  app's audio instead of failing.
- New UserFacingError (HEXShared, app and keyboard): AVAudioSession codes map
  to short sentences ("Another app is using audio. Try again."); other
  microphone errors get "Hex couldn't start the microphone. Try again.";
  transcription and model errors keep their description only when it reads
  as a sentence, never Foundation's "couldn't be completed (… error N)" or a
  Swift type name.
- DictationController: every error it shows (phase .failed and the
  keyboard's message) goes through UserFacingError instead of
  localizedDescription.

Verified: xcodebuild test on the iPhone 18 Pro simulator, 179 tests pass,
including new tests for the code-to-sentence mapping and for a warm session
whose keepalive throws '!int' reaching the app and keyboard snapshot as
"Another app is using audio. Try again.". The background '!int' retry itself
needs a physical iPhone (music playing, Bluetooth mic allowed, warm session).

* iOS: end empty dictations quietly and keep error text human in the keyboard

Tapping Done (or ✕) in the keyboard's dictation screen without speaking
showed "Transcription failed: HEXKeyboard.(unknown context at $1…)": the
in-keyboard path printed the error's domain (a private Swift type name),
code and description for Apple Speech's "no words" result.

- New EmptyCapture rule (HEXShared): a capture under 0.3 s (or none at all)
  is a tap, and an engine's no-speech error or a cancellation means nothing
  was said; both end quietly. Engines say so through NoSpeechError
  (Parakeet's and Apple Speech's emptyTranscript, the keyboard's
  KeyboardSpeech.Failure.empty).
- KeyboardLocalDictation: Done before the microphone came up, or a capture
  too short to hold a word, returns to idle without transcribing (and a late
  "started" from WebKit stops the capture instead of opening a file);
  no-speech ends quietly; a transcription error from a cancelled or
  superseded job is dropped; every other failure shows a sentence from
  UserFacingError or the keyboard's own copy, while the probe log keeps the
  domain and code. KeyboardSpeech.Failure's descriptions are now sentences.
- DictationController (warm session and app): an engine's no-speech error
  returns to Ready with no message instead of failing.

Verified: xcodebuild test on the iPhone 18 Pro simulator after rebasing onto
origin/dev (bba07c64), 210 tests pass (1 skipped as before), including new
tests for the empty-capture rule and for a warm keyboard dictation whose
transcriber hears nothing (Ready, no message) next to one that fails (a
sentence). The in-keyboard WebKit capture path needs a physical iPhone.

* iOS: start Live Activities only for system dictations and end them reliably

Testers saw Hex's activity (dots + ✓, expanded "Copied") stay in the
Dynamic Island, even with Keep mic warm Off. The ✓/Copied look is the
widget's stale rendering of an activity nobody ended: every recording
(keyboard warm dictations and in-app ones included) requested an activity,
and the end ran in a detached task that a suspending app (microphone just
released) could skip, so it went stale and stayed for hours.

- DictationController requests an activity only for dictations that copy
  (Action Button, Control Center, Siri/Shortcuts); the keyboard and the app
  show their own recording UI. Removed: the activity for keyboard and in-app
  dictations.
- LiveActivityCoordinator: ending (cancel, failure, short capture, warm
  session cancel) ends every Hex activity there is at that moment with
  .immediate; Copied still lingers 2 s via .after. Ends run under a UIKit
  background task, and a dictation that is transcribing keeps one until its
  activity ends. A current activity the system already ended is replaced
  rather than updated.
- Stale activities end at launch (DictationController.shared init), when Hex
  becomes active, and when the warm session ends or Keep mic warm is set to
  Off, unless a dictation is recording or transcribing.
- README: when Live Activities show and end.

Verified: xcodebuild test on the iPhone 18 Pro simulator, 210 tests pass
(1 skipped as before). A temporary, uncommitted test on the simulator showed
no activity for a warm keyboard dictation, one for a copy dictation that
ended after Copied, one ended at once on cancel, and an orphan activity
ended by Keep mic warm › Off. The Dynamic Island itself, and background
suspension timing, need a physical iPhone.

* iOS: give the dictation bar's keys the full keyboard's light haptic

The full keyboard taps under the finger (Experiments › Haptics, with Full
Access); the dictation bar's keys had no haptic at all.

- KeyboardViewModel gains onKeyFeedback; every control on the dictation bar
  calls it: Hex mark, Dictate / Set up, ✕ and Done, Swipe and its return-app
  chips, the style and language triggers and chips, the tray's close button,
  space, return, and delete (each repeat, as the full keyboard's delete).
- KeyboardViewController plays a light impact at the full keyboard's 0.65
  intensity, under the same rule the full keyboard now shares
  (allowsKeyHaptics: Full Access and FullKeyboardSettings().haptics).
- Experiments: the Haptics switch moves out of the Full keyboard group so it
  shows whether or not the full keyboard is on; same key
  (settings.keyboardHaptics), subtitle now says it covers Hex's keys.

Verified: xcodebuild test on the iPhone 18 Pro simulator, 210 tests pass (1
skipped as before); Experiments rendered with --snap-state experiments shows
Haptics after Keyboard layout, outside the Full keyboard rows. The haptic
itself (and its Full Access gate) needs a physical iPhone; the simulator has
no Taptic Engine.

* iOS: lowercase a dictation that continues a sentence

A tester typed a few words, dictated, and the dictation started with a
capital as if it began a sentence: speech engines capitalize every result.

- TranscriptFormatter.startsSentence(after:): the text before the cursor ends
  a sentence when it is empty or only spaces, or its last mark is . ! ? … 。
  or a line break (closing quotes and brackets after them count too).
- TranscriptFormatter.continuingSentence: mid-sentence, the first letter goes
  lowercase unless the first word is "I" or an I' contraction, all capitals
  (2+ letters), has capitals inside (McDonald's), or its lowercase form isn't
  in the dictation language's dictionary (likely a name).
- insertionText takes that dictionary (isWord; nil keeps the case), never
  adds a second space after one already there (spaces at the start of the
  dictation are dropped), and adds none after an opening ( [ { " “ ‘ « ¿ ¡.
- KeyboardViewController (the one place both keyboards insert a dictation)
  passes a UITextChecker for the dictation language (Apple Speech's last
  resolved language, else Settings' or Automatic's, else the first
  preferred language), mapped by KeyboardLanguage.spellCheckLanguage(
  forDictation:available:). German and Luxembourgish keep the case: on the
  simulator de_DE accepts "auto" and "haus", so the name guard can't protect
  their nouns. A language with no dictionary keeps the case too.
- README: the rule.

Verified: xcodebuild test on the iPhone 18 Pro simulator, 213 tests pass (1
skipped as before), with new tests for the sentence-start rule, the
exceptions, spacing, and the language-to-dictionary mapping. A temporary,
uncommitted test on the simulator's real UITextChecker found "sarah" and
"london" unknown and "the"/"that's" known in en_US, es and de_DE. How each
host app reports documentContextBeforeInput is unverified on a device.

* iOS: push to talk on the Hex key and the dictation bar's Dictate

A tester asked to hold to talk instead of tapping Dictate and then Done.

Rule: holding still for 0.35 s (HexKeyGesture.holdDuration, the Hex key's
existing hold) starts dictation at once, and lifting the finger finishes it
as Done does. A quick tap keeps tap to start, Done to finish. On the Hex key,
sliding more than 10 pt before the hold fires still opens the style menu;
once dictation has started, moving changes nothing. Push to talk applies
only when dictation runs in the keyboard (warm session or in-keyboard
dictation); a hold that opens Hex stays a tap.

- HexKeyGesture: a fired hold's touch-up is now .release (was ignored).
  Changed: before, a hold started dictation and the release did nothing.
- HexKeyControl calls onHoldRelease; FullKeyboardView finishes the dictation
  the hold started (model.onStopTap, logged as dictation_stop from
  hex_key_release). The recording fade keeps a still-tracking Hex key in
  the window at alpha 0 instead of hiding it, so its touch keeps arriving.
- KeyboardSurfaceView: the well tracks the press with a simultaneous
  DragGesture on the same state machine (the Dictate button stops taking
  touches once recording shows), starts with the light haptic
  (onKeyFeedback: Haptics setting and Full Access), and finishes on release;
  the button's own touch-up action skips a press the hold took.
- Accessibility hints mention holding; README and docs/typing-logs.md
  describe it.

Verified: xcodebuild test on the iPhone 18 Pro simulator, 213 tests pass (1
skipped as before), including the updated HexKeyGesture tests (hold then
release finishes; moving after the hold is ignored). The dictation bar
renders as before (--snap-state keyboard-warm). Real touches (hold timing,
the bar's buttons with the well's gesture, the Hex key's touch surviving the
recording panel) were not driven on the simulator and need a physical
iPhone.
2026-11-20 13:21:16 +00:00
Ryan Vogel bb5875cea8 iOS: Norwegian, Spanish and German full-keyboard layouts
Testers asked for the Norwegian layout and, with it, Spanish and
German. The layout follows the dictation language unless set.

What changed:
- KeyboardLanguage holds each layout's letter rows, long-press accents
  and spelling dictionary: Norsk (a-ring ending row 1, o-slash and ae
  row 2), Espanol (n-tilde ending row 2), Deutsch (QWERTZ, u/o/a-umlaut,
  sharp s on a held S), as Apple lays them out. Adding a language is a
  new case with those three filled in.
- KeyboardLayout takes the language. Rows of 11 keys narrow only the
  character keys; shift, delete and the bottom row keep their English
  (10-key) sizes, and row 3's letters sit centred between shift and
  delete. letterWidth stays the 10-key width; characterWidth is new.
- Experiments > Full keyboard > Keyboard layout: Automatic, English,
  Norsk, Espanol, Deutsch (settings.keyboardLayout). Automatic follows
  the dictation language (no layout yet for Hebrew and others: English),
  else the first iPhone keyboard or preferred language with a layout. A
  pick in the keyboard's language switch changes the layout at once.
- Autocorrect uses the layout's UITextChecker dictionary (nb, es, de);
  without one it offers nothing but your text replacements instead of
  correcting into English. The English contractions and typo list
  (i -> I) apply only to English; neighbouring-key costs follow the
  layout. LanguageSettings.effectiveDictationLanguage now prefers what
  Automatic picks over the last language dictation ran in.
- Typing logs record the layout's language; the snapshot stage takes
  --keyboard-layout, follows --dictation-language and --accent-key.
- TypingLogTests pins English, since FullKeyboardView now reads the
  App Group's dictation language.

Verified: xcodebuild test on the iPhone 18 Pro simulator (iOS 27.2):
206 tests, 0 failures, 1 skipped (the Apple Speech clip test), with
layout tests for every language at 320-440 pt and landscape (rows,
11-key widths, hit-frame tiling, balloons, accent menus) and autocorrect
tests; Release generic-iOS build with CODE_SIGNING_ALLOWED=NO succeeds;
light and dark screenshots of each layout, its accents, shift, the
settings row and the dictation bar's re-shot trigger checked by eye. The
simulator has dictionaries for nb, es and de. Accent lists are from
Apple's layouts as remembered, not checked on a device; the 123 and #+=
pages are English's in every language.
2026-11-20 07:52:48 -05:00
Ryan Vogel 2b225b390b iOS: switch the dictation language from the keyboard
Testers asked to change the dictation language without opening Hex.

What changed:
- Dictation bar: a language trigger beside the style ("Norsk") opens a
  tray of chips: Auto plus up to five languages.
- Full keyboard: the Hex key menu gets a Language row and page with the
  same choices, and the space bar names the dictation language faintly
  ("English", "Norsk"), like Apple's.
- DictationLanguageSwitch picks the choices (the language set, recent
  picks, then the iPhone's preferred languages, each matched to a locale
  Apple Speech has; Nynorsk maps to Bokmal) and names each in its own
  language, adding a region code only where two share a language.
- LanguageSettings keeps recent picks, what Automatic resolves to, and
  Apple's language list (cached when Hex or the keyboard lists it); a
  pick in the keyboard writes the same settings.dictationLanguage as
  Hex's menu. In-keyboard dictation reads it per dictation; a warm Hex
  session follows it in its 100 ms loop and prepares the language
  between dictations; a keyboard handoff and the Action Button pick it
  up before recording.
- Typing logs record the dictation language setting and a
  dictation_language UI event.

Verified: xcodebuild test on the iPhone 18 Pro simulator (iOS 27.2):
195 tests, 0 failures, 1 skipped (the Apple Speech clip test, which
needs HEX_SPEECH_CLIPS), including new switch, recents, follow and
space-bar tests; Release generic-iOS build with CODE_SIGNING_ALLOWED=NO
succeeds; screenshots of the Hex menu's language page (light, dark),
the space bar, the dictation bar's tray and the settings row checked by
eye. The trigger's last change (chevron instead of a globe, which looked
like the next-keyboard key) was compiled, not re-shot. Not tried on a
physical iPhone.
2026-11-20 07:52:38 -05:00
Ryan Vogel 635a6c5a2e iOS: add a Haptics switch to the full keyboard (#2030)
A tester asked to turn off typing haptics. Experiments › Full keyboard › Haptics (on by default) now gates the keyboard's allowsHaptics alongside Full Access, so key presses, accent picks and trackpad steps stop tapping when it's off. Typing logs record the setting.

Verified HEXMobileTests on the iPhone 18 Pro simulator and a render of the toggle.
2026-11-20 11:59:34 +00:00
Ryan Vogel 8c7d01672e iOS: add a Key pop-ups switch to the full keyboard (#2027)
A tester asked to turn off the enlarged letter that pops up over a pressed key. Experiments › Full keyboard › Key pop-ups (on by default) now controls it; off, the pressed key darkens in place as with iOS's Character Preview off. Typing logs record the setting as key_popups.

Verified HEXMobileTests on the iPhone 18 Pro simulator and simulator renders of a held key with the setting on and off.
2026-11-20 11:47:18 +00:00
Ryan Vogel dcc8479cc2 ios: record raw typing logs from Hex's keyboard (experiment) (#2024)
What changed
- Experiments › Record typing logs (off by default, App Group key
  experiments.typingLogs). It is separate from Learn how I type, which is
  unchanged. While it is on, the keyboard writes NDJSON day files to
  TypingLogs/yyyy-MM-dd.ndjson in the App Group. Events:
  session_start/session_end (field traits, keyboard size and safe area,
  orientation, appearance, device, build, locale); layout (every key's frame
  and hit frame, the strip and its suggestion slots, and a hash, with the key
  list written once per file); touch (began/moved/ended/cancelled with raw
  x/y, majorRadius and tolerance, force, UITouch.timestamp, the key the engine
  resolved, and the key under the point when they differ; moves are throttled
  to 2 pt and 8 ms); edit (insert, delete, word delete, autocorrect,
  autocorrect undo, suggestion, double-space period, return and send, paste,
  undo dictation, trackpad cursor); candidates; context
  (documentContextBefore/AfterInput capped at 200 characters, after each word,
  return and dictation and once host changes settle, marked cleared when the
  host empties the field); ui (accents, trackpad, the Hex key and its menu,
  strip chips, globe); dictation (raw, learned fixes, spelling, tagger, Apple
  Intelligence, translation, formatting, inserted text, durations, engine and
  locale, from in-keyboard and app dictation); and dictation_outcome (from a
  second DictationEditTracker). The schema is in ios/docs/typing-logs.md.
- Password, new password, one-time code, every credit card content type, and
  secure fields write one skipped line and nothing else. A new field, or new
  traits, starts a new session.
- TypingLogger buffers events on the main thread. TypingLogStore encodes and
  appends them on a serial utility queue every 2 s, when the keyboard goes
  away, and on a memory warning. Each day is capped at 64 MB with one
  cap_reached line, and files are kept 14 days. When the App Group can't be
  reached (no Full Access), events are dropped silently. Nothing is sent.
- Hooks: FullKeyboardEngine.typingLog, and the KeyboardKeyPlaneView touch
  path, which the UITouch overrides now reach through
  began/moved/ended/cancelled(id:sample:). Every call is skipped while the
  logger is nil, so typing is unchanged with the experiment off.
  KeyboardSnapshot gains optional stages, sent only while the experiment is
  on. KeyboardSpeech.transcribe also returns its locale.
- App: the toggle and a Typing logs screen (each day with its size and event
  count, Share per day, Share all through the share sheet, Delete all with a
  confirmation). Also --snap-state experiments and typing-logs scenes, and an
  --experiment-typing-logs launch argument.

Why
Ryan wants to export a day of raw typing to his Mac, to test and train
prediction models and a per-user touch model (adaptive key targets).

Verified
- xcodebuild test on the iPhone 18 Pro simulator: 177 tests, 0 failures.
  This includes 9 new TypingLogTests: line format and escaping; day rotation
  with layouts in full once per file; Delete all while the writer holds the
  file open; the daily cap (tested at 4 KB) across two writers; 14-day
  retention; the skip rule; a skipped session writing one line; and a real
  FullKeyboardView typing 25 taps through the plane's touch path
  (autocorrect, delete, double-space period, the 123 layer, return, a
  long-press é). That log's edits replay to the field's text, "Don't go yet.
  7 ok\nCafé". An independent Python replay of the same file matched, and
  every lifted key's point was inside its logged hit frame.
- Overhead, simulator Debug, 3,000 taps: about 9 µs a tap on the main thread
  (38 → 47 µs) and about 1 KB a tap (5.2 lines), so about 12 MB an hour at
  200 characters a minute.
- The Release generic-iOS build with CODE_SIGNING_ALLOWED=NO succeeds.
- Simulator screenshots of the toggle and the Typing logs screen.
- Not verified on a physical iPhone: real touch radii and sampling rates, how
  often hosts call textDidChange, whether documentIdentifier stays stable,
  the extension's write cost, and AirDrop from the share sheet.
2026-11-20 02:13:16 +00:00