Files
hex/ios/HEXShared/FullKeyboard/FullKeyboardEngine.swift
Ryan Vogel c14df5a4f0 iOS: spatial autocorrect experiment with Google Books models, plus accent and two-letter fixes (#2059)
* iOS keyboard: Spatial autocorrect experiment (tap + bigram decoder), off by default

What: a noisy-channel word decoder (TypingDecoder) behind Settings ›
Experiments › Spatial autocorrect (experiments.spatialAutocorrect, off
by default; it needs the full keyboard with Autocorrect on). When on, for a layout whose
model is bundled (English, Spanish), it replaces UITextChecker's one-edit
guesses on space and in the strip:
- each tap is scored against every letter with a 2D Gaussian around the
  key (sigma 0.29 x 0.26 of the key pitch), moved by Learn how I type's
  average tap offsets for letters with 20+ taps;
- words come from a beam search over a memory-mapped, sorted word list
  with substitutions, one insertion, deletion or transposition, free
  apostrophes ("dont" -> "don't") and accented letters read from their
  base key (é from e, +1 nat);
- a bigram language model on the word before (stupid backoff) weighs
  them, and a missed space is tried by splitting the taps in two (or
  treating a bottom-row tap as the space);
- what was typed is replaced only when the best reading beats it by 2
  nats, or 7 when it is itself a word the model, UITextChecker, learned
  words or the shorthand list know.
Text replacements, learned fixes, the English contraction/typo table and
words you kept still come first. FullKeyboardEngine now records each
letter's tap point for the word being typed (cleared on anything else).
scripts/build-typing-model.py writes the model format (v2: per-language
alphabet, sorted words, u8 costs, sorted bigram keys); models are kept
out of git (.gitignore). The English prototype model is built from
SymSpell's frequency lists (MIT; Google Books Ngram CC BY 3.0 and SCOWL),
the Spanish one from Tatoeba sentence counts (CC BY 2.0 FR). Typing logs
now record the setting.

Why: Record typing logs show short words and non-words left uncorrected
("tk" for "to"), missed spaces with a typo inside, and o/i taps landing
low on k/j. The UITextChecker path skips 1-2 letter words, never uses
the word before, and can't split a word that also has a substitution.

Verified (anomastudio, iPhone 18 Pro simulator, iOS 27.2): benchmark
harness (kept outside the repo) typing every tap through
FullKeyboardEngine. Synthetic held-out English, 13,870 words: output
word error rate 15.0% today vs 6.7% (6.3% with tap offsets); typos fixed
58% vs 83%; correct words wrongly changed 89 vs 97 of 9,203. Labelled
real typing logs, 172 words: 14.5% vs 4.7%. Spanish held-out, 13,301
words: 26.8% vs 8.1% (Spanish model and test from the same corpus, so
optimistic). Per letter p50 1.0 ms, p95 2.5 ms; mapping the model adds
no measurable phys_footprint. Full HEXMobileTests suite: 221 tests,
0 failures (1 skipped, as before); Release generic-iOS build with
CODE_SIGNING_ALLOWED=NO succeeds with both models bundled. No new tests.

* iOS keyboard: Spatial autocorrect models from Google Books Ngram v3

What: the English and Spanish models (HEXKeyboard/TypingModel/en.hextm,
es.hextm, 3.6 MB each) are now built from Google Books Ngram v3 (CC BY
3.0) word and word-pair counts summed over 1980-2019, keeping only
words SCOWL (sizes up to 70, lowercase, ASCII) or RLA-ES (LibreOffice's
es_ES and es_MX dictionaries, MPL 1.1) accept, and are committed.
- English: 86,133 words, 417,480 word pairs; Spanish: 110,000 words,
  326,366 pairs. Pairs are the ones that tell most over the single-word
  model (count x log of P(b|a) over 0.4 P(b)), as many as fit.
- Format v3: pairs grouped by first word (u32 row starts, u32 second
  word, u8 cost), 5 bytes a pair instead of 9, so ~70% more pairs fit
  in the same size. TypingDecoderModel reads v3 only.
- scripts/build-typing-model.py downloads, counts (each ngram file
  deleted once counted, every stage cached for a resumable rerun) and
  writes a model: python build-typing-model.py <en|es> <workdir> <out>.
- Tuning for the new models (dev sentences): English leans more on the
  word before (lmWeight 1.5; Parameters.tuned(for:)); an accent-only
  change ("esta" -> "está") needs a 2-nat margin instead of 7 (ñ for n
  isn't one: "ano"/"año"); an accented letter also reads from its base
  key when it has its own (ñ from n), whichever is cheaper.
- Settings › Acknowledgements (new row and sheet, AcknowledgementsView)
  and ios/README.md credit Google Books Ngram (CC BY 3.0), SCOWL with its
  and WordNet's notices, and RLA-ES (MPL 1.1).
- AutocorrectEngineTests: decoder ranking cases on the bundled models
  ("need t^o" -> "need to", a missed space with a slip -> "see how",
  correct words kept, Spanish "aqui manana" -> "aquí mañana", "esta"
  kept).

Why: Ryan's licence decision: CC BY with credit, no share-alike, no
non-commercial or GPL/LGPL data, and not SymSpell's word-pair list
(unverified provenance). The Spanish model was built from Tatoeba,
the same corpus as its test set.

Removed: the SymSpell- and Tatoeba-built prototype models and the v2
format (pairs as sorted u64 keys); models are no longer gitignored.

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2; benchmark
harness outside the repo; word error rate on words that aren't names):
English held-out test, 13,870 words: today's engine 13.0%, decoder
3.6% (3.4% with tap offsets), correct words broken 4 -> 2 of 7,716;
labelled real typing logs, 172 words: 14.5% -> 4.1%; the shared
example sentence decodes exactly; Spanish held-out Tatoeba test (no
source shared with the model), 13,301 words: 25.7% -> 8.0%, broken
16 -> 20 of 7,246. Per letter p50 1.0 ms, p95 2.6 ms.
AutocorrectEngineTests 19 passed.

* iOS keyboard: Spatial autocorrect knows your own words

What: the decoder also searches TypingDecoder.UserWords, a small sorted
in-memory list of your words, beside the model: Contacts names and
other UILexicon terms, one-word text replacement outputs, words you
kept twice (LearnedWordsStore) and typo fixes Learn how I type learned.
Each scores as a moderately common word (userWordCost 11 nats) and
comes back as you spell it ("Zorbix", not "zorbix"); typed exactly,
one counts as a word, so it needs the larger margin to change.
AutocorrectEngine rebuilds the list only when a source changes
(revision bumped by loadLexiconEntries, applyLearning, suppressAndKeep)
or the model changes. The search is now generic over TypingDecoderWords
(the model or your words); the model's own narrow moved to it.

Why: a name, brand or word you use that the model lacks could only be
left alone, never reached from a typo.

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2):
AutocorrectEngineTests 20 passed, including a new case: with "Zorbix"
in the lexicon, a slip "zorbiz" becomes "Zorbix" and "zorbix" typed
right stays. Benchmark unchanged (it has no user words).

* iOS keyboard: an accent or tilde is the cheapest autocorrect fix

What: AutocorrectEngine.singleEditKeyboardCost now scores a one-letter
substitution that only adds or changes an accent, diaeresis or tilde
(e/é, u/ü, n/ñ) at 0.15, below every other edit, instead of 1.15 (the
cost of two far-apart keys). New helper sameLetterIgnoringAccents.
Valid words are still never rewritten, so "esta" stays "esta".

Why: Spanish typed without accents was corrected into the wrong word,
because the accented guess cost more than any insertion, deletion or
neighbouring key: "aqui" -> "caqui", "podia" -> "odia", "manana" ->
"manaba". Now they become "aquí", "podía", "mañana".

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, the
benchmark harness kept outside the repo, today's engine with Spatial
autocorrect off): Spanish held-out Tatoeba test, 13,301 words, output
word error rate 26.8% -> 24.7%, words changed into another wrong word
1,015 -> 743, correct words broken 16 -> 16 of 7,246 (names excluded).
English test (13,870 words) unchanged at 15.0%.

* iOS keyboard: fix two-letter typos toward very common two-letter words

What: cautiousReplacement fixes a two-letter non-word (UITextChecker
doesn't know it, not ALL CAPS, no apostrophe) when one neighbouring key
or one swap turns it into a very common two-letter word of the layout's
language (KeyboardLanguage.commonTwoLetterWords, commonest first: "tk"
-> "to", "ot" -> "to", Spanish "nl" -> "no", "ee" -> "de"). English and
Spanish have lists; Norwegian and German have none yet and keep
leaving two-letter words alone. The step numbering in the comments
moves by one.

Why: autocorrect skipped every word under three letters, and the
commonest words are two letters: typing logs show "tk" for "to" left
uncorrected again and again (the o key's taps land low, on k).

Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, benchmark
harness outside the repo, today's engine): English held-out test,
13,870 words, output word error rate 15.0% -> 13.2%; labelled real
typing logs, 172 words, 14.5% -> 10.5%; Spanish test 24.7% -> 22.9%;
correct words broken unchanged (4 of 7,716 English, 16 of 7,246
Spanish, names excluded).
2026-11-20 16:18:07 +00:00

765 lines
30 KiB
Swift
Raw Permalink Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
import QuartzCore
import UIKit
/// The text field the keys type into. `UITextDocumentProxy` in the keyboard; a fake in tests.
@MainActor
protocol KeyboardTextTarget: AnyObject {
var documentContextBeforeInput: String? { get }
var hasText: Bool { get }
var autocapitalizationType: UITextAutocapitalizationType? { get }
var autocorrectionType: UITextAutocorrectionType? { get }
var keyboardType: UIKeyboardType? { get }
var returnKeyType: UIReturnKeyType? { get }
var enablesReturnKeyAutomatically: Bool? { get }
var keyboardAppearance: UIKeyboardAppearance? { get }
var textContentType: UITextContentType? { get }
func insertText(_ text: String)
func deleteBackward()
func adjustTextPosition(byCharacterOffset offset: Int)
}
extension KeyboardTextTarget {
var autocorrectionType: UITextAutocorrectionType? { .default }
var enablesReturnKeyAutomatically: Bool? { false }
var textContentType: UITextContentType? { nil }
func adjustTextPosition(byCharacterOffset offset: Int) {}
}
@MainActor
final class DocumentProxyTextTarget: KeyboardTextTarget {
private let proxy: () -> UITextDocumentProxy?
init(_ proxy: @escaping () -> UITextDocumentProxy?) {
self.proxy = proxy
}
var documentContextBeforeInput: String? { proxy()?.documentContextBeforeInput }
var hasText: Bool { proxy()?.hasText ?? false }
var autocapitalizationType: UITextAutocapitalizationType? { proxy()?.autocapitalizationType }
var autocorrectionType: UITextAutocorrectionType? { proxy()?.autocorrectionType }
var keyboardType: UIKeyboardType? { proxy()?.keyboardType }
var returnKeyType: UIReturnKeyType? { proxy()?.returnKeyType }
var enablesReturnKeyAutomatically: Bool? { proxy()?.enablesReturnKeyAutomatically }
var keyboardAppearance: UIKeyboardAppearance? { proxy()?.keyboardAppearance }
var textContentType: UITextContentType? { proxy()?.textContentType ?? nil }
func insertText(_ text: String) { proxy()?.insertText(text) }
func deleteBackward() { proxy()?.deleteBackward() }
func adjustTextPosition(byCharacterOffset offset: Int) { proxy()?.adjustTextPosition(byCharacterOffset: offset) }
}
/// Everything the keys do, apart from drawing: multitouch with rollover, shift, pages,
/// long-press accents, spacebar cursor trackpad, cautious autocorrect, double-space period,
/// auto-capitalization and accelerating delete. Touches are identified by any stable integer
/// so tests can drive it without `UITouch`.
@MainActor
final class FullKeyboardEngine {
struct ActiveTouch: Equatable {
var key: KeyboardKey
var startPoint: CGPoint
var point: CGPoint
/// A character that already went in because another key went down after it.
var committed: Bool
/// The page a 123/ABC press came from; releasing over a character returns there.
var returnLayer: KeyboardLayer?
/// True once a spacebar press crosses `trackpadActivationDistance`.
var isTrackpad = false
var trackpadAnchorX: CGFloat = 0
/// Characters the trackpad has moved the cursor, for Record typing logs.
var trackpadOffset = 0
}
static let accentHoldDelay: TimeInterval = 0.40
static let accentSlop: CGFloat = 10
static let trackpadActivationDistance: CGFloat = 10
static let trackpadPointsPerCharacter: CGFloat = 7.5
private(set) var shift = ShiftStateMachine()
private(set) var layerState = KeyboardLayerState()
private(set) var field: KeyboardFieldKind = .text
private(set) var touches: [Int: ActiveTouch] = [:]
/// Touch ids in the order they went down.
private var touchOrder: [Int] = []
private var lastSpaceAt: TimeInterval?
/// Where each letter of the word being typed was tapped (nil for one that came another
/// way, such as an accent), for Spatial autocorrect.
private var tokenTaps: [CGPoint?] = []
/// Spatial autocorrect: where you tend to tap each letter, from its centre (Learn how I type).
var tapOffsets: [Character: CGVector] = [:] {
didSet { if tapOffsets != oldValue { syncDecoderKeys() } }
}
private(set) var activeAccentMenu: AccentMenu?
private var activeAccentTouchID: Int?
private(set) var suggestions: [KeyboardSuggestion] = []
let target: KeyboardTextTarget
let autocorrect: AutocorrectEngine
/// Learn how I type (Experiments); nil while off.
var journal: TypingJournal? {
didSet { syncLearning() }
}
private var learning: TypingJournal? {
guard let journal, TypingJournal.allows(field: field, target: target) else { return nil }
return journal
}
/// Record typing logs (Experiments); nil while it is off, in fields it skips, and in the
/// app's fixtures. Every call through it is skipped (arguments too) while nil.
var typingLog: TypingLogger?
var width: CGFloat = 375 { didSet { if width != oldValue { rebuildLayout() } } }
var orientation: KeyboardOrientation = .portrait { didSet { if orientation != oldValue { rebuildLayout() } } }
var showsGlobe = false { didSet { if showsGlobe != oldValue { rebuildLayout() } } }
/// Experiments › Emoji, in fields that take emoji.
var showsEmoji = false { didSet { if showsEmoji != oldValue { rebuildLayout() } } }
var horizontalInset: CGFloat = 0 { didSet { if horizontalInset != oldValue { rebuildLayout() } } }
/// The letters, accents and spelling dictionary (Settings › Keyboard layout).
var language: KeyboardLanguage = .english {
didSet {
guard language != oldValue else { return }
cancelAllTouches()
autocorrect.setLanguage(language)
rebuildLayout(cause: "language")
refreshSuggestions()
}
}
private(set) var layout = KeyboardLayout(width: 375)
var clock: () -> TimeInterval = { CACurrentMediaTime() }
/// Called whenever something visible changed (page, shift, pressed keys, suggestions, accents).
var onChange: (() -> Void)?
/// Called after the keys typed or deleted anything.
var onEdit: (() -> Void)?
/// Called when an accent cell or spacebar trackpad step ticks, for selection haptics.
var onSelectionStep: (() -> Void)?
/// Called just before the return key types its line break, so a send can be noticed.
var onReturn: (() -> Void)?
/// Called when the emoji key is let go.
var onEmojiKey: (() -> Void)?
init(target: KeyboardTextTarget, autocorrect: AutocorrectEngine? = nil) {
self.target = target
self.autocorrect = autocorrect ?? AutocorrectEngine()
rebuildLayout()
}
var isUppercase: Bool { layerState.layer == .letters && shift.isUppercase }
var isTrackpadActive: Bool {
touches.values.contains(where: \.isTrackpad)
}
/// Settings › Key pop-ups. Off: pressed keys only darken, as with iOS's Character Preview off.
var showsKeyPopups = true
/// The key whose balloon shows: the newest character still held (hidden while an accent menu is up).
var previewKey: KeyboardKey? {
guard activeAccentMenu == nil else { return nil }
for id in touchOrder.reversed() {
if let touch = touches[id], touch.key.character != nil, !touch.committed {
return touch.key
}
}
return nil
}
var pressedKinds: Set<KeyKind> {
Set(touches.values.map(\.key.kind))
}
func displayLabel(for key: KeyboardKey) -> String {
guard let character = key.character else { return key.label }
return isUppercase && layerState.layer == .letters ? character.uppercased() : character
}
// MARK: - Field
/// Call when the keyboard appears or the field changes.
func configureForField() {
switch KeyboardFieldRouting.route(for: target.keyboardType) {
case .keys(let kind, let initialLayer):
field = kind
layerState.reset(to: initialLayer)
case .handOff:
field = .text
layerState.reset(to: .numbers)
}
shift.reset()
lastSpaceAt = nil
tokenTaps.removeAll()
autocorrect.clearPendingUndo()
cancelAllTouches()
rebuildLayout(cause: "field")
refreshAutoCapitalization()
refreshSuggestions()
}
/// Call after the text or the cursor changed outside these keys.
func textDidChange() {
lastSpaceAt = nil
tokenTaps.removeAll()
autocorrect.clearPendingUndo()
refreshAutoCapitalization()
refreshSuggestions()
}
func refreshAutoCapitalization() {
let before = shift
shift.refreshAutoCapitalization(
AutoCapitalization.shouldCapitalize(
mode: AutoCapitalization.Mode(target.autocapitalizationType),
before: target.documentContextBeforeInput,
hasText: target.hasText
)
)
if shift != before {
logState("auto")
onChange?()
}
}
func refreshSuggestions() {
syncLearning()
autocorrect.decoderTaps = currentTokenTaps
let next = autocorrect.suggestions(
beforeInput: target.documentContextBeforeInput,
field: field,
autocorrectionType: target.autocorrectionType
)
if next != suggestions {
suggestions = next
typingLog?.suggestionsChanged(next)
onChange?()
}
}
func acceptSuggestion(_ suggestion: KeyboardSuggestion) {
if let log = typingLog {
let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput) ?? ""
log.edit("suggestion", [
TypingLogField("slot", suggestions.firstIndex(of: suggestion).map { .int($0) } ?? .null),
TypingLogField("text", .string(suggestion.displayText)),
TypingLogField("deleted", .string(token)),
TypingLogField("inserted", .string(suggestion.replacement + " ")),
TypingLogField("candidates", TypingLogger.candidates(suggestions)),
])
}
if let journal = learning {
let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput)
if let token, !token.isEmpty, token != suggestion.replacement {
journal.record(.picked, typed: token, final: suggestion.replacement)
} else {
journal.didFinish(word: token)
// Tapping what you typed over the autocorrection space would apply turns it down.
if suggestion.kind == .keepLiteral, let token,
let autoFix = suggestions.first(where: \.willAutoReplace) {
journal.record(.keptTyped, typed: autoFix.replacement, final: token)
}
}
}
if let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput) {
for _ in 0..<token.count {
target.deleteBackward()
}
if suggestion.kind == .keepLiteral {
autocorrect.suppressAndKeep(token)
}
}
autocorrect.clearPendingUndo()
insert(suggestion.replacement + " ", source: nil)
typingLog?.context("suggestion")
}
/// Types an emoji from the emoji panel: no spacing or autocorrect around it.
func insertEmoji(_ text: String, source: String) {
autocorrect.clearPendingUndo()
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
insert(text, source: source)
}
// MARK: - Touches
func touchBegan(id: Int, at point: CGPoint) {
guard let key = layout.hitTest(point) else { return }
// Rollover: a new key going down commits any character or accent picker still held.
for other in touchOrder {
guard var touch = touches[other], touch.key.character != nil, !touch.committed else { continue }
if activeAccentTouchID == other, let menu = activeAccentMenu {
typingLog?.ui("accent_pick", [
TypingLogField("id", .int(other)),
TypingLogField("index", .int(menu.selectedIndex)),
TypingLogField("chosen", .string(menu.selectedOption)),
TypingLogField("rollover", .bool(true)),
])
commitAccentSelection(menu.selectedOption)
activeAccentMenu = nil
activeAccentTouchID = nil
} else {
commitCharacter(touch.key, at: touch.startPoint)
}
touch.committed = true
touches[other] = touch
}
var touch = ActiveTouch(
key: key,
startPoint: point,
point: point,
committed: false,
returnLayer: nil
)
if key.character != nil {
learning?.recordTap(on: key, at: point)
}
switch key.kind {
case .shift:
shift.shiftDown(now: clock())
logState("shift_key")
case .delete:
deleteBackward(allowAutocorrectUndo: true)
case .layerSwitch(let next):
touch.returnLayer = layerState.layer
layerState.switchTo(next)
if next == .letters { shift.reset() }
rebuildLayout(cause: "layer_key")
if next == .letters { refreshAutoCapitalization() }
case .character, .space, .returnKey, .hex, .globe, .emoji:
break
}
touches[id] = touch
touchOrder.append(id)
onChange?()
}
@discardableResult
func openAccentMenuIfEligible(forTouch id: Int) -> Bool {
guard activeAccentMenu == nil,
let touch = touches[id],
!touch.committed,
let base = touch.key.character else { return false }
let dx = touch.point.x - touch.startPoint.x
let dy = touch.point.y - touch.startPoint.y
guard (dx * dx + dy * dy).squareRoot() <= Self.accentSlop else { return false }
guard let options = AccentCatalog.options(for: base, uppercase: isUppercase, language: language),
options.count > 1 else { return false }
activeAccentMenu = AccentMenu.build(
for: touch.key.frame,
rawOptions: options,
in: layout.width,
horizontalInset: layout.horizontalInset
)
activeAccentTouchID = id
typingLog?.ui("accent_open", [
TypingLogField("id", .int(id)),
TypingLogField("key", .string(TypingLogger.keyID(touch.key.kind))),
TypingLogField("options", .strings(activeAccentMenu?.options ?? [])),
])
onChange?()
return true
}
func touchMoved(id: Int, to point: CGPoint) {
guard var touch = touches[id] else { return }
touch.point = point
// 1. Active long-press accent picker: horizontal drag selects the alternate character.
if activeAccentTouchID == id, var menu = activeAccentMenu {
let nextIndex = menu.index(for: point.x)
touches[id] = touch
if nextIndex != menu.selectedIndex {
menu.selectedIndex = nextIndex
activeAccentMenu = menu
onSelectionStep?()
onChange?()
}
return
}
// 2. Spacebar cursor trackpad: horizontal drag >= 10 pt slides the text caret.
if touch.key.kind == .space {
if !touch.isTrackpad, abs(point.x - touch.startPoint.x) >= Self.trackpadActivationDistance {
touch.isTrackpad = true
touch.trackpadAnchorX = point.x
lastSpaceAt = nil
autocorrect.clearPendingUndo()
touches[id] = touch
typingLog?.ui("trackpad_start", [TypingLogField("id", .int(id))])
onSelectionStep?()
onChange?()
return
}
if touch.isTrackpad {
let delta = point.x - touch.trackpadAnchorX
let rawSteps = Int((delta / Self.trackpadPointsPerCharacter).rounded(.towardZero))
if rawSteps != 0 {
touch.trackpadAnchorX += CGFloat(rawSteps) * Self.trackpadPointsPerCharacter
touch.trackpadOffset += rawSteps
touches[id] = touch
typingLog?.edit("cursor", [TypingLogField("offset", .int(rawSteps))])
tokenTaps.removeAll()
target.adjustTextPosition(byCharacterOffset: rawSteps)
onSelectionStep?()
refreshAutoCapitalization()
refreshSuggestions()
return
}
touches[id] = touch
return
}
}
// 3. Sliding across character keys updates the active key and re-anchors the hold timer.
if touch.key.character != nil, !touch.committed,
let under = layout.hitTest(point), under.character != nil, under.kind != touch.key.kind {
touch.key = under
touch.startPoint = point
touches[id] = touch
onChange?()
return
}
touches[id] = touch
}
func touchEnded(id: Int, at point: CGPoint) {
guard var touch = touches.removeValue(forKey: id) else { return }
touchOrder.removeAll { $0 == id }
touch.point = point
if activeAccentTouchID == id, let menu = activeAccentMenu {
let index = menu.index(for: point.x)
let chosen = menu.options[index]
activeAccentMenu = nil
activeAccentTouchID = nil
if !touch.committed {
typingLog?.ui("accent_pick", [
TypingLogField("id", .int(id)),
TypingLogField("index", .int(index)),
TypingLogField("chosen", .string(chosen)),
])
commitAccentSelection(chosen)
}
onChange?()
return
}
switch touch.key.kind {
case .character:
if !touch.committed {
commitCharacter(touch.key, at: touch.startPoint)
}
case .shift:
shift.shiftUp()
logState("shift_key")
case .space:
if touch.isTrackpad {
typingLog?.ui("trackpad_end", [
TypingLogField("id", .int(id)),
TypingLogField("offset", .int(touch.trackpadOffset)),
])
typingLog?.context("trackpad")
} else {
typeSpace()
}
case .returnKey:
if target.enablesReturnKeyAutomatically == true && !target.hasText {
typingLog?.edit("return", [
TypingLogField("returnKeyType", .string(TypingLogTraits.name(target.returnKeyType ?? .default))),
TypingLogField("ignored", .bool(true)),
])
break
}
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
autocorrect.clearPendingUndo()
typingLog?.edit("return", [
TypingLogField("inserted", .string("\n")),
TypingLogField("returnKeyType", .string(TypingLogTraits.name(target.returnKeyType ?? .default))),
TypingLogField("send", .bool(target.returnKeyType == .send)),
])
onReturn?()
insert("\n", source: nil)
typingLog?.context("return")
case .layerSwitch:
// Hold 123, slide to a key, let go: type it and go back.
if let returnLayer = touch.returnLayer,
let under = layout.hitTest(point),
let character = under.character,
under.kind != touch.key.kind {
autocorrect.clearPendingUndo()
insert(character, source: "slide")
layerState.switchTo(returnLayer)
rebuildLayout(cause: "layer_slide")
refreshAutoCapitalization()
}
case .emoji:
onEmojiKey?()
case .delete, .hex, .globe:
break
}
onChange?()
}
func touchCancelled(id: Int) {
if activeAccentTouchID == id {
activeAccentMenu = nil
activeAccentTouchID = nil
typingLog?.ui("accent_cancel", [TypingLogField("id", .int(id))])
}
guard touches.removeValue(forKey: id) != nil else { return }
touchOrder.removeAll { $0 == id }
onChange?()
}
func cancelAllTouches() {
let hadTouches = !touches.isEmpty || activeAccentMenu != nil
touches.removeAll()
touchOrder.removeAll()
activeAccentMenu = nil
activeAccentTouchID = nil
if shift.isHeld {
shift.shiftUp()
logState("cancel")
}
if hadTouches { onChange?() }
}
// MARK: - Delete
/// The delete key repeats after 360 ms, accelerates at 850 ms, and removes whole words after 1.4 s.
static let repeatDelay: TimeInterval = 0.36
static let repeatInterval: TimeInterval = 0.075
static let fastRepeatAfter: TimeInterval = 0.85
static let wordDeleteAfter: TimeInterval = 1.4
func repeatDelete(heldFor: TimeInterval) {
autocorrect.clearPendingUndo()
if heldFor >= Self.wordDeleteAfter {
deleteWordBackward()
} else if heldFor >= Self.fastRepeatAfter {
deleteBackward(allowAutocorrectUndo: false, source: "repeat_fast")
deleteBackward(allowAutocorrectUndo: false, source: "repeat_fast")
} else {
deleteBackward(allowAutocorrectUndo: false, source: "repeat")
}
}
func deleteBackward(allowAutocorrectUndo: Bool = true, source: String = "key") {
if allowAutocorrectUndo,
let undo = autocorrect.consumePendingUndo(),
let before = target.documentContextBeforeInput,
before.hasSuffix(undo.insertedText) {
typingLog?.edit("autocorrect_undo", [
TypingLogField("deleted", .string(undo.insertedText)),
TypingLogField("inserted", .string(undo.original)),
])
for _ in 0..<undo.insertedText.count {
target.deleteBackward()
}
tokenTaps.removeAll()
target.insertText(undo.original)
autocorrect.suppressAndKeep(undo.original)
learning?.record(
.undone,
typed: undo.insertedText.trimmingCharacters(in: .whitespaces),
final: undo.original
)
lastSpaceAt = nil
didEdit()
return
}
autocorrect.clearPendingUndo()
learning?.willErase(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
if let log = typingLog {
let deleted = target.documentContextBeforeInput?.last.map { String($0) }
log.edit("delete", [
deleted.map { TypingLogField("deleted", .string($0)) } ?? TypingLogField("deletedCount", .int(1)),
TypingLogField("src", .string(source)),
])
}
if let last = target.documentContextBeforeInput?.last, last.isLetter || last == "'", !tokenTaps.isEmpty {
tokenTaps.removeLast()
} else {
tokenTaps.removeAll()
}
target.deleteBackward()
lastSpaceAt = nil
didEdit()
}
func deleteWordBackward() {
autocorrect.clearPendingUndo()
guard let before = target.documentContextBeforeInput, !before.isEmpty else {
deleteBackward(allowAutocorrectUndo: false, source: "repeat_word")
return
}
learning?.willErase(word: AutocorrectEngine.currentToken(in: before))
var count = 0
var index = before.endIndex
while index > before.startIndex, before[before.index(before: index)].isWhitespace {
index = before.index(before: index)
count += 1
}
while index > before.startIndex, !before[before.index(before: index)].isWhitespace {
index = before.index(before: index)
count += 1
}
tokenTaps.removeAll()
typingLog?.edit("delete_word", [
TypingLogField("deleted", .string(String(before.suffix(max(1, count))))),
TypingLogField("src", .string("repeat_word")),
])
for _ in 0..<max(1, count) {
target.deleteBackward()
}
lastSpaceAt = nil
didEdit()
typingLog?.context("delete_word")
}
// MARK: - Typing
private func commitCharacter(_ key: KeyboardKey, at point: CGPoint) {
guard let character = key.character else { return }
let text = layerState.layer == .letters && shift.isUppercase ? character.uppercased() : character
if layerState.layer == .letters {
shift.didType()
}
autocorrect.clearPendingUndo()
insert(text, source: "key", at: point)
logState("typed")
}
/// The taps of the word being typed when every letter came from a key, else nil.
private var currentTokenTaps: [CGPoint]? {
let taps = tokenTaps.compactMap { $0 }
return taps.count == tokenTaps.count && !taps.isEmpty ? taps : nil
}
private func commitAccentSelection(_ chosen: String) {
if layerState.layer == .letters {
shift.didType()
}
autocorrect.clearPendingUndo()
insert(chosen, source: "accent")
logState("typed")
}
private func typeSpace() {
let now = clock()
if DoubleSpacePeriod.shouldReplace(before: target.documentContextBeforeInput, lastSpaceAt: lastSpaceAt, now: now) {
autocorrect.clearPendingUndo()
typingLog?.edit("double_space_period", [
TypingLogField("deleted", .string(" ")),
TypingLogField("inserted", .string(". ")),
])
target.deleteBackward()
target.insertText(". ")
tokenTaps.removeAll()
lastSpaceAt = nil
layerState.didInsert(" ")
rebuildLayoutIfNeeded()
didEdit()
typingLog?.context("space")
return
}
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
syncLearning()
autocorrect.decoderTaps = currentTokenTaps
if let (original, replacement) = autocorrect.replacementOnSpace(
beforeInput: target.documentContextBeforeInput,
field: field,
autocorrectionType: target.autocorrectionType
) {
for _ in 0..<original.count {
target.deleteBackward()
}
let inserted = replacement + " "
tokenTaps.removeAll()
typingLog?.edit("autocorrect", [
TypingLogField("from", .string(original)),
TypingLogField("to", .string(replacement)),
TypingLogField("deleted", .string(original)),
TypingLogField("inserted", .string(inserted)),
])
target.insertText(inserted)
autocorrect.recordAppliedCorrection(original: original, insertedText: inserted)
learning?.record(.autocorrected, typed: original, final: replacement)
lastSpaceAt = now
layerState.didInsert(" ")
rebuildLayoutIfNeeded()
didEdit()
typingLog?.context("space")
return
}
autocorrect.clearPendingUndo()
insert(" ", source: "key")
lastSpaceAt = now
typingLog?.context("space")
}
/// `source` names the edit in typing logs; nil when the caller writes its own line.
private func insert(_ text: String, source: String?, at point: CGPoint? = nil) {
if text.count == 1, text.first?.isLetter == true {
tokenTaps.append(point)
} else {
tokenTaps.removeAll()
}
if let source {
typingLog?.edit("insert", [
TypingLogField("inserted", .string(text)),
TypingLogField("src", .string(source)),
])
}
target.insertText(text)
lastSpaceAt = nil
layerState.didInsert(text)
rebuildLayoutIfNeeded()
didEdit()
}
/// Hands what the journal learned (typos, rejected corrections) to autocorrect;
/// nothing while Learn how I type is off.
private func syncLearning() {
autocorrect.applyLearning(journal?.typingLearning ?? TypingLearning())
}
private func didEdit() {
refreshAutoCapitalization()
refreshSuggestions()
onEdit?()
}
// MARK: - Layout
private func rebuildLayoutIfNeeded() {
if layout.layer != layerState.layer {
rebuildLayout(cause: "layer_auto")
}
}
private func rebuildLayout(cause: String = "layout") {
layout = KeyboardLayout(
width: width,
orientation: orientation,
layer: layerState.layer,
field: field,
showsGlobe: showsGlobe,
showsEmoji: showsEmoji,
horizontalInset: horizontalInset,
language: language
)
syncDecoderKeys()
logState(cause)
}
/// The letter keys Spatial autocorrect scores taps against: this layout's, moved by
/// `tapOffsets`. None off the letters page.
private func syncDecoderKeys() {
autocorrect.decoderKeys = TypingDecoder.Keys(layout: layout, offsets: tapOffsets)
}
/// Hands the layout and shift to typing logs, which writes a line when either changed.
func logState(_ cause: String) {
typingLog?.keyboardState(layout: layout, shift: shift, cause: cause)
}
}