mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-22 16:38:03 +08:00
* iOS keyboard: Spatial autocorrect experiment (tap + bigram decoder), off by default
What: a noisy-channel word decoder (TypingDecoder) behind Settings ›
Experiments › Spatial autocorrect (experiments.spatialAutocorrect, off
by default; it needs the full keyboard with Autocorrect on). When on, for a layout whose
model is bundled (English, Spanish), it replaces UITextChecker's one-edit
guesses on space and in the strip:
- each tap is scored against every letter with a 2D Gaussian around the
key (sigma 0.29 x 0.26 of the key pitch), moved by Learn how I type's
average tap offsets for letters with 20+ taps;
- words come from a beam search over a memory-mapped, sorted word list
with substitutions, one insertion, deletion or transposition, free
apostrophes ("dont" -> "don't") and accented letters read from their
base key (é from e, +1 nat);
- a bigram language model on the word before (stupid backoff) weighs
them, and a missed space is tried by splitting the taps in two (or
treating a bottom-row tap as the space);
- what was typed is replaced only when the best reading beats it by 2
nats, or 7 when it is itself a word the model, UITextChecker, learned
words or the shorthand list know.
Text replacements, learned fixes, the English contraction/typo table and
words you kept still come first. FullKeyboardEngine now records each
letter's tap point for the word being typed (cleared on anything else).
scripts/build-typing-model.py writes the model format (v2: per-language
alphabet, sorted words, u8 costs, sorted bigram keys); models are kept
out of git (.gitignore). The English prototype model is built from
SymSpell's frequency lists (MIT; Google Books Ngram CC BY 3.0 and SCOWL),
the Spanish one from Tatoeba sentence counts (CC BY 2.0 FR). Typing logs
now record the setting.
Why: Record typing logs show short words and non-words left uncorrected
("tk" for "to"), missed spaces with a typo inside, and o/i taps landing
low on k/j. The UITextChecker path skips 1-2 letter words, never uses
the word before, and can't split a word that also has a substitution.
Verified (anomastudio, iPhone 18 Pro simulator, iOS 27.2): benchmark
harness (kept outside the repo) typing every tap through
FullKeyboardEngine. Synthetic held-out English, 13,870 words: output
word error rate 15.0% today vs 6.7% (6.3% with tap offsets); typos fixed
58% vs 83%; correct words wrongly changed 89 vs 97 of 9,203. Labelled
real typing logs, 172 words: 14.5% vs 4.7%. Spanish held-out, 13,301
words: 26.8% vs 8.1% (Spanish model and test from the same corpus, so
optimistic). Per letter p50 1.0 ms, p95 2.5 ms; mapping the model adds
no measurable phys_footprint. Full HEXMobileTests suite: 221 tests,
0 failures (1 skipped, as before); Release generic-iOS build with
CODE_SIGNING_ALLOWED=NO succeeds with both models bundled. No new tests.
* iOS keyboard: Spatial autocorrect models from Google Books Ngram v3
What: the English and Spanish models (HEXKeyboard/TypingModel/en.hextm,
es.hextm, 3.6 MB each) are now built from Google Books Ngram v3 (CC BY
3.0) word and word-pair counts summed over 1980-2019, keeping only
words SCOWL (sizes up to 70, lowercase, ASCII) or RLA-ES (LibreOffice's
es_ES and es_MX dictionaries, MPL 1.1) accept, and are committed.
- English: 86,133 words, 417,480 word pairs; Spanish: 110,000 words,
326,366 pairs. Pairs are the ones that tell most over the single-word
model (count x log of P(b|a) over 0.4 P(b)), as many as fit.
- Format v3: pairs grouped by first word (u32 row starts, u32 second
word, u8 cost), 5 bytes a pair instead of 9, so ~70% more pairs fit
in the same size. TypingDecoderModel reads v3 only.
- scripts/build-typing-model.py downloads, counts (each ngram file
deleted once counted, every stage cached for a resumable rerun) and
writes a model: python build-typing-model.py <en|es> <workdir> <out>.
- Tuning for the new models (dev sentences): English leans more on the
word before (lmWeight 1.5; Parameters.tuned(for:)); an accent-only
change ("esta" -> "está") needs a 2-nat margin instead of 7 (ñ for n
isn't one: "ano"/"año"); an accented letter also reads from its base
key when it has its own (ñ from n), whichever is cheaper.
- Settings › Acknowledgements (new row and sheet, AcknowledgementsView)
and ios/README.md credit Google Books Ngram (CC BY 3.0), SCOWL with its
and WordNet's notices, and RLA-ES (MPL 1.1).
- AutocorrectEngineTests: decoder ranking cases on the bundled models
("need t^o" -> "need to", a missed space with a slip -> "see how",
correct words kept, Spanish "aqui manana" -> "aquí mañana", "esta"
kept).
Why: Ryan's licence decision: CC BY with credit, no share-alike, no
non-commercial or GPL/LGPL data, and not SymSpell's word-pair list
(unverified provenance). The Spanish model was built from Tatoeba,
the same corpus as its test set.
Removed: the SymSpell- and Tatoeba-built prototype models and the v2
format (pairs as sorted u64 keys); models are no longer gitignored.
Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2; benchmark
harness outside the repo; word error rate on words that aren't names):
English held-out test, 13,870 words: today's engine 13.0%, decoder
3.6% (3.4% with tap offsets), correct words broken 4 -> 2 of 7,716;
labelled real typing logs, 172 words: 14.5% -> 4.1%; the shared
example sentence decodes exactly; Spanish held-out Tatoeba test (no
source shared with the model), 13,301 words: 25.7% -> 8.0%, broken
16 -> 20 of 7,246. Per letter p50 1.0 ms, p95 2.6 ms.
AutocorrectEngineTests 19 passed.
* iOS keyboard: Spatial autocorrect knows your own words
What: the decoder also searches TypingDecoder.UserWords, a small sorted
in-memory list of your words, beside the model: Contacts names and
other UILexicon terms, one-word text replacement outputs, words you
kept twice (LearnedWordsStore) and typo fixes Learn how I type learned.
Each scores as a moderately common word (userWordCost 11 nats) and
comes back as you spell it ("Zorbix", not "zorbix"); typed exactly,
one counts as a word, so it needs the larger margin to change.
AutocorrectEngine rebuilds the list only when a source changes
(revision bumped by loadLexiconEntries, applyLearning, suppressAndKeep)
or the model changes. The search is now generic over TypingDecoderWords
(the model or your words); the model's own narrow moved to it.
Why: a name, brand or word you use that the model lacks could only be
left alone, never reached from a typo.
Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2):
AutocorrectEngineTests 20 passed, including a new case: with "Zorbix"
in the lexicon, a slip "zorbiz" becomes "Zorbix" and "zorbix" typed
right stays. Benchmark unchanged (it has no user words).
* iOS keyboard: an accent or tilde is the cheapest autocorrect fix
What: AutocorrectEngine.singleEditKeyboardCost now scores a one-letter
substitution that only adds or changes an accent, diaeresis or tilde
(e/é, u/ü, n/ñ) at 0.15, below every other edit, instead of 1.15 (the
cost of two far-apart keys). New helper sameLetterIgnoringAccents.
Valid words are still never rewritten, so "esta" stays "esta".
Why: Spanish typed without accents was corrected into the wrong word,
because the accented guess cost more than any insertion, deletion or
neighbouring key: "aqui" -> "caqui", "podia" -> "odia", "manana" ->
"manaba". Now they become "aquí", "podía", "mañana".
Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, the
benchmark harness kept outside the repo, today's engine with Spatial
autocorrect off): Spanish held-out Tatoeba test, 13,301 words, output
word error rate 26.8% -> 24.7%, words changed into another wrong word
1,015 -> 743, correct words broken 16 -> 16 of 7,246 (names excluded).
English test (13,870 words) unchanged at 15.0%.
* iOS keyboard: fix two-letter typos toward very common two-letter words
What: cautiousReplacement fixes a two-letter non-word (UITextChecker
doesn't know it, not ALL CAPS, no apostrophe) when one neighbouring key
or one swap turns it into a very common two-letter word of the layout's
language (KeyboardLanguage.commonTwoLetterWords, commonest first: "tk"
-> "to", "ot" -> "to", Spanish "nl" -> "no", "ee" -> "de"). English and
Spanish have lists; Norwegian and German have none yet and keep
leaving two-letter words alone. The step numbering in the comments
moves by one.
Why: autocorrect skipped every word under three letters, and the
commonest words are two letters: typing logs show "tk" for "to" left
uncorrected again and again (the o key's taps land low, on k).
Verified (Mac Studio, iPhone 18 Pro simulator, iOS 27.2, benchmark
harness outside the repo, today's engine): English held-out test,
13,870 words, output word error rate 15.0% -> 13.2%; labelled real
typing logs, 172 words, 14.5% -> 10.5%; Spanish test 24.7% -> 22.9%;
correct words broken unchanged (4 of 7,716 English, 16 of 7,246
Spanish, names excluded).
765 lines
30 KiB
Swift
765 lines
30 KiB
Swift
import QuartzCore
|
||
import UIKit
|
||
|
||
/// The text field the keys type into. `UITextDocumentProxy` in the keyboard; a fake in tests.
|
||
@MainActor
|
||
protocol KeyboardTextTarget: AnyObject {
|
||
var documentContextBeforeInput: String? { get }
|
||
var hasText: Bool { get }
|
||
var autocapitalizationType: UITextAutocapitalizationType? { get }
|
||
var autocorrectionType: UITextAutocorrectionType? { get }
|
||
var keyboardType: UIKeyboardType? { get }
|
||
var returnKeyType: UIReturnKeyType? { get }
|
||
var enablesReturnKeyAutomatically: Bool? { get }
|
||
var keyboardAppearance: UIKeyboardAppearance? { get }
|
||
var textContentType: UITextContentType? { get }
|
||
func insertText(_ text: String)
|
||
func deleteBackward()
|
||
func adjustTextPosition(byCharacterOffset offset: Int)
|
||
}
|
||
|
||
extension KeyboardTextTarget {
|
||
var autocorrectionType: UITextAutocorrectionType? { .default }
|
||
var enablesReturnKeyAutomatically: Bool? { false }
|
||
var textContentType: UITextContentType? { nil }
|
||
func adjustTextPosition(byCharacterOffset offset: Int) {}
|
||
}
|
||
|
||
@MainActor
|
||
final class DocumentProxyTextTarget: KeyboardTextTarget {
|
||
private let proxy: () -> UITextDocumentProxy?
|
||
|
||
init(_ proxy: @escaping () -> UITextDocumentProxy?) {
|
||
self.proxy = proxy
|
||
}
|
||
|
||
var documentContextBeforeInput: String? { proxy()?.documentContextBeforeInput }
|
||
var hasText: Bool { proxy()?.hasText ?? false }
|
||
var autocapitalizationType: UITextAutocapitalizationType? { proxy()?.autocapitalizationType }
|
||
var autocorrectionType: UITextAutocorrectionType? { proxy()?.autocorrectionType }
|
||
var keyboardType: UIKeyboardType? { proxy()?.keyboardType }
|
||
var returnKeyType: UIReturnKeyType? { proxy()?.returnKeyType }
|
||
var enablesReturnKeyAutomatically: Bool? { proxy()?.enablesReturnKeyAutomatically }
|
||
var keyboardAppearance: UIKeyboardAppearance? { proxy()?.keyboardAppearance }
|
||
var textContentType: UITextContentType? { proxy()?.textContentType ?? nil }
|
||
func insertText(_ text: String) { proxy()?.insertText(text) }
|
||
func deleteBackward() { proxy()?.deleteBackward() }
|
||
func adjustTextPosition(byCharacterOffset offset: Int) { proxy()?.adjustTextPosition(byCharacterOffset: offset) }
|
||
}
|
||
|
||
/// Everything the keys do, apart from drawing: multitouch with rollover, shift, pages,
|
||
/// long-press accents, spacebar cursor trackpad, cautious autocorrect, double-space period,
|
||
/// auto-capitalization and accelerating delete. Touches are identified by any stable integer
|
||
/// so tests can drive it without `UITouch`.
|
||
@MainActor
|
||
final class FullKeyboardEngine {
|
||
struct ActiveTouch: Equatable {
|
||
var key: KeyboardKey
|
||
var startPoint: CGPoint
|
||
var point: CGPoint
|
||
/// A character that already went in because another key went down after it.
|
||
var committed: Bool
|
||
/// The page a 123/ABC press came from; releasing over a character returns there.
|
||
var returnLayer: KeyboardLayer?
|
||
/// True once a spacebar press crosses `trackpadActivationDistance`.
|
||
var isTrackpad = false
|
||
var trackpadAnchorX: CGFloat = 0
|
||
/// Characters the trackpad has moved the cursor, for Record typing logs.
|
||
var trackpadOffset = 0
|
||
}
|
||
|
||
static let accentHoldDelay: TimeInterval = 0.40
|
||
static let accentSlop: CGFloat = 10
|
||
static let trackpadActivationDistance: CGFloat = 10
|
||
static let trackpadPointsPerCharacter: CGFloat = 7.5
|
||
|
||
private(set) var shift = ShiftStateMachine()
|
||
private(set) var layerState = KeyboardLayerState()
|
||
private(set) var field: KeyboardFieldKind = .text
|
||
private(set) var touches: [Int: ActiveTouch] = [:]
|
||
/// Touch ids in the order they went down.
|
||
private var touchOrder: [Int] = []
|
||
private var lastSpaceAt: TimeInterval?
|
||
|
||
/// Where each letter of the word being typed was tapped (nil for one that came another
|
||
/// way, such as an accent), for Spatial autocorrect.
|
||
private var tokenTaps: [CGPoint?] = []
|
||
/// Spatial autocorrect: where you tend to tap each letter, from its centre (Learn how I type).
|
||
var tapOffsets: [Character: CGVector] = [:] {
|
||
didSet { if tapOffsets != oldValue { syncDecoderKeys() } }
|
||
}
|
||
|
||
private(set) var activeAccentMenu: AccentMenu?
|
||
private var activeAccentTouchID: Int?
|
||
private(set) var suggestions: [KeyboardSuggestion] = []
|
||
|
||
let target: KeyboardTextTarget
|
||
let autocorrect: AutocorrectEngine
|
||
/// Learn how I type (Experiments); nil while off.
|
||
var journal: TypingJournal? {
|
||
didSet { syncLearning() }
|
||
}
|
||
|
||
private var learning: TypingJournal? {
|
||
guard let journal, TypingJournal.allows(field: field, target: target) else { return nil }
|
||
return journal
|
||
}
|
||
/// Record typing logs (Experiments); nil while it is off, in fields it skips, and in the
|
||
/// app's fixtures. Every call through it is skipped (arguments too) while nil.
|
||
var typingLog: TypingLogger?
|
||
var width: CGFloat = 375 { didSet { if width != oldValue { rebuildLayout() } } }
|
||
var orientation: KeyboardOrientation = .portrait { didSet { if orientation != oldValue { rebuildLayout() } } }
|
||
var showsGlobe = false { didSet { if showsGlobe != oldValue { rebuildLayout() } } }
|
||
/// Experiments › Emoji, in fields that take emoji.
|
||
var showsEmoji = false { didSet { if showsEmoji != oldValue { rebuildLayout() } } }
|
||
var horizontalInset: CGFloat = 0 { didSet { if horizontalInset != oldValue { rebuildLayout() } } }
|
||
/// The letters, accents and spelling dictionary (Settings › Keyboard layout).
|
||
var language: KeyboardLanguage = .english {
|
||
didSet {
|
||
guard language != oldValue else { return }
|
||
cancelAllTouches()
|
||
autocorrect.setLanguage(language)
|
||
rebuildLayout(cause: "language")
|
||
refreshSuggestions()
|
||
}
|
||
}
|
||
private(set) var layout = KeyboardLayout(width: 375)
|
||
|
||
var clock: () -> TimeInterval = { CACurrentMediaTime() }
|
||
/// Called whenever something visible changed (page, shift, pressed keys, suggestions, accents).
|
||
var onChange: (() -> Void)?
|
||
/// Called after the keys typed or deleted anything.
|
||
var onEdit: (() -> Void)?
|
||
/// Called when an accent cell or spacebar trackpad step ticks, for selection haptics.
|
||
var onSelectionStep: (() -> Void)?
|
||
/// Called just before the return key types its line break, so a send can be noticed.
|
||
var onReturn: (() -> Void)?
|
||
/// Called when the emoji key is let go.
|
||
var onEmojiKey: (() -> Void)?
|
||
|
||
init(target: KeyboardTextTarget, autocorrect: AutocorrectEngine? = nil) {
|
||
self.target = target
|
||
self.autocorrect = autocorrect ?? AutocorrectEngine()
|
||
rebuildLayout()
|
||
}
|
||
|
||
var isUppercase: Bool { layerState.layer == .letters && shift.isUppercase }
|
||
|
||
var isTrackpadActive: Bool {
|
||
touches.values.contains(where: \.isTrackpad)
|
||
}
|
||
|
||
/// Settings › Key pop-ups. Off: pressed keys only darken, as with iOS's Character Preview off.
|
||
var showsKeyPopups = true
|
||
|
||
/// The key whose balloon shows: the newest character still held (hidden while an accent menu is up).
|
||
var previewKey: KeyboardKey? {
|
||
guard activeAccentMenu == nil else { return nil }
|
||
for id in touchOrder.reversed() {
|
||
if let touch = touches[id], touch.key.character != nil, !touch.committed {
|
||
return touch.key
|
||
}
|
||
}
|
||
return nil
|
||
}
|
||
|
||
var pressedKinds: Set<KeyKind> {
|
||
Set(touches.values.map(\.key.kind))
|
||
}
|
||
|
||
func displayLabel(for key: KeyboardKey) -> String {
|
||
guard let character = key.character else { return key.label }
|
||
return isUppercase && layerState.layer == .letters ? character.uppercased() : character
|
||
}
|
||
|
||
// MARK: - Field
|
||
|
||
/// Call when the keyboard appears or the field changes.
|
||
func configureForField() {
|
||
switch KeyboardFieldRouting.route(for: target.keyboardType) {
|
||
case .keys(let kind, let initialLayer):
|
||
field = kind
|
||
layerState.reset(to: initialLayer)
|
||
case .handOff:
|
||
field = .text
|
||
layerState.reset(to: .numbers)
|
||
}
|
||
shift.reset()
|
||
lastSpaceAt = nil
|
||
tokenTaps.removeAll()
|
||
autocorrect.clearPendingUndo()
|
||
cancelAllTouches()
|
||
rebuildLayout(cause: "field")
|
||
refreshAutoCapitalization()
|
||
refreshSuggestions()
|
||
}
|
||
|
||
/// Call after the text or the cursor changed outside these keys.
|
||
func textDidChange() {
|
||
lastSpaceAt = nil
|
||
tokenTaps.removeAll()
|
||
autocorrect.clearPendingUndo()
|
||
refreshAutoCapitalization()
|
||
refreshSuggestions()
|
||
}
|
||
|
||
func refreshAutoCapitalization() {
|
||
let before = shift
|
||
shift.refreshAutoCapitalization(
|
||
AutoCapitalization.shouldCapitalize(
|
||
mode: AutoCapitalization.Mode(target.autocapitalizationType),
|
||
before: target.documentContextBeforeInput,
|
||
hasText: target.hasText
|
||
)
|
||
)
|
||
if shift != before {
|
||
logState("auto")
|
||
onChange?()
|
||
}
|
||
}
|
||
|
||
func refreshSuggestions() {
|
||
syncLearning()
|
||
autocorrect.decoderTaps = currentTokenTaps
|
||
let next = autocorrect.suggestions(
|
||
beforeInput: target.documentContextBeforeInput,
|
||
field: field,
|
||
autocorrectionType: target.autocorrectionType
|
||
)
|
||
if next != suggestions {
|
||
suggestions = next
|
||
typingLog?.suggestionsChanged(next)
|
||
onChange?()
|
||
}
|
||
}
|
||
|
||
func acceptSuggestion(_ suggestion: KeyboardSuggestion) {
|
||
if let log = typingLog {
|
||
let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput) ?? ""
|
||
log.edit("suggestion", [
|
||
TypingLogField("slot", suggestions.firstIndex(of: suggestion).map { .int($0) } ?? .null),
|
||
TypingLogField("text", .string(suggestion.displayText)),
|
||
TypingLogField("deleted", .string(token)),
|
||
TypingLogField("inserted", .string(suggestion.replacement + " ")),
|
||
TypingLogField("candidates", TypingLogger.candidates(suggestions)),
|
||
])
|
||
}
|
||
if let journal = learning {
|
||
let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput)
|
||
if let token, !token.isEmpty, token != suggestion.replacement {
|
||
journal.record(.picked, typed: token, final: suggestion.replacement)
|
||
} else {
|
||
journal.didFinish(word: token)
|
||
// Tapping what you typed over the autocorrection space would apply turns it down.
|
||
if suggestion.kind == .keepLiteral, let token,
|
||
let autoFix = suggestions.first(where: \.willAutoReplace) {
|
||
journal.record(.keptTyped, typed: autoFix.replacement, final: token)
|
||
}
|
||
}
|
||
}
|
||
if let token = AutocorrectEngine.currentToken(in: target.documentContextBeforeInput) {
|
||
for _ in 0..<token.count {
|
||
target.deleteBackward()
|
||
}
|
||
if suggestion.kind == .keepLiteral {
|
||
autocorrect.suppressAndKeep(token)
|
||
}
|
||
}
|
||
autocorrect.clearPendingUndo()
|
||
insert(suggestion.replacement + " ", source: nil)
|
||
typingLog?.context("suggestion")
|
||
}
|
||
|
||
/// Types an emoji from the emoji panel: no spacing or autocorrect around it.
|
||
func insertEmoji(_ text: String, source: String) {
|
||
autocorrect.clearPendingUndo()
|
||
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
|
||
insert(text, source: source)
|
||
}
|
||
|
||
// MARK: - Touches
|
||
|
||
func touchBegan(id: Int, at point: CGPoint) {
|
||
guard let key = layout.hitTest(point) else { return }
|
||
// Rollover: a new key going down commits any character or accent picker still held.
|
||
for other in touchOrder {
|
||
guard var touch = touches[other], touch.key.character != nil, !touch.committed else { continue }
|
||
if activeAccentTouchID == other, let menu = activeAccentMenu {
|
||
typingLog?.ui("accent_pick", [
|
||
TypingLogField("id", .int(other)),
|
||
TypingLogField("index", .int(menu.selectedIndex)),
|
||
TypingLogField("chosen", .string(menu.selectedOption)),
|
||
TypingLogField("rollover", .bool(true)),
|
||
])
|
||
commitAccentSelection(menu.selectedOption)
|
||
activeAccentMenu = nil
|
||
activeAccentTouchID = nil
|
||
} else {
|
||
commitCharacter(touch.key, at: touch.startPoint)
|
||
}
|
||
touch.committed = true
|
||
touches[other] = touch
|
||
}
|
||
|
||
var touch = ActiveTouch(
|
||
key: key,
|
||
startPoint: point,
|
||
point: point,
|
||
committed: false,
|
||
returnLayer: nil
|
||
)
|
||
if key.character != nil {
|
||
learning?.recordTap(on: key, at: point)
|
||
}
|
||
switch key.kind {
|
||
case .shift:
|
||
shift.shiftDown(now: clock())
|
||
logState("shift_key")
|
||
case .delete:
|
||
deleteBackward(allowAutocorrectUndo: true)
|
||
case .layerSwitch(let next):
|
||
touch.returnLayer = layerState.layer
|
||
layerState.switchTo(next)
|
||
if next == .letters { shift.reset() }
|
||
rebuildLayout(cause: "layer_key")
|
||
if next == .letters { refreshAutoCapitalization() }
|
||
case .character, .space, .returnKey, .hex, .globe, .emoji:
|
||
break
|
||
}
|
||
touches[id] = touch
|
||
touchOrder.append(id)
|
||
onChange?()
|
||
}
|
||
|
||
@discardableResult
|
||
func openAccentMenuIfEligible(forTouch id: Int) -> Bool {
|
||
guard activeAccentMenu == nil,
|
||
let touch = touches[id],
|
||
!touch.committed,
|
||
let base = touch.key.character else { return false }
|
||
let dx = touch.point.x - touch.startPoint.x
|
||
let dy = touch.point.y - touch.startPoint.y
|
||
guard (dx * dx + dy * dy).squareRoot() <= Self.accentSlop else { return false }
|
||
guard let options = AccentCatalog.options(for: base, uppercase: isUppercase, language: language),
|
||
options.count > 1 else { return false }
|
||
activeAccentMenu = AccentMenu.build(
|
||
for: touch.key.frame,
|
||
rawOptions: options,
|
||
in: layout.width,
|
||
horizontalInset: layout.horizontalInset
|
||
)
|
||
activeAccentTouchID = id
|
||
typingLog?.ui("accent_open", [
|
||
TypingLogField("id", .int(id)),
|
||
TypingLogField("key", .string(TypingLogger.keyID(touch.key.kind))),
|
||
TypingLogField("options", .strings(activeAccentMenu?.options ?? [])),
|
||
])
|
||
onChange?()
|
||
return true
|
||
}
|
||
|
||
func touchMoved(id: Int, to point: CGPoint) {
|
||
guard var touch = touches[id] else { return }
|
||
touch.point = point
|
||
|
||
// 1. Active long-press accent picker: horizontal drag selects the alternate character.
|
||
if activeAccentTouchID == id, var menu = activeAccentMenu {
|
||
let nextIndex = menu.index(for: point.x)
|
||
touches[id] = touch
|
||
if nextIndex != menu.selectedIndex {
|
||
menu.selectedIndex = nextIndex
|
||
activeAccentMenu = menu
|
||
onSelectionStep?()
|
||
onChange?()
|
||
}
|
||
return
|
||
}
|
||
|
||
// 2. Spacebar cursor trackpad: horizontal drag >= 10 pt slides the text caret.
|
||
if touch.key.kind == .space {
|
||
if !touch.isTrackpad, abs(point.x - touch.startPoint.x) >= Self.trackpadActivationDistance {
|
||
touch.isTrackpad = true
|
||
touch.trackpadAnchorX = point.x
|
||
lastSpaceAt = nil
|
||
autocorrect.clearPendingUndo()
|
||
touches[id] = touch
|
||
typingLog?.ui("trackpad_start", [TypingLogField("id", .int(id))])
|
||
onSelectionStep?()
|
||
onChange?()
|
||
return
|
||
}
|
||
if touch.isTrackpad {
|
||
let delta = point.x - touch.trackpadAnchorX
|
||
let rawSteps = Int((delta / Self.trackpadPointsPerCharacter).rounded(.towardZero))
|
||
if rawSteps != 0 {
|
||
touch.trackpadAnchorX += CGFloat(rawSteps) * Self.trackpadPointsPerCharacter
|
||
touch.trackpadOffset += rawSteps
|
||
touches[id] = touch
|
||
typingLog?.edit("cursor", [TypingLogField("offset", .int(rawSteps))])
|
||
tokenTaps.removeAll()
|
||
target.adjustTextPosition(byCharacterOffset: rawSteps)
|
||
onSelectionStep?()
|
||
refreshAutoCapitalization()
|
||
refreshSuggestions()
|
||
return
|
||
}
|
||
touches[id] = touch
|
||
return
|
||
}
|
||
}
|
||
|
||
// 3. Sliding across character keys updates the active key and re-anchors the hold timer.
|
||
if touch.key.character != nil, !touch.committed,
|
||
let under = layout.hitTest(point), under.character != nil, under.kind != touch.key.kind {
|
||
touch.key = under
|
||
touch.startPoint = point
|
||
touches[id] = touch
|
||
onChange?()
|
||
return
|
||
}
|
||
touches[id] = touch
|
||
}
|
||
|
||
func touchEnded(id: Int, at point: CGPoint) {
|
||
guard var touch = touches.removeValue(forKey: id) else { return }
|
||
touchOrder.removeAll { $0 == id }
|
||
touch.point = point
|
||
|
||
if activeAccentTouchID == id, let menu = activeAccentMenu {
|
||
let index = menu.index(for: point.x)
|
||
let chosen = menu.options[index]
|
||
activeAccentMenu = nil
|
||
activeAccentTouchID = nil
|
||
if !touch.committed {
|
||
typingLog?.ui("accent_pick", [
|
||
TypingLogField("id", .int(id)),
|
||
TypingLogField("index", .int(index)),
|
||
TypingLogField("chosen", .string(chosen)),
|
||
])
|
||
commitAccentSelection(chosen)
|
||
}
|
||
onChange?()
|
||
return
|
||
}
|
||
|
||
switch touch.key.kind {
|
||
case .character:
|
||
if !touch.committed {
|
||
commitCharacter(touch.key, at: touch.startPoint)
|
||
}
|
||
case .shift:
|
||
shift.shiftUp()
|
||
logState("shift_key")
|
||
case .space:
|
||
if touch.isTrackpad {
|
||
typingLog?.ui("trackpad_end", [
|
||
TypingLogField("id", .int(id)),
|
||
TypingLogField("offset", .int(touch.trackpadOffset)),
|
||
])
|
||
typingLog?.context("trackpad")
|
||
} else {
|
||
typeSpace()
|
||
}
|
||
case .returnKey:
|
||
if target.enablesReturnKeyAutomatically == true && !target.hasText {
|
||
typingLog?.edit("return", [
|
||
TypingLogField("returnKeyType", .string(TypingLogTraits.name(target.returnKeyType ?? .default))),
|
||
TypingLogField("ignored", .bool(true)),
|
||
])
|
||
break
|
||
}
|
||
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
|
||
autocorrect.clearPendingUndo()
|
||
typingLog?.edit("return", [
|
||
TypingLogField("inserted", .string("\n")),
|
||
TypingLogField("returnKeyType", .string(TypingLogTraits.name(target.returnKeyType ?? .default))),
|
||
TypingLogField("send", .bool(target.returnKeyType == .send)),
|
||
])
|
||
onReturn?()
|
||
insert("\n", source: nil)
|
||
typingLog?.context("return")
|
||
case .layerSwitch:
|
||
// Hold 123, slide to a key, let go: type it and go back.
|
||
if let returnLayer = touch.returnLayer,
|
||
let under = layout.hitTest(point),
|
||
let character = under.character,
|
||
under.kind != touch.key.kind {
|
||
autocorrect.clearPendingUndo()
|
||
insert(character, source: "slide")
|
||
layerState.switchTo(returnLayer)
|
||
rebuildLayout(cause: "layer_slide")
|
||
refreshAutoCapitalization()
|
||
}
|
||
case .emoji:
|
||
onEmojiKey?()
|
||
case .delete, .hex, .globe:
|
||
break
|
||
}
|
||
onChange?()
|
||
}
|
||
|
||
func touchCancelled(id: Int) {
|
||
if activeAccentTouchID == id {
|
||
activeAccentMenu = nil
|
||
activeAccentTouchID = nil
|
||
typingLog?.ui("accent_cancel", [TypingLogField("id", .int(id))])
|
||
}
|
||
guard touches.removeValue(forKey: id) != nil else { return }
|
||
touchOrder.removeAll { $0 == id }
|
||
onChange?()
|
||
}
|
||
|
||
func cancelAllTouches() {
|
||
let hadTouches = !touches.isEmpty || activeAccentMenu != nil
|
||
touches.removeAll()
|
||
touchOrder.removeAll()
|
||
activeAccentMenu = nil
|
||
activeAccentTouchID = nil
|
||
if shift.isHeld {
|
||
shift.shiftUp()
|
||
logState("cancel")
|
||
}
|
||
if hadTouches { onChange?() }
|
||
}
|
||
|
||
// MARK: - Delete
|
||
|
||
/// The delete key repeats after 360 ms, accelerates at 850 ms, and removes whole words after 1.4 s.
|
||
static let repeatDelay: TimeInterval = 0.36
|
||
static let repeatInterval: TimeInterval = 0.075
|
||
static let fastRepeatAfter: TimeInterval = 0.85
|
||
static let wordDeleteAfter: TimeInterval = 1.4
|
||
|
||
func repeatDelete(heldFor: TimeInterval) {
|
||
autocorrect.clearPendingUndo()
|
||
if heldFor >= Self.wordDeleteAfter {
|
||
deleteWordBackward()
|
||
} else if heldFor >= Self.fastRepeatAfter {
|
||
deleteBackward(allowAutocorrectUndo: false, source: "repeat_fast")
|
||
deleteBackward(allowAutocorrectUndo: false, source: "repeat_fast")
|
||
} else {
|
||
deleteBackward(allowAutocorrectUndo: false, source: "repeat")
|
||
}
|
||
}
|
||
|
||
func deleteBackward(allowAutocorrectUndo: Bool = true, source: String = "key") {
|
||
if allowAutocorrectUndo,
|
||
let undo = autocorrect.consumePendingUndo(),
|
||
let before = target.documentContextBeforeInput,
|
||
before.hasSuffix(undo.insertedText) {
|
||
typingLog?.edit("autocorrect_undo", [
|
||
TypingLogField("deleted", .string(undo.insertedText)),
|
||
TypingLogField("inserted", .string(undo.original)),
|
||
])
|
||
for _ in 0..<undo.insertedText.count {
|
||
target.deleteBackward()
|
||
}
|
||
tokenTaps.removeAll()
|
||
target.insertText(undo.original)
|
||
autocorrect.suppressAndKeep(undo.original)
|
||
learning?.record(
|
||
.undone,
|
||
typed: undo.insertedText.trimmingCharacters(in: .whitespaces),
|
||
final: undo.original
|
||
)
|
||
lastSpaceAt = nil
|
||
didEdit()
|
||
return
|
||
}
|
||
autocorrect.clearPendingUndo()
|
||
learning?.willErase(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
|
||
if let log = typingLog {
|
||
let deleted = target.documentContextBeforeInput?.last.map { String($0) }
|
||
log.edit("delete", [
|
||
deleted.map { TypingLogField("deleted", .string($0)) } ?? TypingLogField("deletedCount", .int(1)),
|
||
TypingLogField("src", .string(source)),
|
||
])
|
||
}
|
||
if let last = target.documentContextBeforeInput?.last, last.isLetter || last == "'", !tokenTaps.isEmpty {
|
||
tokenTaps.removeLast()
|
||
} else {
|
||
tokenTaps.removeAll()
|
||
}
|
||
target.deleteBackward()
|
||
lastSpaceAt = nil
|
||
didEdit()
|
||
}
|
||
|
||
func deleteWordBackward() {
|
||
autocorrect.clearPendingUndo()
|
||
guard let before = target.documentContextBeforeInput, !before.isEmpty else {
|
||
deleteBackward(allowAutocorrectUndo: false, source: "repeat_word")
|
||
return
|
||
}
|
||
learning?.willErase(word: AutocorrectEngine.currentToken(in: before))
|
||
var count = 0
|
||
var index = before.endIndex
|
||
while index > before.startIndex, before[before.index(before: index)].isWhitespace {
|
||
index = before.index(before: index)
|
||
count += 1
|
||
}
|
||
while index > before.startIndex, !before[before.index(before: index)].isWhitespace {
|
||
index = before.index(before: index)
|
||
count += 1
|
||
}
|
||
tokenTaps.removeAll()
|
||
typingLog?.edit("delete_word", [
|
||
TypingLogField("deleted", .string(String(before.suffix(max(1, count))))),
|
||
TypingLogField("src", .string("repeat_word")),
|
||
])
|
||
for _ in 0..<max(1, count) {
|
||
target.deleteBackward()
|
||
}
|
||
lastSpaceAt = nil
|
||
didEdit()
|
||
typingLog?.context("delete_word")
|
||
}
|
||
|
||
// MARK: - Typing
|
||
|
||
private func commitCharacter(_ key: KeyboardKey, at point: CGPoint) {
|
||
guard let character = key.character else { return }
|
||
let text = layerState.layer == .letters && shift.isUppercase ? character.uppercased() : character
|
||
if layerState.layer == .letters {
|
||
shift.didType()
|
||
}
|
||
autocorrect.clearPendingUndo()
|
||
insert(text, source: "key", at: point)
|
||
logState("typed")
|
||
}
|
||
|
||
/// The taps of the word being typed when every letter came from a key, else nil.
|
||
private var currentTokenTaps: [CGPoint]? {
|
||
let taps = tokenTaps.compactMap { $0 }
|
||
return taps.count == tokenTaps.count && !taps.isEmpty ? taps : nil
|
||
}
|
||
|
||
private func commitAccentSelection(_ chosen: String) {
|
||
if layerState.layer == .letters {
|
||
shift.didType()
|
||
}
|
||
autocorrect.clearPendingUndo()
|
||
insert(chosen, source: "accent")
|
||
logState("typed")
|
||
}
|
||
|
||
private func typeSpace() {
|
||
let now = clock()
|
||
if DoubleSpacePeriod.shouldReplace(before: target.documentContextBeforeInput, lastSpaceAt: lastSpaceAt, now: now) {
|
||
autocorrect.clearPendingUndo()
|
||
typingLog?.edit("double_space_period", [
|
||
TypingLogField("deleted", .string(" ")),
|
||
TypingLogField("inserted", .string(". ")),
|
||
])
|
||
target.deleteBackward()
|
||
target.insertText(". ")
|
||
tokenTaps.removeAll()
|
||
lastSpaceAt = nil
|
||
layerState.didInsert(" ")
|
||
rebuildLayoutIfNeeded()
|
||
didEdit()
|
||
typingLog?.context("space")
|
||
return
|
||
}
|
||
learning?.didFinish(word: AutocorrectEngine.currentToken(in: target.documentContextBeforeInput))
|
||
syncLearning()
|
||
autocorrect.decoderTaps = currentTokenTaps
|
||
if let (original, replacement) = autocorrect.replacementOnSpace(
|
||
beforeInput: target.documentContextBeforeInput,
|
||
field: field,
|
||
autocorrectionType: target.autocorrectionType
|
||
) {
|
||
for _ in 0..<original.count {
|
||
target.deleteBackward()
|
||
}
|
||
let inserted = replacement + " "
|
||
tokenTaps.removeAll()
|
||
typingLog?.edit("autocorrect", [
|
||
TypingLogField("from", .string(original)),
|
||
TypingLogField("to", .string(replacement)),
|
||
TypingLogField("deleted", .string(original)),
|
||
TypingLogField("inserted", .string(inserted)),
|
||
])
|
||
target.insertText(inserted)
|
||
autocorrect.recordAppliedCorrection(original: original, insertedText: inserted)
|
||
learning?.record(.autocorrected, typed: original, final: replacement)
|
||
lastSpaceAt = now
|
||
layerState.didInsert(" ")
|
||
rebuildLayoutIfNeeded()
|
||
didEdit()
|
||
typingLog?.context("space")
|
||
return
|
||
}
|
||
autocorrect.clearPendingUndo()
|
||
insert(" ", source: "key")
|
||
lastSpaceAt = now
|
||
typingLog?.context("space")
|
||
}
|
||
|
||
/// `source` names the edit in typing logs; nil when the caller writes its own line.
|
||
private func insert(_ text: String, source: String?, at point: CGPoint? = nil) {
|
||
if text.count == 1, text.first?.isLetter == true {
|
||
tokenTaps.append(point)
|
||
} else {
|
||
tokenTaps.removeAll()
|
||
}
|
||
if let source {
|
||
typingLog?.edit("insert", [
|
||
TypingLogField("inserted", .string(text)),
|
||
TypingLogField("src", .string(source)),
|
||
])
|
||
}
|
||
target.insertText(text)
|
||
lastSpaceAt = nil
|
||
layerState.didInsert(text)
|
||
rebuildLayoutIfNeeded()
|
||
didEdit()
|
||
}
|
||
|
||
/// Hands what the journal learned (typos, rejected corrections) to autocorrect;
|
||
/// nothing while Learn how I type is off.
|
||
private func syncLearning() {
|
||
autocorrect.applyLearning(journal?.typingLearning ?? TypingLearning())
|
||
}
|
||
|
||
private func didEdit() {
|
||
refreshAutoCapitalization()
|
||
refreshSuggestions()
|
||
onEdit?()
|
||
}
|
||
|
||
// MARK: - Layout
|
||
|
||
private func rebuildLayoutIfNeeded() {
|
||
if layout.layer != layerState.layer {
|
||
rebuildLayout(cause: "layer_auto")
|
||
}
|
||
}
|
||
|
||
private func rebuildLayout(cause: String = "layout") {
|
||
layout = KeyboardLayout(
|
||
width: width,
|
||
orientation: orientation,
|
||
layer: layerState.layer,
|
||
field: field,
|
||
showsGlobe: showsGlobe,
|
||
showsEmoji: showsEmoji,
|
||
horizontalInset: horizontalInset,
|
||
language: language
|
||
)
|
||
syncDecoderKeys()
|
||
logState(cause)
|
||
}
|
||
|
||
/// The letter keys Spatial autocorrect scores taps against: this layout's, moved by
|
||
/// `tapOffsets`. None off the letters page.
|
||
private func syncDecoderKeys() {
|
||
autocorrect.decoderKeys = TypingDecoder.Keys(layout: layout, offsets: tapOffsets)
|
||
}
|
||
|
||
/// Hands the layout and shift to typing logs, which writes a line when either changed.
|
||
func logState(_ cause: String) {
|
||
typingLog?.keyboardState(layout: layout, shift: shift, cause: cause)
|
||
}
|
||
}
|