* add fluid speech pro to the speech model catalog
FluidVoice's Fluid Speech Pro, a Parakeet TDT 0.6B v3 retrain that also
declares Moondream's Parakeet Ultra as a base, for the same 25 European
languages. TalkMate converted it to a Q8_0 transcribe.cpp GGUF at d59dc79
(741,819,456 bytes, sha256 c3848e98...); the file says
parakeet/tdt-0.6b-fluid-speech-pro and announces its extra pre-Conformer
block as stt.parakeet.encoder.pre_block_kernel, so it needs a transcribe.cpp
that reads enc.pre_block.
Like Parakeet v3 it takes no language hint, detects the language under Auto,
does not stream, and is never recommended or badged. Its card publishes no
LibriSpeech test-clean figure, so it carries no single word error rate and no
benchmark record; its per-language FLEURS figures and the CC BY 4.0 credit
are in its details.
* bump(transcribe.cpp): 295c7e6, fluid speech pro's pre-conformer block
Fluid Speech Pro's GGUF announces stt.parakeet.encoder.pre_block_kernel and
carries eight enc.pre_block.* tensors: a residual FrameCleaner block between
the subsampler and Conformer layer 0. The old pin's graph never read them, so
the model loaded as plain Parakeet v3 and its encoder was wrong.
Both transcribe-cpp crates move to 295c7e601 on hex/combined-fluid, a
fast-forward of bcb639f that runs the block on pre_encode before xscaling
when the KV is present, and fails the load rather than skipping tensors when
it is only partly there. Cargo.lock changes only those two sources
(`getrandom`'s tempfile pointer was reverted to what it was), and
nix/package.nix carries the hash from a failed `nix build .#hex`, then a full
one whose check phase passed.
`hex model install fluid-speech-pro` followed by `hex model transcribe` on
whisper.cpp's jfk.wav prints the expected sentence, as does Parakeet v3 on
the same clip.
* fluid speech pro: snake_case id, FLEURS-only figures, blurb, notices
* local api: native segment and word timestamps behind a Labs toggle
POST /transcriptions takes timestamps=none|segment|word. Absent means
none and the response is unchanged. segment adds timestampGranularity
and segments; word adds segments and words, with text, startMs, and
endMs in whole milliseconds from the submitted audio's start.
The times are the model's own: the GGUF capabilities metadata decides
what it can produce, Apple Speech gives segments, and nothing estimates
or downgrades a level. A model without the requested level answers 422
unsupported-timestamps, decided after the model resolves on both the
shared dictation engine and the service's own copy. Chunked long clips
add each chunk's offset once from its sample position, and ends padded
past the decoded audio are clamped to durationMs.
Labs › Timestamps in API transcripts (labs.api_timestamps) gates it and
is off by default; off, a timed request answers 403 timestamps-disabled
before the audio body is read. The SDK takes timestamps and decodes the
new fields.
* local api timestamps: review follow-ups
* local api timestamps: correct embedded helper docs
* local api timestamps: align labs doc wording with the service spec
* local api: optional output processing behind a lab
* local api: share one processing state with listeners
* local api: process one transcript at a time
* docs: clarify the processing route checks
* feat(windows): /dictations records with Hex's own microphone, and a dictation can stream its live text
Windows Hex started its local API without a dictation control, so
POST /dictations answered 404 service-capture-unavailable and apps had to
record the microphone themselves for /streams, ignoring the microphone
chosen in Hex.
The Windows listener now drives the same dictation::service owner as
Linux: one capture at a time beside the shortcut, the saved microphone,
pre-roll, While dictating, the lease, levels, and the HUD.
A start may also ask for the text so far ("partials": true). Hex then
holds a live-text demand for the recording's stream and sends
{stable, tentative} on GET /dictations/{id}/partials until the dictation
ends. /capabilities reports dictationPartials: true on Linux and Windows;
macOS doesn't stream a client's recording, so it reports false.
* fix(macos): no dead /dictations partials code where the recognition loop has none
The local API's running-app dictation now records on the Linux listener with
the macOS contract: owner tokens, the ten-second lease, levels, the audio tap,
and the same error codes. The protocol, completion record, lease and meter move
to dictation::service, which both platforms share; Linux drives them through
ServiceDictations. A client's dictation and a shortcut or hex dictation capture
never end each other: a start while the shortcut records is dictation-busy, and
the shortcut does nothing while the client's dictation records.
* wip: Nemotron 3.5 ASR
* wip: Nemotron numbers
* Drop the temporary bench
* Docs for Nemotron 3.5 ASR
* Move the catalog's artifacts out of models.rs; fix tests for Nemotron
* Use speech models already on this computer (#594)
* Keep found-model state together
* CLI copy for found models
* Docs and onboarding hold test for models found on this computer
* Labs: Parakeet Ultra catalog entry with details and attribution (#648)
* Tests, SDK id
* Say Parakeet Ultra needs its Lab
* Scene for Parakeet Ultra's details
* Docs, notices, and details never choose the model
* wip: import your own speech model (Labs)
* fix debug
* Show the imported file's name
* clippy
* Say why an import can't be used
* Golden for the picker's Add model file
* Docs, feature map, AGENTS.md for imported speech models
* Arm Remove before removing an import
* Add ivrit.ai's Hebrew Whisper large-v3-turbo and Hebrew (#752)
* Move benchmark records into their own module
* Import ErrorMetric where the test uses it
* feat: add Underdog Bark 3.0 Pro and Clean transcription models
Part of #343
* fix: use word_error_rate for Bark models
* fix: explicit wire names for Bark models
* Run personal commands in embedded QuickJS instead of Bun
Hex now runs ~/.config/hex/hex.config.ts in-process: rquickjs on one worker
thread, TypeScript erased to same-position JavaScript with oxc's parser,
per-call time limits and a runtime memory limit, and no filesystem, network,
process, or timer API. Handlers return actions that Hex validates and runs on
its native executor. The Bun host, managed SDK provisioning, bun update, the
sdk/commands workspace, and their scripts and CI steps are gone; Hex writes
hex.d.ts typings instead, and hex commands check replaces bun run check.
Plan and measurements: docs/plans/quickjs.md.
* Spell out a range check that a new dependency's impls made ambiguous
* Rename voice-control to hex everywhere, without migrations
The Cargo package and binary are now hex, every OS keeps its data in a
hex folder, and VOICE_CONTROL_* variables are HEX_*. Hex is a fresh
app: the macOS voice-control rename, its symlink, and the Windows and
Linux beta settings imports are gone. The Windows installer gets a new
AppId so it never takes over an old voice-control install's entry.
* Format
* Drop the Windows paste-last default only the beta import used
A gap or interruption while a programmatic dictation recorded threw the
capture away, so Seer's /finish got dictation-not-active and showed "Hex
heard nothing". The audio owner now finishes a programmatic capture at the
last audio delivered before the loss and hands the clip over with the event;
recognition holds it under the dictation's id, where /finish transcribes it,
/cancel drops it, and heartbeats and the lease still apply. With no audio
(the microphone never opened), /finish answers 409 microphone-lost. Hotkey,
voice, and Voice Action captures still fail as before.
Kit presses Escape inside Seer all the time while a note records (closing a
lightbox, leaving the reply box, a picker), and Hex's tap cancelled the
capture each time, so Seer showed "Hex heard nothing". Hex no longer claims
Escape for a programmatic dictation or its pending job: the key reaches the
owner app, which cancels through /cancel, and the lease still ends an
abandoned capture.
Work lands on dev, the default branch. production holds what was released
(app-v2.3.1) and moves only by a fast-forward from dev. Pushes to dev run the
checks, test builds, and the dev site deploy; stable releases run only from
production or an app-v tag, and the TypeScript SDK's version pull request
targets dev while publishing waits for production.
* feat(macos): move the data folder from voice-control to hex
* fix(macos): match Hex's folder precedence in SDK and scripts; re-create a missing compat symlink
* chore(deps): update compatible Rust dependencies
* chore(deps): update rubato, dirs, base64, ed25519-dalek, and x11rb
* chore(deps): update Effect 4.0.0, Vitest 5, and Changesets 3 in the SDK workspace
* chore(deps): update website dependencies
* ci: update GitHub Actions to current major versions
* chore(deps): hold libc at 0.2.189 for gpui's Linux xattr dependency
Settings controls commit a full candidate before changing the saved UI choice,
so a failed write keeps shortcut capture open and preserves pending debounced
edits. The macOS Transcriber owns backend input padding for ordinary,
voice-protocol, and segmented transcription. Moonshine stream cleanup and the
Linux client share one owner-side path. The command SDK shares capture
normalization between constructors and host registration, and the public SDK
decodes model progress and dictation levels with one bounded SSE framer. The
signing scripts share one identity helper, and historical release evidence
moves out of the feature map.
raw-window-handle was declared in the macOS dependency table but never
imported anywhere in the tree. sdk/commands/test/fixtures/host-config.ts
was never imported by any test, including at its introduction.
Align desktop projections, processing fallback, history retention, contributor setup, Linux requirements, SDK contracts, and personal command examples with the implementation. Clarify prototype cleanup and historical validation limits without rewriting published release notes.
Move generation bodies and credentials off argv, match active service registrations, and align catalog IDs and model variants with the audited V2 contract.\n\nPreserve onboarding across imported settings, serialize paste settling, bound pending outputs, and fix capture, SDK, and Linux startup regressions. Add regression coverage, dependency patches, icon notices, and the V2 compatibility baseline.
Add bounded digit, letter, choice, union, and text captures across the TypeScript SDK, host protocol, Rust grammar, and command runtime. Validate capture schemas and invocation payloads before activation, preserve exact inferred handler types, and document the personal-command workflow.\n\nKeep modifier-only dictation events passive so Finder and other applications still receive Option state changes. Also emit balanced synthetic modifier transitions, tighten local installation behavior, and refine recording HUD responsiveness.
A personal command phrase may end in one {name} placeholder, e.g.
"search amazon for {query}". The grammar compiles it to a literal prefix
plus a bounded trailing free-text capture (24 words / 512 bytes), rejects
overlaps with any coexisting phrase sharing the prefix, and delivers the
normalized remainder through Action::InvokeHandler into the Bun host as a
bounded captures map. Handlers receive it as captures.name in both the
Promise and Effect APIs. Capture phrases require a handler; native actions
stay static.