44 Commits
Author SHA1 Message Date
Dax 4d28a692d8 add fluid speech pro to the speech model catalog (#2007)
* add fluid speech pro to the speech model catalog

FluidVoice's Fluid Speech Pro, a Parakeet TDT 0.6B v3 retrain that also
declares Moondream's Parakeet Ultra as a base, for the same 25 European
languages. TalkMate converted it to a Q8_0 transcribe.cpp GGUF at d59dc79
(741,819,456 bytes, sha256 c3848e98...); the file says
parakeet/tdt-0.6b-fluid-speech-pro and announces its extra pre-Conformer
block as stt.parakeet.encoder.pre_block_kernel, so it needs a transcribe.cpp
that reads enc.pre_block.

Like Parakeet v3 it takes no language hint, detects the language under Auto,
does not stream, and is never recommended or badged. Its card publishes no
LibriSpeech test-clean figure, so it carries no single word error rate and no
benchmark record; its per-language FLEURS figures and the CC BY 4.0 credit
are in its details.

* bump(transcribe.cpp): 295c7e6, fluid speech pro's pre-conformer block

Fluid Speech Pro's GGUF announces stt.parakeet.encoder.pre_block_kernel and
carries eight enc.pre_block.* tensors: a residual FrameCleaner block between
the subsampler and Conformer layer 0. The old pin's graph never read them, so
the model loaded as plain Parakeet v3 and its encoder was wrong.

Both transcribe-cpp crates move to 295c7e601 on hex/combined-fluid, a
fast-forward of bcb639f that runs the block on pre_encode before xscaling
when the KV is present, and fails the load rather than skipping tensors when
it is only partly there. Cargo.lock changes only those two sources
(`getrandom`'s tempfile pointer was reverted to what it was), and
nix/package.nix carries the hash from a failed `nix build .#hex`, then a full
one whose check phase passed.

`hex model install fluid-speech-pro` followed by `hex model transcribe` on
whisper.cpp's jfk.wav prints the expected sentence, as does Parakeet v3 on
the same clip.

* fluid speech pro: snake_case id, FLEURS-only figures, blurb, notices
2026-11-19 15:19:06 -05:00
Dax 002b575c95 Add KB-Whisper Medium for Swedish (#1991) (#1992) 2026-11-19 05:44:56 -05:00
Dax a38f6cb709 local api: native segment and word timestamps behind a labs toggle (#1658)
* local api: native segment and word timestamps behind a Labs toggle

POST /transcriptions takes timestamps=none|segment|word. Absent means
none and the response is unchanged. segment adds timestampGranularity
and segments; word adds segments and words, with text, startMs, and
endMs in whole milliseconds from the submitted audio's start.

The times are the model's own: the GGUF capabilities metadata decides
what it can produce, Apple Speech gives segments, and nothing estimates
or downgrades a level. A model without the requested level answers 422
unsupported-timestamps, decided after the model resolves on both the
shared dictation engine and the service's own copy. Chunked long clips
add each chunk's offset once from its sample position, and ends padded
past the decoded audio are clamped to durationMs.

Labs › Timestamps in API transcripts (labs.api_timestamps) gates it and
is off by default; off, a timed request answers 403 timestamps-disabled
before the audio body is read. The SDK takes timestamps and decodes the
new fields.

* local api timestamps: review follow-ups

* local api timestamps: correct embedded helper docs

* local api timestamps: align labs doc wording with the service spec
2026-11-10 17:07:52 -05:00
Dax 1843603caa local api: optional output processing behind a lab (#1597)
* local api: optional output processing behind a lab

* local api: share one processing state with listeners

* local api: process one transcript at a time

* docs: clarify the processing route checks
2026-11-09 10:35:05 -05:00
Dax 62a40669fe add orukeet to the speech model catalog (#1545)
* add orukeet to the model catalog

* orukeet: notice links, restored notice sentence, spec id, translations
2026-11-08 10:28:20 -05:00
Luke Parker 7f902061ef feat(windows): /dictations records with Hex's own microphone, and a dictation can stream its live text (#1474)
* feat(windows): /dictations records with Hex's own microphone, and a dictation can stream its live text

Windows Hex started its local API without a dictation control, so
POST /dictations answered 404 service-capture-unavailable and apps had to
record the microphone themselves for /streams, ignoring the microphone
chosen in Hex.

The Windows listener now drives the same dictation::service owner as
Linux: one capture at a time beside the shortcut, the saved microphone,
pre-roll, While dictating, the lease, levels, and the HUD.

A start may also ask for the text so far ("partials": true). Hex then
holds a live-text demand for the recording's stream and sends
{stable, tentative} on GET /dictations/{id}/partials until the dictation
ends. /capabilities reports dictationPartials: true on Linux and Windows;
macOS doesn't stream a client's recording, so it reports false.

* fix(macos): no dead /dictations partials code where the recognition loop has none
2026-11-07 16:21:52 -06:00
Dax 8a327aa90d Add Granite Speech 5.0 470M TurboCTC (#1252) (#1256)
* Add Granite Speech 5.0 470M TurboCTC (#1252)

* fmt
2026-11-04 06:19:55 -05:00
Dax 637b1f933c Add NB-Whisper Medium for Norwegian (#1150) (#1173)
* Add NB-Whisper Medium for Norwegian, and list Norwegian (#1150)

* Embed q5_0 Vulkan shaders for NB-Whisper's file

* fmt
2026-11-03 12:43:26 -05:00
Dax 6b9ddff6e6 Bring /dictations to Linux (#981) (#984)
The local API's running-app dictation now records on the Linux listener with
the macOS contract: owner tokens, the ten-second lease, levels, the audio tap,
and the same error codes. The protocol, completion record, lease and meter move
to dictation::service, which both platforms share; Linux drives them through
ServiceDictations. A client's dictation and a shortcut or hex dictation capture
never end each other: a start while the shortcut records is dictation-busy, and
the shortcut does nothing while the client's dictation records.
2026-11-02 06:38:58 -05:00
Dax efa1aa065a Add GigaAM v3 e2e RNN-T for Russian (#952) (#954) 2026-11-01 23:57:19 -05:00
Dax caa29177be Add Nemotron 3.5 ASR streaming 0.6B (#834) (#839)
* wip: Nemotron 3.5 ASR

* wip: Nemotron numbers

* Drop the temporary bench

* Docs for Nemotron 3.5 ASR

* Move the catalog's artifacts out of models.rs; fix tests for Nemotron
2026-11-01 09:48:16 -05:00
Dax 54d8f5387b Use speech models already on this computer (#594) (#819)
* Use speech models already on this computer (#594)

* Keep found-model state together

* CLI copy for found models

* Docs and onboarding hold test for models found on this computer
2026-11-01 07:24:24 -05:00
Dax dff0d7e005 Labs: Parakeet Ultra with a details view (#766)
* Labs: Parakeet Ultra catalog entry with details and attribution (#648)

* Tests, SDK id

* Say Parakeet Ultra needs its Lab

* Scene for Parakeet Ultra's details

* Docs, notices, and details never choose the model
2026-11-01 01:24:54 -04:00
Dax 5c5f0f66b4 Labs: use your own speech model (import a GGUF) (#759)
* wip: import your own speech model (Labs)

* fix debug

* Show the imported file's name

* clippy

* Say why an import can't be used

* Golden for the picker's Add model file

* Docs, feature map, AGENTS.md for imported speech models

* Arm Remove before removing an import
2026-11-01 01:03:12 -04:00
Dax d86d494c3b Add Hebrew with ivrit.ai's Whisper large-v3-turbo fine-tune (#755)
* Add ivrit.ai's Hebrew Whisper large-v3-turbo and Hebrew (#752)

* Move benchmark records into their own module

* Import ErrorMetric where the test uses it
2026-11-01 00:33:38 -04:00
Dax ee8de33bf2 Add QwenCleo-ASR 1.7B for Egyptian Arabic (#467) (#577) 2026-10-31 07:29:50 -04:00
Dax 9c91096d29 Local API, /streams, and the SDK host on Linux and Windows (#537) 2026-10-30 22:07:30 -04:00
Dax fdd6ccdc4a Add Underdog Bark 3.0 Pro and Clean transcription models (#345)
* feat: add Underdog Bark 3.0 Pro and Clean transcription models

Part of #343

* fix: use word_error_rate for Bark models

* fix: explicit wire names for Bark models
2026-10-29 01:53:35 -04:00
Dax 3434eb5f35 Rename voice-control to hex everywhere, without migrations (#293)
* Rename voice-control to hex everywhere, without migrations

The Cargo package and binary are now hex, every OS keeps its data in a
hex folder, and VOICE_CONTROL_* variables are HEX_*. Hex is a fresh
app: the macOS voice-control rename, its symlink, and the Windows and
Linux beta settings imports are gone. The Windows installer gets a new
AppId so it never takes over an old voice-control install's entry.

* Format

* Drop the Windows paste-last default only the beta import used
2026-10-28 13:08:44 -04:00
Kit Langton ed6ce75205 Keep a programmatic dictation's audio when the microphone fails
A gap or interruption while a programmatic dictation recorded threw the
capture away, so Seer's /finish got dictation-not-active and showed "Hex
heard nothing". The audio owner now finishes a programmatic capture at the
last audio delivered before the loss and hands the clip over with the event;
recognition holds it under the dictation's id, where /finish transcribes it,
/cancel drops it, and heartbeats and the lease still apply. With no audio
(the microphone never opened), /finish answers 409 microphone-lost. Hotkey,
voice, and Voice Action captures still fail as before.
2026-10-27 11:13:36 -07:00
Kit Langton da85830567 Escape never cancels a programmatic dictation
Kit presses Escape inside Seer all the time while a note records (closing a
lightbox, leaving the reply box, a picker), and Hex's tap cancelled the
capture each time, so Seer showed "Hex heard nothing". Hex no longer claims
Escape for a programmatic dictation or its pending job: the key reaches the
owner app, which cancels through /cancel, and the lease still ends an
abandoned capture.
2026-10-27 11:13:36 -07:00
Kit Langton 0ed8354414 Move to dev and production branches
Work lands on dev, the default branch. production holds what was released
(app-v2.3.1) and moves only by a fast-forward from dev. Pushes to dev run the
checks, test builds, and the dev site deploy; stable releases run only from
production or an app-v tag, and the TypeScript SDK's version pull request
targets dev while publishing waits for production.
2026-10-25 06:46:14 -07:00
Kit Langton e76afaaa47 feat(macos): move the data folder from voice-control to hex (#190)
* feat(macos): move the data folder from voice-control to hex

* fix(macos): match Hex's folder precedence in SDK and scripts; re-create a missing compat symlink
2026-10-21 18:11:47 -07:00
Kit Langton f249917ef6 refactor: simplify codebase and align desktop UI with OpenCode 2026-10-06 23:29:09 -07:00
Kit Langton 929cee7ca6 chore(deps): update app, SDK, website, and CI dependencies (#115)
* chore(deps): update compatible Rust dependencies

* chore(deps): update rubato, dirs, base64, ed25519-dalek, and x11rb

* chore(deps): update Effect 4.0.0, Vitest 5, and Changesets 3 in the SDK workspace

* chore(deps): update website dependencies

* ci: update GitHub Actions to current major versions

* chore(deps): hold libc at 0.2.189 for gpui's Linux xattr dependency
2026-10-02 17:09:39 -07:00
github-actions[bot] 2ef6b9ec01 chore: release TypeScript SDK 0.3.1 (#102)
Release the bounded shared SSE framing fix through the configured Changesets workflow.
2026-09-25 18:36:56 -07:00
Kit Langton 7b0c6f659c refactor: remove redundant desktop and transcription work 2026-09-22 17:24:01 -07:00
Kit Langton 04a42dae3d refactor: consolidate settings commits, transcription padding, and SDK framing
Settings controls commit a full candidate before changing the saved UI choice,
so a failed write keeps shortcut capture open and preserves pending debounced
edits. The macOS Transcriber owns backend input padding for ordinary,
voice-protocol, and segmented transcription. Moonshine stream cleanup and the
Linux client share one owner-side path. The command SDK shares capture
normalization between constructors and host registration, and the public SDK
decodes model progress and dictation levels with one bounded SSE framer. The
signing scripts share one identity helper, and historical release evidence
moves out of the feature map.
2026-09-22 17:08:43 -07:00
Kit Langton 9e9da53aa9 docs: audit guides against current behavior
Align desktop projections, processing fallback, history retention, contributor setup, Linux requirements, SDK contracts, and personal command examples with the implementation. Clarify prototype cleanup and historical validation limits without rewriting published release notes.
2026-09-01 09:56:40 -04:00
Kit Langton 7353e511d9 refactor: remove unused desktop and capture bookkeeping (#60)
Remove unused desktop projections, constant ingestion returns, and redundant recording-session state. Preserve capture behavior, shortcut recovery, and public SDK interfaces.
2026-09-01 09:45:51 -04:00
github-actions[bot] 190ddf0676 chore: release TypeScript SDK 2026-08-31 19:12:09 -04:00
Kit Langton c5bffe32fa feat(sdk): add ready model-bound transcribers (#57)
* feat(sdk): add ready model-bound transcribers

* fix(sdk): simplify progress and preserve cleanup failures
2026-08-31 18:49:57 -04:00
github-actions[bot] 955dcfc727 Version Packages (#40)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-28 16:26:55 -04:00
Kit Langton e1637536fc fix: recover startup audio and update Effect SDKs (#39)
* fix(audio): recover from startup microphone failures

* fix(sdk): support Effect 4.0.0-rc.112

* chore(release): prepare Hex 2.1.4

* fix(commands): migrate managed Effect dependencies safely

* fix(commands): refresh cached local SDK dependency metadata
2026-08-28 16:24:25 -04:00
github-actions[bot] 39de364a02 Version Packages (#27)
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
2026-08-27 19:41:27 -04:00
Kit Langton c5ba7d47cd fix: harden dictation and OpenCode integration
Move generation bodies and credentials off argv, match active service registrations, and align catalog IDs and model variants with the audited V2 contract.\n\nPreserve onboarding across imported settings, serialize paste settling, bound pending outputs, and fix capture, SDK, and Linux startup regressions. Add regression coverage, dependency patches, icon notices, and the V2 compatibility baseline.
2026-08-26 16:59:53 -04:00
Kit Langton 5a295346a5 fix: publish SDK without private provenance 2026-08-06 20:10:52 -04:00
Kit Langton 5b1d089612 fix: reject incompatible capture APIs 2026-08-06 20:09:39 -04:00
Kit Langton 135d9a0f8d feat: expose running app dictation SDK 2026-08-06 17:46:33 -04:00
Kit Langton 62f06ac2d0 feat: default to Parakeet Unified English 2026-08-04 16:33:52 -04:00
Kit Langton d9e5066ce7 feat: add automatic transcription language detection 2026-07-30 13:16:49 -04:00
Kit Langton 4fcc3ce9ec docs(sdk): record the zero-TCC permissions decomposition 2026-07-24 15:38:18 -04:00
Kit Langton c691f74941 feat: unify desktop modes and platform runtime 2026-07-23 17:33:22 -04:00
Kit Langton 3e166524bc feat: expand dictation tooling and TypeScript SDK 2026-07-22 20:58:27 -04:00