* add fluid speech pro to the speech model catalog FluidVoice's Fluid Speech Pro, a Parakeet TDT 0.6B v3 retrain that also declares Moondream's Parakeet Ultra as a base, for the same 25 European languages. TalkMate converted it to a Q8_0 transcribe.cpp GGUF at d59dc79 (741,819,456 bytes, sha256 c3848e98...); the file says parakeet/tdt-0.6b-fluid-speech-pro and announces its extra pre-Conformer block as stt.parakeet.encoder.pre_block_kernel, so it needs a transcribe.cpp that reads enc.pre_block. Like Parakeet v3 it takes no language hint, detects the language under Auto, does not stream, and is never recommended or badged. Its card publishes no LibriSpeech test-clean figure, so it carries no single word error rate and no benchmark record; its per-language FLEURS figures and the CC BY 4.0 credit are in its details. * bump(transcribe.cpp): 295c7e6, fluid speech pro's pre-conformer block Fluid Speech Pro's GGUF announces stt.parakeet.encoder.pre_block_kernel and carries eight enc.pre_block.* tensors: a residual FrameCleaner block between the subsampler and Conformer layer 0. The old pin's graph never read them, so the model loaded as plain Parakeet v3 and its encoder was wrong. Both transcribe-cpp crates move to 295c7e601 on hex/combined-fluid, a fast-forward of bcb639f that runs the block on pre_encode before xscaling when the KV is present, and fails the load rather than skipping tensors when it is only partly there. Cargo.lock changes only those two sources (`getrandom`'s tempfile pointer was reverted to what it was), and nix/package.nix carries the hash from a failed `nix build .#hex`, then a full one whose check phase passed. `hex model install fluid-speech-pro` followed by `hex model transcribe` on whisper.cpp's jfk.wav prints the expected sentence, as does Parakeet v3 on the same clip. * fluid speech pro: snake_case id, FLEURS-only figures, blurb, notices
HEX TypeScript Client
@kitlangton/hex provides local transcription through a native HEX helper, with
Promise and optional Effect APIs. The host application owns recording, microphone
permission, settings, and what happens to the resulting text. HEX owns model
preparation and inference.
The package is ESM-only and requires Node.js >=20.3.0 or a compatible host
runtime.
Distribution status: a bundled native helper is not published yet. Embedded
consumers must currently supply an explicit native command. The examples below
use the HEX executable's service --embedded mode; users do not need to launch
the HEX desktop app. This package alone is not yet a turnkey native distribution.
Promise API
Pass a model to create() to get a ready-to-use transcriber:
import * as Hex from "@kitlangton/hex"
const transcriber = await Hex.create({
command: ["/path/to/hex-service", "service", "--embedded"],
model: "parakeet_unified_en",
language: "en",
onProgress: console.log,
})
try {
// WAV is an ArrayBuffer or Uint8Array supplied by your recording code.
const result = await transcriber.transcribe(wav, {
signal: abortController.signal,
})
console.log(result.transcript, result.durationMs)
} finally {
await transcriber.close()
}
create({ model }) starts the helper and waits for download (if necessary),
verification, and model loading. Progress can report downloading, verifying, and
loading. Model and language are captured when creation starts; language defaults
to en, independently of desktop settings. A failed or cancelled creation closes
the helper before rejecting. The optional creation signal covers startup and
preparation, not the returned transcriber's entire lifetime.
Keep the transcriber alive across recordings to reuse its loaded model. It exposes
the bound model, language, and diagnostic pid; audio requests need only WAV
bytes. transcribe() never initiates a download. Inference remains bounded by the
native service's admission limits; simultaneous requests can receive a busy error
instead of entering an unbounded client queue.
Cancellation and ownership
close() is idempotent. It closes the helper's stdin lease, waits for graceful
shutdown, then terminates the exact child if it does not exit within the bounded
deadline. Close when the feature or application shuts down, not after every clip.
No model files are deleted.
Cancelling an in-flight transcribe() closes this transcriber's helper before
rejecting. This is intentional: aborting HTTP alone cannot interrupt native
inference. All other work on that transcriber also ends; create a new transcriber
before the next recording. An already-aborted request is rejected without closing
an otherwise usable transcriber. Ordinary request errors do not close it.
This distinction lets hosts that retain recording resources until cancellation
settles know that native inference has stopped. Cleanup failures remain visible;
they are not reported as successful cancellation. The host owns temporary audio
cleanup and any operation deadline, for example through AbortSignal.timeout().
Effect API
Effect is an optional peer. Use Effect >=4.0.0-rc.112 <5.0.0 for this entrypoint;
older v4 betas are not supported. Promise-only consumers do not need Effect and
can adapt the Promise API to their own framework/runtime version.
import { Effect } from "effect"
import * as Hex from "@kitlangton/hex/effect"
const program = Effect.scoped(Effect.gen(function* () {
const transcriber = yield* Hex.create({
command: ["/path/to/hex-service", "service", "--embedded"],
model: "parakeet_unified_en",
language: "en",
onProgress: console.log,
})
return yield* transcriber.transcribe(wav)
}))
const result = await Effect.runPromise(program)
The scope owns helper cleanup; there is no manual close() on the Effect
transcriber. Put that scope around the voice feature's lifetime when reusing the
model. Model preparation is interruptible after bounded helper startup, and
interrupting active transcription waits for helper cleanup even when the owning
scope remains open. Errors are schema-backed tagged failures; a scope finalizer
that cannot close the helper fails as a defect rather than silently leaking it.
The progress callback is synchronous, as in the Promise API.
Low-Level Client
Existing create() calls without a model still return { pid, client, close }
in the Promise API and a scoped { pid, client } in the Effect API. This is useful
for catalog browsing and explicit model management:
const host = await Hex.create({
command: ["/path/to/hex-service", "service", "--embedded"],
})
try {
const models = await host.client.models.list()
await host.client.models.prepare("parakeet_unified_en", {
language: "en",
onProgress: console.log,
})
const result = await host.client.transcribe({
audio: { data: wav, contentType: "audio/wav" },
model: "parakeet_unified_en",
language: "en",
})
} finally {
await host.close()
}
The low-level client retains request-only cancellation; callers own helper
shutdown if native work must stop. Effect's low-level client.models.prepare()
remains a progress Stream. Hex.layer(options) still supplies the low-level
Hex.Service and owns its helper for the layer scope.
On the running desktop app, client.transcribe({ ..., process: "global" })
also asks for the text after your Global mode's output processing (Labs ›
Process API transcripts) and returns it as processed. An embedded
hex-service installs no processor and answers processing-unavailable;
without process, results are raw transcripts as before. capabilities()
reports outputProcessing when the host supports it; older hosts omit it.
With Labs › Timestamps in API transcripts on, the same call takes
timestamps: "segment" | "word" and returns the speech model's own times:
const result = await client.transcribe({
audio: { data: wav, contentType: "audio/wav" },
model: "parakeet_unified_en",
timestamps: "word",
})
for (const word of result.words ?? []) {
console.log(word.text, word.startMs, word.endMs) // ms from the clip's start
}
segment adds segments; word adds segments and words. Each is a
TimedText: text, startMs, endMs. "none" and an absent option send no
parameter, so the response is the untimed one. A model that cannot produce the
level asked for answers unsupported-timestamps; the times are never estimated
or cut down from a coarser level, and the Lab being off answers
timestamps-disabled. An embedded helper reads the lab from saved settings
once, on its first request, so restart it after turning the lab on; connect()
reads the running app's value live.
Use models.list() to discover available models rather than treating the exported
MODEL_IDS as an availability list. apple_speech remains in the public type but
is currently unavailable in the native catalog. The optional language argument
to models.list() affects readiness metadata, not language filtering; inspect
each model's languages and supportsLanguageDetection fields. source says
where the file comes from: download, hex, imported, or
found: owner/name@abcdef1 for a model another app downloaded to the Hugging Face
cache, which Hex uses where it is.
Connect to HEX Desktop
Alternatively, reuse the microphone and engine owned by the running HEX desktop app through the Promise entrypoint:
import { connect } from "@kitlangton/hex"
const hex = await connect()
const recording = await hex.dictation.start({ source: "my-app" })
void (async () => {
for await (const { rmsDb, peakDb } of recording.levels) {
console.log({ rmsDb, peakDb })
}
})()
const { transcript, durationMs } = await recording.finish()
connect() finds the running Hex on macOS, Linux, and Windows. Desktop capture
(hex.dictation) works on all three while Hex is listening, with the
microphone the user chose in Hex. A recording never ends the user's shortcut
dictation: starting one while the shortcut records is refused with
dictation-busy, and the
shortcut does nothing while yours records. Outside macOS there is no
packaged helper yet; pass command: ["hex", "service", "--embedded"] to
create() to run an installed Hex as the direct child.
Call recording.cancel() to discard; Escape doesn't cancel a recording, so the
key stays with your app. If the microphone fails mid-recording, finish() on
macOS still transcribes the audio captured before the failure, or rejects with
remote code microphone-lost when there was none; on Linux and Windows it
always rejects with microphone-lost. Subscribing to recording.audio enables a
bounded best-effort mono Float32 PCM tap at recording.sampleRate. Desktop capture
returns raw text and never pastes. capabilities().serviceCapture distinguishes
desktop capture from the transcription-only helper. The recording handle renews
a ten-second lease with a heartbeat every three seconds; capture expires when
renewal stops. Dropping the handle does not stop the heartbeat: always finish or
cancel explicitly. Its unguessable ownerToken scopes capture operations and
observations automatically.
Host Boundary
- Use the SDK in Node or Electron main code, not an untrusted renderer. Expose only narrow, validated recording/model operations across the renderer bridge.
- The direct-child helper does not capture microphone audio. Permissions,
NSMicrophoneUsageDescription, the recording indicator, and recording consent belong to the host application. Desktopconnect()instead uses HEX's capture. - Encode PCM WAV in the host. Do not send compressed recordings such as M4A or WebM and label them as WAV. The service accepts 8-192 kHz audio with 1-8 channels, subject to a 64 MiB upload limit and separate source-frame and normalized-audio budgets. See the service spec for the exact limits; there is no promise of unlimited clip length.
- Persist the host's model selection in the host, not in HEX desktop preferences.
- Spawn the signed helper directly. Do not use LaunchServices,
open, orNSWorkspace. Validate native packaging/signing in the final consuming app. - Never expose the startup bearer token to renderer or web content. Credentials are exchanged over the child pipe and used for authenticated loopback requests.