mirror of
https://github.com/OpenWhispr/openwhispr.git
synced 2026-10-20 06:30:20 +08:00
codex/note-header-responsive
130
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Chad Piha
|
c4ae6071e8 |
fix(scripts): let download scripts exit after following a redirect (#2410)
* fix(scripts): let download scripts exit after following a redirect downloadFile followed redirects without reading the redirect body. On Node 24 the global agent keeps sockets alive and only frees (and unrefs) a socket once its response ends, so the unread 302 from huggingface.co pinned the socket and kept `download-whisper-vad-model.js` alive for minutes, stalling the `predev:main` chain before Electron started. Drain redirect bodies in downloadFile and fetchJson, drain fetchJson's non-200 bodies, and destroy a timed-out fetchJson request instead of only rejecting. * fix(scripts): keep a followed redirect from cancelling the live download downloadFile's error and timeout handlers acted on whichever request was current, so a reset on a redirect whose body was still draining destroyed the request that replaced it and forced a retry. Each handler now only acts on its own request; a stale one that times out is just destroyed. Also adds a test for fetchJson's timeout closing the stalled request. |
||
|
Chad Piha
|
bcd9f1a8f7 |
fix(build): ship sherpa-onnx without the GPL espeak-ng library (#2340)
* fix(build): ship sherpa-onnx without the GPL espeak-ng library The default sherpa-onnx release archives are built with TTS on, which statically links espeak-ng (GPL-3.0) into sherpa-onnx-c-api. The app never loads that library: the ASR and diarization executables link only ONNX Runtime. It was bundled only because every library in the archive was copied. - Download the upstream -no-tts archives (SHERPA_ONNX_ENABLE_TTS=OFF) on macOS, Windows and Linux. On macOS the three executables are byte-identical to the TTS build, and on Windows/Linux they differ only in build metadata - Copy only the ONNX Runtime libraries out of the archive - Record the archive name in the install marker so existing 1.13.8 installs from the TTS archive re-extract and drop the stale libraries * fix(build): remove unshipped sherpa-onnx libraries left by pre-marker installs Installs before v1.7.6 wrote no install marker but copied every library in the archive, so marker-based cleanup never removed their sherpa-onnx C API library (which carries espeak-ng) and packaging kept shipping it. Delete any archive library we no longer ship from resources/bin regardless of marker ownership. |
||
|
Chad Piha
|
942df2a3f7 |
feat(mobile): Use the Yowza brand font (#2358)
* feat(scripts): Let download-brand-fonts fetch into another directory Add an --output-dir option so the mobile app can fetch the licensed Yowza files into its own git-ignored assets with the same script, instead of keeping a second copy of the release name, tag and file list. The skip message no longer names Noto Sans, since mobile falls back to Space Grotesk. * feat(mobile): Embed the Yowza brand font in native builds Fetch Yowza on npm install through the shared download script and embed it in the native binary with the expo-font config plugin, the same way desktop bundles it into app.asar. The files stay git-ignored, and embedding them natively keeps them out of the JS bundle, so OTA updates never upload them to the update CDN. The fonts are embedded only when every face is present, so builds without access to the private release still work. Production EAS builds set BRAND_FONTS_REQUIRED=1 so a release cannot ship without the brand font; they need a GH_TOKEN secret with read access to OpenWhispr/brand-assets. * feat(mobile): Set the app in Yowza with Yowza Soft headings Use Yowza for body text when the running binary embeds it, and fall back to Space Grotesk otherwise. The check asks the binary rather than the config, because an OTA update or dev bundle can run on a native build made with or without the fonts. As on desktop, Medium covers semibold because the family has no Semibold. Headings use Yowza Soft Medium, matching desktop's h1-h6: native header titles, and any Text with accessibilityRole="header". Screen and sheet titles and Markdown h1-h3 now carry that role, which also helps VoiceOver. The SpaceGrotesk constant becomes AppFont now that it names either family. * feat(mobile): Set emphasized text in Yowza Soft Medium Yowza's own Medium and Bold read too heavy at mobile sizes, so every medium, semibold and bold weight now uses Yowza Soft Medium, the face headings already use. Regular text stays Yowza. This departs from desktop on purpose. Yowza Medium and Bold are no longer embedded. A nested span without its own weight now keeps its parent's face instead of dropping to regular mid-line, which also keeps accent words in headings in the heading face. Fixes from a simulator pass: tab bar labels use the app font, and the Notes top bar, permission alert, speaker sheets and keyboard-switch onboarding title are marked as headings so they get the heading face. |
||
|
|
f7ea87216f |
feat(notes): tables in the note editor, plus a formatting toolbar (#2255)
* feat(notes): tables in the note editor, saved as GFM pipe tables Notes are stored as Markdown, and the editor had no table node, so any table in a note was flattened into one paragraph on the first desktop edit (#1948). Tables now render and edit, and are kept to what a GFM pipe table can store: a header first row, one line per cell, no merged cells, no alignment or widths. Anything else would be written by tiptap-markdown as the literal text "[table]". Cells hold one paragraph and pasted cells are flattened while parsing; a normalizer fixes header rows, merged cells and line breaks after each change; cells serialize in their own state with pipes escaped. Enter moves down a row and leaves the table on an empty last row, a typed "| a | b |" line becomes a table, and pastes into a cell are flattened unless they are a table or copied rows. A floating menu inserts and deletes rows and columns. * feat(notes): let note actions write tables when asked The wrapper prompts in front of custom actions and Generate Notes forbade tables because the editor could not show them. They now allow a table only when the action's instructions ask for one, so default output stays the same. Detailed Notes and Follow-up email are standalone and unchanged. * fix(search): keep table syntax out of Cmd+K note previews Drop delimiter rows and cell pipes, and collapse whitespace, so a note that holds a table previews as its text. * feat(notes): formatting toolbar on selected text and empty lines One toolbar with bold, italic, strikethrough, code, text style, lists, quote and Insert table. It shows above selected text (Tiptap BubbleMenu) and beside the caret on an empty top-level line (FloatingMenu). Tables keep their own menu, and code blocks get none: the marks don't apply there and a list would take the fence apart. The toolbar and the table menu now hide when focus leaves them from inside, or after a click on them; Tiptap's own blur handling misses both. The hide waits a microtask and skips open dropdowns: Chromium fires focusout while it removes a focused element, and hiding from there broke Tiptap's removal of the menu and skipped the note's save. A dropdown inside a menu is controlled and closes with it. Tiptap hides a menu by detaching its element without React unmounting what's inside, so a dropdown open when the note changed from elsewhere (dictation, a note action) stayed open over a scroll-locked page that swallowed the next click. The table menu had this before the toolbar existed. The nine formatting glyphs are drawn in primitives.tsx, beside the status marks, rather than vendored: the sync script needs a Nucleo license on the machine that runs it. Its generated index re-exports them, so a later vendor pass can replace them without touching the toolbar. * docs(notes): say why table-cell pipe escaping leaves backslashes alone * chore(deps): bump the adm-zip override to 0.6.1 0.6.0 picked up two advisories (symlink extraction, uncontrolled memory allocation), so `npm audit --audit-level=high` fails the quality job on every branch. onnxruntime-node pulls it in. * fix(notes): Close editor menus when editing is disabled --------- Co-authored-by: boseq <307062491+boseq@users.noreply.github.com> Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Nathan RollandChadpiha
|
67dee550db |
feat(dictation): stream Orukeet through managed Cloud sessions (#2231)
* feat(dictation): stream Orukeet through managed Cloud sessions * Bound Orukeet capacity retry timers to the final deadline * Select fixed local retry delays for Orukeet capacity backoff * fix(dictation): Keep managed Orukeet rollouts recoverable on the desktop Four gaps would have made the staged rollout unsafe to operate: - Language gate: the managed route ignored the user's language and the wire protocol carries none, so a Japanese or Chinese user in the cohort would stream to a model that covers 25 languages. An explicitly selected language outside the registry list now takes the batch path, which honors the language end to end. "auto" keeps the model's own detection, as every other streaming provider does. - Start-time fallback: a refused session (403 FEATURE_NOT_ENABLED, 429 RATE_LIMITED, any 5xx) surfaced as a "Failed to start streaming" error and no recording started. It now starts a batch recording, the same path NO_API already took. Auth, policy, quota and upgrade denials still surface. Scoped to the managed Orukeet route. - Config refresh: the STT config was fetched once and pinned for the life of the window, so a server-side rollback never reached a running app. A copy older than 15 minutes is refreshed in the background at recording start, and a FEATURE_NOT_ENABLED denial drops it so the next start refetches before deciding. - Fallback telemetry: batch uploads from an eligible user carried no reason. They now send streamingFallbackReason (feature_disabled, rate_limited, session_unavailable, language_unsupported, stream_no_final) so the rollout's fallback rate is measurable. Backend contract this relies on: the session route's cohort denial is 403 with code FEATURE_NOT_ENABLED and its mint limit is 429 with code RATE_LIMITED. * fix(dictation): Keep streaming fallback telemetry through cloud cleanup With cloud cleanup on, the upload asks /api/transcribe not to log (sendLogs=false) because /api/reason writes the combined STT log, but the cleanup request never carried streamingFallbackReason, so every cleanup user's streaming-to-batch fallback went untracked. Forward the reason through cloud-reason into that combined log request. Translation alone no longer suppresses the transcribe log. It writes no combined STT log, so suppressing it dropped the row and its fallback reason along with it. * fix(dictation): Declare the Orukeet capability and refetch config uncached The backend (OpenWhispr/openwhispr-api#218) now advertises and mints Orukeet only for clients that send x-openwhispr-capabilities: orukeet, since a version number cannot prove a build ships the Orukeet client: a release without it would map the unknown provider to openai-realtime and silently reroute dictation. Send the capability on every policy request through withPolicyRequestHeaders. Fetch cloud config with cache: "no-store", so the refetch after a refused session cannot come back from Chromium's HTTP cache and re-advertise a route the server already withdrew. Tag stream_no_final only when the stream was Orukeet, so the rollout's fallback rate is not mixed with other providers'. The batch fallback itself still runs for every provider. * fix(dictation): Fall back to batch on any Orukeet start failure A managed Orukeet start only fell back to batch for FEATURE_NOT_ENABLED, RATE_LIMITED and 5xx. A network error, the 10 s session timeout, a WebSocket handshake failure or timeout, a rejected session body or a cookie-only session instead showed a "Streaming Error" toast and lost the recording, so an Orukeet outage past its token endpoint broke dictation for the whole cohort. Fall back as session_unavailable on every start failure except the denials the user has to act on: expired or missing sign-in, an account switch mid-start, account required, upgrade required, the word limit and any workspace policy. Still scoped to the managed Orukeet route, so no other provider changes. The fallback log line now carries the error message. * fix(dictation): Show the word limit and upgrade prompt on an Orukeet start At the weekly word limit the session route answers 429 with limitReached, but the desktop dropped that flag, so a cohort user saw "Streaming Error: Failed to start streaming: Weekly word limit reached" with no way to upgrade. Raise the same LIMIT_REACHED code the batch upload uses, carrying wordsUsed and limit, so the error card gets the batch limit title and the server's message, and open the existing upgrade prompt through notifyLimitReached. A 429 RATE_LIMITED from the mint cap is unchanged and still falls back to batch. * fix(dictation): Mint one Orukeet session per re-warm instead of two Every managed Orukeet connection spends a single-use GPU token, and after each dictation both re-warm paths (audioManager and useAudioRecording) call dictation-realtime-warmup at once. The warmup always tore down the current socket and reconnected, so the second call discarded the first's fresh socket and minted again: two tokens per dictation, which also let a normal dictation pace approach the 20-per-minute mint cap and fall back to batch. Run warmups one at a time and have a warmup reuse a connected Orukeet socket that has carried no audio and was opened for the same route, reporting alreadyWarm. A socket that has carried audio is never reused. Other providers keep reconnecting as before. The connection key that start, connect and warmup compare now comes from one helper. * Expose Orukeet audio language metadata for cloud fallback * Bound Orukeet idle sessions and support account-bound tokens --------- Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
JoshuaandClaude Fable 5.1
|
6d56d75e7e |
fix(tinfoil): delist the retired deepseek-v4-flash — canary probe, startup remap, fixtures (#2250)
* fix(canary): probe deepseek-v4-1-flash on Tinfoil, the id that replaced deepseek-v4-flash Tinfoil's live catalog no longer serves deepseek-v4-flash (canary run 35322067138 on 2026-09-18: HTTP 404 "The model does not exist"); the replacement id is deepseek-v4-1-flash. The registry never pinned the old id, so this is a stale probe, not user-facing drift. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tinfoil): retire deepseek-v4-flash in the startup remap and test fixtures Tinfoil retired deepseek-v4-flash for deepseek-v4-1-flash. The startup sweep still mapped deepseek-v4-pro onto the retired id, and its sentinel shipped in v1.10.0, so both ids now map to the successor under a rotated key. Fixtures that called the retired id "what Tinfoil serves first" follow the live catalog. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(tinfoil): name the rotated sentinel in the quota-failure test; correct the seed history Review findings on #2250: the quota-failure test still asserted the pre-rotation key, which nothing writes any more, and the retired-table comment carried "through v1.7.6" for two ids whose seeds actually ended in v1.7.4 (deepseek-v4-pro, dropped by |
||
|
Emanuele TonelloandChadpiha
|
ecd9459121 |
chore(transcription): update sherpa-onnx to 1.13.8 (#1951)
* chore(deps): update sherpa-onnx to 1.13.7 * fix(sherpa): recover from malformed install markers * chore(deps): update sherpa-onnx to 1.13.8 * fix(transcription): Repair sherpa runtime validation and retries --------- Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Gabriel Stein
|
94aeecb75a |
fix(notes): give note formatting a ten-minute deadline and stop retrying timeouts (#2204)
## Problem Generate AI Summary with a bring-your-own-key provider sends the whole transcript to a model that may reason for a minute or more before it writes. Every BYOK request shared the 30-second deadline meant for dictation cleanup, and an expired deadline was treated like a dropped connection and retried three more times. A 2,170-word transcript through GPT-5.6 Terra spun for about 1.5 minutes, was billed for four requests, and ended with "Request timed out after 30s" and no note. 1.10.1 exposed this: raising the reasoning-model output cap to 25k tokens let Terra think long enough to cross 30s. Enterprise reasoning (Bedrock, Azure, Vertex) had the same bug behind a different constant: the main-process handler applied its own 60-second default because the renderer never sent a deadline. ## Fix - `getLlmRequestTimeoutSeconds` takes the request's inference scope. `noteFormatting` gets 600s (the OpenAI SDK's own default); dictation cleanup keeps 30s; streaming keeps its 60s idle deadline. - `llmRequestTimeoutError` carries `LLM_REQUEST_TIMEOUT_CODE`, and `createApiRetryStrategy` refuses to retry it. The same request under the same deadline expires again, and every attempt is billed. - OpenAI, Gemini, Tinfoil and the generic chat-completions path (Groq, LAN, custom endpoints) pass their scope and raise the coded error. Tinfoil maps the SDK's `APIConnectionTimeoutError` onto it so it is not re-sent either. - The enterprise provider sends `timeoutMs` from the same helper. The handler already honoured that field. Note: this moves enterprise dictation cleanup from 60s to 30s, matching every other provider. - Note prompt wrappers and `NOTE_OUTPUT_MAX_TOKENS` move from the action store to `builtinActions.js` so the live canary sends exactly what the app sends. ## Tests - Every inference scope must be classified as interactive (≤ 60s) or background (≥ 300s); a new scope with no verdict fails. - OpenAI and Gemini note requests survive the 30s tick, fail once at 600s, and issue exactly one POST. Cleanup still times out at 30s. - A note request through the real `ReasoningService` dispatch outlives the dictation deadline and is sent once. - Tinfoil receives the long deadline, maps the SDK timeout to the coded error, and does not re-send. - Enterprise forwards 600s for notes and 30s for cleanup to the main process. - `withRetry` makes exactly one attempt for a coded timeout. - The note store's requests carry the `noteFormatting` scope. ## Canary `llm-canary` now runs one meeting-sized Detailed Notes request per keyed provider (gpt-5.6-terra, gemini-3.5-flash) through the app's real provider code under the note-formatting deadline, reporting elapsed time, tokens, and a slow warning past half the budget. This is the only layer that can catch a provider or model change crossing the deadline. |
||
|
|
685fceabdf |
fix(linux): keep the dictation pill hoverable after the first hover, and visible after Retry (#2161)
* fix(linux): keep the dictation pill clickable after the first hover
setMainWindowInteractivity returned the pill to click-through on mouse-leave
with setIgnoreMouseEvents(true, { forward: true }). Electron ignores `forward`
outside macOS: on Linux it sets a 1x1 X11 input region and nothing forwards
hover back, so the window never saw another mouseenter. The cancel X appeared
on the first hover and never again, and a recording could no longer be
cancelled from the pill.
Only macOS can forward hover through a click-through window, so treat Linux
like Windows and keep normal hit-testing. This is the same Electron limitation
that stranded the meeting card's buttons in #1456, and the companion pill
already guards it this way.
Reproduced on Ubuntu 26.04 GNOME Wayland (the app forces XWayland) with real
uinput pointer events: hover #1 logged setIgnoreMouseEvents(false) and revealed
the cancel button; after the mouse-leave put the window back into click-through,
hovers #2 and #3 produced no native calls at all.
Closes #2136
Claude-Session: https://claude.ai/code/session_01Rs6g9RUbpHn5Hg8gBRPdWx
* fix(dictation): keep the pill visible when Retry starts a new recording
Retry on the dictation error card starts the next dictation, and a successful
start dismisses the card. Leaving the error footprint releases the pill
visibility handoff, whose shouldAutoHide only asked whether "Auto-hide when
idle" was enabled — so it hid the whole pill window while that new recording
was running. The recording kept going with nothing on screen until the user
pressed the hotkey again. The same happens when the hotkey (rather than Retry)
starts a dictation while the card is up.
Feed the hook the live dictation state and let auto-hide act only when nothing
is recording or processing, which is the idle condition App.jsx's own auto-hide
effect already applies.
Reproduced on Ubuntu 26.04 GNOME Wayland: after the Retry click the main
process logged hide() and reported the window not visible for the rest of the
recording, while a capture stream stayed live.
Closes #2141
Claude-Session: https://claude.ai/code/session_01Rs6g9RUbpHn5Hg8gBRPdWx
* fix(linux): Preserve click-through around the dictation pill
Recover hover from native pointer coordinates so the pill and cancel control
remain usable without blocking clicks through the transparent window padding.
Suspend sampling while hidden and preserve capture during menus, errors,
panels, and dragging. Keep the tooltip aligned with recovered hover state.
Cover pointer recovery, visibility changes, stale replies, zoom conversion,
IPC sender validation, and unchanged Windows and macOS input handling.
* fix(linux): Restore pill hover with native input regions
Use X11 input shaping to keep the pill interactive without relying on cached
cursor coordinates. Preserve click-through padding and sequence overlay capture
through the same native writer.
Cover native hit testing, scaling, lifecycle, and helper failure fallback.
---------
Co-authored-by: boseq <307062491+boseq@users.noreply.github.com>
Co-authored-by: Chadpiha <chadpiha23@gmail.com>
|
||
|
Gabriel Stein
|
a7f22e07c3 |
feat(notes): space settings modal, in-app emoji picker, membership UI cleanup (#2186)
* feat(notes): space settings modal, in-app emoji picker, membership UI cleanup - Replace the OS emoji panel with a searchable, categorised, virtualised in-app picker (emojibase data per locale, emoji-version probe, recents) - Add a Space settings modal (General | Members) behind a single Settings menu item; name and emoji save in place, members can leave, admins delete through the existing type-the-name confirm - Merge role and remove into one quiet menu per roster row and collapse the add-people list until focused - Onboarding: an existing macOS Screen Recording grant now counts as the screen-context opt-in; the setup card shows all four local model marks - Remove the show-emoji-panel IPC path and the inline space rename * fix(notes): keep focus in the delete confirm and select the space name once per open |
||
|
Chad Piha
|
9789e46646 |
fix(transcription): repair BYOK Deepgram and AssemblyAI streaming (#2149)
* fix(transcription): authenticate BYOK Deepgram with the Token scheme Deepgram binds the Authorization scheme to the kind of credential: a raw API key is only accepted as `Token`, while `Bearer` is reserved for the short-lived credential /v1/auth/grant mints. The streaming client presented every credential as a `Bearer`, so bring-your-own-key dictation and Note Recording failed the handshake with 401 while OpenWhispr Cloud — which receives a real grant token — kept working. providerConnectionTest.js already used `Token`, which is why the same key tested green in Settings. Pick the scheme from the mode adoptMode() already normalizes, and forward that mode into both meeting connectOpts, which carried `preconfigured` but not `mode` and so re-authenticated Note Recording as managed. Snapshot the saved options in scheduleRewarm(), as scheduleProactiveRefresh already does: cleanupWarmConnection() drops them without cancelling the timer, so spreading a null one lost the mode and re-warmed a BYOK key as a Bearer. The canary now takes both its header and its URL from the client so the probe cannot drift from what ships. That mirroring did not hide this bug — the old probe sent the same wrong header the client did; the row has simply never run, for want of a configured STT_CANARY_DEEPGRAM_KEY. Fixes #2140 * fix(meetings): keep AssemblyAI audio frames inside its duration window AssemblyAI accepts 50-1000 ms per frame and closes the session otherwise. The minimum frame size was derived from this module's hardcoded 16 kHz, but Note Recording opens the socket at MEETING_STREAM_SAMPLE_RATE (24 kHz), where the same 1600 bytes lasts 33 ms. With the AEC helper's 10 ms frames the gate emitted exactly 40.0 ms — the figure in the bug report — and the session died seconds in. Dictation, pinned to 16 kHz, was unaffected. Derive both bounds from the rate the socket was actually opened at. The ceiling now matters: the macOS tap and the Windows/Linux loopback helpers forward raw stdout chunks, so a stalled main process yields a 64 KB pipe read — 1365 ms at 24 kHz — which the floor fix would otherwise have exposed for the first time, since these sessions never survived long enough to reach it. Oversized frames are sliced and any sub-floor remainder is carried forward rather than sent short. Compare sampleRate alongside model when deciding whether a warm socket can be reused; the rate is pinned at open, so reusing one across rates would stream 24 kHz PCM down a socket declaring 16 kHz and garble the transcript silently. Fixes #2140 * docs: trim the #2140 comments to what the code cannot say itself Drop the historical narrative from the canary header, the scheme explanation duplicated onto the Deepgram test from the helper it describes, and the notes restating what the adjacent line already shows. What remains is the provider behaviour that cannot be recovered by reading the code: Deepgram binding scheme to credential kind, AssemblyAI's 50-1000 ms window, the coalescing stdout reads, and the two test traps (an unanchored source match accepts a commented-out key; node:test has no default timeout). No behaviour change. * test(transcription): pin the #2140 behaviours that shipped uncovered Three of the fixes landed without a test, and each one fails silently: the carry-forward drops audio, the sample-rate guard streams 24 kHz PCM down a socket declaring 16 kHz, and the re-warm guard re-authenticates a BYOK key as a Bearer. Reverting any of them left the suite green. collectFrames now takes the chunk sizes, so a case can leave a sub-floor tail instead of only sending exact multiples of the floor — neither existing frame test ever reached the carry-forward branch. The re-warm guard is timer-gated. node:test's mock timers deadlock the ws server's teardown, so captureScheduledCallback stubs setTimeout for the single synchronous scheduleRewarm() call and hands back the callback; the rewarm reaches warmup() before its first await, so the assertions settle without waiting on a handshake. * fix(meetings): split AssemblyAI audio at its documented frame ceiling AssemblyAI hard-closes the session outside 50-1000 ms but documents 50-250 ms as the supported shape, and slicing at the hard limit put the first piece of a coalesced 64 KB read on exactly 1000.0 ms — the bound the split exists to stay under. The floor already kept a margin (Math.ceil), the ceiling did not. A 64 KB read now goes out as 5x250 ms plus a 115 ms tail rather than 1000 ms plus 365 ms. Dictation is untouched: at 16 kHz the accumulator flushes at 1600 bytes and never reaches the ceiling at either value. * fix(transcription): cold-start Deepgram when the warm socket was opened for other options buildWebSocketUrl pins sample_rate, the language — which also selects nova-3 vs nova-2 and the keyterm parameter name — and the keyterms, and the socket carries them for its whole life. connect() promoted a warm connection on nothing but its existence, so a session could run entirely under another session's settings, and Deepgram had no reason to object to the audio. Dictation reaches this on every translation: warmupStreamingConnection() runs at startup and after each transcription, before setTranslationRequested() can say the source language is wanted, so the socket is always warmed on the UI language. Aligning the two call sites is not possible — the intent does not exist yet at warmup — so the mismatch has to be caught here, as the AssemblyAI client already does for its model and sample rate. A dictionary edit between warmup and start reaches the socket the same way. The identity folds "auto" and empty keyterms exactly as the builder does, so the ordinary dictation path still rides the warm socket and keeps its instant start. |
||
|
Zeljaa
|
32541ce2ac |
fix(macos): use the arm64-native ONNX Runtime slice for sherpa-onnx on Apple silicon (#2116)
sherpa-onnx's macOS archives bundle a universal2 libonnxruntime whose arm64 slice runs INT8 models ~3x slower than the arm64-only build of the same ONNX Runtime version from the same maintainer. On macOS hosts the download script now fetches that build, verifies its SHA-256, and rebuilds libonnxruntime.1.27.0.dylib with its arm64 slice while keeping the x86_64 slice, so the file stays universal2 and Intel Macs are unchanged. Linux and Windows are untouched. The install marker records the new slice so existing dev installs re-extract. Apple M5, 11 s clip, warm median of 10 app-level calls, identical transcripts: orukeet-v0.1.0 605 ms -> 173 ms parakeet-tdt-0.6b-v3 665 ms -> 230 ms |
||
|
Gabriel Stein
|
f27f318cfb |
feat(ui): adopt Yowza brand font and Nucleo icon set (#2117)
Replace lucide-react with vendored Nucleo core outline icons behind the existing lucide-style names (src/components/icons), generated from nucleo-map.json by scripts/sync-nucleo-icons.js so the bundle is offline and license-free. Circle and Square are hand-drawn primitives; the retry icon uses ui/redo-2. Set the UI in Yowza with Yowza Soft for headings. The font files are licensed and never committed: scripts/download-brand-fonts.js fetches them from the private OpenWhispr/brand-assets release during the dev and build pre-steps, src/brandFonts.ts registers whatever is present, and vite bundles them inside app.asar (excluded from extraResources). Builds without access fall back to Noto Sans; release CI requires them via BRAND_FONTS_REQUIRED=1. Weight 600 maps to Medium. |
||
|
Nathan Roll
|
7e3e0f785b |
feat(transcription): add Orukeet through the existing Parakeet ONNX runtime (#2085)
* Add Orukeet as the recommended local ASR model with native hardware acceleration * Run native qualification in Electron and retain the onboarding provider scope * Keep Orukeet available below the ONNX macOS floor and fix Windows test cleanup * Parse benchmark options independently of Electron platform flags * Separate benchmark models from Chromium profile caches * Register Orukeet in the existing Parakeet ONNX runtime * Show Oruk branding and recommendation in local model selection * Fix Windows bzip2 extraction for fresh local-model installs * Speed up Orukeet with the existing sherpa runtime * Show the no-speech message for silent local uploads * Keep live preview updating across consecutive recordings * Preserve existing provider order and stock upload labels |
||
|
Gabriel Stein
|
61c6fdec96 |
feat(transcription): add gpt-transcribe as the OpenAI BYOK batch default (#2076)
OpenAI deprecated whisper-1, gpt-4o-transcribe and gpt-4o-mini-transcribe on 2026-08-26 (removal 2027-02-26); gpt-transcribe is the batch successor. - Registry: gpt-transcribe is the first OpenAI transcription model, so new BYOK selections land on it. The three deprecated models stay selectable and reach the API untouched; no stored selection is rewritten. - resolveByokModel accepts the gpt-transcribe prefix and falls back to it for an empty or foreign OpenAI selection. - The custom dictionary rides gpt-transcribe's keywords[] channel (one field per term, scrubbed of the characters OpenAI rejects) instead of the prompt, which keeps only the Chinese script bias. Legacy models keep sending prompt. - shouldStreamTranscription includes gpt-transcribe; its SSE events match the existing reader. - stt-canary posts gpt-transcribe with a keywords[] entry on the OpenAI key. - Locale description in all ten languages. |
||
|
Gabriel Stein
|
82742ae058 |
feat(transcription): Deepgram and AssemblyAI as streaming-only BYOK providers (#2063)
* feat(transcription): add Deepgram and AssemblyAI as streaming-only BYOK providers * feat(assemblyai): offer Universal-3.5 Pro and warn on silent model downgrade * fix(byok): forward-compatible policy validation, streaming-only guards, mode-scoped token caches, chat mic gating |
||
|
Gabriel Stein
|
9c41a8705e |
feat(transcription): Gemini Live streaming for dictation (#2062)
* feat(transcription): add Gemini Live streaming for dictation * fix(gemini-live): flush cold-start audio on readiness and pin the live model id * fix(gemini-live): classify close codes, honor the final-transcript budget, serialize warmup * test(canary): pace the Gemini Live probe like the mic worklet * fix(gemini-live): honor a turn the server closed before the hotkey release |
||
|
|
60bc697c01 |
feat: add analytics leaderboard (#2011)
* feat(analytics): add production leaderboard experience * feat(analytics): move leaderboard to production tab * fix(analytics): review iteration 1 — 8 issues (leaderboard consent, privacy) - Leaderboard participation is an explicit join, never inherited from the Insights-sync toggle; the reconcile effect is read-only and policy-guarded - Opt-out applies locally before the network call and surfaces failures - Share card shows name or email local part, never full colleague emails - Jump-to-rank scrolls on the current page instead of relying on setPage - Signed-out users get a preview with a sign-in CTA - Dropped dead LeaderboardPreview variant prop and data attributes - Day counts use i18next plural keys (dayValue_one/_other) Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 2 — 7 issues (join consent order, leave action) - joinLeaderboard waits for the Insights-sync claim dialog and PATCHes participation only after sync is actually on; declining never joins - Leave button on the leaderboard header mirrors join via leaveLeaderboard - Settings row no longer shows leaderboard errors; a failed opt-out toasts - A 403 from the leaderboard endpoints re-reads participation or access based on its code instead of offering a Retry that cannot succeed - memberValue lives once in helpers/leaderboard.ts; pageForRank/pageCount take the response's pageSize, and refresh honours refreshAfterSeconds - Participation is read only from LeaderboardView, not Settings/Insights Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 3 — 3 issues (leave gating, invite scope) - The Leave control follows the account's participation state, not the local Insights-sync toggle, so a still-published account (failed opt-out or a second device) can always leave; a failed leave toasts on the board - Inviting a teammate no longer switches the app-wide active workspace - The Settings opt-out is a silent best-effort call; leaderboard errors are reported only where participation is known Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 4 — 4 issues (403 loop, stale participation) - A failed participation read fails closed (participationEnabled false), so the SYNC_REQUIRED re-read path terminates instead of looping 403s - LeaderboardView re-reads the account's participation when the local Insights-sync toggle flips, so a Settings opt-out settles the board at once - Drop the unused setInsightsSyncEnabled destructure in SettingsPage - WEEKLY_METRICS/ALL_TIME_METRICS live once in helpers/leaderboard.ts beside the selection rules that use the same lifetime-metric fact Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 5 — 2 issues (join/read race, signed-out opt-out) - A successful join or leave PATCH publishes its answer through the hook's read-id guard, so a participation GET that started earlier (for example the one the sync toggle flip fires during join) can no longer overwrite it - Turning Insights sync off while signed out stops at the device instead of issuing a PATCH that is guaranteed to fail auth validation Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * feat(analytics): durable leaderboard opt-out and retry on failed participation read - A leave that fails or happens offline is persisted per account as a pending-leave record and retried on every sync pass and participation read until the PATCH lands; the retry only ever leaves, never joins, and an explicit Join clears it. Meanwhile the board shows the user as not participating and a neutral "we'll finish on the account" toast - A failed participation read is distinguished from "not a participant": it renders an error card with Retry instead of offering Join to someone who may already be joined - One shared LeaderboardRetryCard backs the access, load and read errors Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 7 — 2 issues (pending-leave race, per-account record) - Join retires the queued leave right after the consent gate and before its PATCH, so the toggle-driven participation re-read can no longer flush a PATCH false that races the join - Pending leaves are stored as a per-account set under one key, so a second account's failed leave never erases the first account's undelivered opt-out, and flush only ever sends the signed-in user's entry Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 8 — 2 issues (shared participation store) - Leaderboard participation state (enabled, ready, error, updating, the read-id guard, refresh/publish/join/leave) now lives in one zustand store that every useInsightsSyncOptIn instance reads and writes, so an opt-out from Settings reaches the leaderboard view instead of a private copy - refresh() defers while a write is in flight, so the sync-toggle flip no longer dispatches a read that races the leave PATCH; the hook's public API and all consumers are unchanged Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 9 — 1 issue (reset participation store on account purge) - The account-scope purge in useAuth resets the leaderboard participation store beside the Insights-sync reset, so a sign-out or account switch clears in-memory participation and retires in-flight reads even when the leaderboard view is not mounted; the per-account pending-leave record is deliberately left intact Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * fix(analytics): review iteration 10 — 1 issue (reset clears updating, stale writes dropped) - reset() clears the updating flag, and join/leave capture the store's read generation so a write started by a departing account can never publish into the next account's participation state; the account-tagged pending-leave record still survives a reset Claude-Session: https://claude.ai/code/session_018ze5JCkitdstyDfiDz4rum * feat(analytics): unlock leaderboards across plans * feat(analytics): refine leaderboard experience * chore(dev): add leaderboard v1 profile harness * feat(leaderboard): add solo invitation empty state * refactor(leaderboard): model free access surfaces * feat(leaderboard): tailor access state surfaces * feat(leaderboard): complete workspace entry flows * feat(leaderboard): make scope selection deterministic * feat(leaderboard): nudge inactive teammates * test(leaderboard): cover free access surfaces * chore(dev): harden leaderboard profile harness * fix(leaderboard): close production review gaps * Remove local leaderboard QA harness * fix(leaderboard): close review gaps * fix(leaderboard): address final review findings * feat(leaderboard): refine ranking controls * fix(leaderboard): clarify participation copy * fix(leaderboard): allow leaving from empty state * feat(insights): unify leaderboard participation * fix(insights): align leaderboard tabs and join copy * fix(leaderboard): reconcile access and participation * Polish Insights sync controls * fix(leaderboard): harden consent and failure flows * fix: decouple leaderboard participation from Insights Sync * fix: simplify leaderboard auth transitions * feat: refine leaderboard privacy controls * fix(leaderboard): keep ready boards accessible * fix(leaderboard): align count copy and paging * fix(i18n): complete Russian leaderboard plurals and interpolate counts Russian needs _few and _many, and i18next falls back to English when the form it selected is missing, so counts of 2, 5 and 40 rendered English text inside the Russian UI. The _one forms also wrote the digit 1, which Russian selects again at 21 and 31 -- a 21-member workspace read "1 участник". French and Portuguese select _one for 0 as well, so they carried the same hardcoded digit. Every locale's _one now interpolates {{count}}; where the category only ever means exactly one, rendering is unchanged. check-i18n now asserts both rules: a plural base must define every category a realistic count selects, and a form covering counts other than 1 must interpolate. Probing 0-200 keeps the millions-only 'many' of es/fr/it/pt out of scope, so no pre-existing key is affected. * fix(leaderboard): keep a compensating leave only when the join outcome is unknown A failed join banked a compensating leave unconditionally. Against an API without the participation route -- the staggered-rollout window the 404 read fallback exists for -- the retry PATCH 404s too, so the record was never cleared, and SyncService treats a pending leave as a reason to bypass the sync throttle. One tap on the Settings toggle therefore ran a full syncAll on every focus, visibility and online event, indefinitely and across restarts. A 4xx is the server's verdict on a request it declined to act on, so there is nothing to compensate. No response, a server error, or a body this device could not read still bank the leave: the join may have committed before the failure surfaced, and the user was told it had not. * fix(leaderboard): reject week values and refresh intervals the renderer cannot use weekStart and availableWeekStarts were validated as strings, but the week picker formats them with Intl in the render body. An empty or unparseable value threw RangeError past every async fallback, and with no boundary around the lazy LeaderboardView that reaches the app-level one, which replaces the whole control panel. They are now checked as real calendar dates by round trip, so a well-shaped day that does not exist is refused too. refreshAfterSeconds had a lower bound but no upper one and drives setInterval, which clamps a delay past a signed 32-bit millisecond count to 1ms -- turning an implausible value into a request loop against an endpoint that re-aggregates the roster on every call. Both now fail validation inside the request, where the existing catch turns them into the retry card. * fix(leaderboard): bound an undeliverable opt-out and withhold domain addresses A pending leave let SyncService bypass the sync throttle, the in-flight guard and the ambient backoff. Only the join path checked whether the failure left anything to compensate, so a leave the API refused banked a record that no retry could clear: every focus, visibility change, online event and interval then ran an unthrottled full syncAll for the life of the install, across restarts. The outcome check now covers both paths and classifies what the write actually did. cloudPatchForAuthGeneration raises CloudApiError for every transport and HTTP failure, so any other error escaped while reading a body the API had already answered 2xx with -- the write landed. Banking a compensating leave there silently reverted a join that had succeeded, and a test pinned that as intended; it now asserts the opposite. A request the main process fenced before the wire compensates nothing either. The record itself still outlives every outcome that leaves the row untouched, because only a delivered PATCH may retire an opt-out. What expires is its claim on priority: after five undelivered attempts it keeps retrying on ordinary passes without commandeering them. Domain boards no longer carry member addresses. A domain roster is assembled from a shared mail suffix and nothing else, and the suffix is only as good as the provider deny-list, so withholding the address keeps a walkable directory of strangers' real emails off the wire even when eligibility is wrong. Workspace rosters are administered and keep theirs. Also rewrites a bare-hostname includes() assertion that CodeQL reads as incomplete URL sanitization, which was failing the required check. * fix(leaderboard): say when an opt-out is still owed, and keep its retry off the sync lock A leave the network refused published the same answer as one the account had taken: enabled false, error null. The toggle read off, the Insights chip said "disabled", and after the one-shot toast nothing on any surface distinguished the two -- while the account stayed on every colleague's board until a retry landed, or forever if none could. The store now carries leavePending, derived from whether the durable record survived the attempt (an API that has already applied the leave, or has no participation route, retires it), and the Settings description, the Insights chip and the Join card all say the account is still coming off the boards instead of calling it done. The retry itself ran ahead of every analytics pass, awaited under SYNC_ALL_LOCK. Cloud requests carry no timeout, so one hung PATCH held every window's sync behind the lock for as long as the socket stayed open. Nothing in the pass depends on whether that leave has landed, so it now runs beside the pass rather than in front of it; the account-changed rejection it can raise is the one the pass's own gate already handles. * fix(leaderboard): stop the solo funnel, the page turn and an unvalidated session dead-ending Three ways the board left a user with nowhere to go. A scope of one resolved to the solo empty state before participation was consulted, so an account that never opted in was answered "invite someone" by a surface that carries no way to join. Create workspace -- the funnel's own CTA -- landed exactly there, under an Insights chip still reading "Leaderboard is disabled". Participation now decides first: a solo scope asks a non-participant to join, and becomes the invite state once they have. A page turn, metric change or week change blanked the whole card, because the board was shown only while the loaded key matched the selected one. That took the podium, the table, the period picker, Share and the pagination control the user had just clicked, then snapped it all back. The previous answer now stays on screen, dimmed and aria-busy, until the next one arrives; a different scope is a different board and still clears. Failures keep the surface: the generic path leaves the previous board in state, so the guard is "no fresh board" rather than "no board". useAuth deliberately keeps presenting a session whose refetch failed, which leaves a signed-in user holding no validated credential for as long as that lasts. loadAccess treated it as loading, so the tab sat on a bare spinner with no message and no retry -- while the Usage tab beside it falls back to device-scoped data. It now waits only while the session is unsettled, and once settled without a credential it renders the retry card, which already offers Sign in. --------- Co-authored-by: boseq <307062491+boseq@users.noreply.github.com> Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Chad Piha
|
4d6077ded8 |
fix(parakeet): ship ONNX Runtime under a private DLL name on Windows (#2074)
* build(win32): add a PE import-name reader and patcher Read and rewrite module names in a PE image's import and delay-load directories. Names are patched in place, so a replacement can never be longer than the original. Needed to point the bundled sherpa-onnx binaries at a privately named ONNX Runtime (#2054). * fix(parakeet): ship ONNX Runtime as ow-onnxrt.dll on Windows (#2054) Windows 11 ships an older onnxruntime.dll (1.17) in System32. The bundled sherpa-onnx 1.13.4 binaries are built against ORT 1.27 and bind the runtime by bare module name, and on some machines the loader resolves that name to the System32 copy even though ours sits beside the exe. The exe then fails with "requested API version [27] is not available" before it can listen, which the app reports as "parakeet-ws process died during startup". Rename the extracted runtime to ow-onnxrt.dll at download time and rewrite the import tables of every sherpa image to match, so no name-based loader rule (PreferSystem32Images, KnownDlls, an already loaded module) can substitute the system copy. Record the rename in the install marker so checkouts extracted before this change re-download. Upstream's archive lists each DLL twice (bin/ and lib/), so the library list is deduplicated. Remove once microsoft/onnxruntime#27893 (version-suffixed DLL names) lands and sherpa-onnx adopts it. * build(win32): fail packaging if onnxruntime.dll would ship The Windows package must contain ow-onnxrt.dll and no onnxruntime.dll. A stray upstream DLL or a missing private one means the rename step in download-sherpa-onnx.js was skipped (stale cache, older marker) and Parakeet would die at startup for users (#2054). * fix(parakeet): Invalidate install markers before automatic repairs Clear the completion marker before every reinstall so a failed DLL patch cannot leave a partially patched bundle marked complete. Add regression coverage for patch failure and successful automatic retry. |
||
|
|
ba2a6414a9 |
feat(dictation): liquid cancel button, pill spacing, and overlay smoothness (#1886)
* feat(dictation): liquid cancel button and pill spacing polish The hover cancel control now pours out of the recording pill as a fused liquid skin: an SDF smooth-union (smin) of the pill capsule and the button circle, traced by marching squares into one SVG outline, animated over 280ms on the house curve. The blend ramps with a smoothstep so the skin matches the bare pill exactly at mount, and the emergence rAF loop re-renders only the leaf component. Inside the Live Transcript panel the button keeps its classic bordered look with the same emergence motion. The compact pill widens 92 -> 98 for deliberate 6/12 edge insets (icon / waveform side), the icon-waveform gap becomes an owned constant, and the RECORDING window widens 128 -> 142 so pill + gap + cancel fit exactly at the 4px dock insets - the cancel button previously clipped outside the window on hover. Live-transcript pill vars and the panel grow apertures move with the footprint contract. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(dictation): harden liquid cancel emergence * fix(dictation): match the liquid skin's stroke to the pill state it replaces The fused skin hardcoded the recording chrome (border-hover stroke), so hovering during processing/thinking — whose pills wear the softer border/60 — popped the outline color the moment the skin took over. The stroke now comes from VOICE_PILL_STATE_STROKE, the SVG mirror of STATE_APPEARANCE's border classes (kept in voicePillPresentation.js so the component file stays Fast Refresh-clean). Fill stays surface-1: every state the skin can appear over shares it. * fix(dictation): tween the liquid skin between pill footprints The skin's capsule geometry came straight from the footprint constants, which flip discretely, while the real pill animates width/height over GROW_TRANSITION (300ms) — and a fused pill is headless, so the skin is the only chrome on screen. Clicking the pill to start recording popped a full 98px outline the moment the entrance expansion began (overshooting the growing pill by up to ~58px), and stopping or cancelling snapped the skin to a 40px circle while the pill's contents were still shrinking under it. The leaf now chases the footprint contract with the pill's own duration and curve (usePillFootprintTween), and keeps chasing while hidden so an emergence that starts mid-flight picks up the pill's current in-between size. Reduced motion snaps, matching the global rule that drops width/height from transition-property. The t=0 tangency the tween relies on (skin ≡ pill for any capsule ≥28px tall) is pinned for in-between footprints, and stitch()'s eps assumption is documented. * fix(dictation): drive the liquid skin's chrome from CSS so it tracks pill state The inline stroke map couldn't express everything the pill's own chrome does: the hovered pill sits on surface-3 (the skin's hardcoded surface-1 fill popped brighter when a de-fusing skin unmounted over a hovered pill), the idle pill wears a softer border in dark theme (dark:border-border/50), and the pill transitions border/background colors over 220ms while the skin's snapped. The skin's path is now styled by .liquid-cancel-skin[data-pill-state] rules that mirror STATE_APPEARANCE per state with the same 220ms color transition (fill and stroke transitions deliberately survive the global reduced-motion rule). The VOICE_PILL_STATE_STROKE map goes away with its only consumer. * fix(dictation): drop the inline hover shadow while the liquid skin owns the chrome The fused rule's box-shadow: none is a stylesheet rule, so the inline floatingHover shadow outranked it and painted a phantom capsule shadow over the skin for the 280ms a de-fusing skin lingers on a hovered pill. * fix(dictation): stop resizing the pill window on recording edges Resizing the transparent always-on-top window paints one compositor frame of the stale texture inside the new bounds before the renderer catches up, and no resize mask can cover it — every recording start grew the window and every stop shrank it back, kicking the pill each time (intermittently on start, reliably on stop). - BASE and RECORDING now share one 208x120 box, so recording edges never call setBounds; the ladder keys stay distinct for ranking - the restore-to-BASE path short-circuits when bounds are unchanged instead of running the mask handshake and a no-op setBounds - the entrance hold drops 420ms -> 260ms for the floating pill now that it no longer hides a native grow; recording under the open assistant panel keeps the 420ms hold so the expansion still waits out the footer's actions->pill handoff * fix(dictation): linger the swallowed cancel skin under the pill's returning chrome Fuse-in was already masked: the skin mounts under the pill while its chrome fades out over 220ms, and the two capsules are identical so the paint never changes. Un-fuse had no mask — at t=0 the skin unmounted instantly while the pill's chrome faded back in from transparent, dipping the capsule for the fade duration on every hover-out. Keep painting the swallowed (t=0, geometry-identical) skin under the returning chrome until the fade lands, then unmount it. Re-fusing during the linger cancels it. * fix(dictation): mask the panel-return shrink behind a pill fade Closing the assistant or live-transcript panel snapped the native window from panel bounds back to the pill box the instant the panel unmounted — racing the pill's 320ms dock travel by a frame, and the big shrink paints one compositor frame of the old texture inside the new bounds, blanking the just-settled pill (the artifact the dictation-error handoff already documents). Panel returns now reuse that handoff pattern: fade the pill out over its 150ms opacity transition, resize while nothing is visible, chase any superseding menu/toast target, and reveal once bounds settle. A panel reopening mid-fade re-takes the window and reveals the pill without shrinking under it. * fix(dictation): let the panel exit fade ride the contraction The surface faded out in 140ms while its clip-path contraction runs 280ms, so closing read as a blink-vanish two frames in — the panel never visibly condensed into the pill. Split the durations by direction: the entrance keeps the fast 140ms fade (solid quickly, the morph carries), the exit fades over 240ms so the condense motion stays visible to the end. * fix(overlay): only re-apply Spaces membership when it actually drifted setupAlwaysOnTop lands on hot paths (assistant panel open/close, window show) and re-applied setVisibleOnAllWorkspaces every time. Re-asserting unchanged collection behavior makes the window server momentarily pull the window out of the active Space, which can blink the entire visible window. Guard it (and setFullScreenable) behind the current state so enforcement is idempotent. * fix(assistant): never request app activation for the panel on macOS Opening the panel called focus(), which asks NSApp to activate; when the user has another OpenWhispr window (control panel) on a different Space, macOS answers a granted activation by sliding the whole desktop over to it — and slides back when close blurs. That whole-screen slide was the 'massive flash' on panel open and close. It reproduces only after real user interaction (macOS denies synthetic activations), which is why it was intermittent. The overlay is a non-activating panel: skip focus() and blur() on macOS. Clicking the panel's input still makes it key (typing and Escape work from then on) without activating the app, and opening the panel no longer steals the keyboard from the app the user was in. Windows/Linux keep the explicit focus/blur handoff. * fix(agent): give the companion pill the liquid cancel button The opposite-edge companion still carried the pre-liquid cancel — a bordered circle popping in with no motion, next to a main pill whose cancel pours out as a fused skin. Adopt LiquidCancelButton with the same wiring as App.jsx: footprint targets from the shared contract, fused everywhere except inside the Live Transcript panel, and the pill goes headless while the skin owns the fused surface. * fix(assistant): close as one beat — fold the pill into the panel exit The close choreography read as two beats: the surface condensed into the pill's corner, and then the pill immediately faded out for the masked window shrink and faded back in. Suppress the pill for the whole of assistant.closing instead: it fades with the closing content, stays hidden through the contraction, travel, and shrink (the panel-return mask picks up at unmount without a visible gap), and materializes once at its settled dock. * fix(onboarding): manual titlebar drag for the transparent control panel The control panel window is transparent on macOS (for the compact onboarding frame's rounded corners), and Electron ignores -webkit-app-region: drag entirely on transparent windows — the hiddenInset titlebar's native drag dies with it. Onboarding (login, permissions, every step) and the signed-in panel have been undraggable on macOS since #1670 introduced the transparency. Verified with synthetic CGEvent drags: identical input moves the window when opaque and does nothing when transparent. Recreate the titlebar in JS: a mousedown in the top 48px that is not aimed at an interactive control moves the native window through the shared DragManager (which gains a per-drag window override; the pill's target-window flow and its position-ownership bookkeeping are untouched). Mounted once at the panel-window root, covering onboarding, reauthentication, and the control panel. Where app-region does work (the opaque Windows/Linux onboarding strip), the system consumes the mousedown first and the hook stays idle. Verified end-to-end with synthetic drags against the transparent window: full-fidelity tracking on both axes, work-area clamping intact. * fix(onboarding): make the compact hero draggable window chrome The compact screens' dithered hero reads as the window's titlebar, but only the top 48px was draggable — grabbing the visually obvious band (welcome screen, permissions) did nothing. Size the shell's drag strip to the full 192px hero on compact screens and declare it as a data-window-drag-zone, which the JS drag fallback honors beyond its universal 48px band. Interactive overlays in the band (compact Back, permissions Skip) already carry no-drag and stay clickable through the fallback's interactive-element exclusion. Also move the fallback's mousedown listener to the capture phase so a step's own mouse handling cannot swallow the titlebar gesture, and keep the dev CDP hook behind OW_DEBUG_CDP for future diagnostics. * style: prettier format for useControlPanelWindowDrag * refactor(dictation): drop redundant pill interaction flag pillVisuallySuppressed now includes assistant.closing, so the separate pillInteractionSuppressed alias was identical to it. * fix(onboarding): keep the compact Back reachable under the widened drag band The shell's drag strip grew 48px -> the full 192px hero on compact screens, so it now covers the sign-in/sign-up screen's Back control at y=52. That control predates the widening (#1670) and sits inside the step tree, where nothing can out-stack the z-50 band: the step wrapper's entry animation keeps a filling transform, so it owns a stacking context and caps every descendant regardless of z-index. The result was a dead end — elementFromPoint returned the band, a real click delivered zero events to the button, and the only other way off that screen is an sr-only toggle. Portal it to body at z-60, the same recipe CompactPermissionsStep already uses for its Continue; no-drag keeps both the JS drag fallback and the Windows/Linux app-region off it. Embedded in SignInDialog there is no band, so it stays inline. The hero keeps its full 192px drag footprint. AuthenticationStep's shallow-render test has no DOM, so it gains a react-dom mock whose createPortal returns its children (the harness walks elements rather than mounting) plus a scoped document stub. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * fix(onboarding): scope the titlebar drag fallback to macOS and require movement Two narrowings to the JS titlebar drag. Platform: the fallback exists because a transparent window drops `-webkit-app-region` entirely, and the control panel is only transparent on macOS. Windows/Linux keep an opaque window where app-region works, so the hook was pure overshoot there — its universal 48px band is wider than ControlPanel's own 40px strips, adding an 8px sliver of drag just below the real titlebar. It now stays off outside darwin instead of second-guessing regions the system already owns. Threshold: the drag started on mousedown, so a plain click on the titlebar ran a full start/stop cycle whose first tracking tick applies DragManager's clampToWorkArea — snapping a window the user had parked past a screen edge. It now waits for 3px of travel. No fidelity cost: DragManager captures its cursor-to-window offset when it starts, so beginning at first movement still picks the window up exactly under the cursor. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * refactor(dictation): give the pill cancel geometry one definition The cancel control's 28px size and 8px gap were written out in LiquidCancelButton, in WINDOW_SIZES.RECORDING's fit comment, and again in the fusion test — three copies of a cross-process contract that only comments tied together. The footprint transition was duplicated too: VoicePill built it from LISTENING_ENTRANCE_TIMING plus a literal curve, while the skin's tween re-derived the same duration independently, so a change to one would silently drift the fused outline off the real pill. VOICE_PILL_CANCEL and VOICE_PILL_GROW_TRANSITION now live beside VOICE_PILL_FOOTPRINT, where the contract is already documented, and every consumer reads them. windowConfig's comment points at that source instead of restating the numbers, and the fusion test pins its fixtures against it so they cannot drift from what the component renders. Values are unchanged; the rendered geometry is identical. Also drops an `overflow: visible` that was already the default. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * test(assistant): pin both platform paths for the panel focus contract The panel-open expectation branched on the host's own process.platform, so whichever path CI was not running went unverified — and darwin, the platform this contract was written for, is the one no Linux runner covers. A withPlatform helper (the idiom already used elsewhere in this file) now runs both, plus the close/blur side that had no coverage at all. Also corrects the method's doc comment: it still claimed the panel "needs keyboard focus so Escape can dismiss it reliably", directly above code that deliberately never focuses on macOS. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * refactor: name the pill visibility handoff for both its callers createDictationErrorPillHandoff now backs two unrelated handoffs — the dictation-error return and the panel return this branch added — so the name was wrong at one of its two call sites and its doc comment described only the error case. Pure rename of the factory, its file, its test, and the options interface; no behaviour change. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * fix(dictation): let a dictation error bail the panel-return chase The panel-return handoff chases the size owner's latest target before revealing the pill, but unlike its dictation-error twin it had no escape when an error takes the window over on its own path. Add the same actionCountRef guard so the two chases behave alike and the loop cannot keep re-requesting a size another owner is already moving. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * fix(dictation): keep a live recording visible through the panel close Folding the pill into the panel exit ORed assistant.closing straight into pillVisuallySuppressed, which overrode the close-intent ownership handoff two other pieces of code exist to guarantee: voicePillOwnsActivity hands a running recording back to the pill at close INTENT precisely because beginClose hides the companion on that same tick, and shouldSuppressPillForAssistantActions carves out assistantClosing && hasLiveActivity for the same reason. With the unconditional term both were unreachable, so a plain dictation started during the ~580ms close (setAssistantPanelOpen(false) fires at close intent, so the main process accepts one again) recorded with no visible owner at all — the "both-hidden gap" that carve-out is named after. Compose the suppression in a pure resolvePillVisualSuppression instead, so the rule lives beside the two policies it has to agree with and is testable: nothing checked it at any level, which is how it got dropped. Live activity opts out of the closing fold; an idle close still reads as one beat. The panel-return mask stays unconditional — it is bounded by the native shrink it covers, not by the panel's exit choreography, and guarding it would hold a visible pill at its new dock inside still-panel sized bounds. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * fix(dictation): size the cancel button from the shared contract VOICE_PILL_CANCEL was introduced as the one definition of the cancel geometry, but the rendered button kept a literal size-7 the skin cannot see. The outline traces its circle at CANCEL_BUTTON_SIZE / 2 and the slot reserves CANCEL_BUTTON_SIZE, so changing the constant would have moved the fused skin and the slot while leaving the real control at 28px. Size the element from the constant instead, finishing what the extraction started. Also stop gating the plain-mode discard on the emergence. The slot-width guard exists because the fused bud overlaps the pill until its slot clears one button width, but plain mode scales its own hit area and is the Live Transcript panel's persistent discard control — it was clickable at mount before the emergence motion existed, and the shared guard left it inert for the first ~218ms of the 280ms animation. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * fix(onboarding): stop the widened drag band swallowing the step content Three related holes in the macOS titlebar fallback. The drag latch outlived a lost button. Deferring the drag to the first mousemove armed an `origin` that only mouseup and blur cleared, so a release the window never saw left the next unpressed move free to start a drag with no button held. Bail (and disarm) on event.buttons === 0, and take mouseup on the capture phase like its two siblings so a handler in between cannot strand the gesture. data-no-window-drag matched nothing. The selector shipped with the hook but no element ever carried it, while the panel's real chrome opts out with -webkit-app-region: no-drag — which the fallback cannot read, and which is the only drag path on macOS since the window is transparent there. A press on those wrappers' padding dragged the window; only one landing on the inner button was spared by closest("button"). Mark the three of them. The compact band swallowed events on every platform. It carries app-region: drag, so it is hit-testable by design and a sibling of the step scroller inside an overflow-hidden main: at 192px it ate wheel scrolling over the top of a compact step, reachable because setOnboardingWindowMode clamps the compact window to the work area height and the permissions list then overflows. Keep the native strip at 48px on every platform and declare the hero separately as a pointer-events-none zone, so only the macOS fallback widens. That means matching declared zones by their bounds rather than by hit testing the target, which is what lets a zone stay out of the event path entirely. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 * test(auth): assert the compact Back really leaves the step tree The portal exists so the compact Back can out-stack the onboarding shell's drag band, but the harness mocked createPortal as a passthrough and stubbed document to { body: {} }, so the test passed identically with the portal removed — verified by deleting it. Record the container the mock is handed and assert it is document.body, which does fail on that mutation. Claude-Session: https://claude.ai/code/session_017f33LXZQ3jgYcZobmkqKm4 --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Marshall Bose <mashallbose@gmail.com> Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com> Co-authored-by: boseq <307062491+boseq@users.noreply.github.com> |
||
|
|
9e86381d45 |
fix(tinfoil): move the default chat model from glm-5-2 to glm-5-3 (#2018)
* fix(tinfoil): move the default chat model from glm-5-2 to glm-5-3 Tinfoil retires GLM-5.2 on 2026-09-10 (replaced by glm-5-3) and its live /v1/models list already omits it. The default lookup missed, so every reconciled Tinfoil selection landed on whatever the enclave listed first (deepseek-v4-flash today) instead of the intended default. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tinfoil): resolve the default model by name, not by list position The glm-5-3 constant only governed the retirement reconcile. Two other paths resolved a Tinfoil default from `models[0]`, and Tinfoil's catalog is fetched live and cached at module load with no age gate, so on any profile that had ever opened the Tinfoil picker both landed on whatever the enclave served first (deepseek-v4-flash today) rather than the flagship: - an org policy that pins Tinfoil, via defaultLlmModel - picking Tinfoil during onboarding Providers now name their default in the registry and one shared helper reads it, with list order kept as the rule for the static catalogs that are already ordered best-first. Behaviour for every other provider is unchanged. A persisted glm-5-2 selection also had no deterministic repair: the live-catalog reconcile only runs after a successful fetch, so it misses an offline launch and loses the race against the first request of a session, which 404s. The Groq-only startup migration is generalised into a provider-keyed sweep that repairs it before the store is read. Groq keeps its shipped sentinel and its one-shot semantics; Tinfoil gets its own, so an installed app's Groq flag no longer suppresses it. Also drops the deepseek-v4-pro description key, retired from Tinfoil's catalog, from the map and all ten locales, and adds the guard that would have caught it; and probes glm-5-3 in the LLM canary, which never exercised the default users actually land on. * fix(models): Cover every retired Tinfoil model and restore the switch notice The startup sweep repaired one retired id, but three Tinfoil chat models the app has offered are gone from its /v1/models: glm-5-2, deepseek-v4-pro (whose description key this branch already drops) and kimi-k2-6 (retired by #1722). All three shipped as seed entries — deepseek-v4-pro and kimi-k2-6 through v1.7.6, glm-5-2 until this release — so anyone who simply took a default can still be on one, hitting the exact failure the sweep exists to prevent: a 404 on the first request of an offline launch. Map each to the successor in its own family. Coverage was also a one-time decision without saying so. The sweep short- circuits on a per-provider sentinel, so an id added under a provider whose key a released build already writes reaches nobody who has launched it. Say that where the table is edited, and snapshot each sentinel with the id set it covers so editing either forces the rotation into the diff. The snapshot cannot verify the rotation happened, only surface the decision; the Tinfoil key has not shipped, so the set was still free to grow here. Repointing a selection also went silent. Tinfoil is the one provider that tells the user their model was switched out, via reconcileSelectedModels; the sweep now gets there first and leaves nothing stale for it to announce. The sweep reports what it rewrote and the store announces the Tinfoil moves itself, once per retired model rather than once per scope. Names resolve offline from the user's own catalog cache and the seed, falling back to the raw id as the reconcile already does. Two consequences of announcing from the sweep. A replacement outside the curated seed is named by id. And the notice can now fire at cold start rather than mid-session after a fetch, where TinfoilModelSwitchToastListener answers it with showDictationPanel({reposition:true}): affected users see the pill move to the active display once, and users with floatingIconAutoHide on see it appear until the toast clears. One-time and self-healing, but new. A storage failure took the renderer with it. The sweep writes at module scope during settings-store evaluation with no guard, unlike sweepRetiredCustomPrompts two functions above it, so a rejected write threw out of module evaluation. Guard per provider: the sentinel stays unset and the next launch retries. Finally, defaultLlmModel and the onboarding provider step each repeated a "defaultModel" in provider narrowing dance to read a field the registry already names, and reconcileRetiredCloudModelSelections hand-rolled the same lookup by list position. Add a shared pickDefaultModelId that takes the provider and use it at all three. No provider but Tinfoil names a default yet, so behaviour is unchanged today. --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
|
fad475c6f4 |
fix(whisper): extract MSVC runtime DLLs beside win32 whisper-server (#1687)
* fix(whisper): extract MSVC runtime DLLs beside win32 whisper-server The win32 CPU whisper-server.exe links the dynamic MSVC runtime (msvcp140/vcruntime140/vcruntime140_1/vcomp140) but shipped as a bare exe: download-whisper-cpp.js copied only the binary out of the release zip, so on machines without the VC++ 2015-2022 redistributable the Windows loader killed every spawn with 0xC0000135 (STATUS_DLL_NOT_FOUND) before main(), with empty stderr — local transcription dead on arrival (CUS-113). - pin whisper.cpp 0.0.10, the first release whose win32 zips bundle the DLLs - give the win32 config the same libPattern DLL-companion extraction the llama/sherpa download scripts already use, consolidating their duplicated findLibrariesInDir/matchesPattern helpers into lib/download-utils.js - prep whisperVulkanManager with libPattern so the 0.0.10 vulkan zip's CRT DLLs are extracted too (its "statically linked" comment was wrong re CRT) electron-builder's extraResources filter already passes *.dll and cleanupFiles already spares libraries, so no packaging-config change needed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(downloads): validate cached binaries and traversal errors * fix(whisper): Preserve migration and propagate download failures Keep legacy Windows Vulkan packs migratable until the DLL-bearing release and digest update lands. Make all-platform downloads exit nonzero after any platform failure and remove the unused numeric library-search options shim. * fix(gpu): Require Windows whisper runtime libraries Pin CUDA and Vulkan packs to verified 0.0.10 assets and reject incomplete caches or staged installs. Preserve the existing pack when validation fails. * fix(deps): Upgrade Tiptap past attribute injection advisory Align the full Tiptap package family at 3.30.4, the first patched release for GHSA-cp6q-959q-f8rh. Restore the npm audit gate without unrelated dependency upgrades. * fix(deps): Patch xmldom serialization advisory Upgrade the existing xmldom override to 0.8.15, the first patched 0.8.x release for GHSA-6gmq-8vp8-gcm6, so the npm audit gate remains clean. --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Gabriel Stein
|
ccf837c8fc |
feat(transcription): add Gemini as a BYOK cloud transcription provider (#1899)
* feat(transcription): add Gemini as a BYOK cloud transcription provider * fix(gemini): provider-aware BYOK size gating and transcription edge cases * test(gemini): add batch STT canary probes and map Google's invalid-key 400 |
||
|
voidandChadpiha
|
5a0f9d9065 |
Restore Voice Assistant auto-paste into focused inputs (#1859)
* fix(assistant): restore focused-input auto-paste * fix(build): compile Windows text monitor from source * fix(assistant): copy responses when no input is focused * fix(assistant): exclude terminals from caret delivery targets * fix(chat): keep partial replies from unmounted streams in history * fix(assistant): keep open-panel follow-ups panel-first * fix(assistant): settle editable probe on first output line * docs: sync CLAUDE.md with caret response delivery * fix(assistant): refuse caret delivery when the focused field has a live selection The clipboard-based capture cannot see a selection whose text already matches the clipboard (or a full-line selection in a line-copy editor), so a "none" read could upgrade to a caret and paste the generated answer over the user's highlighted text. The editable probes now read the focused element's own selection state (UIA TextPattern.GetSelection, AT-SPI text selections, AXSelectedText) and report NOT_EDITABLE when a non-empty selection exists, at capture and at paste-time revalidation. Also gate the probe on the auto-paste setting via a probeEditable flag on capture-selected-text (the renderer discards the caret otherwise, so the binary spawn was pure waste), and drop the unreachable EDITABLE_NONE: branch from the AppleScript fallback, which never emits that verdict — panel-first by design. * chore: trim assistant implementation comments * test: consolidate assistant delivery coverage * fix(chat): make unmount-persisted partial replies reachable The unmount cleanup routes through cancelStream, which bumps the send generation, so cancelled() was always true whenever mountedRef was false and the unmount-persist branch could never run: every mid-stream unmount dropped the partial reply from the saved conversation. Stamp the generation of cancels issued while still mounted and key persistence on that, so an explicit Esc/Stop cancel still drops the partial while an unmount keeps it. Pinned by a live-root unmount test. * fix(assistant): Refuse terminals on the Linux AT-SPI caret path AT-SPI targets carry only a pid, so the terminal-signature check saw an empty signature and a Wayland terminal's empty prompt could verify as a writable caret — pasting a multi-line answer into a shell would execute it. Resolve the pid's executable name before probing (the same ps helper macOS uses; its path parsing degrades to Linux's bare comm name), and refuse ATSPI_ROLE_TERMINAL inside both probe sources so VTE and Qt terminals are excluded at the source as well. --------- Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Gabriel Stein
|
db8b95e6af | Merge remote-tracking branch 'origin/main' into fix/862-parakeet-macos-compatibility | ||
|
Gabriel Stein
|
9b3f57a4c6 |
fix: retire dead cloud models and make the canary suites trustworthy (#1722)
* fix(models): drop cloud models retired upstream and remap saved Groq selections Groq shut down qwen/qwen3-32b, llama-3.3-70b-versatile, and llama-3.1-8b-instant on 2026-08-16, and Tinfoil replaced kimi-k2-6 with kimi-k3 — all four registry entries now 404 at the provider, so a BYOK user picking one gets model_not_found (caught by the first keyed llm-canary run). Remove the entries, their locale description keys, and the kimi entry in the Tinfoil description-key map. Persisted scope selections pointing at a retired Groq model are remapped by a one-shot migration to the gpt-oss replacement Groq recommends; non-Groq providers that still serve those ids (OpenRouter) are left alone. Tinfoil needs no migration: its live-catalog reconcile already moves users off retired models. The qwen entry was the only registry model flagged disableThinking, so the request-shape matrix row pinning that flag is removed with it; the suppression mechanic itself stays (any future registry or catalog entry can set the flag). * fix(canary): validate post-handshake auth, normalize Gemini ids, probe registry models Three fixes so a red canary means provider news, not canary bugs: - The openai-realtime STT probe resolved on WebSocket open, but OpenAI completes the upgrade before checking the key and only then sends an error event — a garbage key passed (proven green in run 32156015458). The probe now waits for the first server event and fails on error or close-before-event. The tinfoil-realtime row cannot reach the network without the SDK's enclave attestation, so its report cell now says key present (transport not probed) instead of a bare check. - Gemini's OpenAI-compat /models returns models/-prefixed ids, so every registry id read as vanished from the live catalog while the probe itself passed. Strip the prefix before diffing. - The LLM probe list pinned gpt-4o-mini, which is not in the registry, so the suppression gate never disarmed and OpenAI 400'd on reasoning_effort + chat_template_kwargs — a red the app never hits (it self-heals via fetchWithParamFallback; the canary deliberately does not). Probe gpt-4.1-mini instead, and replace Groq's retired qwen/llama probes with the gpt-oss pair the registry still serves. Verified locally: STT canary with a garbage OpenAI key now exits 1 (server error event: invalid_api_key). |
||
|
Gabriel Stein
|
1d5c1091d4 | Merge remote-tracking branch 'origin/main' into fix/862-parakeet-macos-compatibility | ||
|
Chadpiha
|
0c271fefd5 |
fix(parakeet): Gate incompatible macOS runtime
Prevent Parakeet selection, startup, downloads, and transcription when the packaged ONNX Runtime cannot load on the current macOS version. Validate universal ONNX Runtime deployment targets in macOS release artifacts so runtime capability and packaged binaries cannot drift. Fixes #862 |
||
|
Chad Piha
|
41cf1d4617 |
fix(local-llm): Phase 0 correctness + hygiene fixes for local model params and canaries (#1714)
* fix(settings): derive local-provider migration lists from the model registry migrateProviderSettings and migrateAgentMode classified a legacy byok selection as "local" from two hand-maintained provider lists that missed liquidai when LFM landed (#1176). Both migrations now use the registry-derived localLlmProviderIds set that already exists in the store, so the next catalog addition cannot repeat the omission. Latent for existing users (the sentinel-guarded migrations predate the LFM catalog) — this is hygiene, pinned by a test that walks every registry local provider through both migrations. * fix(models): route non-registry gemma ids to the local provider getModelProvider's id-heuristic fallback (used only for ids outside the registry) listed qwen/llama/mistral/lfm2/gpt-oss but not gemma, so a renamed or not-yet-synced Gemma GGUF failed closed to "" instead of resolving "local". Catalog gemma ids were unaffected (the registry lookup runs first). Pinned by a test that walks a non-registry id of every local family through the fallback and checks gemini ids still resolve to gemini. * fix(local-llm): keep explicit 0 params and forward requireCompleteOutput on the IPC path localReasoningBridge built its inference config with `||`, so an intentional temperature 0 (cloud cleanup parity), maxTokens 0, or any other falsy override was silently swapped for the hardcoded default — local cleanup ran at 0.7 no matter what the caller sent. Switch the five defaults to `??`. The same allowlist also dropped requireCompleteOutput, so the selection-edit truncation guard in llamaServer.inference (reject on finish_reason length/max_tokens) could never fire on the primary local path. Forward it. Both are pinned by a test that drives the real bridge → modelManagerBridge → llamaServer chain against an HTTP stub and asserts on the body that reaches the wire. * fix(reasoning): lan suppression takes the family effort floor instead of a flat none Self-hosted (lan) cleanup on a gpt-oss model produced a contradictory body: the family cleanup pin set reasoning_effort "low" while the lan dialect's suppression object said reasoning {effort: "none"} — a value the gpt-oss family has no switch for. The lan branch now reads the effort from modelFamilyConstraints like the generic branch does, so gpt-oss gets {effort: "low"} and qwen / unknown families keep the existing {effort: "none"} (#1021 shape unchanged; the object stays nested because flat reasoning_effort trips vLLM on lan, #1611). Two lan + gpt-oss-20b-mxfp4 rows in the request-shape matrix pin the cleanup and suppression bodies; a dialect unit test pins the shape. * fix(canary): fail the LLM and STT canaries when no secrets are configured Both canaries skipped providers without a key and, when every provider was skipped, still printed "All probes passed." and exited 0 — so a workflow whose secrets were never set (or silently dropped) reads as a healthy weekly check while probing nothing. An all-skip run now counts as a failure and exits 1, filing the canary issue like any other failure; per-provider skips are still allowed and listed. Pinned by a test that spawns each script with every *_CANARY_* variable scrubbed and asserts the non-zero exit. * refactor(models): drop dead prompt-template and inference-config code llama-server has run with --jinja (the GGUF's embedded chat template) since the server-based local path landed, so the registry's per-provider promptTemplate strings and ModelRegistry's formatPrompt / createPromptFormatter were computed and never read. Likewise src/config/InferenceConfig.ts (a sampling-presets manager) and src/utils/llamaOutputParser.js (a pre-llama-server CLI output cleaner) have no importers or tests. Remove all four and correct the CLAUDE.md model-registry section, which still documented promptTemplate as live. Repo-wide grep confirms zero remaining references. * test(models): guard registry cloud gemma ids against the local fallback Gemini and Tinfoil also serve Gemma (gemma-4-31b-it, gemma-4-26b-a4b-it, gemma4-31b). Those ids resolve through the registry lookup, which runs before the id heuristics, so the new gemma → local fallback never sees them — pin that ordering so a future reorder of getModelProvider cannot route a cloud Gemma selection to the local provider. |
||
|
Chad Piha
|
bc306d27d4 |
feat(meetings): Auto-end forgotten meeting recordings (#1494)
* feat(meetings): Auto-end forgotten meeting recordings Track reliable external microphone activity and show a protected 60-second countdown after meeting activity ends. Preserve recording during reconnects, unreliable detection, and Keep recording actions. * fix(meetings): Harden auto-end against stale binaries and races * fix(meetings): Keep auto-end alive through coverage loss and CI races Code review found that every major failure path killed the feature silently. All fixes preserve the fail-safe property: degradation can only delay or disable auto-end, never stop a live recording. - Windows: a single unattributable session or flaky endpoint made the helper exit, with no respawn and a polling fallback that also lost meeting prompts. It now announces CAPABILITY AGGREGATE once and keeps emitting best-effort events; unattributable sessions refcount under pid 0, which the JS side can never exclude as its own, so they count as external capture and park the countdown. - macOS: a process vanishing between enumeration and its property query permanently downgraded PID mode to aggregate, and the 5s heartbeat re-rolled that race for the whole session. Skip the vanished object and let the next reconcile retry it instead. - Linux: one pactl source-output without application.process.id (echo-cancel, loopbacks) dropped PID reliability entirely. Skip module streams; client streams always carry the pid. - CI: the notarize build raced the listener-publishing workflow and could cache a stale binary under a fresh source hash. The release tag is now pinned to MIC_LISTENER_VERSION in the C source on both ends, and CI downloads fail hard instead of silently falling back. - Renderer: a rejected auto-end stop was swallowed; it is now reported through a required onError hook wired to the meeting logger. * style(notes): Revert drive-by Prettier reflows Restore ShareNoteDialog, OverviewNoteList, and useNoteDragAndDrop to their merge-base content. The reflows were format-only, and the ShareNoteDialog hunk conflicts with the policy-enforcement merge on main for no benefit. * fix(meetings): Exclude own audio helpers and persist auto-end stops Auto-end never fired when system audio was captured: the capture helpers (macOS audio tap, Linux portal, Windows loopback) are plain child processes absent from app.getAppMetrics(), so the OS attributed their capture to non-excluded pids and externalMicActive never went false for the whole recording — verified empirically on macOS via the mic listener reporting the tap helper's pid. Feed the helpers' live pids into the detector's exclusion provider. Transcript persistence was keyed to mounted views, and an auto-end stop is the first stop that can fire while the notes view is closed: the final tail since the last 30s flush was lost and diarization results were dropped. Persist the final transcript in the store's stop path after the main result arrives, move the periodic saver to the always-mounted MeetingRecordingMount, and persist diarization results session-scoped to the note that was recorded. NoteEditor's handler is now display-only and gated to the recording note, so enrichment can no longer land on whichever note happens to be open. Toast when an auto-end request ends the live session — including when a Keep click loses the race with the countdown — so the user learns the recording ended instead of assuming it was kept. New notes.meeting.autoEnded key in all 10 locales. * feat(meetings): Confirm auto-end with audio silence and add a fallback Mic ownership alone had two blind spots: an app that releases the mic on mute could stop a live call while remote voices still played, and the feature was inert wherever ownership is unreliable (older macOS, coverage loss) or never armed (in-person meetings). Combine the two signals in a tick-driven controller. In ownership mode the countdown runs only while the system-audio channel is also quiet, so remote speech defers it; the mic channel is deliberately ignored there so room noise, typing, or talking to a colleague after the call can't mask the end. When ownership is unreliable or no app ever held the mic, both channels quiet for 60s prompts instead, tightened to 10s when the last tracked meeting app exits. Every raw meeting PCM chunk of both channels feeds a pure energy monitor via a single hook in sendMeetingAudio, sharing the echo-leak detector's thresholds through an alignment-safe RMS helper. Chunks are ignored until the controller session exists, since system audio starts streaming before the session is registered. Sleep gaps discard accumulated silence instead of stopping on wake, and a mic hold shorter than 30s no longer arms the mic-ignoring path on an in-person recording. Keep recording is now per episode with a 5-minute cooldown, the card explains which signal fired, and Settings → Meetings gains an "Auto-end forgotten recordings" toggle that takes effect immediately. * fix(meetings): Make auto-end recording fail safe * fix(meetings): Queue detections while a meeting recording session is live The engine's user-recording flag is shared with dictation, so a dictation that ends mid-meeting (or a cancel-hotkey press) clears it while the meeting recording is still live. From then on a detection — most plausibly the calendar reminder for a back-to-back meeting, which fires one minute before its start, right inside the auto-end countdown — bypassed the queue, and its prompt replaced the visible countdown card. The replacement path never notifies the controller, so the countdown ran on unseen and stopped the recording without any chance to keep it. Gate detections on the engine's own recording session as well, which nothing outside the recording lifecycle can reset: new detections are queued behind it like the user-recording gate, and a post-dictation cooldown flush holds the queue until the recording ends. This also stops the pre-existing "Take notes?" prompt that could appear during the user's own manual recording after a dictation. * fix(meetings): Persist the final transcript before awaiting the main-side stop Moving the final transcript save into the store's stop path (so auto-end and owner-loss stops persist without the notes view mounted) also moved it after `cleanup()` and the `meetingTranscriptionStop` round trip. The note editor swaps from live segments to the stored transcript the moment `isRecording` flips, so for the duration of that stop — up to a few seconds with a local Parakeet flush — the tail since the last 30s periodic save disappeared from the note and reappeared once the stop returned. On main the notes-view effect wrote it in the same commit as the state flip. Persist the captured segments before awaiting the stop; they are final once the session is released. Only a segment-less recording still waits for main's final transcript, as before. `persistFinalTranscriptAroundStop` replaces `buildFinalMeetingTranscript`, whose only caller was this path. * fix: clear meeting flag after recording session ends |
||
|
Gabriel Stein
|
c0202dfc41 |
fix(dictation): single source of truth for realtime STT provider routing — fixes #1624 (#1631)
* fix(dictation): single source of truth for realtime STT provider routing (#1624) The openai-realtime dictation path never sent a provider field, and 1.8.2's fail-closed token allowlist rejected undefined — breaking default cloud dictation. Provider resolution and session options now live in one pure renderer module (dictationStreamingRouting), token acquisition in one main-process registry (realtimeTokenProviders), the boundary defaults a missing provider to openai-realtime for older renderers, the IPC contract is typed, and a settings×context matrix test plus cross-boundary closure tests pin the whole surface. Warmup failures now log at error level with the provider named and a consecutive-failure streak. * ci(stt): weekly realtime provider credential and handshake canary |
||
|
Gabriel Stein
|
961a1b6933 |
refactor(llm): single source of truth for provider/model request params — dialects, matrix tests, fail-soft, live canary (#1620)
* fix(reasoning): family constraint table, deepseek/cerebras dialects, registry capability gaps Hoist model-family facts (gpt-oss effort floor, qwen enum, magistral omission) into modelFamilyConstraints so no family rule lives inside a provider branch again (the #1611 bug class, seen 11 times since 1.7.x). - deepseek endpoint dialect: thinking {type: disabled}, never the rejected reasoning_effort none (#1260) - cerebras endpoint dialect: strict like Groq, no chat_template_kwargs (#831, gap noted in #990) - registry disableThinking honored on every provider, not just groq - OpenRouter vendor-prefixed ids inherit the upstream model's supportsTemperature so Claude 4.7+ stops getting temperature (#1417) - gemini-3-flash-preview / gemini-3.1-pro-preview marked supportsThinking so disable-thinking sends thinkingConfig minimal and thinking tokens stop eating the output budget (#1341) * refactor(reasoning): one param builder for every chat-completions transport, bounded param fallback applyChatCompletionsParams decides token param, temperature, family effort, and suppression once; the non-streaming, agent-streaming, openai-chat, and tinfoil transports all consume it, so the same model on the same provider now gets the same body everywhere. The AI SDK agent path's groq providerOptions reads the family table too (gpt-oss got the nonexistent reasoning_effort none before). fetchWithParamFallback replaces the reasoning-object-only retry: on a 400/422 it strips the reasoning object blind (old Ollama proxies), then strips exactly the shaped params the error names, logging each strip as PARAM_FALLBACK — degraded beats broken, and openai.ts gets the ladder for the first time. The request-shape matrix test pins the full param set for 23 provider x model x purpose cells, so any future dialect or registry change is a visible one-line diff. * feat(reasoning): weekly live canary for provider request shapes and catalog drift Sends a ~1-token request per provider x constrained family using the real shaping code and asserts 2xx, then diffs each provider's /models against modelRegistryData.json (vanished pinned ids, unprobed constrained-family arrivals — how Tinfoil+gpt-oss slipped through). Scheduled weekly, never part of PR CI; failures file or update an llm-canary issue. Providers without an LLM_CANARY_*_KEY secret are skipped and listed. * style(reasoning): fix prettier formatting in openai and tinfoil providers |
||
|
Gabriel Stein
|
77357d6d7d |
fix(linux): link -lgobject-2.0 explicitly instead of pkg-config --static
--static pulls in the full transitive Requires.private closure, including dbus-1's -lsystemd, which fails to link on Ubuntu (including the ubuntu-latest CI runner) unless libsystemd-dev is installed. Appending -lgobject-2.0 explicitly links the one missing symbol source and builds cleanly on Ubuntu 24.04 with only libatspi2.0-dev and libglib2.0-dev, and on Arch. |
||
|
iSparshandClaude Sonnet 5
|
1d1bcf8dc5 |
fix(linux): link libgobject-2.0 for AT-SPI2 text monitor build
atspi-2.pc lists gobject-2.0 under Requires.private rather than Requires, so a plain `pkg-config --cflags --libs atspi-2` omits -lgobject-2.0. Linkers that default to --as-needed (current Ubuntu/Arch toolchains, including the GitHub Actions runner) then fail with "undefined reference to symbol 'g_object_unref'". Use `pkg-config --cflags --libs --static atspi-2` instead, which pulls in Requires.private too. This is very likely why the "Build Linux Text Monitor" workflow's only run failed in ~18s back in February, leaving no prebuilt binary published since — silently disabling auto-learn correction on Linux for anyone relying on the download path. Also strengthens the CI "Verify binary" step, which previously only checked the file existed (not that it was executable or correctly linked) and so couldn't have caught this class of bug, and documents the separate `toolkit-accessibility` gsetting requirement for non-GNOME sessions in TROUBLESHOOTING.md. Fixes #1531 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019SJosYcDfwrp35KErSdkrf |
||
|
Chadpiha
|
610a515d17 | feat: voice driven selection editing | ||
|
Gabriel SteinandChadpiha
|
5d69b75a2e |
feat(calendar): Apple Calendar (EventKit) integration for macOS (#1237)
* feat(calendar): Apple Calendar (EventKit) integration for macOS Add a second calendar provider that reads the local EventKit store — covering iCloud, Google, Exchange, and Outlook accounts already in Calendar.app — with one native permission prompt and push-based updates instead of OAuth + polling. - New macos-calendar-listener Swift helper (mic-listener pattern): emits permission + calendars/events snapshots as line-delimited JSON, re-emits on EKEventStoreChanged, stdin "sync", and a 5-min window roll; --request triggers the TCC prompt attributed to OpenWhispr - Build script patches dev Electron's Info.plist with calendar usage strings; packaged builds get them via electron-builder extendInfo, plus the calendars entitlement for hardened runtime - calendar_events gains a provider column; new apple_calendars table; Google deletes are provider-scoped; shared read queries suppress Apple copies of Google-mirrored meetings (datetime-normalized, Google rows never collapsed) so reminders fire once per meeting - Meeting reminder scheduling extracted from GoogleCalendarManager into a provider-agnostic CalendarReminderScheduler shared by both managers - AppleCalendarManager owns the helper lifecycle; meeting join links are extracted from url/location/notes into hangout_link at sync time so Join buttons and note linking work unchanged - Integrations UI gets an Apple Calendar row (macOS only) with connect, denied → System Settings flow, source list, and disconnect; i18n keys added to all 10 locales Downstream consumers (Upcoming Meetings, overlay reminders, agent calendar tool, note linking) read the shared table and work for both providers with no changes. * refactor(calendar): Harden EventKit integration and extract shared helpers * fix(calendar): Show the calendar permission prompt in dev builds * chore(calendar): Clean up after dev TCC fix * fix: tentative meeting responses and stale state fix * fix: dev script * fix(tests): broken test after rebase --------- Co-authored-by: Chadpiha <chadpiha23@gmail.com> |
||
|
Gabriel Stein
|
f4f4fa2c28 |
fix(whisper): stop local transcription splitting words mid-word (#1348) (#1369)
Local Whisper started inserting a stray space inside words in 1.7.4
("abschalten" -> "abs chalten", "отключил" -> "отк лючил"). The breaks land
on BPE token boundaries roughly every 60 characters, in any language.
No source change caused it. download-whisper-cpp.js tracked the latest
release of the binaries repo, so the 1.7.4 build silently picked up
whisper.cpp v1.9.1 (0.0.7, Jun 25) where 1.7.3 had shipped v1.8.3 (0.0.6).
v1.8.3's server enabled token timestamps only for verbose_json:
wparams.token_timestamps = !no_timestamps && response_format == vjson_format;
v1.9.1 enables them for every request, and both versions force
`max_len = 60` when it is unset. max_len wrapping only runs on the token
timestamp path, so v1.9.1 wraps every segment at 60 characters with
split_on_word off — the wrap lands mid-token. We join segments into one
string and normalizeWhitespace turns the "\n" into a space.
Pass --no-timestamps, which turns the wrap off with them; we only read
`text`, never per-token timings. Verified against the shipping 0.0.8 binary:
output matches v1.8.3 exactly. Per-request form fields (token_timestamps,
max_len, split_on_word) are ignored by the server, so this has to be a
startup flag.
Also pin the binaries to 0.0.8 instead of tracking latest, matching how
download-llama-server.js pins llama.cpp. Pinning in the script rather than
in CI keeps one source of truth, gives local prebuild the same binary as CI,
and lets the workflow cache key (hashFiles on scripts/download-*.js)
invalidate on a bump. 0.0.8 is what 1.7.6 already ships, so no rollback.
Fixes #1348
Fixes #1158
|
||
|
Idris GadiandGabriel Stein
|
c2d2eb5d3e |
fix(linux): align Wayland development server with packaged behavior and use XWayland (#1323)
* fix: initialze all wayland sessions to start with xwayland for dev/local server * refactor(linux): share the XWayland launch rule between main and dev launcher main.js still gated its self-relaunch on a KDE/GNOME/COSMIC allowlist, so a direct (non-wrapper) launch on Hyprland or any wlroots compositor kept native Wayland. Move the rule into src/helpers/xwayland.js and use it from both main.js and scripts/run-electron.js so the three launch paths agree with the packaged wrapper, which already forces XWayland for every Wayland session. * refactor: trim redundant comments on the XWayland helper --------- Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com> |
||
|
Gabriel Stein
|
a4d654e4b4 |
fix: isolate dev qdrant storage from production app (#1253)
Dev and production both pointed qdrant at ~/.cache/openwhispr/qdrant-data, so whichever instance started second crashed on the WAL file lock. Dev now uses qdrant-data-dev, matching the transcriptions-dev.db convention. Also: - report qdrant's actual panic line (stdout ERROR) in the startup death message instead of the first stderr line (jemalloc warning) - skip qdrant download only when the installed binary matches the release version, via a marker file like sherpa-onnx (stale binaries re-download) - drop the predev hook duplicating predev:main; the prep chain now runs once per npm run dev instead of twice |
||
|
|
f94079978f |
fix(media): pause/resume on macOS 15.4+ via vendored mediaremote-adapter (#806)
* fix(media): pause/resume on macOS 15.4+ via vendored mediaremote-adapter Bundle ungive/mediaremote-adapter v0.7.6 (BSD-3, pruned of its test client) and load it through /usr/bin/perl — the only path Apple still trusts to call MediaRemote since 15.4 closed the daemon to user processes. The previous CGEvent fallback was an osascript F8 keystroke, which is not a media key on modern Macs and was silently dropped by every player. - Vendor + clang framework build (no cmake dep, hash-gated) - afterPack registers the .framework bundle for codesign - macOS pause/resume now async via spawn (no main-thread block) - Swift helper rewritten to post a real NSEventSubtypeMediaKey via CGEventPost as the last-resort fallback (replacing F8) Verified: pause/resume against live Spotify on macOS 26.3 through both the perl path (state-aware) and from inside a packed .app bundle. Closes #794. Same root cause as #508 and #632. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(media): production cleanup for mediaremote-adapter integration - skip artwork payload on the pause probe (get --no-artwork) - drop a resolver candidate path that no packaging layout produces - verify the packed framework and perl entry point in the macOS release workflow - condense comments and prettier-format the adapter build script * chore(ci): drop workflow change; will land on main directly --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com> Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com> |
||
|
Alcahest Malus DienheimandGabriel Stein
|
7a9d13a5e8 |
feat: URL audio download, batch upload, and speaker diarization (#754)
* feat: add URL audio download and transcription
Download audio from YouTube videos and direct audio URLs for transcription.
Uses yt-dlp via youtube-dl-exec for YouTube, HTTP fetch for direct URLs.
Adds Videos default folder, progress UI, and i18n for 10 languages.
* fix: use file.fromUrl flag instead of downloadedTempPath for note title
* feat: add batch upload queue and speaker diarization toggle
Batch queue processes multiple files/URLs sequentially with per-item
progress. Speaker diarization toggle uses existing sherpa-onnx pipeline
with folder selector and graceful fallback on failure.
* feat: GPU-accelerated diarization with cross-platform EP detection
Uses ONNX Runtime execution providers (CUDA on Linux, CoreML on macOS,
DirectML on Windows) with CPU fallback. Parallelizes transcription and
diarization via Promise.all. Falls back to sherpa-onnx binary on failure.
* feat: cross-platform CUDA setup with GPU detection and auto-install
GPU diarization uses ONNX Runtime with CUDA/CoreML/DirectML and CPU
fallback. cuDNN bundled for AppImage/tar.gz users. deb/rpm after-install
detects NVIDIA GPU and installs CUDA runtime from package manager with
support for Ubuntu, Debian, Fedora (with repo version fallback), and
openSUSE. CI workflows download cuDNN for Linux builds.
* feat: add CUDA setup guide in Settings for AppImage and tar.gz users
Shows GPU diarization readiness checks with per-distro install commands
in a dialog, matching the Wayland Paste Setup pattern. Green status when
CUDA runtime and cuDNN are both available.
* fix: polish URL input with dynamic icons and Settings-matching toggle
YouTube URLs show red YouTube logo, audio file URLs show audio icon,
generic URLs show link icon. Toggle matches Settings page style.
Number input spinner removed, height aligned with URL input.
* feat: add cloud diarization for OpenAI and Mistral providers
OpenAI overrides model to gpt-4o-transcribe-diarize. Mistral adds
diarize=true to Voxtral at no extra cost. Groq shows a warning that
cloud diarization is unavailable. Provider-specific info notes shown
when speaker detection is active.
* fix: local GPU diarization with fast-path and speaker-text merge
Fast-path GPU pipeline uses segmentation directly without slow embedding
extraction. Speaker-text merge runs in main process via IPC to avoid
CommonJS/ESM incompatibility. Binary fallback timeout increased to 60min.
* chore: include onnxWorkerClient cuDNN detection and updated tests
* fix: use URL parsing instead of substring match for provider detection
Resolves CodeQL incomplete URL substring sanitization alerts. Checks
hostname via new URL() instead of includes() for YouTube, OpenAI,
and Mistral detection.
* ci: remove cuDNN download step from CI workflows
cuDNN is handled by the after-install script and local dev setup,
not needed in CI builds.
* ci: pass GITHUB_TOKEN to npm ci for youtube-dl-exec postinstall
The youtube-dl-exec package downloads yt-dlp from GitHub releases
during postinstall. Without authentication, CI runners hit the
GitHub API rate limit.
* fix: escape shell variables in after-install to avoid electron-builder macro parsing
* fix: address code review findings (SSRF, diarization, React races, i18n, packaging)
Security:
- SSRF hardening: HTTPS-only for direct downloads, DNS resolution with private IP rejection (RFC1918/loopback/link-local), redirect cap at 3, 500MB download limit, dropped octet-stream content-type
- delete-temp-file: realpathSync + path.relative validation to prevent symlink traversal
- BYOK diarize: explicit isOpenAi hostname check, reject unsupported providers instead of blindly injecting gpt-4o-transcribe-diarize
- Removed user-writable ~/.cache path from cuDNN LD_LIBRARY_PATH candidates
Correctness:
- GPU diarization: replaced fast path with overlap alignment, builds co-occurrence matrix across 5s window overlap to map local speaker IDs to global IDs via greedy matching
- useBatchQueue: moved queueRef write into useEffect (React purity), clearQueue sets cancelledRef=true to stop in-flight loop, replaced one-shot snapshot with live find to pick up items added mid-run
- check-cuda-diarization-ready: no longer returns ready=true on macOS/Windows without probing EP usability
- Videos folder migration: INSERT OR IGNORE + re-flag is_default on pre-existing user folder
- as any cast replaced with "diarized" in transcriptionResult narrowing
Robustness:
- GPU diarize wrapped in Promise.race with 10 min timeout, falls back to binary
- diarizeLoad timeout raised to 600s for CUDA cold start
- ONNX worker heap raised from 512MB to 2048MB for 60-min audio PCM
- formatDiarTime and formatTimestamp handle hours
- speakerMerge sentence splitting uses Intl.Segmenter with regex fallback for CJK
- URL batch deduplication + 50-item cap
Packaging:
- Added download:cudnn to prebuild:linux, prepack, predist so deb/rpm/AppImage ship libcudnn.so.9
- Arch Linux pacman branch in after-install.sh
- Fixed ${SUSE_VER} electron-builder macro in openSUSE section
i18n:
- Extracted 10 hardcoded strings from CUDA setup (SettingsPage) and speaker detection notes (UploadAudioView) into translation keys across all 10 locales
* fix: address code review findings (SSRF, diarization, React races, i18n, packaging)
Disable GPU diarization pipeline and use sherpa-onnx binary directly.
The GPU path loads the embedding model but never calls it — segmentation
produces local per-window speaker labels that are inconsistent across
windows. Without the embedding+clustering step the output is unreliable.
The sherpa-onnx binary handles the full pipeline correctly.
Security: eliminate DNS rebinding TOCTOU in SSRF guard by using Node's
lookup option at connection time, add IPv4-mapped IPv6 check, remove
noCheckCertificates from yt-dlp, add path validation to transcribe and
diarize IPC handlers, restrict open-external to http/https, replace
execSync with spawnSync in download-cudnn, add gpgcheck to zypper.
Logic: fix BYOK diarization detection in batch queue (was parsing empty
base URL), guard single-file progress listener against batch cross-talk,
support multi-file drag-and-drop, snapshot API key at batch start.
i18n: replace hardcoded English strings in BatchQueueView, useBatchQueue
error codes, and SettingsPage CUDA guide with translation keys across
all 10 languages.
* chore: remove cuDNN download and CUDA auto-install from build pipeline
GPU diarization pipeline is disabled (missing embedding+clustering step).
No need to download ~200MB of cuDNN libraries or auto-install CUDA
runtime during package install until the pipeline is complete.
* feat: auto-download diarization models on toggle and add Cloud messaging
The speaker detection toggle no longer requires a separate download step.
Clicking it when models are missing triggers the download automatically
and enables detection once complete. Cloud users see a note that speaker
detection runs locally on their device.
* chore: hide GPU diarization setup guide in Settings
GPU pipeline is disabled (missing embedding+clustering step), no reason
to show the CUDA setup guide or probe GPU status on every Settings load.
* chore: remove GPU diarization setup guide from Settings entirely
Remove dead CUDA status check, state, and the entire setup guide UI
block. Recoverable from git history when the GPU pipeline is complete.
* fix: address code review findings (SSRF, diarization, React races, i18n, packaging)
Remove dead GPU diarization scaffolding entirely: gpuDiarization.js,
download-cudnn.js, ONNX worker GPU handlers, cuDNN path detection,
CUDA IPC handler/preload bridge, electron-builder CUDA Suggests, and
110 unused GPU translation keys. Revert worker heap from 2048MB to
512MB since the worker only handles MiniLM embeddings now.
Write diarization WAV to temp dir instead of next to user's source
file, with finally-based cleanup. Restore 25MB BYOK file-size limit
regardless of diarize flag. Gate single-URL download input while
batch is processing.
Tighten isPrivateIp: block 0.0.0.0/8, CGNAT 100.64/10, multicast
224/4, reserved 240/4, broadcast, and IPv6 multicast ff00::/8.
Drain HEAD response socket. Resolve relative redirect URLs against
the original. Validate youtu.be video IDs to 11-char alphanumeric.
Add 18 regression tests covering isPrivateIp for IPv4, IPv6, mapped
addresses, CGNAT, multicast, ssrfSafeLookup reject/allow, and
youtu.be ID validation.
* fix: address uriel review findings (input validation, cleanup, minor hardening)
* fix: harden IPC validation, HEAD redirect handling, deep link matching, and test runner
* fix: add path validation to cloud/BYOK IPC handlers and harden URL downloader
* fix: unpack youtube-dl-exec from ASAR and resolve binary path for packaged builds
* fix: harden SSRF protection and IPC input validation
* fix(desktop): clear saveTimeoutRef after debounced save fires
* fix: guard against malformed URL encoding and missing yt-dlp binary
* fix: harden IPC handlers, fix stale closure in transcription flow, and align batch diarization logic
* fix: clean up dead i18n keys, gate Enter-key during batch, and tighten BYOK diarize handling
* fix: address review (path validation, SSRF, proxy, yt-dlp sidecar + self-update)
* fix(logging): avoid externally-controlled format string in debug logger
* fix: address review (diarized_json, CI token scope, batch flow parity, proxy redirects, path allowlist)
* fix(url-download): pick the real yt-dlp output over leftover artifacts
* fix(url-download): harden yt-dlp pipeline (checksums, JS runtime, self-heal, timeouts)
- Pin per-asset SHA-256 in download-yt-dlp.js and verify after download; bump to 2026.07.04
- Wire yt-dlp's EJS JS runtime to the app's own Electron binary via ELECTRON_RUN_AS_NODE
- Self-update on the nightly channel; checksum-verify the user-writable cache copy before every spawn (0700 dir)
- Extend stale-extractor self-heal to the metadata step where those failures actually surface
- Map YouTube bot-check/403/login walls to YOUTUBE_BLOCKED with an honest retry-later message
- Hard timeouts on yt-dlp runs; HEAD fallback to GET for servers that reject HEAD; User-Agent on direct/proxy paths; immediate abort during stalls; --max-filesize skips surface as FILE_TOO_LARGE; fe80::/10 link-local range; drop stale plan-doc comment markers
* fix(ipc): narrow audio path allowlist and add per-download abort registry
- Drop $HOME from the transcribe/diarize path allowlist; user-picked files are individually approved via the dialog/drag-drop flow
- Replace the single global URL-download abort slot with an id-keyed registry so a new download can no longer abort an unrelated one; progress events carry downloadId
- Revert dead timeoutMs param on onnxWorkerClient.request (no caller passed it)
- Consolidate speakerMerge imports; extract isInvitationDeepLink in main.js
* fix(upload): unify single/batch pipelines, cancel semantics, and error UX
- Extract shared transcribe+diarize+merge pipeline into fileTranscription.ts (transcribeFileWithSpeakers); both flows now pass the real audio duration instead of the last diarization segment's end
- BYOK-native diarization no longer requires local sherpa-onnx models in the batch flow
- Batch cancel discards in-flight results: no note is saved after Cancel All, current item shows Cancelled
- Queue ref updated synchronously (drain loop can no longer miss items added as the last item settles)
- Batch errors localized via shared DOWNLOAD_ERROR_KEYS map incl. NO_SPEECH_DETECTED; new YOUTUBE_BLOCKED and FILE_TOO_LARGE messages; progress bar counts failed items as settled with a failed-count label
- Batch file notes get the same generated titles as the single flow
- Single/batch mutual exclusion: files and URLs join the queue while one exists or is processing; single Transcribe disabled during a batch; per-download ids wired through so flows can't cancel each other's downloads
- Retry after a URL failure re-runs the download instead of silently no-oping
- Speaker toggle flips on first click, auto-downloads/heals models, and gets switch semantics (role, aria-checked, aria-label)
- Videos folder routed by name so user-created folders are respected (migration no longer promotes them)
- Skipped/over-cap pasted URLs surface a notice instead of dropping silently (cap 50)
- Dedupe folder Select into FolderSelect, shared upload field classes, drop dead cleanupProgress branch and inline YouTube-URL IIFE
- 5 new i18n keys across 10 locales
* test(url-download): cover checksum verification, nightly channel, fe80::/10, block heuristic, and merge fallbacks
* chore: drop unused eslint-disable
* fix(upload): address end-to-end review findings
- Self-hosted mode: never routes diarization to BYOK (local diarization works again), batch size gate exempted like the single flow, provider diarize notes hidden
- Speaker toggle skips the local model download entirely when BYOK-native diarization covers it; mount-time heal rolls the toggle back if the download fails
- Batch Open note passes the destination folder so the note actually opens
- Cancel All flips the in-flight item immediately instead of leaving it spinning
- runId guard after saveNote (cancel during save no longer blanks the panel)
- Batch URL notice renders without a queue, clears on reset/clear-queue
- Drop the frozen per-item bar during transcription and the dead converting stage/i18n key
* fix(url-download): close update-path checksum bypass and connect-phase cancel gap
- maybeUpdateYtDlp verifies the cache copy before executing it (discard+reseed on mismatch) and only re-records the checksum after a successful update exit, so the update path can no longer execute or bless a tampered binary
- Abort signal wired into HEAD/GET requests and checked at streamToFile entry: cancel now lands during the connect phase instead of being silently lost
- youtube.com/live/<id> URLs accepted; metadata timeouts surface as retryable download failures instead of 'video unavailable'
- Startup sweep for orphaned ow-url-*/ow-diarize-* temp files older than 24h
- resolveYtDlpBinary falls back to the cache copy when no bundle exists during an update window
- Renderer rejects http:// direct URLs upfront (main is HTTPS-only); http YouTube links still accepted and coerced
- Folder pull-sync converges any name collision by adopting identity (not just defaults), preventing a permanent sync wedge when a user-created cloud 'Videos' meets the seeded default
- Tests: tampered-cache update path, unverified-copy discard; ensureCacheBinary records checksums
* fix(url-download): harden batch, sync, and downloader flows from end-to-end review
- Hide per-item "Open note" while a batch runs (opening a note unmounted
the upload view and silently cancelled the remaining queue)
- Bound downloadViaProxy's connect/header wait with CONNECT_TIMEOUT_MS so
a proxy that never answers can't wedge the batch queue
- Seed the Videos folder once (PRAGMA user_version) so deleting or
renaming it no longer resurrects an undeletable default on relaunch
- Restrict pull-side folder identity adoption to unlinked folders with
exact-name match first; case-insensitive fallback only for defaults
- Default batch URL uploads to the Videos folder like the single-URL flow
- Surface partial-transcription warnings on completed batch items
- Degrade BYOK diarization to a plain transcript when the base URL is not
OpenAI/Mistral instead of failing the whole upload
- Kill the whole yt-dlp process tree on cancel/timeout (detached group on
Unix, taskkill /T on Windows)
- Memoize yt-dlp cache checksum verification (one hash per binary change)
- Report real YouTube download progress (--newline + percent parsing) and
pulse indeterminate bars when size is unknown
- Track concurrent diarization processes in a set with collision-free
temp names and a guarded pid-file slot
- Cover downloadDirect's redirect/SSRF/size/cancel branches with tests;
drop an unused export and stale comments
* fix(upload): move batch queue to a zustand store so batches survive navigation
Follows the actionProcessingStore pattern: queue state and the drain loop
live at module level, so switching views (or opening a completed note
mid-batch) no longer unmounts and silently cancels the remaining items —
the "Open note" guard is no longer needed. Cancel all now orphans the run
via a run-id bump instead of waiting on the un-abortable transcription
IPC: the UI unlocks immediately and the stale run's late results are
discarded on arrival. Drops the unused currentItemId state.
---------
Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com>
|
||
|
|
e4e5d19ddb |
feat(parakeet): add NVIDIA Nemotron streaming ASR models (EN + multilingual 3.5) (#1131)
* feat: add Nemotron ASR model Add Nemotron Speech Streaming EN 0.6B through sherpa-onnx online websocket support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * fix(parakeet): bump sherpa-onnx to 1.13.4 for Nemotron streaming support The Nemotron model artifact in the registry (sherpa-onnx-nemotron-speech-streaming-en-0.6b-560ms-int8-2026-04-25) was re-exported alongside runtime updates in sherpa-onnx v1.13.0; the previously pinned v1.12.23 runtime produces degraded transcripts on Nemotron streaming models (verified empirically: 1.12.x mis-decodes words that 1.13.4 gets right on the model's own test wavs). Also correct the Nemotron registry size to the extracted model size (632MB, matching the convention used by the other Parakeet entries — this value feeds the pre-download disk-space check), and document the new model and the offline/online runtime split in CLAUDE.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * feat(parakeet): add multilingual Nemotron 3.5 ASR Streaming 0.6B model Adds nemotron-3.5-asr-streaming-0.6b to the local model registry alongside the English Nemotron model. Cache-aware streaming FastConformer with per-stream automatic language detection; 15 transcription-ready languages. Uses the same online websocket runtime added for the English model — no code changes needed, only registry + i18n + docs. Verified against sherpa-onnx 1.13.4: the published int8 export (sherpa-onnx-nemotron-3.5-asr-streaming-0.6b-560ms-int8-2026-06-11) loads with the exact encoder/decoder/joiner/tokens flags OpenWhispr passes and decodes English, German, and Japanese test wavs with language auto-detection. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * feat(parakeet): stream Nemotron live preview over a persistent connection The Live Transcription Preview previously transcribed buffered 1.5s chunks independently, which discards a cache-aware streaming model's context at every chunk boundary. When the selected NVIDIA model has runtime "online" (Nemotron), the preview now opens one persistent websocket stream to the online sherpa-onnx server for the whole recording: worklet PCM is converted to float32 and forwarded as it is captured, partial results replace the preview text live, and stopping the preview flushes the stream ("Done" / "Done!" handshake) for the final text. Audio captured while the streaming server is warming up is buffered and fed into the stream once it opens. Offline models keep the existing chunked path, and the online path falls back to it if the stream cannot start. Covered by test/helpers/parakeetOnlineStream.test.js using a mock websocket server that speaks the sherpa-onnx online protocol (partials, endpointed segments, finish handshake, abort). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * refactor(parakeet): code-quality pass on Nemotron integration Applies the findings from a four-angle review (reuse, simplification, efficiency, altitude) of the Nemotron PR diff: - Single registry access point: new parakeetModelInfo.js owns getModelRuntime and REQUIRED_MODEL_FILES; parakeetWsServer no longer reads the model registry, and the getModelRuntime pass-throughs on both manager layers are gone. ipcHandlers now asks a capability question (supportsOnlineStreaming) instead of comparing runtime strings. - Runtime-only binary API: getWsBinaryPath/isAvailable take a runtime (the model-name-or-runtime union sniffing is gone), callers resolve model to runtime once, hasAnyWsBinary replaces the hand-rolled offline-OR-online checks, and per-runtime path memoization is restored. - One online-protocol implementation: _transcribeOnline is now a thin wrapper over createOnlineStream instead of a ~55-line duplicate of it. - Incremental result accumulation: createOnlineAccumulator parses each websocket message once (the previous per-message re-parse of the whole history was quadratic over a recording) and live preview updates are emitted only when the text actually changes. parseOnlineMessages keeps its API as a wrapper over the accumulator. - Wire format stays in the ws layer: the stream handle exposes sendPcm16 (int16 capture format, converted internally) so ipcHandlers no longer performs float32 conversion; abort now settles a pending finish(). - Preview session staleness is tracked by one generation counter instead of three heuristic condition sets. - pcm16ToFloat32 is the canonical converter (returns Float32Array); meetingEchoLeakDetector and liveSpeakerIdentifier drop their local copies. - Sidecar reaper matches parakeet against its two exact ws binary prefixes instead of the whole "sherpa-onnx-" vendor prefix, which also overlapped the diarization entry. - Download script reuses the shared cleanupFiles (extended to accept multiple keep prefixes) and copyBinary drops its redundant label and basename handling. Verified on Node 24 with full dependencies: 278 tests, 270 pass, 0 fail (8 skipped), tsc --noEmit clean, eslint clean, prettier clean. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AdfxroZFwgsVyKMhum1UAo * fix(parakeet): repair sherpa-onnx 1.13.4 binary downloads - Windows release assets are now suffixed (-MD-Release); old name 404s - ad-hoc re-sign macOS binaries/dylibs: upstream ships libonnxruntime with an invalid arm64 signature that dyld SIGKILLs - rm before copy so a stale symlink can't redirect writes into an old inode - re-point the unversioned lib symlink and prune stale versioned libs - run download:sherpa-onnx in predev/prestart (marker-gated, fast no-op) * refactor(parakeet): condense comments --------- Co-authored-by: Joel Tony <github@jaytau.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Alcahest Malus DienheimandGabriel Stein
|
f629b07bab |
fix: meeting stream reconnect on session expiry + onnx worker packaging (#830)
* fix: include onnx worker in production builds * fix: auto-reconnect meeting streams on OpenAI 60min session expiry * test: add session expiry and pre-connect buffer tests for realtime streaming * test: add zero-audio-loss verification for session reconnect flow * fix(meeting): harden stream reconnect after merging main - Pass a real sender to fetchRealtimeToken: the null event crashed getAuthHeader in preconfigured mode, so cloud reconnects always failed - Arm the pre-connect buffer (beginConnecting) on the swapped-in instances so audio arriving during the token fetch is not dropped - Mirror connectRealtimeStreaming's connect options (environment, tenant, keyterms, sampleRate) and the actual stream topology instead of re-deriving system-audio capability - On reconnect failure, tear down the half-connected instances and restore the old ones so a proactive reconnect keeps transcribing until the hard session limit retries - Tear down freshly connected streams when the recording was stopped mid-reconnect instead of leaking open sockets - Skip the reconnect-count increment when connection context is gone * build: fail packaged builds when the onnx worker is missing from app.asar.unpacked Extends the afterPack unpacked-binary verification to the ONNX worker entry point so this packaging failure breaks the build instead of crash-looping the utility process at runtime. Also aligns the worker path rewrite with the ffmpegUtils regex so Windows backslash paths are handled. * fix(realtime): keep dictation error flow on session expiry and fork onnx worker via asar path - Gate the session_expired reconnect path on an onSessionExpired handler: consumers without one (dictation) keep the onError/onSessionEnd flow, so a 60-minute dictation still auto-stops with the accumulated transcript. - Fork the onnx worker via the asar-virtual path again: forking from app.asar.unpacked breaks require resolution for asar-only pure-JS deps (onnxruntime-common); Electron already redirects the unpacked file read. --------- Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com> |
||
|
Gabriel Stein
|
7d1674cab8 |
fix: bundled binaries missing from app.asar.unpacked break Windows local transcription (#1124)
* fix(build): unpack ps-list vendor executable from asar ps-list@9 resolves vendor/fastlist-*.exe relative to __dirname with no asar awareness. Without an asarUnpack entry the exe stays inside app.asar and Windows process detection (meeting-app detection) fails on every poll. * fix(stt): never spawn ffmpeg from inside app.asar fs.existsSync returns true for files inside an asar, so when the unpacked ffmpeg-static binary was missing, getFFmpegPath returned the in-asar path as spawnable and every local transcription failed with a cryptic spawn ENOENT. Fall through to the system-candidate and PATH scan instead, and make the convertToWav rejection message actionable. * build: fail packaging when unpacked binaries are missing Assert in afterPack that app.asar.unpacked contains the ffmpeg-static binary (all platforms) and the ps-list fastlist-*.exe vendor executable (Windows targets - electron-builder strips *.exe from node_modules on non-Windows builds). Throw instead of warn: a build missing these must fail loudly instead of shipping broken local transcription. |
||
|
Alcahest Malus Dienheim
|
43b4c16880 | fix(linux): fall back to --no-sandbox when user namespaces are restricted (#1042) | ||
|
Gabriel Stein
|
01f8557b0c |
Native Windows system audio capture for meeting notes (#960)
* feat(meeting): native Windows system audio capture via WASAPI process loopback Windows previously captured system audio only through Chromium's display-media loopback, which hears just the default render device. When a meeting app played to a non-default output (a common case), the loopback streamed silence with no error and meeting notes missed the other side of the call. Add a native WASAPI process-loopback helper (windows-system-audio-helper) that runs in EXCLUDE mode against OpenWhispr's own process tree, so it hears every application on every output device while never re-capturing the app's own sounds. It mirrors the linux-system-audio-helper contract: 24 kHz mono s16le PCM on stdout, line-delimited JSON events on stderr, managed by windowsLoopbackAudioManager (modeled on linuxPortalAudioManager). The new "wasapi-loopback" strategy is used when the helper is available; if it is missing (Windows < 10 2004) or fails to start, the strategy is downgraded to "loopback" and the renderer transparently falls back to the pre-existing Chromium loopback path — no behavior change for that case. Also harden the fallback path: the win32 setDisplayMediaRequestHandler now handles empty source lists and rejections (previously it could hang meeting start forever) and skips thumbnail generation for the discarded video; and the renderer now surfaces a "continuing with microphone only" error when system-audio capture fails entirely. The helper binary is built in CI (build-windows-system-audio-helper.yml), downloaded during prebuild:win, and bundled like the other Windows binaries. The capability cache is warmed at startup so the first meeting start pays no probe cost. macOS and Linux paths are unchanged. * fix(i18n): add missing Hyprland hotkey translations for 9 locales The Hyprland config-reload fix (#919) added three English-only keys (hyprlandConfigWriteWarningTitle/Description, hyprlandUnbindDescription). Add translations for de, es, fr, it, ja, pt, ru, zh-CN, zh-TW so i18n:check passes and these warnings render localized text. * fix(meeting): harden Windows system-audio helper edge cases - helper.c: check CreateThread result so a failed stdin monitor surfaces a warning instead of silently disabling parent-death shutdown, and close the thread handle to avoid a leak - windowsLoopbackAudioManager: cache only definitive capability probe results so a transient probe failure (timeout/spawn error) no longer pins 'unavailable' for the whole session and forces the Chromium fallback - download script: pin via exact release tag when an override is set (matching the documented reproducible-build behavior) and report the actually-searched tag in the not-found message * fix(meeting): harden Windows system-audio capture from review findings - Surface meeting recording errors as a toast; the 'continuing with microphone only' notice was written to store state no UI read - Cache transient capability-probe failures (activation timeout, spawn failure) with a short TTL instead of pinning the native path unavailable for the whole session - Resolve stop() only after the helper process exits so a stale exit event can't error a later capture session - Skip the generic exit error when the helper already reported a specific fatal error - Fill wall-clock silence gaps only while the stream is idle; rebase for clock drift during continuous audio instead of splicing silence - Return an explicit fallback from getMeetingSystemAudioPlan instead of falling through to undefined - release.yml: download and verify the helper in the Windows build and key the bin cache on the helper source |
||
|
Alcahest
|
3f4aad6549 | chore: bump bundled llama.cpp to b9763 (#995) | ||
|
Gabriel Stein
|
70a0d677b4 |
feat(onboarding): use-case intent capture, meeting step, and finish step (#928)
* feat(onboarding): use-case intent capture, meeting step, and finish step Reworks the onboarding wizard around an id-keyed dynamic step machine: - New "Why are you here?" step: multi-select use-case cards (OptionCard) plus an optional note, persisted to settings and synced fire-and-forget to /api/onboarding-intent for signed-in users - Permission copy rewritten to explain outcomes; system audio badge flips to Recommended when the user came for meeting notes - New conditional meeting step introducing auto-detection and the existing meeting hotkey (shown when system audio is granted or meetings selected) - New finish step: OpenWhispr Cloud summary with Open Settings deep-link (Settings > Transcription) or Skip; local/BYOK variant for skip-auth users; registry-gated Corti partner screen for medical intent - All new strings translated across the 10 locale files * refactor(lint): fix react-refresh and hook-deps warnings, run formatter - Move pending-invitation token helpers out of AcceptInvitationModal into src/utils/pendingInvitationToken.ts so the component file only exports components (react-refresh/only-export-components) - Memoize the invitations fallback in ShareNoteDialog so its callback dependencies are stable (react-hooks/exhaustive-deps) - Prettier reflow in clipboard.js * fix(onboarding): add back-to-sign-in escape from email verification Without it, users who never receive the verification email are stuck on the polling screen with no way back to choose another email or sign-in method. * style(onboarding): match back-to-sign-in button to auth screen idiom Align the verification back button with AuthenticationStep's continue- without-account styling; prettier array reflow in zh locale files. * refactor(auth): use Button ghost variant for subtle text actions Replace bespoke raw buttons (back-to-sign-in, continue-without-account) with the design-system Button, gaining focus-visible rings and proper disabled styling. * feat(onboarding): broaden medical use case to healthcare, clarify multi-select - Rename use-case id medical -> healthcare; card now reads "Medical, clinical & therapy transcription / Dictate patient notes, session notes and reports" so therapy and psychology users self-identify - Add "Select all that apply" hint and switch OptionCard's selection indicator from radio circle to checkbox square so multi-select is obvious - Update Corti finish title to match; all 10 locales updated * fix(onboarding): allow proceeding with Corti credentials in setup step The Corti provider tab (#929) now appears in the onboarding transcription picker, but canProceed only recognized openai/groq/mistral/custom — a user selecting Corti was blocked on Next unless an OpenAI key happened to be set. Gate on client id + secret, matching shouldUseStreaming's readiness rule. * feat(onboarding): preselect Corti provider when opening settings from finish step The Corti finish screen's Set up in Settings CTA now sets cloudTranscriptionProvider to corti so the settings picker opens on the Corti tab with its credential fields. Safe for signed-in users — the provider only takes effect in BYOK mode. * feat(onboarding): inline Corti setup on the finish step Replace the settings detour with credentials entered in place: honest copy (clinical-grade, HIPAA compliant), a visible $50 free credit badge next to the create-account link, Client ID/Secret fields reusing ApiKeyInput, and a US/EU data-region select (defaults prefilled from the store). Use Corti activates the provider in BYOK mode and completes onboarding; Not now falls through to the default finish. * refactor(onboarding): remove quit button from title bar The power/quit button only ever rendered in onboarding (TitleBar's sole consumer) and duplicates OS-level quit (Cmd+Q, dock, window controls). Remove it end to end: TitleBar quit UI + confirm dialog, app-quit IPC handler, appQuit preload bridge and type, and the titleBar i18n block in all 10 locales. * feat(onboarding): link Corti homepage with UTM and add setup steps Point the create-account link at corti.ai with the partnership UTM params instead of the console, and add a three-step guide (sign up, create API credentials in the console, paste below) since the homepage no longer lands users on signup. * feat(onboarding): refine use-case step copy and move progress into macOS title bar - Subtext constrained to readable width and rewritten to say plainly that the user's picks set up the app and steer what we build next - Select-all hint restyled to the existing uppercase section-label idiom - Note placeholder reduced to "Optional" — no canned examples - TitleBar gains a centered no-drag slot; on macOS the step progress renders there (reclaiming the progress strip's height), Windows/Linux keep the strip alongside their window controls * feat(onboarding): add upload-audio use case and skip button for optional steps - New "Uploading audio files" use-case option (id: upload) with FileAudio icon - Skip button beside Next on optional steps (use case, voice agent, meeting) - Add common.skip and the upload option copy across all 10 locales * feat(transcription): add Corti provider icon Register the Corti logo in PROVIDER_ICONS so the Corti tab and provider chips show a real icon; monochrome (currentColor) like xai. * refactor(meetings): consolidate meeting detection into the notification toggle The meeting-detection notification was the only consumer of detection, so a detector running with notifications off did nothing but burn CPU. Drive the audio detector from the notification toggle (notificationsEnabled && notifyMeetingDetection) and remove the redundant standalone Audio Detection setting, its UI section, and the now-dead calendar.detection translations. * fix(settings): don't grab the mic when opening settings MicrophoneSettings called getUserMedia on mount to read device labels, which activates a mic input session and interrupts other audio (pauses music on macOS) every time Settings opens. Enumerate devices first and only fall back to getUserMedia when labels are missing (permission not yet granted). * feat: add UTM tracking to Corti 'get a key' link Point the Corti transcription provider's 'Get a key' link at the corti.ai home page with referral UTM params, matching the canonical CORTI_SIGNUP_URL used in onboarding instead of console.corti.app. * chore: apply prettier formatting to pre-existing files Format 21 docs, build scripts, and test files that were not prettier-compliant on main. Pure formatting, no logic changes. Kept separate from the onboarding feature work. * chore(release): 1.7.3 Bump version to 1.7.3 and document the onboarding upgrade, Corti provider polish, simplified meeting-detection setting, and the Settings microphone fix in the changelog. * docs(changelog): capture full 1.7.3 scope Expand the 1.7.3 entry to cover everything merged since v1.7.2, not just the onboarding branch: Corti BYOK and xAI transcription providers, Snippets, Voice Agent hotkey, dictionary redesign, notification controls, Linux PipeWire/paste fixes, new AI models, and the broader fix batch. * fix(onboarding): keep step progress labels on one line The stepper renders inside TitleBar's absolutely-positioned `left-1/2` center slot, whose shrink-to-fit width is capped at the space from center to edge. That squeezed the row and wrapped the longest label ("Voice Agent") onto two lines. Adding `whitespace-nowrap` lets the wrapper resolve to the content's full width and center it cleanly, fixing the layout across all platforms (shared StepProgress component). * fix(auth): recover stale-bearer cloud requests via session cookie The main process authenticates cloud requests with a single saved bearer token that only refreshes on sign-in. After an email/password login (which, unlike OAuth, never force-refreshes it) that token can go stale, so main-process requests 401 while the renderer stays signed in via its session cookie. onboarding-intent was the visible casualty: it fires right after the use-case step and failed silently, leaving intent unsaved. Retry cloud-api-request once with the window's session cookie when a bearer request returns 401. Cookie-only on retry, since a tagging-along bearer overwrites the cookie server-side. * fix(onboarding): hide voice agent step for continue-without-account users Continue-without-account users have no LLM configured, so the dictation agent can't run — skip the voice agent onboarding step for them and show it only to signed-in account users. * fix(i18n): add Hyprland hotkey locale strings * fix(clipboard): prevent stale clipboard restores during paste * fix(dictation-agent): reach cloud agent from voice hotkey without a selected model The voice agent hotkey resolved to "skip" for signed-in OpenWhispr cloud users because agentReachable required a non-empty dictationAgentModel, which cloud mode never sets — pasting the raw transcript instead of running the agent. Mirror the cleanup cloud fallback: the dictation agent is now reachable in cloud (openwhispr) and self-hosted (lan) modes without an explicit model, matching ReasoningService.processText's empty-model allowance, and the agent route forces provider "openwhispr" in cloud mode so it no longer depends on the cleanup scope. Also pass voiceAgentRequested on the streaming-STT path so the hotkey works there too. Reachability logic is extracted into a pure, tested helper (resolveDictationAgentReachability) alongside resolveDictationRouteKind. * feat(onboarding): preview the real meeting notification in the meeting step Replace the abstract "meeting detected" explainer with a faithful preview of the actual notification, rendered from a shared MeetingNotificationCard that MeetingNotificationOverlay now also consumes so the two never drift. Add the onboarding.meeting.notification.{title,body,cta} keys across all 10 locales. * fix(note-formatting): use cloud provider when no model is selected Note formatting passed no provider to processText, so cloud users fell back to getModelProvider(""), which resolves from the cleanup scope — throwing "No reasoning model selected" when cleanup uses a BYOK/local provider. Pin the provider to openwhispr in cloud mode for both enhancement and title generation. * feat(onboarding): temporarily hide the meeting step for all users * fix(dev): diagnose missing Electron binary * feat(transcription): dedicated Audio Upload speech-to-text settings Give audio upload its own transcription context, mirroring how Note Recording was split from Dictation. Adds an Audio Upload tab under Settings -> Speech-to-Text with independent upload* settings; the upload page is now settings-only (inline picker removed). A one-time migration copies existing users' dictation transcription preference into the new context; new users default to OpenWhispr Cloud. - store: 9 upload* keys, selectResolvedUploadTranscription (falls back to base dictation values), migrateUploadTranscription (runs after provider migration) - settings: new UploadTranscriptionPanel (openwhispr/providers/local) - upload page: read-only consumer of resolved upload settings; no-provider CTA routes to the Audio Upload tab - i18n: tabs.upload across all 10 locales; remove orphaned upload setup strings - fix cross-window boolean sync for meeting/upload useLocalWhisper (BOOLEAN_SETTINGS) - extract shared useStartOnboarding hook; drop dead uploadSetupComplete write * fix(history): inline discarded toggle and stop list flash on refresh Move the "Show Discarded" toggle inline to the left of "Clear All" in the first day header so the pair reveals together on hover; keep it reachable above the empty state. Gate the loading spinner on the initial load only (no data yet) so toggling discarded keeps the current list mounted and swaps in new data instead of unmounting to a spinner and flashing. * feat(upload): cancel button during audio-file transcription Add a Cancel transcription action beneath the progress bar on the upload page (shown for every transcription). Cancelling returns to the upload screen and discards the result so nothing is saved; a run-id token guards the async path so a cancelled or superseded run never persists a note. * docs(changelog): document late-breaking 1.7.3 changes * fix(onboarding): apply Corti to all transcription scopes on finish Choosing Corti during onboarding only set the dictation cloud provider, so note recording and audio upload kept their previous provider and every Settings tab still showed OpenWhispr Cloud — each tab displays the InferenceMode field, which was never updated. setCloudTranscriptionForAllScopes now applies the chosen provider to dictation, note recording, and audio upload, and sets each scope's InferenceMode via a shared deriveTranscriptionMode helper (also reused by the provider-settings migration). * refactor(corti): rework streaming close handshake with ack tracking Replace the single closeResolve callback with a pendingAck/waitForAck mechanism that resolves on the matching server ack (flushed/ended) or socket close, times out cleanly, and closes the socket with code 1000. * feat(onboarding): rename activation step to Dictation and add voice agent test area - Rename the activation onboarding step to "Dictation" in the progress nav and "Dictation Setup" in the step header - Rename the voice agent step header to "Voice Agent Setup" - Add a test area under the voice agent hotkey with example spoken commands and a textarea to try it - Add and translate the new strings across all 10 locales * fix(meeting): send true 24kHz sample rate to cloud streaming providers Meeting capture runs at 24kHz end-to-end, but Corti/Deepgram/AssemblyAI were told 16kHz, so they read the PCM ~1.5x too slow and degraded accuracy. Thread the real rate through the meeting connect path and teach cortiStreaming to honor options.sampleRate (matching Deepgram/AssemblyAI). OpenAI already declares 24kHz; dictation (16kHz) and the local downsample path are untouched. * fix(dictation-agent): default to OpenWhispr Cloud for signed-in users The dictation agent scope defaulted to "providers" (BYOK) with no model, unlike the cleanup and chat-agent scopes which default to OpenWhispr Cloud. As a result, a signed-in user who never explicitly configured the dictation agent had it unreachable, so the voice agent hotkey silently typed the raw transcript instead of running the command. Default dictationAgentMode and dictationAgentCloudMode to "openwhispr" so the voice agent works out of the box on Cloud. Users who explicitly chose BYOK, local, or self-hosted are unaffected (persisted values are read first). * perf(corti): pre-warm dictation socket to cut startup latency and first-word loss Corti dictation paid the full token-mint + socket-open + CONFIG_ACCEPTED cost at hotkey time (~1s), and audio captured before the socket existed was dropped. Give CortiStreaming a real warmup() that opens the socket and finishes the config handshake ahead of recording, keeps it alive with WS pings, and lets connect() promote the warm socket for an instant start — mirroring Deepgram/AssemblyAI. Falls back to a cold connect when no warm socket exists, so behavior is never worse than before. Meeting mode (fresh instance, no warmup) is unaffected. |
||
|
janandAlcahest
|
0e8f8ef641 |
feat(linux): implement PipeWire system audio capture for transcriptions (#904)
* feat(linux): implement PipeWire system audio capture for meeting transcription * fix(blockers): PipeWire compile & CI helpers * refactor(linux): replace portal flow with direct PipeWire loopback capture Drop the ScreenCast portal path for system audio and use direct PipeWire sink capture instead. Add format negotiation (S16LE/24kHz/mono), nonblocking audio transport, atomic JSON events, and default-sink targeting via PipeWire metadata. * fix(linux): gate finish_start, log dropped PCM frames * refactor(linux): remove dead browser-portal path and portal grant leftovers --------- Co-authored-by: Alcahest <xaris@gzgd.info> |
||
|
perry2of5andGabriel Stein
|
2ea4af18e8 |
Expose controls for VAD settings from Solero (#718)
* Expose controls for VAD settings from Solero/whisper.cpp
* fix(vad): wire up whisper.cpp Silero VAD end-to-end
Previous commit exposed the VAD tuning UI but whisper-server was never
told to enable VAD: --vad and --vad-model were missing from the spawn
args, the ggml-silero model wasn't bundled or resolved, and toggling the
UI had no effect on transcription output.
- Bundle ggml-silero-v5.1.2.bin (864 KB) via scripts/download-whisper-vad-model.js
hooked into all prebuild/predev/prestart scripts; resolved by
WhisperManager.getVadModelPath() with the standard process.resourcesPath
+ dev fallback pattern (matches diarization.js).
- buildWhisperServerArgs now emits --vad and --vad-model when both
vadEnabled and vadModelPath are present; isVadActive() guards every
branch; VAD signature includes the model path so server restarts on path
change.
- Collapse three single-purpose IPC handlers
(whisper-vad-set-{dictation,note-recording,meeting}-enabled) into the
existing setWhisperVadConfig payload; renderer setters and electron.ts
type declarations follow suit.
- Move shared VAD defaults + limits into src/constants/whisperVad.json
imported by both whisperVadConfig.js (main) and settingsStore.ts
(renderer); drop the duplicated WHISPER_VAD_DEFAULTS + clampVadValue
block in the store.
- Unify the SettingsPage section on settingsPage.transcription.vad.*
(component previously read from two namespaces); add proper
fields.{label,info} keys with full translations in all 10 locales; drop
the dead params.* keys (only contained flag names); fix umlaut and typo
regressions in the original locale work.
- Remove unused Building2 import; drop the unexplained
ignoreDeprecations + baseUrl tsconfig escape hatches (typecheck is
clean without them).
- Update tests for the new vadModelPath arg + verify VAD args drop out
when the model is missing.
* fix(diarization): enforce max speaker cap when expected count is set
The diarizer was being hinted with `numSpeakers` but no post-hoc cap was
applied when an explicit expected count was provided, only on the default
fallback. When sherpa-onnx produced more clusters than requested (which
it can — observed 80+ for a 2-attendee meeting), they flowed straight
through to the UI.
Set `cap: numSpeakers` on all three explicit branches so
`capSpeakerClusters` collapses any excess into the dominant speaker.
* fix(vad): move Voice Activity Detection out of Preferences into Speech-to-Text
The section was buried under Preferences (sidebar label for the
"general" tab), so users couldn't find it. Move it into
Speech-to-Text → Dictation, rendered below the model picker, only when
local mode is selected and the local provider is whisper (VAD has no
effect on cloud, self-hosted, or Parakeet paths).
Extracted the JSX into a renderWhisperVadSettings helper inside
SettingsPage so the Dictation render callback stays readable.
* fix(vad): also render Voice Activity Detection in Note Recording sub-tab
The toggles apply to dictation, note recording, and meetings — so the
section should be visible from the Note Recording sub-tab too, not just
Dictation. Same gate (local Whisper provider only) so it stays hidden
when VAD has no effect.
* style: apply prettier line-width formatting
---------
Co-authored-by: Gabriel Stein <gabrielstein416@gmail.com>
|