124 Commits
Author SHA1 Message Date
Kit Langton 9f078b9a40 feat(history): find dictations and captures together on Mac (#1932)
* Stage clipboard-preserving capture insertion for Paste Last on Mac

* Update Rune and its Nix source hash for the picker work

* Select the latest ready paste by owner identity and revalidate it

* Exercise a held mixed paste picker with real bundles and a private Mac pasteboard

* Publish ready Annotate identity and queue owner-specific paste requests

* Wire the existing Mac paste shortcut and controls to the opt-in mixed picker

* Recheck the selected field and cover Paste Last with a disposable Mac receiver

* Refresh the Rune revision for the unified History pane

* Unify Mac History and keep only spoken words after a deliberate dictation cancel

* History and Paste Last call a Hex capture an annotation (t1612)

A finished Hex capture is an Annotation in History, its details and the
Paste Last picker, and the type filter is Annotations. The Captures pane no
longer rewrites its own label when the unified History is on, and the search
index drops the old synonym. Scrawl stays only as Scrawl.app and the on-disk
.scrawl format; a Hex-owned format is separate file and engine work.

* Gate the unified History behind a Labs toggle and give it one row shape (t1612)

Labs › Paste Last picker (labs.paste_last_picker; macOS, Dev audience, off by
default) replaces HEX_EXPERIMENTAL_PASTE_LAST and is read where the work
starts: the picker and its gesture, History's unified list, and the cancel
path's word writer. Off, Paste Last and History behave exactly as before.

History's rows now share one shape for dictations, annotations, screenshots
and recordings; a capture's detail card carries its picture, words, facts and
actions; a search or the type filter never leaves a stale card open; an empty
History has one empty state; and the type filter shows only when there is more
than one kind to filter.

The picker's rows sit on its title's left edge.

* Show the Paste Last picker card on macOS only (t1612)

* Refresh the Labs goldens for the Paste Last picker card (t1612)

* Keep the unified History's capture helpers to macOS builds (t1612)

* Keep the unified History's fixture pictures behind the Mac capture gate (t1612)

* Point the docs at the Paste Last picker lab (t1612)
2026-11-19 19:52:09 -08:00
Dax b33db219c9 fix annotate pen stopping short on gnome at fractional scale (#1995) (#1996)
The Annotate ink overlay on GNOME Wayland is Rune's XWayland window, which
lays out in its own points (X11 pixels over Xft.dpi). capture_host::draw sized
the engine's target frame from the ScreenCast portal's stream `size` instead,
the desktop's logical layout: 1536x960 for a 1920x1200 screen at 125%. Every
pointer past 1536 was outside the target, so the pen stopped at 1536/1920 of
the screen (and 960/1200 down); at 100% the two spaces coincide.

The XWayland target is now the overlay's own size in points, read from its
laid-out root element (Cx::layout_size), so it covers the whole screen whether
a compositor scales XWayland natively or traditionally. Native Wayland layer
surfaces keep the portal/named-output logical size unchanged, and the recorded
pixels over the overlay's scale stand in until the view has laid it out.

Known gap, recorded in the map: on XWayland the ink is one full-screen
primary-monitor window, so it cannot follow a named output.
2026-11-19 10:18:13 -05:00
Dax ebdd971ce1 fix(dictation): release the warm mic while the display sleeps or the screen is locked (#1956)
* fix(dictation): release the warm mic while the display sleeps or the screen is locked (#1953)

* fix(dictation): address review on warm mic away release

* docs: fix hex microphone resume wording
2026-11-18 02:34:47 -05:00
Kit Langton 434acbdf6b fix(annotate): show Paste and ask before sending suggestions (#1945)
* Separate local annotation paste from a routed send

* Render the Paste chip with the capsule theme

* Exercise the ready capsule fixtures on the Linux golden runner

* Keep the displayed result on handoff and test pointer and keyboard selection

* Preserve matched rules through finalization and update checked capsule goldens

* Give the ready annotation one named action and a quiet search control

* Check the two ready surfaces, their accessible names and delivery states

* Put the ready line and its list on shared edges, with Paste a 28 pt button

The preview sat 12 pt from the wall while the list's header and names sat
16 pt in, and Paste was a 32 pt fill 4 pt from the top and bottom beside
28 pt icon boxes. The preview now starts on the list's edge, and Paste,
Search, Inspect and the close button are one row of 28 pt boxes 6 pt apart
and 6 pt from the walls. Paste's 32 pt chip still takes the click.

* Update the Linux references for the ready capsule's shared edges

* Nest the annotation preview in the capsule's left corners

The preview sat 16 pt from the left wall and 10 pt from the top and bottom,
so its 4 pt corners turned about a point 6 pt from where the shell's 14 pt
corners do. It now clears all three walls by 10 pt, the shell's radius less
its own, and the list's 16 pt margin stays for words. The guard that held
the preview on that margin now checks the corner centres instead.

* Update the Linux references for the nested annotation preview
2026-11-17 20:35:10 -08:00
Dax 9a79a25475 make the labs card copy plain (#1929)
* make the labs card copy plain

Rewrite the Labs card titles and lines in words a regular user understands:
YOLO mode says the message sends itself, Live text and Failure notices name
the small recording indicator, Vocabulary help says which speech models it
affects, and the key-side cards name Option, Command, Shift, and Control.
Audit every other card, updating the CLI descriptions, all nine translated
languages, the website lab catalog, the feature docs, and the render goldens.

* fix labs copy review findings

Restore the ISO qualifier on the section-key card, spell out that
double-tap and hold-through shortcuts contain no letter or number key,
cover all Global mode stages in the API text card, and name the print
command in the print-start copy. Restore the developer API card titles,
fix the Annotate capture wording, and retranslate the affected strings.
Remove every replaced translation key and stale lab-name reference.

* fix second labs copy review

Update the live typing test and YOLO goldens. Add a Windows variant of the
hold-through-keys copy naming Ctrl, Alt, Shift, and the Windows key. Remove
the last unused rewrite-by-voice translation key, keep Annotate untranslated
in the replay title, and clean the remaining old lab-name comments.

* use the Polish Annotate pane name in its replay title

Polish calls the Annotate pane Adnotacje, so the Annotate last 15 seconds
title says Ostatnie 15 sekund w Adnotacjach again. Other languages keep
Annotate because their pane name is Annotate.
2026-11-16 22:32:02 -05:00
Dax 8cffd94db7 annotate: count words from transcript text when the model has no word timings (#1918)
* annotate: count words from transcript text when the model has no word timings

A model that requests no timestamps (GigaAM v3) returns its text with an
empty word list, so `transcript.words.len()` read 0: lists and JSON said
"0 words" and the capsule called it "no speech". `transcribe::word_count`
counts the model's own word times, or the words in the text when it has
none, and every capture count goes through it (finished captures, the
iPhone and screen paths, replay, and `hex capture list|show`).

* annotate: spread untimed words over the clip for the editor

A model that gives no word times (GigaAM v3) left a capture's words empty,
so a recording or annotation opened from the Captures pane read "Nothing
was said" and its Send used the placeholder text. The words path now
spreads the text over the clip with estimated times, as `hex` does out of
process, and the pane spreads a bundle already on disk the same way.
`word_count` still counts those bundles when they are listed.

* annotate: spread each untimed piece over its own audio

A model with no word times (GigaAM v3) had a multi-minute capture's whole
text spread over the entire clip, so words landed minutes away from where
they were said. Each `audio.wav` piece now spreads its own untimed text
over its own 30 s, on the same offset base as measured words, and the
estimate reaches `transcript()` so it never spreads the text again; a clip
held in memory still spreads over the whole of it.

Also fix the Captures pane's wording: an annotation's times stay on the
capture's clock, not the movie's.
2026-11-16 16:15:24 -05:00
Dax cdb96f585e capture: keep all audio in long captures (#1795) (#1894)
* capture: stream audio to disk so long captures keep all of it

A capture's timeline now writes 16 kHz mono PCM16 to its bundle's
audio.wav as it records, with the leading offset and gaps as silence, so
memory stays flat and the 20-minute drop is gone. An 8-hour guard bounds
disk use only; past it the rest is one recorded gap.

A finished capture is transcribed from the WAV in 30 s pieces, the
reader hex transcribe already used (lifted to transcription/pieces.rs),
and each piece's words move onto the capture clock. The Parakeet, Apple
(path-based), and GGUF paths keep every word without holding the
session. The pipeline never rewrites a streamed audio.wav from empty
samples, an iPhone narration is read back where it must be re-based,
and a cancelled capture closes and removes its file.

* capture: say where a capture's audio stopped

A streamed capture keeps all of its audio, but the 8-hour disk guard (or
a failed audio write) still stops it early, and only the process log
said so. The timeline now carries the stop point onto RecordedAudio,
the bundle stores it as audioStoppedAtMs (serde default, so old bundles
load unchanged), and three places say it plainly: summary.md, `hex
capture show`, and `hex capture status`.

The guard records its point at a 1 s test limit, and the summary line is
tested at 12:34 and 8:00:00 so both clock shapes are covered.

* capture: fill the replay's stop point from its bundle

The 15 s replay builds Written too, on macOS only, so the new field
broke the macOS clippy job while Linux never compiled that function.

* capture: keep a recording out of recovery until it stops

Review fixes for the streamed timeline.

A capture now streams to `audio.wav.partial` and renames it to `audio.wav`
in `Timeline::finish`, so a bundle with a movie and an in-progress audio
file is never mistaken for a capture whose build never ran (recover's
orphan rule, Scrawl.app's `scrawl recover`) and retention cleans a crashed
one like any other bundle. Cancel and discard remove the partial; a failed
publish reads it back into memory rather than pointing at a missing file.

Also: a write failure's stop point is the last sample that reached the
file, not the end of the failed chunk; a trim failure counts every written
sample; the capture's words keep one running frame total instead of
adding truncated per-piece millis; an iPhone narration's `since` drains the
decoded samples instead of copying them and treats a missing file as no
audio; the phone and replay paths carry the audio stop point into their
bundles; and empty audio answers `source: "none"`, as `transcribe_with`
does.

Tests: a movie plus partial audio is never owed, retention trashes a
crashed bundle's media, a write failure records its stop point, a missing
streamed file shifts to no samples, and the replay shift moves the stop
point with the clock.

* capture: publish audio.wav only once its build journals the capture

Follow-ups from review.

A capture now streams to audio.wav.partial and keeps that name after it
stops: the partial is renamed to audio.wav right after the build writes
unfinished.json, so audio.wav never exists without a journal, and a
stopped capture waiting in the finish queue no longer looks like an
orphan to Scrawl.app's recover. A partial that cannot take the name is
read back and written by the build, and its partial name removed; the
replay removes the unshifted partial once the narration is read for
shifting.

Also: a periodic header flush error is now a write failure (the rest of
the capture is a gap) rather than a log line; the midpoint of a failed
write records where the file stopped; and the words test cuts pieces at
odd sample counts and computes offsets from the running frame total, so
per-piece millisecond drift would fail it.

Tests: a streamed audio takes its final name only when the build settles
it (and falls back when it cannot), a capture that has not been
journaled is never owed a rebuild, a mid-capture write failure stops
where the file stopped, and the word offsets are exact at odd piece
lengths.

* capture: consume a streamed recording's partial when it is read

Follow-ups from review.

`RecordedAudio::since` and `into_memory` now remove the partial once its
samples are read: an iPhone narration (capture.rs's `host_narration`
path) used to leave the whole streamed file in the finished bundle. The
replay relies on the same path instead of removing it by hand, and the
light zip leaves `audio.wav.partial` out defensively. A read that fails
is an error to `into_memory` (so a build keeps the only copy) while
`since` still treats a missing file as no audio for an iPhone.

A failed publish now reads the audio back only when it produced the
samples the recording had; a failed read keeps the partial and removes
any half-copied `audio.wav`. `publish_audio` takes `&mut self` and points
the recording at the published file.

A journaled bundle whose `audio.wav` is missing adopts `audio.wav.partial`
before rebuilding. Docs say the guarantee is a streamed capture's, and
that a crash while recording or queued is retention's job, not
recovery's.

Tests: reading a streamed recording consumes the partial; a finished
iPhone bundle keeps none; a streamed audio is published only when its
build settles it, including the read-back and unreadable-partial cases; a
journaled capture adopts its partial.

* capture: keep a failed audio publish recoverable

Follow-ups from review.

The publish fallback read the partial with `into_memory`, which deletes
it, and then ignored whether its own write landed: a failed write left
the audio nowhere. It now reads through `read_streamed` (no deletion),
writes `audio.wav` and checks the result, and removes the partial only
once that write succeeded. A failed read or write removes any half
`audio.wav` and fails the build, which is the pipeline's existing failed
path: `recover::finished` runs only for an `Ok` build, so `unfinished.json`
stays and the next launch's recovery adopts the partial instead of the
capture quietly finishing without its audio.

`publish_audio` also treats a landed copy as success even when deleting
the partial fails, logging it rather than reporting failure and letting
the fallback overwrite a good `audio.wav`.

Tests: a blocked target keeps the partial and fails the build; a
fallback write that lands consumes the partial and writes `audio.wav`;
an unreadable partial is kept.
2026-11-16 10:28:54 -05:00
Dax 1cf2c651e1 docs(capture): note off-main accessibility reads of Hex's own overlay (#1859) (#1861) 2026-11-15 11:51:04 -05:00
Dax 687ea32d65 Bump Rune to 1899e64: overlays pace frames with the swap chain on Windows (#1832)
* Bump Rune to 1899e64: overlays pace frames with the swap chain on Windows

Windows HUD memory grew and the app could hang after dictation because the
overlay presented frames without waiting for swap chain room after Rune #844;
Rune #985 makes an overlay's frames wait for room in its swap chain, as a
window's do.

nix: vendor each Cargo source in its own directory. Rune's tiny-skia-path is a
path dependency inside Rune (a Git crate at 0.11.4 here), the same name and
version as resvg's from crates.io, and cargo's directory source keys a crate by
name and version alone.

Goldens: Rune's text drawing changed over the 152 commits since d6715352.

* Review: patch accesskit_macos, and make the nix vendor split name what breaks

Rune #975 put accesskit_macos in its [patch.crates-io], so Hex patches it too
(Rune's README asks consumers to); it is the fifth and last crate Rune patches.

The vendor script override now checks each fragment it rewrites and names the
missing one, its comment says crates.io goes to registry/ and Git sources to
git/, nixfmt formats the file, and check-nix-git-pins.py's docstring says the
hashes reach importCargoLock through the cargoDeps override.

* linux: keep Hex running when the window can't start

Rune r399 makes App::run print a startup error and exit 1 on native targets
instead of panicking, so a Linux Hex with no display died, the next launch
became a fresh instance, and Check Linux's single-instance exercise saw
`hex start` hang and `hex app --hidden` reset.

run_app now returns Rune's try_run error and run_window reports it and waits,
as it did for the panic (#979).

* macos, windows: take rune's startup error instead of exiting

Rune r399 prints a startup error and exits the process from `run` on
native targets, which skipped Hex's own stop: macOS's drain (workers,
event-log flush, a recording kept for the next launch) and Windows' GPU
draw-marker disarm, so the launch that failed before drawing was read as
a crash while drawing. Both apps take `try_run`'s error now, log it, and
stop the way Quit does; Windows also reports it from `hex app`. The
Linux window already took the error (#979).

* macos: a launch whose window never started exits 1

Taking Rune's try_run error lets the app stop through the drain, but
process_exit::finish ends with _exit(0), so a failed GUI launch reported
success. finish_with takes the code; the window's failure exits 1 while a
normal quit keeps 0.

* docs: overlays wait for swap-chain room on windows since rune #985

Rune #985 makes every overlay's frames wait for room in its swap chain, so
the notes that said only a window waits and that the HUD's 60 Hz timer worked
around a suspected cause were stale. The timer is a cap on top of the slot
wait now. The macOS and Windows startup-error comments cited #979, which is
the Linux window issue; they describe Rune r399 instead.

* windows: a failed start keeps the GPU draw marker

disarm() means a frame reached the screen or Hex quit cleanly; a start that
never drew is neither, so it keeps the marker and the next launch falls back to
the high-performance GPU.

* docs: say what was measured about overlay pacing

* docs: say what stays bounded on windows, not what shrinks

The meetings map and the dictation map described the fix as memory that no
longer grows, and as Rune holding a frame's slot and skipping it. Rune #985
defers the frame until its swap chain has room, then draws it, which bounds
the frames in flight; the growth itself hasn't been re-measured on Windows
with the fix.
2026-11-14 20:40:20 -05:00
Kit Langton 804c2af789 feat(capture): Annotate suggests the OpenCode session open in a Ghostty pane, and never guesses between two (#1514)
* Annotate: route Ghostty captures by the terminal's OpenCode session and folder

* fix(capture): build the argv parser only where it's used (t1451)

* fix(capture): a TUI that moved off its command line's session names none (t1509)
2026-11-12 16:47:14 -08:00
Dax ec7b6b283d labs: copy captures as a zip (#1721)
* labs: copy captures as a zip

A finished Annotate capture can go on the clipboard as one light zip
file (Labs > Copy captures as a zip, off by default): the bundle's
manifest, summary, contact sheet, and keyframes, with summary.md's
bundle path written as ".", and without its video, audio, full-size
frames, delivery markers, or an earlier copy.

crates/capture/src/slim_zip.rs writes it with STORED local headers, a
central directory, and an end-of-central-directory record (CRC-32,
UTF-8 flag, DOS timestamp; no dependency). It lands inside the bundle
beside the other files, renamed into place, so retention removes it
with the bundle.

The clipboard holds only the file: a macOS file URL, Windows CF_HDROP,
or Linux text/uri-list (X11 target or wl-copy). A failed zip build or
clipboard write logs and falls back to the normal copy.

Checks: slim_zip unit tests (CRC-32 vector, central-directory read-back,
excludes, summary rewrite, a second copy), linux/paste file_uri, the
labs switch test, and the regenerated Labs goldens. Not verified: a
real paste of the zip into a chat or agent app on any OS.

* labs: leave the light zip out of the agent host copy

The SSH light-files tar copies a bundle wholesale to the agent host, so
a capture's <stem>.scrawl.slim.zip would travel with it. Share the name
pattern as scrawl-router's SLIM_ZIP_SUFFIX/SLIM_ZIP_TEMP_SUFFIX, use it
from slim_zip, and exclude both from light_files_tar. The existing tar
test now puts the zip and its temp file in the fixture and checks they
stay out.

No other path copies a bundle wholesale: phone ingest builds a new
bundle from the movie and timeline, retention shrinks or trashes named
media, and zoom/export write their own files.

* labs: tidy the capture zip copy

Generate the zipped summary with the bundle folder as ".", so only its
header names the copy's location and transcript text is never touched.

Use strict limits before ZIP64 (0xFFFF files, 0xFFFFFFFF bytes), a
Windows path from encode_wide rather than lossy text, and the current
time for an entry whose summary.md is missing. DOS times stay UTC: the
crate has no local-time facility without a new dependency.

Show the lab row only where Annotate itself is available, under the same
Annotate policy lock, with a folder icon and a shorter description that
fits its card. Docs note the terminal paste gets no text and that the
manifest and summary still name media the zip leaves out.

* labs: match the capture zip description

Use the same shortened sentence in the Labs description the card shows.
2026-11-12 09:14:48 -05:00
Dax 9ca6bf8d77 capture: name why a capture has no screen on linux (#1709)
* capture: name why a capture has no screen on linux

A Linux capture with no screen frames was saved with the generic "the
screen recorded no frames" even when the screen source had failed, so the
summary blamed an empty screen instead of naming the real reason.

The failure is now kept when the person stops the capture rather than
dropped, so it survives into the stop. A PipeWire stream that sends no
frames fails with that reason (or says the stream sent no frames), an X11
sampler that grabs nothing fails instead of returning quietly, and the
summary's no-screen reason comes from the reason recorded for the capture.

The PipeWire buffers param also asks for at least two buffers, four
preferred, since `newest_locked` holds the last one until the next frame
arrives and a single buffer would stall.

* capture: keep the generic reason for an early x11 stop, document early stream failure

An X11 sampler only grabs nothing when the person stopped before its first grab, so it returns quietly again and the generic no-frames reason stands. The test now checks the source's failure is kept and, on Linux, that no failure falls back to the platform's literal reason.
2026-11-11 20:56:55 -05:00
Kit Langton 4ac1a79dbf fix(capture): a page that names its session beats a site rule for its host (#1627) 2026-11-09 19:37:11 -08:00
Kit Langton 82f1e4feab feat(macos): key shortcuts keep working while another app holds secure input (#1625) 2026-11-09 18:32:16 -08:00
Dax ceb07a1a5c annotate: use OpenCode in WSL when the lab is on (#1586)
* annotate: send through OpenCode in WSL when the lab is on

* annotate: address review

* annotate: dropped wsl replies read as no answer
2026-11-09 08:08:57 -05:00
Dax 1099e264ca fix: stop re-reading screen recording every 2s while idle (#1592) 2026-11-09 07:53:27 -05:00
Dax ece7cf9bfb Linux: Annotate on GNOME lab (capsule and ink float on XWayland) (#1529)
* add a Labs toggle for Annotate overlays on GNOME

Rune won't back an overlay with an XWayland window when it asks for keys
on demand, so Annotate's recording capsule, ink, and picker never show on
GNOME Wayland: a capture records with nothing on screen. With Labs ›
Annotate on GNOME (`labs.gnome_annotate`, off by default), open them on an
already running XWayland like the meeting capsules do, with the same
checks at each show and without ever starting XWayland. The ink is made
with pointer regions and no keys, since it must take the pointer later
and Rune refuses an OnDemand overlay there; Esc can't stop drawing, so
the capsule's pen button or the Annotate key does.

idea and diagnosis from @pkkid in #1526

* Bump Rune to f2fb51ea

XWayland overlays now update their input shape on every drawn frame

* fix the Annotate lab copy and the XWayland checks

Esc can't stop drawing on the GNOME path, and a tap of the Annotate key
stops the recording, not the pen; the capsule's pen button or letting go
of a held key ends drawing. Use each language's existing name for the
Annotate feature, evaluate the ink's XWayland check once at install, and
skip the picker's check when nothing shows.
2026-11-08 04:13:40 -05:00
Dax bc996e85a8 linux: open the annotate pen on the recorded output (#1515)
* linux: open the annotate pen on the output the screencast portal records

* linux: leave the pen's display alone when the named output isn't found
2026-11-07 23:14:58 -05:00
Dax b0ecc0a809 Keep capture media safe when Linux retention can't trash it (#1516)
* Keep capture media safe when Linux retention can't trash it

* docs(capture): describe staged media recovery
2026-11-07 23:05:30 -05:00
Kit Langton 8d998ab173 fix(capture): Shift in the Annotate list marks the row under the pointer (#1503)
* Capture capsule: Shift follows the row under the pointer, and no shown row moves (t1440)

* Capture capsule goldens: Shift over the pointed row (t1440)
2026-11-07 19:02:18 -08:00
Dax e463421b7f fix: fit the capture capsule's notch shell to the measured notch (#1461)
* fix: fit the capture capsule's notch shell to the measured notch

The notch flow drew a black shell from a fixed 185 pt pretend notch centred
on the panel, so on macOS it could sit a pixel off the hardware notch and
show desktop beside it.

Measure the pointer screen's notch the way the dictation HUD does, snap its
edges and the panel origin to the backing-pixel grid, and place the panel's
left edge on the display instead of centring it. The surface takes the
geometry so the tucked shell is the notch, the resting shell covers it with
12 pt either side, and the content centres on it; a pending entry or a rest
already computed is refreshed when the geometry changes. A screen that can't
be read keeps the pretend notch and today's centred placement.

* fix: hold the notch shell to its items and remeasure it every poll

The notch branch of `layout_at` replaced the shell's x with the notch's
middle, so a recording with its chips shown, a row that hangs left of its
middle for the lead dot and the clock, pushed the dot past its left wall.
Widen about the shell's own middle, and only when it is narrower than the
notch with 12 pt either side.

Measure the pointer screen's notch every poll while the capsule hangs from
it, not only when the anchor changes: Rune resolves `UnderPointer` on show
and on every display change while the picker follows the pointer, so a
MacBook's inset stayed on an external display and a second capture's picker
used the first one's measurement. `set_notch` and `place` run only when the
measurement changed.

`set_notch` now respects Reduce Motion: a pending entry waits at rest and a
hidden capsule stays where it left instead of jumping to the tucked pose.

`pointer_screen` uses Rune's `UnderPointer` boundary rule (open at a
screen's bottom, closed at its top); the fallback stays the main screen.

docs/features/capture.md lists the new tests and no longer suggests a flat
display keeps today's exactly centred placement on macOS.

* fix: keep a showing capsule put and pin its measured display

The notch read moved to every poll made a showing capsule jump displays
whenever the pointer did: Rune resolves UnderPointer when it is placed, so
a re-place put the panel on whichever screen the pointer was on then. Read
the notch only when the panel may go to the pointer's display: a new
capture, an anchor change, or the picker following the pointer. A showing
capsule that isn't following keeps its place, whatever the notch reads
now. `move_of` makes that decision pure, and four tests cover it.

Read the screen's NSScreenNumber with the notch and pin the panel to it
with DisplayTarget::Native(NativeDisplayId::MacOS), so the panel and the
notch it is fitted to cannot disagree about the display. Skip measuring and
placing without an overlay, so the playground and stills keep the pretend
notch.

docs/features/capture.md no longer says the notch is re-read every poll,
names the new tests, and says following across displays may show one frame
at the previous inset before it re-places.

* fix: place the capsule when it comes back from hidden

A pinned display stays pinned while the capsule is hidden, and Rune's
show places the window on it. Ways back that aren't a new capture, such
as the permission helper or stopping after a close, showed on the last
display instead of the pointer's. Work out whether the overlay will be
up before the place block, treat hidden to shown as a place, and place
before Rune shows it in the same poll.

Also describe how following really works once the display is pinned:
Rune's follow only runs for UnderPointer, so Hex's poll re-places
display and inset together while picking; a stale inset is only
possible on the fallback where the screen number couldn't be read.

* fix: wake the capsule at the picker's pace while it crosses displays

Pinned to the measured screen, Rune's own follow never runs for the notch
capsule, and the poll that should re-place it during Selecting had no
wake: pointer events only dirty an overlay while the pointer is over the
panel, draw returns early when idle and not dirty, and the flow asks for
no timer. So the picker, which re-checks the display every 100 ms, moved
to the next screen while the capsule stayed behind. Ask for a wake at the
picker's own cadence while following a pinned capsule with an overlay, so
the changed read re-places display and inset within one check of the
picker.

`follows_display` makes that decision pure and its test covers the cases;
the wake itself and the live crossing still need two real displays.

Correct the comments in mod.rs and docs/features/capture.md: pinned, Rune
doesn't follow, and Hex's poll crosses the capsule with the picker within
~100 ms rather than being woken by pointer events.
2026-11-06 14:10:46 -05:00
Kit Langton 86ed547288 Menu bar: meeting first, then actions, then the app; Speech Model; shortcuts shown (#1409) 2026-11-05 11:54:18 -08:00
Kit Langton 1f8701bd78 Labs cards show their own settings; Voice Filter and Playground are dev-only (#1400)
* Labs: Voice Filter and Playground are dev-only; cards show their settings inline, no drawer

* Labs: regenerate mac-labs-link goldens; drop an unused test helper
2026-11-05 11:36:37 -08:00
Kit Langton 081b252f2c feat(annotate): add 15 s replay buffer Lab on macOS 27 (t1289) (#1342)
* feat(annotate): prototype 15 s rolling replay buffer on macOS 27 (t1289)

* fix(capture): gate macOS-only replay imports and helpers on non-macOS (t1289)
2026-11-04 19:05:17 -08:00
Kit Langton f294a437a5 fix(capture): typing over the Send to suggestions searches every session by title, folder, or id (t1326) (#1322) 2026-11-04 14:36:54 -08:00
Kit Langton 632bbe13ce Merge remote-tracking branch 'origin/dev' into annotate-words-undo
# ------------------------ >8 ------------------------
# Do not modify or remove the line above.
# Everything below it will be ignored.
#
# Conflicts:
#	src/rune_ui/capture_capsule/mod.rs
2026-11-04 12:10:02 -08:00
Kit Langton 3c412c50a3 Capture capsule: the Undo ring is the chip's border, drop the dial (t1296) 2026-11-04 12:04:40 -08:00
Kit Langton 11d6d87c03 Capture capsule: the pointer holds the Undo countdown and a rule's send (t1296) 2026-11-04 10:44:29 -08:00
Kit Langton a7cab72325 Annotate's capsule moves on the dictation HUD's springs, and only fades under Reduce Motion (t1293) (#1292) 2026-11-04 08:43:09 -08:00
Kit Langton c01a263e0f Capture capsule: Undo ring, plain words, Copied, empty search, lone undo keeps the list (t1296) 2026-11-04 08:22:11 -08:00
Kit Langton 3ebf6ab395 Annotate cues follow the chosen sound set (t1292) (#1275) 2026-11-04 06:46:37 -08:00
Luke Parker 3f8aa05af1 perf(windows): tray Hex makes no GPU device at rest; the HUD still shows fast (Rune c448a35a) (#1229)
* perf(windows): tray Hex makes no GPU device until something shows

Rune 4c039671. The capture capsule, picker, and ink keep their windows from
launch but make their GPU surfaces only to show (release_gpu_when_hidden).
gpu-draw.pending is written when Rune starts the launch's first GPU device
(rune::on_gpu) and cleared by its first frame, so a tray launch that never
draws leaves no marker.

* perf(windows): the first HUD after rest shows without waiting for the GPU

Rune c448a35a. A first surface on a new device now takes about 210 ms to its
first present instead of 630 to 850 ms (precompiled DX12 uber shader, one
adapter enumeration). The HUD keeps its window from launch, keeps its GPU
surface for 5 minutes after its last hide (release_gpu_after), then gives the
device back. When the dictation shortcut's modifiers go down, Rune makes the
device and the HUD's pipeline on a thread (warm_gpu_with), so the HUD that
follows shows in about 15 ms.

* docs(capture): the picker is prewarmed again; it holds no GPU surface while hidden
2026-11-04 01:49:43 -06:00
Luke Parker aba7efbbf3 Windows: make the capture picker at its first open (#1226)
On Windows a prewarmed overlay keeps no pixels while hidden, only its
surface. The picker's surface covers the whole primary display, so a tray
Hex held it in GPU memory from launch. The picker is now made the first
time it opens. It shows only before the capture it starts, so that
capture still knows its window. The idle memory case also reports
gpu.vram.
2026-11-03 21:30:04 -06:00
Ryan Vogel 5d8ff5bb6d feat(capture): record Linux screen captures at 60 fps (#1212)
Raise Linux Annotate screen recording from 30 fps to 60 fps across the
PipeWire/VA-API, PipeWire/x264, and X11/x264 paths (step L1):

- Raise `linux::media::FPS` from 30 to 60 and propagate it to both
  sampler loops, `ffmpeg` mux/encode args, and VA-API H.264 sequence
  and VUI timing (`time_scale = 2 * FPS = 120`), replacing the 30 fps
  cap (`time_scale = 60`).
- Offer `SPA_FORMAT_VIDEO_maxFramerate` 60/1 (range 1/1..60/1) and a
  60/1 default on `SPA_FORMAT_VIDEO_framerate` in PipeWire `enum_format`,
  keeping linear modifier negotiation unchanged for now.
- Compute VA-API H.264 `level_idc` from macroblocks and frame rate per
  Table A-1 (e.g. 4.2 at 1080p60, 5.1 at 1440p60, 5.2 at 4K60, replacing
  hardcoded level 5.1), allocate coded surfaces padded to 16 px
  macroblocks while setting SPS `frame_cropping_flag` and crop offsets
  (replacing `fit(..., 16)` truncation that squashed 1920x1080 to
  1920x1072 and float-truncating 3840x2160 to 2560x1438), and scale the
  `small` candidate surface on demand in `small_rgba` at
  `FRAME_SAMPLE_INTERVAL` instead of on every frame in `load`.
- Switch CPU `Writer::x264` from fixed `veryfast` to `superfast` up to
  1440p and `ultrafast` above, raise the pipe buffer to 1 MiB, and
  timestamp piped frames on the wall clock into `fps=60` inside `ffmpeg`
  instead of duplicating multi-MB raw frames across `pipe:0` when a
  screen is still or a 4K software frame runs past 16.7 ms.
- Borrow the X11 MIT-SHM segment directly into `Writer` instead of
  copying each frame into a `Vec<u8>`, and cache the XFixes cursor image
  across frames using `XFixesSelectCursorInput` (`XFixesCursorNotify`),
  pipelining `QueryPointer` with `shm_get_image` instead of fetching the
  cursor bitmap every frame.
- Update `docs/features/capture.md` (`capture.linux`) with the new
  behaviour, checks, and T4 verification limits.

Verified:
- `cargo clippy --locked -p scrawl-capture --target x86_64-unknown-linux-gnu -- -D warnings`
  and `python3 scripts/code-shape.py --check`.
- FFmpeg `trace_headers` bit-level parse of generated VA-API SPS/PPS/IDR
  headers at 1920x1080@60 (level 42, crop_bottom 4), 2560x1440@60
  (level 51), and 3840x2160@60 (level 52), all `time_scale = 120`.
- On `hex-linux-nvidia` (`g4dn.2xlarge`, Tesla T4, Ubuntu 24.04):
  273/275 `scrawl-capture` unit tests pass (the other 2 fail identically
  on clean `dev`); native Xorg 1080p60 recorded 598 frames in 9.97 s at
  60/1 (`1920x1080`, 597 grabs, 16.5% capture CPU); Xorg 1280x720 Area
  recorded 298 frames in 4.97 s at 60/1 (297 grabs); rootful Xwayland 4K
  recorded `2560x1440` at 60/1 (598 frames in 9.97 s from 282 raw 4K
  grabs); PipeWire 1080p60 SHM -> x264 recorded 598 frames in 9.97 s at
  60/1 (594 delivered). VA-API hardware encoding was not exercised live
  because Tesla T4 has no VA-API encoder.
2026-11-03 18:04:52 -05:00
Kit Langton fb1e06bc54 perf(annotate): show destinations while routing finishes (#1204)
* perf(annotate): show routes before remote preparation finishes

* test(annotate): refresh countdown snapshots and check remote identity
2026-11-03 13:56:25 -08:00
Kit Langton 9af39c0add Speed up Annotate routing lookups and remote checks (#1199) 2026-11-03 12:31:51 -08:00
DaxandHex issue pipeline c99d6b9f1d Fix #1179: annotate screen capture (VA-API DMA-BUF import) (#1182)
Automated fix from the Hex issue pipeline (run 1179-2026-11-03t19-19-51-347z).

Fixes #1179

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-11-03 14:38:52 -05:00
DaxandHex issue pipeline cfe30da71a Fix #1131: Annotate capture routing to OpenCode (Windows/WSL) (#1132)
Automated fix from the Hex issue pipeline (run 1131-2026-11-03t13-35-50-619z).

Fixes #1131

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-11-03 08:39:09 -05:00
Dax 04f7ed0a7f Split suppression.rs under the code-shape cap (#1080)
* Split the dictation shortcut state machine and tests out of suppression.rs

No behaviour change: DictationHotkey and its state move to
suppression/hotkey.rs and the module tests to suppression/tests.rs, bringing
suppression.rs under the size cap; its exception is removed.

* Format history.rs (cargo fmt --check fails on dev since #1070)

* Gate Again::finished like the Rune UI tests that use it (macOS test build fails on dev since #1070)

* Keep recovery from counting as neutral evidence; fix the Hold-mode cancel test's Escape flags

Stale-key recovery rebuilds only the held modifier flags (#1076): its
non-atomic scan must not vouch for keys whose delayed edges may still
arrive. The Hold-mode case of the cancel-shortcut test sent Escape without
the held Option, which reads as a release.
2026-11-02 22:23:12 -05:00
DaxandHex issue pipeline 00ab1af8e5 Fix #1042: annotate tool voice trigger words / language support (#1044)
Automated fix from the Hex issue pipeline (run 1042-2026-11-02t16-42-11-215z).

Fixes #1042

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-11-02 11:46:28 -05:00
Kit Langton 4b4fce89e4 docs: record native hidden-overlay smoke results 2026-11-02 08:03:29 -08:00
Kit Langton a893b34310 fix(macos): use Rune hidden-panel ordering for Spaces 2026-11-02 08:03:29 -08:00
DaxandHex issue pipeline 687fbb36ba Fix #999: annotate pen overlay on fractional scaling (#1001)
Automated fix from the Hex issue pipeline (run 999-2026-11-02t13-09-43-508z).

Fixes #999

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-11-02 08:15:33 -05:00
DaxandHex issue pipeline 859137b681 Fix #853: Annotate screen capture (video encoding) (#854)
Automated fix from the Hex issue pipeline (run 853-2026-11-01t16-47-52-189z).

Fixes #853

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-11-01 11:53:38 -05:00
Dax 3a2304386d Remove dictation cancel: Escape ends nothing (#786) (#795)
* Remove dictation cancel: Escape ends nothing (#786)

* Format and refresh the sound sets and commands goldens

* Release Escape before finishing in the held-capture test
2026-11-01 06:16:03 -05:00
Kit Langton 008fec4531 Scrawl sends one contact sheet, not the keyframes (#754) 2026-10-31 21:58:10 -07:00
Kit Langton af5a419d97 fix(macos): clear stale permission rows and offer one-click recovery (#715)
* fix(macos): clear stale permission rows and offer one-click recovery

* test(rune_ui): add Linux goldens for stale permission rows

* refactor(rune_ui): keep rune_ui.rs under code-shape ceiling
2026-10-31 20:53:36 -07:00
Dax 4661987182 Annotate on X11: record the screen and areas without a ScreenCast portal (#690) (#694)
On an X11 session (DISPLAY set, WAYLAND_DISPLAY empty; never XWayland) Screen and Area are read from the X server: MIT-SHM GetImage each frame (plain GetImage without it), the pointer blended in from XFixes, 30 fps to x264 through ffmpeg. The screen is the RandR monitor under the pointer; `hex annotate start --area X,Y,W,H` records an area. i3 and other desktops without a ScreenCast backend now get video. A Window still needs the portal's picker and fails saying to record the screen or an area, never widening to the screen. When X11 capture can't start (no display, unusual depth, no ffmpeg/x264) the capture fails at once with why in the capsule instead of writing one with no picture.

Checked in a headless Xvnc + i3 session on apollo with no ScreenCast portal: an 800x500 area recorded 154 frames over 5.1 s, the 1920x1080 screen 153, each with a keyframe and the cursor. Also fixes scrawl-capture's own tests, which no longer compiled (ScreenCaptureSession.failure).

Reported and diagnosed by @nefthy.
2026-10-31 18:19:36 -04:00
Kit Langton c430bc1b70 Update docs for transcription/, dictation/, desktop/, and microphone_activity/ 2026-10-31 15:13:56 -07:00
DaxandHex issue pipeline eef51da7e0 Fix #681: Annotate screen capture and dictation hotkey on GNOME Wayland (#683)
Automated fix from the Hex issue pipeline (run 681-2026-10-31t21-36-18-652z).

Fixes #681

Co-authored-by: Hex issue pipeline <noreply@anoma.ly>
2026-10-31 17:40:50 -04:00