mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-21 22:46:49 +08:00
* capture: stream audio to disk so long captures keep all of it A capture's timeline now writes 16 kHz mono PCM16 to its bundle's audio.wav as it records, with the leading offset and gaps as silence, so memory stays flat and the 20-minute drop is gone. An 8-hour guard bounds disk use only; past it the rest is one recorded gap. A finished capture is transcribed from the WAV in 30 s pieces, the reader hex transcribe already used (lifted to transcription/pieces.rs), and each piece's words move onto the capture clock. The Parakeet, Apple (path-based), and GGUF paths keep every word without holding the session. The pipeline never rewrites a streamed audio.wav from empty samples, an iPhone narration is read back where it must be re-based, and a cancelled capture closes and removes its file. * capture: say where a capture's audio stopped A streamed capture keeps all of its audio, but the 8-hour disk guard (or a failed audio write) still stops it early, and only the process log said so. The timeline now carries the stop point onto RecordedAudio, the bundle stores it as audioStoppedAtMs (serde default, so old bundles load unchanged), and three places say it plainly: summary.md, `hex capture show`, and `hex capture status`. The guard records its point at a 1 s test limit, and the summary line is tested at 12:34 and 8:00:00 so both clock shapes are covered. * capture: fill the replay's stop point from its bundle The 15 s replay builds Written too, on macOS only, so the new field broke the macOS clippy job while Linux never compiled that function. * capture: keep a recording out of recovery until it stops Review fixes for the streamed timeline. A capture now streams to `audio.wav.partial` and renames it to `audio.wav` in `Timeline::finish`, so a bundle with a movie and an in-progress audio file is never mistaken for a capture whose build never ran (recover's orphan rule, Scrawl.app's `scrawl recover`) and retention cleans a crashed one like any other bundle. Cancel and discard remove the partial; a failed publish reads it back into memory rather than pointing at a missing file. Also: a write failure's stop point is the last sample that reached the file, not the end of the failed chunk; a trim failure counts every written sample; the capture's words keep one running frame total instead of adding truncated per-piece millis; an iPhone narration's `since` drains the decoded samples instead of copying them and treats a missing file as no audio; the phone and replay paths carry the audio stop point into their bundles; and empty audio answers `source: "none"`, as `transcribe_with` does. Tests: a movie plus partial audio is never owed, retention trashes a crashed bundle's media, a write failure records its stop point, a missing streamed file shifts to no samples, and the replay shift moves the stop point with the clock. * capture: publish audio.wav only once its build journals the capture Follow-ups from review. A capture now streams to audio.wav.partial and keeps that name after it stops: the partial is renamed to audio.wav right after the build writes unfinished.json, so audio.wav never exists without a journal, and a stopped capture waiting in the finish queue no longer looks like an orphan to Scrawl.app's recover. A partial that cannot take the name is read back and written by the build, and its partial name removed; the replay removes the unshifted partial once the narration is read for shifting. Also: a periodic header flush error is now a write failure (the rest of the capture is a gap) rather than a log line; the midpoint of a failed write records where the file stopped; and the words test cuts pieces at odd sample counts and computes offsets from the running frame total, so per-piece millisecond drift would fail it. Tests: a streamed audio takes its final name only when the build settles it (and falls back when it cannot), a capture that has not been journaled is never owed a rebuild, a mid-capture write failure stops where the file stopped, and the word offsets are exact at odd piece lengths. * capture: consume a streamed recording's partial when it is read Follow-ups from review. `RecordedAudio::since` and `into_memory` now remove the partial once its samples are read: an iPhone narration (capture.rs's `host_narration` path) used to leave the whole streamed file in the finished bundle. The replay relies on the same path instead of removing it by hand, and the light zip leaves `audio.wav.partial` out defensively. A read that fails is an error to `into_memory` (so a build keeps the only copy) while `since` still treats a missing file as no audio for an iPhone. A failed publish now reads the audio back only when it produced the samples the recording had; a failed read keeps the partial and removes any half-copied `audio.wav`. `publish_audio` takes `&mut self` and points the recording at the published file. A journaled bundle whose `audio.wav` is missing adopts `audio.wav.partial` before rebuilding. Docs say the guarantee is a streamed capture's, and that a crash while recording or queued is retention's job, not recovery's. Tests: reading a streamed recording consumes the partial; a finished iPhone bundle keeps none; a streamed audio is published only when its build settles it, including the read-back and unreadable-partial cases; a journaled capture adopts its partial. * capture: keep a failed audio publish recoverable Follow-ups from review. The publish fallback read the partial with `into_memory`, which deletes it, and then ignored whether its own write landed: a failed write left the audio nowhere. It now reads through `read_streamed` (no deletion), writes `audio.wav` and checks the result, and removes the partial only once that write succeeded. A failed read or write removes any half `audio.wav` and fails the build, which is the pipeline's existing failed path: `recover::finished` runs only for an `Ok` build, so `unfinished.json` stays and the next launch's recovery adopts the partial instead of the capture quietly finishing without its audio. `publish_audio` also treats a landed copy as success even when deleting the partial fails, logging it rather than reporting failure and letting the fallback overwrite a good `audio.wav`. Tests: a blocked target keeps the partial and fails the build; a fallback write that lands consumes the partial and writes `audio.wav`; an unreadable partial is kept.
401 lines
13 KiB
Rust
401 lines
13 KiB
Rust
use serde_json::Value;
|
|
|
|
use super::{
|
|
CONTROL_VERSION, ControlReply, ControlResponse, EXIT_NO_SPEECH, EXIT_NOT_RUNNING, EXIT_REFUSED,
|
|
Output, fail,
|
|
};
|
|
use crate::control::{ModelInfo, StatusReport, settings::SettingValue};
|
|
use crate::dictation::vocabulary::VocabularyEntry;
|
|
use scrawl_capture::model::format_length;
|
|
|
|
/// Prints `reply` and returns its exit code.
|
|
pub(crate) fn print(reply: ControlReply, output: Output) -> u8 {
|
|
let code = exit_code(&reply);
|
|
if output.json {
|
|
let response = ControlResponse {
|
|
version: CONTROL_VERSION,
|
|
reply,
|
|
};
|
|
match serde_json::to_string_pretty(&response) {
|
|
Ok(json) => println!("{json}"),
|
|
Err(error) => return fail(&error.to_string()),
|
|
}
|
|
return code;
|
|
}
|
|
match reply {
|
|
ControlReply::Done { message, .. } => println!("{message}"),
|
|
ControlReply::Status(report) => print!("{}", human_status(&report)),
|
|
ControlReply::Settings { settings } => print!("{}", human_settings(&settings)),
|
|
ControlReply::Models { models } => print!("{}", human_models(&models)),
|
|
ControlReply::Vocabulary { entries } => print!("{}", human_vocabulary(&entries)),
|
|
ControlReply::ScrawlCapture(report) => print!("{}", human_scrawl_capture(&report)),
|
|
ControlReply::ScrawlCaptures { captures } => print!("{}", human_captures(&captures)),
|
|
ControlReply::ScrawlCaptureDetail(detail) => print!("{}", human_capture_detail(&detail)),
|
|
ControlReply::ScrawlCaptureRoute { explanation } => print!("{explanation}"),
|
|
ControlReply::Transcript { text } => println!("{text}"),
|
|
ControlReply::Error { message, .. } => eprintln!("hex: {message}"),
|
|
}
|
|
code
|
|
}
|
|
|
|
pub(super) fn human_scrawl_capture(report: &crate::control::ScrawlCaptureReport) -> String {
|
|
use crate::control::ScrawlCaptureState;
|
|
let mut out = String::new();
|
|
let bundle = |bundle: &Option<std::path::PathBuf>| {
|
|
bundle
|
|
.as_ref()
|
|
.map_or_else(String::new, |bundle| bundle.display().to_string())
|
|
};
|
|
match (report.state, report.changed) {
|
|
(ScrawlCaptureState::Recording, true) => {
|
|
out.push_str(&format!("Capturing into {}\n", bundle(&report.bundle)));
|
|
}
|
|
(ScrawlCaptureState::Recording, false) => out.push_str(&format!(
|
|
"Capturing into {} ({} ms)\n",
|
|
bundle(&report.bundle),
|
|
report.elapsed_ms.unwrap_or_default()
|
|
)),
|
|
(ScrawlCaptureState::Idle, true) => {
|
|
out.push_str(&format!("Ended the capture {}\n", bundle(&report.bundle)));
|
|
}
|
|
(ScrawlCaptureState::Idle, false) => out.push_str("No capture is recording\n"),
|
|
}
|
|
for finishing in &report.finishing {
|
|
out.push_str(&format!("Writing {}\n", finishing.display()));
|
|
}
|
|
if let Some(last) = &report.last {
|
|
let words = last
|
|
.words
|
|
.map_or_else(String::new, |words| format!(", {words} words"));
|
|
let gaps = last
|
|
.gaps
|
|
.filter(|gaps| *gaps > 0)
|
|
.map_or_else(String::new, |gaps| format!(", {gaps} audio gaps"));
|
|
let missing = last
|
|
.missing_transcript
|
|
.as_ref()
|
|
.map_or_else(String::new, |why| format!(", no transcript ({why})"));
|
|
let stopped = last.audio_stopped_at_ms.map_or_else(String::new, |at| {
|
|
format!(", audio stopped at {}", format_length(at))
|
|
});
|
|
out.push_str(&format!(
|
|
"Last: {} {}{words}{gaps}{stopped}{missing}\n",
|
|
last.stage,
|
|
last.bundle.display()
|
|
));
|
|
}
|
|
out
|
|
}
|
|
|
|
pub(super) fn human_captures(captures: &[crate::control::ScrawlCaptureSummary]) -> String {
|
|
if captures.is_empty() {
|
|
return "No captures\n".into();
|
|
}
|
|
captures
|
|
.iter()
|
|
.map(|capture| format!("{}\n", capture_line(capture)))
|
|
.collect()
|
|
}
|
|
|
|
/// `1792933491953.scrawl 2026-10-26 14:02 0:04 window · Brave 12 words → Hex`
|
|
pub(super) fn capture_line(capture: &crate::control::ScrawlCaptureSummary) -> String {
|
|
let seconds = capture.duration_ms / 1_000;
|
|
let mut line = format!(
|
|
"{} {} {}:{:02} {}",
|
|
capture.id,
|
|
local_minute(capture.created_at_ms),
|
|
seconds / 60,
|
|
seconds % 60,
|
|
capture.mode
|
|
);
|
|
if let Some(app) = &capture.app {
|
|
line.push_str(&format!(" · {app}"));
|
|
}
|
|
line.push_str(&format!(" {} words", capture.words));
|
|
if let Some(sent) = &capture.sent_to {
|
|
line.push_str(&format!(" → {}", sent.title));
|
|
}
|
|
if capture.queued {
|
|
line.push_str(" (queued)");
|
|
}
|
|
line
|
|
}
|
|
|
|
pub(super) fn human_capture_detail(detail: &crate::control::ScrawlCaptureDetail) -> String {
|
|
let mut out = format!(
|
|
"{}\n{}\n",
|
|
capture_line(&detail.summary),
|
|
detail.summary.bundle.display()
|
|
);
|
|
if let Some(title) = &detail.window_title {
|
|
out.push_str(&format!("Window: {title}\n"));
|
|
}
|
|
if let Some(sent) = &detail.summary.sent_to {
|
|
out.push_str(&format!("Sent to: {} ({})\n", sent.title, sent.session));
|
|
}
|
|
out.push_str(&format!(
|
|
"{} marks, {} keyframes\n",
|
|
detail.marks, detail.keyframes
|
|
));
|
|
for (label, path) in [
|
|
("Video", &detail.video),
|
|
("Audio", &detail.audio),
|
|
("Contact sheet", &detail.contact_sheet),
|
|
] {
|
|
if let Some(path) = path {
|
|
out.push_str(&format!("{label}: {}\n", path.display()));
|
|
}
|
|
}
|
|
if let Some(note) = &detail.note {
|
|
out.push_str(&format!("\nNote: {note}\n"));
|
|
}
|
|
if let Some(at) = detail.audio_stopped_at_ms {
|
|
out.push_str(&format!(
|
|
"\nAudio stopped at {}; the rest of this capture has no audio.\n",
|
|
format_length(at)
|
|
));
|
|
}
|
|
match (&detail.missing_transcript, detail.transcript.trim()) {
|
|
(Some(why), _) => out.push_str(&format!("\nNo transcript: {why}\n")),
|
|
(None, "") => out.push_str("\nNo words\n"),
|
|
(None, text) => out.push_str(&format!("\n{text}\n")),
|
|
}
|
|
out
|
|
}
|
|
|
|
/// `2026-10-26 14:02` in local time where Hex knows it.
|
|
pub(super) fn local_minute(ms: u64) -> String {
|
|
let iso = crate::meeting::query::iso_local(ms);
|
|
iso.get(..16)
|
|
.map_or(iso.clone(), |minute| minute.replace('T', " "))
|
|
}
|
|
|
|
pub(super) fn exit_code(reply: &ControlReply) -> u8 {
|
|
match reply {
|
|
ControlReply::Error { code, .. } if code == "not-running" => EXIT_NOT_RUNNING,
|
|
ControlReply::Error { code, .. } if code == "no-speech" => EXIT_NO_SPEECH,
|
|
ControlReply::Error { .. } => EXIT_REFUSED,
|
|
_ => 0,
|
|
}
|
|
}
|
|
|
|
pub(super) fn human_value(value: &Value) -> String {
|
|
match value {
|
|
Value::String(text) => text.clone(),
|
|
Value::Null => "none".into(),
|
|
other => other.to_string(),
|
|
}
|
|
}
|
|
|
|
pub(super) fn human_settings(settings: &[SettingValue]) -> String {
|
|
if let [setting] = settings {
|
|
return format!("{}\n", human_value(&setting.value));
|
|
}
|
|
let width = settings
|
|
.iter()
|
|
.map(|setting| setting.key.len())
|
|
.max()
|
|
.unwrap_or(0);
|
|
settings
|
|
.iter()
|
|
.map(|setting| {
|
|
let mut value = human_value(&setting.value);
|
|
if value.len() > 60 {
|
|
value = format!("{}…", value.chars().take(59).collect::<String>());
|
|
}
|
|
format!(
|
|
"{:width$} {value}{}\n",
|
|
setting.key,
|
|
if !setting.available {
|
|
" (not on this OS)"
|
|
} else if setting.writable {
|
|
""
|
|
} else {
|
|
" (read-only)"
|
|
}
|
|
)
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
pub(super) fn human_models(models: &[ModelInfo]) -> String {
|
|
let width = models.iter().map(|model| model.id.len()).max().unwrap_or(0);
|
|
let name_width = models
|
|
.iter()
|
|
.map(|model| model.name.len())
|
|
.max()
|
|
.unwrap_or(0);
|
|
models
|
|
.iter()
|
|
.map(|model| {
|
|
let languages = if model.languages.len() > 6 {
|
|
format!("{} languages", model.languages.len())
|
|
} else {
|
|
model.languages.join(", ")
|
|
};
|
|
format!(
|
|
"{} {:width$} {:name_width$} {:>8} {:13} {languages}{}\n",
|
|
if model.selected { "*" } else { " " },
|
|
model.id,
|
|
model.name,
|
|
model.size,
|
|
if model.installed {
|
|
"installed"
|
|
} else {
|
|
"not installed"
|
|
},
|
|
if model.source.is_empty() {
|
|
String::new()
|
|
} else {
|
|
format!(" ({})", model.source)
|
|
},
|
|
)
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
pub(super) fn human_vocabulary(entries: &[VocabularyEntry]) -> String {
|
|
if entries.is_empty() {
|
|
return "The vocabulary is empty; `hex vocabulary add <term>` adds one\n".into();
|
|
}
|
|
entries
|
|
.iter()
|
|
.map(|entry| match entry.sounds_like.as_slice() {
|
|
[] => format!("{}\n", entry.term),
|
|
spellings => format!("{} (sounds like {})\n", entry.term, spellings.join(", ")),
|
|
})
|
|
.collect()
|
|
}
|
|
|
|
pub(super) fn duration(seconds: u64) -> String {
|
|
if seconds >= 3600 {
|
|
format!(
|
|
"{}:{:02}:{:02}",
|
|
seconds / 3600,
|
|
seconds / 60 % 60,
|
|
seconds % 60
|
|
)
|
|
} else {
|
|
format!("{}:{:02}", seconds / 60, seconds % 60)
|
|
}
|
|
}
|
|
|
|
pub(super) fn on(enabled: bool) -> &'static str {
|
|
if enabled { "on" } else { "off" }
|
|
}
|
|
|
|
pub(super) fn human_status(report: &StatusReport) -> String {
|
|
let memory = report
|
|
.memory_footprint_bytes
|
|
.map(|bytes| format!(", {} MB", bytes / 1_000_000))
|
|
.unwrap_or_default();
|
|
let capture = match report.capture.recording {
|
|
Some(kind) => format!(
|
|
"recording {}",
|
|
serde_json::to_value(kind)
|
|
.ok()
|
|
.and_then(|value| value.as_str().map(str::to_owned))
|
|
.unwrap_or_default()
|
|
),
|
|
None => serde_json::to_value(report.capture.state)
|
|
.ok()
|
|
.and_then(|value| value.as_str().map(str::to_owned))
|
|
.unwrap_or_default(),
|
|
};
|
|
let pending = match report.capture.pending_jobs {
|
|
0 => String::new(),
|
|
jobs => format!(" ({jobs} pending)"),
|
|
};
|
|
let meeting = match (&report.meeting.title, report.meeting.elapsed_seconds) {
|
|
(Some(title), Some(elapsed)) => {
|
|
format!(
|
|
"{} “{title}” for {}",
|
|
report.meeting.state,
|
|
duration(elapsed)
|
|
)
|
|
}
|
|
_ if report.features.meetings => report.meeting.state.clone(),
|
|
_ => "off".into(),
|
|
};
|
|
let model = &report.model;
|
|
let mut model_line = format!(
|
|
"{} ({}){}",
|
|
model.name,
|
|
model.language,
|
|
if model.installed {
|
|
""
|
|
} else {
|
|
", not installed"
|
|
}
|
|
);
|
|
if let Some(fallback) = &model.fallback {
|
|
use crate::transcription::custom_models::FallbackReason;
|
|
let reason = match fallback.reason {
|
|
FallbackReason::Missing => "it is missing",
|
|
FallbackReason::Changed => "it changed since it was added",
|
|
FallbackReason::LabsOff => "its Labs toggle is off",
|
|
};
|
|
model_line.push_str(&format!(
|
|
"; using {} ({}) because {reason}",
|
|
fallback.name, fallback.language
|
|
));
|
|
}
|
|
if let Some(preparing) = &model.preparing {
|
|
model_line.push_str(&format!(
|
|
"; preparing {preparing} ({} MB)",
|
|
model.downloaded_bytes / 1_000_000
|
|
));
|
|
}
|
|
if let Some(error) = &model.error {
|
|
model_line.push_str(&format!("; error: {error}"));
|
|
}
|
|
let features = &report.features;
|
|
let permissions = &report.permissions;
|
|
let permission_line = if permissions.ready {
|
|
"ready".to_string()
|
|
} else {
|
|
format!(
|
|
"microphone {}, accessibility {}, model {}",
|
|
permissions.microphone,
|
|
permissions.accessibility,
|
|
if permissions.transcription_model {
|
|
"ready"
|
|
} else {
|
|
"missing"
|
|
}
|
|
)
|
|
};
|
|
let window = match (&report.window.pane, report.window.open) {
|
|
(Some(pane), true) => format!("open ({pane})"),
|
|
(None, true) => "open".into(),
|
|
_ => "closed".into(),
|
|
};
|
|
format!(
|
|
"Hex {} (pid {}{memory})\n\
|
|
Listener {}\n\
|
|
Capture {capture}{pending}\n\
|
|
Meeting {meeting}\n\
|
|
Model {model_line}\n\
|
|
Features meetings {}, release mic while idle {}, history {}\n\
|
|
Permissions {permission_line}\n\
|
|
Window {window}\n\
|
|
Update {}\n\
|
|
Labs {}\n",
|
|
report.app_version,
|
|
report.pid,
|
|
if report.listener.running {
|
|
format!("running: {}", report.listener.status)
|
|
} else {
|
|
format!("waiting: {}", report.listener.status)
|
|
},
|
|
on(features.meetings),
|
|
on(features.release_microphone_while_idle),
|
|
features.history_retention,
|
|
report.update,
|
|
match report.labs.as_slice() {
|
|
[] => "none".to_string(),
|
|
names => names.join(", "),
|
|
},
|
|
)
|
|
}
|