mirror of
https://github.com/anomalyco/hex.git
synced 2026-11-21 13:51:48 +08:00
Hex signature sounds: one voice and motif for start, stop, done, cancel, error, onboarding (#261)
This commit is contained in:
@@ -247,6 +247,8 @@ the relaunch guard and resume marker, grants waiting for a restart
|
||||
`onboarding_downloads_the_model_through_the_service`), and every macOS step
|
||||
is a scene (`mac-onboarding`, `-setup`, `-progress`, `-denied`, `-restart`,
|
||||
`-download-failed`, `-try`, `-listening`, `-landed`, `-reopen`, `-done`).
|
||||
Reaching the last step in a live tour plays the onboarding success sound
|
||||
(`remember_step`; fixtures stay silent), checked by reading the code.
|
||||
The default model comes from `desktop_i18n::preferred_language` (macOS:
|
||||
`NSLocale.preferredLanguages`) through `recommended_selection`; existing
|
||||
settings keep their saved model.
|
||||
@@ -1039,11 +1041,13 @@ Capture // hex app and hex dictate
|
||||
│ tail is still captured; a muted output unmutes first // dictate.stop-tone
|
||||
│ └── Clip turns out near-silent -> Stop sound already played, nothing pasted
|
||||
├── Brief discarded tap -> Start sound, but no stop sound or transcription
|
||||
├── Dictation or Voice Action pasted -> Done sound; failures stay silent
|
||||
└── Cancel active capture -> Cancel sound; idle Escape stays silent
|
||||
```
|
||||
|
||||
The shared [feedback player](../../src/feedback.rs) uses the bundled recording
|
||||
sounds on both platforms; only macOS wake/sleep/error tones use `afplay`.
|
||||
The shared [feedback player](../../src/feedback.rs) uses Hex's bundled sounds
|
||||
([Feedback](dictation.md#feedback)) on every platform; only macOS wake/sleep
|
||||
use `afplay`.
|
||||
Decoding happens before the Linux capture loop; the output device opens for
|
||||
the first tone and is released five minutes after the last one, as described
|
||||
in [macOS feedback](dictation.md#feedback). Playback admission is bounded and
|
||||
@@ -1056,7 +1060,7 @@ requests it when the capture's audio reaches 300 ms past the press, and not
|
||||
during a release tail. Until 2026-10-22 the service muted at the press. This
|
||||
ordering is checked by reading the loop, not by a test or a muted device.
|
||||
|
||||
Checks: `bundled_recording_sounds_decode_without_an_audio_device` and
|
||||
Checks: `bundled_sounds_decode_without_an_audio_device` and
|
||||
`feedback_admission_never_waits_for_playback` in the player, plus
|
||||
`files_that_predate_a_setting_keep_the_beta_defaults` and
|
||||
`a_muted_beta_keeps_the_remembered_volume` in
|
||||
|
||||
@@ -737,6 +737,9 @@ Record -> Release -> Transcribing/processing/paste -> Finished // dictate.feedba
|
||||
-> Voice Action takeover -> Retracted; Voice Action rings at its own hold
|
||||
-> Release microphone while idle -> Output warms on press; tone once
|
||||
the microphone is open and the hold counts
|
||||
Dictation or Voice Action reply pasted -> Done tone // dictate.done-tone
|
||||
-> Empty, failed, cancelled, Paste Last, programmatic -> No done tone
|
||||
Failure (macOS) -> Error tone; wake/sleep keep macOS Pop/Tink
|
||||
Tone requested -> Playback worker opens the default output, then plays
|
||||
-> Output stays open five minutes past the last tone, then releases
|
||||
-> Released output holds no macOS idle-sleep assertion
|
||||
@@ -747,6 +750,23 @@ Record -> Release -> Transcribing/processing/paste -> Finished // dictate.feedba
|
||||
-> Loader keeps running; later tones play once the output opens
|
||||
```
|
||||
|
||||
The sounds are Hex's own (2026-10-27), one kalimba-like FM voice playing one
|
||||
motif, D-B-A-D (a sixth, since "hex" is six): start D5 to B5 (116 ms), stop B5
|
||||
to A5 (230 ms), done D6 over A5 (330 ms), cancel B5 down to D5 (225 ms), error
|
||||
D4 then a minor D4+Bb4 (280 ms), and onboarding success, the whole motif
|
||||
landing on a D sixth chord (1.1 s), played when the first-run tour reaches
|
||||
its last step. They are 48 kHz mono WAVs in `resources/audio/`, each starting
|
||||
on its first sample, made by
|
||||
[`scripts/sounds/synth.py`](../../scripts/sounds/synth.py)
|
||||
(`uv run scripts/sounds/synth.py` rewrites them). Their levels match the
|
||||
sounds they replaced at the same volume: start like the old start clicks
|
||||
(-25.5 LUFS momentary), stop like the old swish, error like macOS Basso;
|
||||
cancel is far quieter than the old thud. Windows and Linux play start, stop,
|
||||
done, and cancel; their failures stay silent as before. Check:
|
||||
`bundled_sounds_are_short_and_sound_immediately` (each sound under its length
|
||||
budget, audible within 1 ms). The sounds were checked by waveform, spectrum
|
||||
and loudness only, never played aloud on a Mac.
|
||||
|
||||
Feedback admission is advisory on every platform: a cold audio stack can exceed
|
||||
the two-second preload timeout, so the recognition worker logs a warning and
|
||||
continues rather than stopping. The loader thread keeps running and registers
|
||||
|
||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -0,0 +1,155 @@
|
||||
# /// script
|
||||
# requires-python = ">=3.11"
|
||||
# dependencies = ["numpy", "scipy"]
|
||||
# ///
|
||||
"""Hex's feedback sounds, synthesized reproducibly.
|
||||
|
||||
Run: uv run scripts/sounds/synth.py (rewrites resources/audio/*.wav)
|
||||
|
||||
One voice, a kalimba-like FM tine, and one motif, D - B - A - D: up a major
|
||||
sixth, a step down, home ("hex" is six, so the sixth is Hex's interval).
|
||||
Start, stop and done play the motif in order across a dictation; cancel
|
||||
collapses the sixth; error turns it minor; onboarding success plays the whole
|
||||
motif and lands on a D sixth chord.
|
||||
|
||||
Every sound is 48 kHz mono 16-bit, starts on its first sample (no leading
|
||||
silence), begins and ends at zero with raised-cosine edges, and is normalized
|
||||
so its loudest 400 ms window (BS.1770 K-weighted, EBU momentary loudness)
|
||||
reads its target in TARGETS. The targets match the sounds they replaced, so a
|
||||
volume setting sounds as loud as it did: start -25.5 LUFS like the old start
|
||||
clicks, stop like the old swish, error like macOS Basso.
|
||||
"""
|
||||
from pathlib import Path
|
||||
|
||||
import numpy as np
|
||||
from scipy.io import wavfile
|
||||
from scipy.signal import butter, lfilter, sosfilt
|
||||
|
||||
SR = 48000
|
||||
CEILING = 10 ** (-1.5 / 20) # sample peak
|
||||
OUT = Path(__file__).resolve().parents[2] / "resources" / "audio"
|
||||
rng = np.random.default_rng(6) # fixed seed: same noise every run
|
||||
|
||||
# file name -> target LUFS (max momentary)
|
||||
TARGETS = {
|
||||
"dictationStart": -25.5,
|
||||
"dictationStop": -27.0,
|
||||
"dictationDone": -27.0,
|
||||
"dictationCancel": -26.5,
|
||||
"error": -26.5,
|
||||
"onboardingSuccess": -25.0,
|
||||
}
|
||||
|
||||
|
||||
def hz(note: str) -> float:
|
||||
names = {"C": -9, "C#": -8, "D": -7, "Eb": -6, "E": -5, "F": -4, "F#": -3,
|
||||
"G": -2, "G#": -1, "A": 0, "Bb": 1, "B": 2}
|
||||
name, octave = note[:-1], int(note[-1])
|
||||
return 440.0 * 2 ** ((names[name] + 12 * (octave - 4)) / 12)
|
||||
|
||||
|
||||
def t_axis(dur):
|
||||
return np.arange(int(dur * SR)) / SR
|
||||
|
||||
|
||||
def attack(n_samples, ms):
|
||||
a = np.ones(n_samples)
|
||||
k = min(n_samples, max(1, int(ms / 1000 * SR)))
|
||||
a[:k] = 0.5 - 0.5 * np.cos(np.pi * np.arange(k) / k)
|
||||
return a
|
||||
|
||||
|
||||
def mix(events, dur):
|
||||
"""Sum (time, signal) events into one buffer of `dur` seconds."""
|
||||
out = np.zeros(int(dur * SR))
|
||||
for at, sig in events:
|
||||
sig = sig.copy()
|
||||
f = min(len(sig) // 3, int(0.008 * SR)) # release every event: no truncation clicks
|
||||
sig[-f:] *= 0.5 + 0.5 * np.cos(np.pi * np.arange(f) / f)
|
||||
i = int(at * SR)
|
||||
n = min(len(sig), len(out) - i)
|
||||
out[i:i + n] += sig[:n]
|
||||
return out
|
||||
|
||||
|
||||
def band(x, lo, hi, order=2):
|
||||
return sosfilt(butter(order, [lo, hi], btype="band", fs=SR, output="sos"), x)
|
||||
|
||||
|
||||
def tine(freq, dur, tau, amp=1.0, index=2.2, index_tau=0.014, ratio=4.0, tick_amt=0.25):
|
||||
"""A kalimba-like FM tine (bright 1:4 attack that settles to a pure sine)
|
||||
with a tiny high noise tick on the onset."""
|
||||
t = t_axis(dur)
|
||||
mod = index * np.exp(-t / index_tau) * np.sin(2 * np.pi * freq * ratio * t)
|
||||
x = np.sin(2 * np.pi * freq * t + mod) * np.exp(-t / tau)
|
||||
if tick_amt:
|
||||
n = band(rng.standard_normal(len(t)), 5000, 14000) * np.exp(-t / 0.0007)
|
||||
x += tick_amt * n / (np.max(np.abs(n)) + 1e-12)
|
||||
return amp * x * attack(len(t), 1.5)
|
||||
|
||||
|
||||
def sounds():
|
||||
D4, Bb4, D5, Fs5, A5, B5, D6 = map(hz, ["D4", "Bb4", "D5", "F#5", "A5", "B5", "D6"])
|
||||
s = {}
|
||||
s["dictationStart"] = mix([(0, 0.75 * tine(D5, 0.05, 0.02)), (0.04, tine(B5, 0.076, 0.026))], 0.116)
|
||||
s["dictationStop"] = mix([(0, 0.8 * tine(B5, 0.07, 0.026)), (0.06, tine(A5, 0.17, 0.05))], 0.23)
|
||||
s["dictationDone"] = mix([(0, tine(D6, 0.32, 0.08)),
|
||||
(0.006, 0.45 * tine(A5, 0.32, 0.075, tick_amt=0))], 0.33)
|
||||
s["dictationCancel"] = mix([(0, 0.8 * tine(B5, 0.07, 0.025, index=1.4)),
|
||||
(0.055, tine(D5, 0.17, 0.045, index=1.2))], 0.225)
|
||||
s["error"] = mix([(0, 0.8 * tine(D4, 0.09, 0.03, index=1.0, tick_amt=0.15)),
|
||||
(0.09, tine(D4, 0.19, 0.05, index=1.0, tick_amt=0.15)),
|
||||
(0.09, 0.7 * tine(Bb4, 0.19, 0.05, index=1.0, tick_amt=0))], 0.28)
|
||||
ev = [(0.00, 0.7 * tine(D5, 0.2, 0.05)),
|
||||
(0.08, 0.8 * tine(B5, 0.2, 0.05)),
|
||||
(0.16, 0.75 * tine(A5, 0.2, 0.06))]
|
||||
land = 0.26
|
||||
ev += [(land, tine(D6, 0.85, 0.28))]
|
||||
for i, (f, a) in enumerate([(D5, 0.35), (Fs5, 0.3), (A5, 0.3), (B5, 0.32)]):
|
||||
ev.append((land + 0.012 * i, a * tine(f, 0.85, 0.3, index=1.2, tick_amt=0)))
|
||||
s["onboardingSuccess"] = mix(ev, land + 0.86)
|
||||
return s
|
||||
|
||||
|
||||
_K1 = ([1.53512485958697, -2.69169618940638, 1.19839281085285], [1.0, -1.69065929318241, 0.73248077421585])
|
||||
_K2 = ([1.0, -2.0, 1.0], [1.0, -1.99004745483398, 0.99007225036621])
|
||||
|
||||
|
||||
def max_momentary_lufs(x):
|
||||
pad = np.zeros(int(0.4 * SR))
|
||||
y = np.concatenate([pad, x, pad])
|
||||
y = lfilter(*_K2, lfilter(*_K1, y))
|
||||
w = int(0.4 * SR)
|
||||
c = np.concatenate([[0], np.cumsum(y ** 2)])
|
||||
ms = (c[w:] - c[:-w])[:: SR // 100] / w # 10 ms hop
|
||||
return -0.691 + 10 * np.log10(ms.max() + 1e-20)
|
||||
|
||||
|
||||
def finish(x, target):
|
||||
# trim the tail below -60 dB of peak (+10 ms), then fade the last 6 ms
|
||||
thr = np.max(np.abs(x)) * 10 ** (-60 / 20)
|
||||
last = np.where(np.abs(x) > thr)[0][-1]
|
||||
x = x[: min(len(x), last + int(0.010 * SR))]
|
||||
f = min(len(x) // 4, int(0.006 * SR))
|
||||
x[-f:] *= 0.5 + 0.5 * np.cos(np.pi * np.arange(f) / f)
|
||||
x[0] = 0.0
|
||||
x[-1] = 0.0
|
||||
x *= 10 ** ((target - max_momentary_lufs(x)) / 20)
|
||||
assert np.max(np.abs(x)) <= CEILING, "a sound would clip at its target"
|
||||
return x
|
||||
|
||||
|
||||
def main():
|
||||
for name, x in sounds().items():
|
||||
x = finish(x.astype(np.float64), TARGETS[name])
|
||||
pcm = np.round(np.clip(x, -1, 1) * 32767).astype(np.int16)
|
||||
wavfile.write(OUT / f"{name}.wav", SR, pcm)
|
||||
first = int(np.argmax(np.abs(pcm) > 0.01 * 32767))
|
||||
print(f"{name:18s} {len(pcm) / SR * 1000:5.0f} ms "
|
||||
f"{max_momentary_lufs(pcm / 32767):6.2f} LUFS "
|
||||
f"peak {20 * np.log10(np.max(np.abs(pcm)) / 32767):6.2f} dBFS "
|
||||
f"onset {first / SR * 1000:.2f} ms")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+147
-86
@@ -10,7 +10,7 @@ use std::time::{Duration, Instant};
|
||||
use color_eyre::eyre::{Result, WrapErr, eyre};
|
||||
use rodio::buffer::SamplesBuffer;
|
||||
use rodio::cpal::BufferSize;
|
||||
use rodio::{Decoder, DeviceSinkBuilder, MixerDeviceSink, Source};
|
||||
use rodio::{DeviceSinkBuilder, MixerDeviceSink, Source};
|
||||
|
||||
#[derive(Clone, Copy, Debug, Eq, PartialEq)]
|
||||
pub enum Tone {
|
||||
@@ -22,7 +22,11 @@ pub enum Tone {
|
||||
Error,
|
||||
DictationStart,
|
||||
DictationStop,
|
||||
/// A dictation or Voice Action reply was pasted.
|
||||
DictationDone,
|
||||
Cancel,
|
||||
/// The first-run tour reached its last step.
|
||||
OnboardingSuccess,
|
||||
/// A Scrawl capture started recording.
|
||||
#[cfg(hex_capture)]
|
||||
CaptureStart,
|
||||
@@ -162,40 +166,13 @@ pub fn preload() -> Result<()> {
|
||||
let (sender, receiver) = mpsc::sync_channel::<Request>(8);
|
||||
let (ready_sender, ready_receiver) = mpsc::sync_channel(1);
|
||||
thread::spawn(move || {
|
||||
let result = (|| -> Result<_> {
|
||||
let start = decode(include_bytes!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/resources/audio/startRecording.mp3"
|
||||
)))?;
|
||||
let stop = decode(include_bytes!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/resources/audio/stopRecording.mp3"
|
||||
)))?;
|
||||
let cancel = decode(include_bytes!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/resources/audio/cancel.mp3"
|
||||
)))?;
|
||||
#[cfg(hex_capture)]
|
||||
let capture = capture_sounds()?;
|
||||
Ok((
|
||||
start,
|
||||
stop,
|
||||
cancel,
|
||||
#[cfg(hex_capture)]
|
||||
capture,
|
||||
))
|
||||
})();
|
||||
#[cfg(hex_capture)]
|
||||
let Ok((start, stop, cancel, capture)) = result else {
|
||||
let _ = ready_sender.send(result.map(|_| ()));
|
||||
LOADER_STARTED.store(false, Ordering::Release);
|
||||
return;
|
||||
};
|
||||
#[cfg(not(hex_capture))]
|
||||
let Ok((start, stop, cancel)) = result else {
|
||||
let _ = ready_sender.send(result.map(|_| ()));
|
||||
LOADER_STARTED.store(false, Ordering::Release);
|
||||
return;
|
||||
let sounds = match Sounds::load() {
|
||||
Ok(sounds) => sounds,
|
||||
Err(error) => {
|
||||
let _ = ready_sender.send(Err(error));
|
||||
LOADER_STARTED.store(false, Ordering::Release);
|
||||
return;
|
||||
}
|
||||
};
|
||||
let mut output = FeedbackOutput::new();
|
||||
// The device opens for the first tone, not at startup, so an idle app
|
||||
@@ -242,28 +219,13 @@ pub fn preload() -> Result<()> {
|
||||
None => None,
|
||||
};
|
||||
if let Some(tone) = tone {
|
||||
let sound = match tone {
|
||||
Tone::DictationStart => &start,
|
||||
Tone::DictationStop => &stop,
|
||||
Tone::Cancel => &cancel,
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureStart => &capture[0],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureStop => &capture[1],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureSent => &capture[2],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureUndo => &capture[3],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureFailed => &capture[4],
|
||||
#[cfg(target_os = "macos")]
|
||||
Tone::Wake | Tone::Sleep | Tone::Error => continue,
|
||||
};
|
||||
output.play(sound.clone(), tone);
|
||||
if let Some(sound) = sounds.get(tone) {
|
||||
output.play(sound.clone(), tone);
|
||||
}
|
||||
} else if timed_out && start_tone.is_due(Instant::now()) {
|
||||
// Only an empty queue reaches here, so a retraction sent while
|
||||
// the device opened always wins over the tone.
|
||||
let (sound, retracted) = Retractable::new(start.clone());
|
||||
let (sound, retracted) = Retractable::new(sounds.start.clone());
|
||||
if output.play(sound, Tone::DictationStart) {
|
||||
start_tone.sounding(retracted);
|
||||
} else {
|
||||
@@ -283,11 +245,87 @@ pub fn play(tone: Tone) {
|
||||
}
|
||||
match tone {
|
||||
#[cfg(target_os = "macos")]
|
||||
Tone::Wake | Tone::Sleep | Tone::Error => play_system_sound(tone),
|
||||
Tone::Wake | Tone::Sleep => play_system_sound(tone),
|
||||
_ => enqueue(DICTATION_PLAYER.get(), Request::Play(tone)),
|
||||
}
|
||||
}
|
||||
|
||||
/// Hex's own sounds, one voice and motif made by `scripts/sounds/synth.py`,
|
||||
/// which regenerates these files.
|
||||
macro_rules! bundled_sound {
|
||||
($name:literal) => {
|
||||
include_bytes!(concat!(
|
||||
env!("CARGO_MANIFEST_DIR"),
|
||||
"/resources/audio/",
|
||||
$name,
|
||||
".wav"
|
||||
))
|
||||
.as_slice()
|
||||
};
|
||||
}
|
||||
|
||||
const START_SOUND: &[u8] = bundled_sound!("dictationStart");
|
||||
const STOP_SOUND: &[u8] = bundled_sound!("dictationStop");
|
||||
const DONE_SOUND: &[u8] = bundled_sound!("dictationDone");
|
||||
const CANCEL_SOUND: &[u8] = bundled_sound!("dictationCancel");
|
||||
#[cfg(any(target_os = "macos", test))]
|
||||
const ERROR_SOUND: &[u8] = bundled_sound!("error");
|
||||
const ONBOARDING_SUCCESS_SOUND: &[u8] = bundled_sound!("onboardingSuccess");
|
||||
|
||||
/// The decoded sounds the playback worker plays. Wake and sleep stay the
|
||||
/// quieter macOS system sounds.
|
||||
struct Sounds {
|
||||
start: SamplesBuffer,
|
||||
stop: SamplesBuffer,
|
||||
done: SamplesBuffer,
|
||||
cancel: SamplesBuffer,
|
||||
#[cfg(target_os = "macos")]
|
||||
error: SamplesBuffer,
|
||||
onboarding_success: SamplesBuffer,
|
||||
#[cfg(hex_capture)]
|
||||
capture: [SamplesBuffer; 5],
|
||||
}
|
||||
|
||||
impl Sounds {
|
||||
fn load() -> Result<Self> {
|
||||
Ok(Self {
|
||||
start: decode_wav(START_SOUND)?,
|
||||
stop: decode_wav(STOP_SOUND)?,
|
||||
done: decode_wav(DONE_SOUND)?,
|
||||
cancel: decode_wav(CANCEL_SOUND)?,
|
||||
#[cfg(target_os = "macos")]
|
||||
error: decode_wav(ERROR_SOUND)?,
|
||||
onboarding_success: decode_wav(ONBOARDING_SUCCESS_SOUND)?,
|
||||
#[cfg(hex_capture)]
|
||||
capture: capture_sounds()?,
|
||||
})
|
||||
}
|
||||
|
||||
fn get(&self, tone: Tone) -> Option<&SamplesBuffer> {
|
||||
Some(match tone {
|
||||
Tone::DictationStart => &self.start,
|
||||
Tone::DictationStop => &self.stop,
|
||||
Tone::DictationDone => &self.done,
|
||||
Tone::Cancel => &self.cancel,
|
||||
Tone::OnboardingSuccess => &self.onboarding_success,
|
||||
#[cfg(target_os = "macos")]
|
||||
Tone::Error => &self.error,
|
||||
#[cfg(target_os = "macos")]
|
||||
Tone::Wake | Tone::Sleep => return None,
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureStart => &self.capture[0],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureStop => &self.capture[1],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureSent => &self.capture[2],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureUndo => &self.capture[3],
|
||||
#[cfg(hex_capture)]
|
||||
Tone::CaptureFailed => &self.capture[4],
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
/// The capture tones, in [`Tone`]'s order: start, stop, sent, undo, failed.
|
||||
#[cfg(hex_capture)]
|
||||
fn capture_sounds() -> Result<[SamplesBuffer; 5]> {
|
||||
@@ -504,7 +542,6 @@ fn play_system_sound(tone: Tone) {
|
||||
let sound = match tone {
|
||||
Tone::Wake => "Pop",
|
||||
Tone::Sleep => "Tink",
|
||||
Tone::Error => "Basso",
|
||||
_ => return,
|
||||
};
|
||||
let volume = volume().to_string();
|
||||
@@ -525,8 +562,27 @@ fn play_system_sound(tone: Tone) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Decodes one of Hex's bundled 16-bit PCM sounds.
|
||||
fn decode_wav(bytes: &'static [u8]) -> Result<SamplesBuffer> {
|
||||
let reader =
|
||||
hound::WavReader::new(Cursor::new(bytes)).wrap_err("could not decode feedback audio")?;
|
||||
let spec = reader.spec();
|
||||
let channels = std::num::NonZero::new(spec.channels)
|
||||
.ok_or_else(|| eyre!("feedback audio has no channels"))?;
|
||||
let sample_rate = std::num::NonZero::new(spec.sample_rate)
|
||||
.ok_or_else(|| eyre!("feedback audio has no sample rate"))?;
|
||||
let samples = reader
|
||||
.into_samples::<i16>()
|
||||
.map(|sample| sample.map(|sample| f32::from(sample) / f32::from(i16::MAX)))
|
||||
.collect::<Result<Vec<_>, _>>()
|
||||
.wrap_err("could not decode feedback audio")?;
|
||||
Ok(SamplesBuffer::new(channels, sample_rate, samples))
|
||||
}
|
||||
|
||||
#[cfg(hex_capture)]
|
||||
fn decode(bytes: &'static [u8]) -> Result<SamplesBuffer> {
|
||||
let decoder = Decoder::new(Cursor::new(bytes)).wrap_err("could not decode feedback audio")?;
|
||||
let decoder =
|
||||
rodio::Decoder::new(Cursor::new(bytes)).wrap_err("could not decode feedback audio")?;
|
||||
let channels = decoder.channels();
|
||||
let sample_rate = decoder.sample_rate();
|
||||
Ok(SamplesBuffer::new(
|
||||
@@ -546,35 +602,41 @@ mod tests {
|
||||
}
|
||||
}
|
||||
|
||||
fn leading_silence_ms(bytes: &'static [u8]) -> u128 {
|
||||
let sound = decode(bytes).unwrap();
|
||||
/// Every bundled sound with the longest it may run.
|
||||
const BUNDLED: [(&str, &[u8], Duration); 6] = [
|
||||
("start", START_SOUND, Duration::from_millis(130)),
|
||||
("stop", STOP_SOUND, Duration::from_millis(250)),
|
||||
("done", DONE_SOUND, Duration::from_millis(350)),
|
||||
("cancel", CANCEL_SOUND, Duration::from_millis(250)),
|
||||
("error", ERROR_SOUND, Duration::from_millis(300)),
|
||||
(
|
||||
"onboarding success",
|
||||
ONBOARDING_SUCCESS_SOUND,
|
||||
Duration::from_millis(1_200),
|
||||
),
|
||||
];
|
||||
|
||||
fn leading_silence(sound: SamplesBuffer) -> Duration {
|
||||
let channels = usize::from(sound.channels().get());
|
||||
let rate = sound.sample_rate().get() as usize;
|
||||
let rate = sound.sample_rate().get();
|
||||
let samples = sound.collect::<Vec<f32>>();
|
||||
let first = samples
|
||||
.iter()
|
||||
.position(|sample| sample.abs() > 0.01)
|
||||
.unwrap_or(samples.len());
|
||||
(first / channels * 1_000 / rate) as u128
|
||||
Duration::from_secs_f64((first / channels) as f64 / f64::from(rate))
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn feedback_tones_sound_immediately() {
|
||||
for (name, bytes) in [
|
||||
(
|
||||
"start",
|
||||
include_bytes!("../resources/audio/startRecording.mp3").as_slice(),
|
||||
),
|
||||
(
|
||||
"stop",
|
||||
include_bytes!("../resources/audio/stopRecording.mp3").as_slice(),
|
||||
),
|
||||
] {
|
||||
let silence = leading_silence_ms(bytes);
|
||||
eprintln!("{name} tone leading silence: {silence} ms");
|
||||
fn bundled_sounds_are_short_and_sound_immediately() {
|
||||
for (name, bytes, budget) in BUNDLED {
|
||||
let sound = decode_wav(bytes).unwrap();
|
||||
let length = sound.total_duration().unwrap();
|
||||
assert!(length <= budget, "{name} runs {length:?}, over {budget:?}");
|
||||
let silence = leading_silence(sound);
|
||||
assert!(
|
||||
silence < 40,
|
||||
"{name} tone starts after {silence} ms of silence"
|
||||
silence <= Duration::from_millis(1),
|
||||
"{name} starts after {silence:?} of silence"
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -836,7 +898,7 @@ mod tests {
|
||||
.and_then(|device| rodio::DeviceTrait::description(&device).ok())
|
||||
.map(|description| description.name().to_owned());
|
||||
eprintln!("default output: {device:?}");
|
||||
let start = decode(include_bytes!("../resources/audio/startRecording.mp3")).unwrap();
|
||||
let start = decode_wav(START_SOUND).unwrap();
|
||||
let open = || {
|
||||
if legacy {
|
||||
let mut sink = DeviceSinkBuilder::open_default_sink().unwrap();
|
||||
@@ -892,16 +954,15 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bundled_recording_sounds_decode_without_an_audio_device() {
|
||||
for bytes in [
|
||||
include_bytes!("../resources/audio/startRecording.mp3").as_slice(),
|
||||
include_bytes!("../resources/audio/stopRecording.mp3").as_slice(),
|
||||
include_bytes!("../resources/audio/cancel.mp3").as_slice(),
|
||||
] {
|
||||
let mut sound = decode(bytes).unwrap();
|
||||
assert!(sound.total_duration().unwrap() > Duration::ZERO);
|
||||
assert!(sound.any(|sample| sample.abs() > 0.0));
|
||||
fn bundled_sounds_decode_without_an_audio_device() {
|
||||
let sounds = Sounds::load().unwrap();
|
||||
for (name, bytes, _) in BUNDLED {
|
||||
let mut sound = decode_wav(bytes).unwrap();
|
||||
assert!(sound.total_duration().unwrap() > Duration::ZERO, "{name}");
|
||||
assert!(sound.any(|sample| sample.abs() > 0.0), "{name}");
|
||||
}
|
||||
assert!(sounds.get(Tone::DictationDone).is_some());
|
||||
assert!(sounds.get(Tone::OnboardingSuccess).is_some());
|
||||
}
|
||||
|
||||
#[test]
|
||||
|
||||
@@ -517,6 +517,7 @@ pub(crate) fn run(
|
||||
pending = pending.saturating_sub(1);
|
||||
match result {
|
||||
Ok(text) => {
|
||||
feedback::play(Tone::DictationDone);
|
||||
events.emit(&VoiceEvent::Transcript {
|
||||
timestamp_ms: now_ms(),
|
||||
phase: TranscriptPhase::Completed,
|
||||
|
||||
@@ -3256,6 +3256,7 @@ fn handle_dictation_event(
|
||||
TranscriptionTarget::Paste | TranscriptionTarget::Send => DictationPhase::Pasted,
|
||||
TranscriptionTarget::Service => return Ok(()),
|
||||
};
|
||||
feedback::play(Tone::DictationDone);
|
||||
events.processed_dictation(phase, text, processing)?;
|
||||
if let Some(indicator) = indicator {
|
||||
indicator.send(DictationIndicatorEvent::JobCompleted {
|
||||
|
||||
@@ -159,13 +159,17 @@ pub(super) fn input(ui: &mut Ui, input: Input) {
|
||||
remember_step(ui, step);
|
||||
}
|
||||
|
||||
/// Saves a new step so a tour quit part-way resumes there.
|
||||
/// Saves a new step so a tour quit part-way resumes there, and sounds the
|
||||
/// success chime on reaching the last one.
|
||||
fn remember_step(ui: &Ui, before: Step) {
|
||||
if let Some(tour) = &ui.tour
|
||||
&& tour.flow.step() != before
|
||||
&& ui.live()
|
||||
{
|
||||
crate::onboarding::record_step(tour.flow.step());
|
||||
if tour.flow.step() == Step::Done {
|
||||
crate::feedback::play(crate::feedback::Tone::OnboardingSuccess);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -537,6 +537,9 @@ pub fn run(
|
||||
match result.result {
|
||||
Ok(()) => {
|
||||
let text = result.text.unwrap_or_default();
|
||||
if !matches!(result.kind, OutputKind::Repaste) {
|
||||
crate::feedback::play(crate::feedback::Tone::DictationDone);
|
||||
}
|
||||
match result.kind {
|
||||
OutputKind::Dictation => {
|
||||
events.emit(&VoiceEvent::Transcript {
|
||||
|
||||
Reference in New Issue
Block a user