mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
fix(logs): count omitted levels from dropped lines (#4141)
## Description Closes #4070. The omission marker previously counted severities in the whole input, including retained errors. It now counts ERROR/FAIL/WARN/INFO only on dropped line identities. Selection and whole-input result.stats are unchanged. If no dropped line has a recognized severity, the marker is simply `[N lines omitted]`, not a severity breakdown. CCR retains the exact original. ## Type of Change - [x] Bug fix (non-breaking) - [x] Documentation update ## Changes Made - Keep the existing retained-line identity set and a single-pass fixed stack histogram; no omitted-line collection or repeated full scans. - Restore complete structured-marker assertions in Rust and Python, including bounded failure names, repeated ids, whole-input totals, and non-severity omissions. - Restore parity fixture `3bc015edc0a36387.json` using its original input, explicit configuration and input digest. Output and compression ratio were re-recorded through the freshly built native extension rather than manually copied. New runtime-frame defaults remain implicit in the legacy configuration. - Remove the constructor-only result-copy test; retain consumer behavior regressions. - Document omitted-versus-input counts and the exact bare-marker form in the existing text/log guide. No manual changelog change. ## Testing - [x] Scoped Rust and Python tests passed - [x] Normal touched-file Ruff and repository mypy hooks passed - [x] Actual native recorder and parity CLI exercised Head `6c472f6b` includes upstream main `f1bc2609`. - Focused Rust log-compressor tests: **41 passed**. - Python public-transform tests: **11 passed**, using the freshly built native extension. - Actual parity CLI: **20 matched, 0 skipped, 0 differences**, including the restored original-input fixture: ```text cargo run --locked -p headroom-parity --bin parity-run -- run --fixtures tests/parity/fixtures --only log_compressor [log_compressor ] total=20 matched=20 skipped=0 diffed=0 ``` ## Real Behavior Proof - Environment: Linux x86_64, Python 3.13.14, native extension freshly built from this worktree. - Exact command / steps: Re-record the original log fixture through LogCompressor.compress with its original explicit configuration, then run the actual parity CLI shown above. - Observed result: The real native recorder produced `[297 lines omitted: 297 INFO]` while whole-input statistics remained 300 INFO / one ERROR / 305 total / eight selected. The exact original was unchanged and the measured compression ratio is `0.024777201057682892`. The original fixture digest remains `3bc015edc0a36387194d075139845c120a308afb314bae436ab454906b431d84`. Normal commit and full pre-push hooks passed without bypass: Rust formatting, warnings-as-errors Clippy, workspace tests, native editable rebuild/import verification, **181 critical Python tests**, and branch-range commitlint. The critical Python run reported five dependency/provider warnings, no failures. These local results do not claim that hosted checks have finished. - Not tested: New Windows/macOS runs, live-provider sessions and the full Python suite. Earlier before/after actual-native CCR qualification showed the retained ERROR was not counted as omitted, whole-input totals stayed unchanged, and retrieval recovered the exact original. That historical implementation evidence is in the review discussion; the explicit results above qualify this follow-up head. No new Windows/macOS or live-provider qualification is claimed. ## Runtime Rollout Safety - Rollout-managed feature(s): Existing LogCompressor and CCR; no new feature or gate. - Minimum rollout channel: Unchanged (stable); no channel promotion. - Stable/default behavior changed: Yes, omitted-severity accounting is corrected; selection and stored originals remain unchanged. - Kill switch / disable path: ContentRouterConfig(enable_log_compressor=False) disables the log route; existing CCR settings disable retrieval separately. - Unsafe override required: No. - Qualification impact: The scoped native/Python/parity proof and required normal push gate above cover this marker path. No new platform/live-provider qualification is claimed. - Rollback path: Revert the fix; no schema or stored-original migration. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review - [x] Existing style and documentation conventions followed - [x] Consumer-visible regressions and actual native/CLI paths exercised - [x] Normal hooks passed; no bypass - [x] No manual CHANGELOG edit AI-assisted: omp / GPT-6.1 Sol. This review follow-up was completed without subagents. No manual human-validation or full Python-suite claim; scoped tests and the required push gate are reported separately. --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
89545fbdf3
commit
023dd1bc54
@@ -1532,35 +1532,50 @@ impl LogCompressor {
|
||||
|
||||
let omitted = all_lines.len().saturating_sub(selected.len());
|
||||
if omitted > 0 {
|
||||
// Keep public stats as input totals; describe only dropped identities here.
|
||||
let mut omitted_levels = [0u64; 4];
|
||||
for line in all_lines {
|
||||
if selected_numbers.contains(&line.line_number) {
|
||||
continue;
|
||||
}
|
||||
match line.level {
|
||||
LogLevel::Error => omitted_levels[0] += 1,
|
||||
LogLevel::Fail => omitted_levels[1] += 1,
|
||||
LogLevel::Warn => omitted_levels[2] += 1,
|
||||
LogLevel::Info => omitted_levels[3] += 1,
|
||||
_ => {}
|
||||
}
|
||||
}
|
||||
let mut summary_parts: Vec<String> = Vec::new();
|
||||
for (label, key) in [
|
||||
("ERROR", "errors"),
|
||||
("FAIL", "fails"),
|
||||
("WARN", "warnings"),
|
||||
("INFO", "info"),
|
||||
for (label, n) in [
|
||||
("ERROR", omitted_levels[0]),
|
||||
("FAIL", omitted_levels[1]),
|
||||
("WARN", omitted_levels[2]),
|
||||
("INFO", omitted_levels[3]),
|
||||
] {
|
||||
let n = stats.get(key).copied().unwrap_or(0);
|
||||
if n > 0 {
|
||||
summary_parts.push(format!("{} {}", n, label));
|
||||
}
|
||||
}
|
||||
if !summary_parts.is_empty() {
|
||||
let omitted_names = if omitted_short_summary_ids.is_empty() {
|
||||
String::new()
|
||||
let omitted_names = if omitted_short_summary_ids.is_empty() {
|
||||
String::new()
|
||||
} else {
|
||||
let shown = omitted_short_summary_ids
|
||||
.iter()
|
||||
.take(5)
|
||||
.copied()
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let overflow = omitted_short_summary_ids.len().saturating_sub(5);
|
||||
if overflow > 0 {
|
||||
format!("; omitted: {shown}, +{overflow} more")
|
||||
} else {
|
||||
let shown = omitted_short_summary_ids
|
||||
.iter()
|
||||
.take(5)
|
||||
.copied()
|
||||
.collect::<Vec<_>>()
|
||||
.join(", ");
|
||||
let overflow = omitted_short_summary_ids.len().saturating_sub(5);
|
||||
if overflow > 0 {
|
||||
format!("; omitted: {shown}, +{overflow} more")
|
||||
} else {
|
||||
format!("; omitted: {shown}")
|
||||
}
|
||||
};
|
||||
format!("; omitted: {shown}")
|
||||
}
|
||||
};
|
||||
if summary_parts.is_empty() {
|
||||
output.push(format!("[{} lines omitted{}]", omitted, omitted_names));
|
||||
} else {
|
||||
output.push(format!(
|
||||
"[{} lines omitted: {}{}]",
|
||||
omitted,
|
||||
@@ -1963,12 +1978,6 @@ mod tests {
|
||||
|
||||
let (output, _) = c.format_output(&selected, &parsed);
|
||||
assert!(!output.contains("; omitted: "));
|
||||
if !keep_summary_lines {
|
||||
assert_eq!(
|
||||
output,
|
||||
"FAILED outside.py::test_0\nFAILED outside.py::test_9\nERROR outside.py::test_10\nERROR outside.py::test_19\n[67 lines omitted: 10 ERROR, 10 FAIL, 51 INFO]"
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -2366,7 +2375,79 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn format_output_emits_summary_with_omitted_count() {
|
||||
fn omitted_marker_counts_only_dropped_lines_on_live_path() {
|
||||
let mut lines = (0..400)
|
||||
.map(|i| format!("INFO worker processed item {i}"))
|
||||
.collect::<Vec<_>>();
|
||||
let error = "ERROR worker failed: connection reset";
|
||||
lines.insert(200, error.to_string());
|
||||
let c = LogCompressor::new(LogCompressorConfig {
|
||||
enable_ccr: false,
|
||||
..Default::default()
|
||||
});
|
||||
let (result, _) = c.compress(&lines.join("\n"), 1.0);
|
||||
let marker = result
|
||||
.compressed
|
||||
.lines()
|
||||
.find(|line| line.contains("lines omitted:"))
|
||||
.expect("large input must have omitted lines");
|
||||
let pattern = Regex::new(r"(\d+) (ERROR|FAIL|WARN|INFO)").unwrap();
|
||||
let mut level_counts = pattern.captures_iter(marker);
|
||||
let count = level_counts.next().expect("omitted INFO count");
|
||||
|
||||
assert!(result.compressed.lines().any(|line| line == error));
|
||||
assert_eq!(&count[2], "INFO");
|
||||
assert_eq!(
|
||||
count[1].parse::<usize>().unwrap(),
|
||||
result.original_line_count - result.compressed_line_count
|
||||
);
|
||||
assert!(level_counts.next().is_none());
|
||||
assert_eq!(
|
||||
marker,
|
||||
format!("[{} lines omitted: {} INFO]", &count[1], &count[1])
|
||||
);
|
||||
assert_eq!(result.stats["errors"], 1);
|
||||
assert_eq!(result.stats["info"], 400);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn omitted_non_severity_lines_keep_count_notice_on_live_path() {
|
||||
use std::fmt::Write as _;
|
||||
|
||||
let count_pattern = Regex::new(r"(?m)^\[(\d+) lines omitted[^\]]*\]$").unwrap();
|
||||
let level_pattern = Regex::new(r"\d+ (ERROR|FAIL|WARN|INFO)\b").unwrap();
|
||||
let error = "ERROR retained failure";
|
||||
for prefix in ["DEBUG ", "TRACE ", ""] {
|
||||
let mut input = String::with_capacity(4096);
|
||||
for i in 0..100 {
|
||||
writeln!(input, "{prefix}progress step {i}").unwrap();
|
||||
}
|
||||
input.push_str(error);
|
||||
let c = LogCompressor::new(LogCompressorConfig {
|
||||
enable_ccr: false,
|
||||
..Default::default()
|
||||
});
|
||||
let (result, _) = c.compress(&input, 1.0);
|
||||
let marker = count_pattern
|
||||
.captures(&result.compressed)
|
||||
.expect("dropped lines must retain an omission count");
|
||||
assert_eq!(
|
||||
marker[1].parse::<usize>().unwrap(),
|
||||
result.original_line_count - result.compressed_line_count
|
||||
);
|
||||
assert!(!level_pattern.is_match(marker.get(0).unwrap().as_str()));
|
||||
assert_eq!(
|
||||
marker.get(0).unwrap().as_str(),
|
||||
format!("[{} lines omitted]", &marker[1])
|
||||
);
|
||||
assert!(result.compressed.lines().any(|line| line == error));
|
||||
assert_eq!(result.stats["errors"], 1);
|
||||
assert_eq!(result.stats["total"], 101);
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn format_output_preserves_input_totals() {
|
||||
let c = cmp();
|
||||
let all_lines = vec![
|
||||
LogLine::new(0, "ERROR a"),
|
||||
@@ -2388,9 +2469,9 @@ mod tests {
|
||||
.collect::<Vec<_>>();
|
||||
let selected = vec![all_lines[0].clone()];
|
||||
let (output, stats) = c.format_output(&selected, &all_lines);
|
||||
assert!(output.contains("[3 lines omitted: 1 ERROR, 1 WARN, 2 INFO]"));
|
||||
assert_eq!(stats["errors"], 1);
|
||||
assert_eq!(stats["info"], 2);
|
||||
assert_eq!(output, "ERROR a\n[3 lines omitted: 1 WARN, 2 INFO]");
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -2415,11 +2496,9 @@ mod tests {
|
||||
} else {
|
||||
String::new()
|
||||
};
|
||||
assert!(
|
||||
output.ends_with(&format!(
|
||||
"[{count} lines omitted: {count} FAIL, 1 INFO; omitted: {shown}{overflow}]"
|
||||
)),
|
||||
"{output}"
|
||||
assert_eq!(
|
||||
output.lines().last().unwrap(),
|
||||
format!("[{count} lines omitted: {count} FAIL; omitted: {shown}{overflow}]")
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -2442,11 +2521,9 @@ mod tests {
|
||||
];
|
||||
|
||||
let (output, stats) = cmp().format_output(&selected, &all_lines);
|
||||
assert!(
|
||||
output.ends_with(
|
||||
"[3 lines omitted: 1 ERROR, 3 FAIL, 1 INFO; omitted: tests/test_ids.py::test_error, tests/test_ids.py::test_repeat, tests/test_ids.py::test_repeat]"
|
||||
),
|
||||
"{output}"
|
||||
assert_eq!(
|
||||
output.lines().last().unwrap(),
|
||||
"[3 lines omitted: 1 ERROR, 2 FAIL; omitted: tests/test_ids.py::test_error, tests/test_ids.py::test_repeat, tests/test_ids.py::test_repeat]"
|
||||
);
|
||||
assert_eq!(stats["errors"], 1);
|
||||
assert_eq!(stats["fails"], 3);
|
||||
@@ -2482,26 +2559,12 @@ mod tests {
|
||||
);
|
||||
|
||||
let (output, _) = c.format_output(&selected, &all_lines);
|
||||
assert!(
|
||||
output.ends_with(
|
||||
"[4 lines omitted: 4 FAIL, 1 INFO; omitted: tests/test_ids.py::test_1, tests/test_ids.py::test_2]"
|
||||
),
|
||||
"{output}"
|
||||
assert_eq!(
|
||||
output.lines().last().unwrap(),
|
||||
"[4 lines omitted: 2 FAIL, 1 INFO; omitted: tests/test_ids.py::test_1, tests/test_ids.py::test_2]"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn format_output_marker_is_unchanged_without_omitted_short_summary_entries() {
|
||||
let c = cmp();
|
||||
let mut error = LogLine::new(0, "ERROR a");
|
||||
error.level = LogLevel::Error;
|
||||
let mut info = LogLine::new(1, "INFO b");
|
||||
info.level = LogLevel::Info;
|
||||
|
||||
let (output, _) = c.format_output(&[error.clone()], &[error, info]);
|
||||
assert_eq!(output, "ERROR a\n[1 lines omitted: 1 ERROR, 1 INFO]");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn score_line_caps_at_one_point_zero() {
|
||||
let line = LogLine {
|
||||
|
||||
@@ -93,6 +93,11 @@ print(f"Compression ratio: {result.compression_ratio:.1%}")
|
||||
- Verbose success output
|
||||
- Repeated patterns
|
||||
|
||||
Omission markers count `ERROR`, `FAIL`, `WARN`, and `INFO` levels only among the dropped lines, not the full input. An error retained in the compressed output is therefore not advertised as omitted. Debug, trace, and unclassified lines are not listed in the level breakdown.
|
||||
When only debug, trace, or unclassified lines are dropped, the marker is simply `[N lines omitted]`, without a severity breakdown.
|
||||
|
||||
`result.stats` continues to report whole-input totals; the marker's narrower counts do not change those statistics. CCR still retains the exact original log when enabled.
|
||||
|
||||
## DiffCompressor
|
||||
|
||||
Compresses unified diffs while keeping the actual changes and enough context to understand them.
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -1,9 +1,10 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
import pytest
|
||||
|
||||
from headroom.transforms.log_compressor import (
|
||||
LogCompressionResult,
|
||||
LogCompressor,
|
||||
LogCompressorConfig,
|
||||
LogFormat,
|
||||
@@ -185,18 +186,37 @@ def test_real_compress_ignores_malformed_short_summary_diagnostic() -> None:
|
||||
"FAILED outside.py::test_9\n"
|
||||
"ERROR outside.py::test_10\n"
|
||||
"ERROR outside.py::test_19\n"
|
||||
"[67 lines omitted: 10 ERROR, 10 FAIL, 51 INFO]"
|
||||
"[67 lines omitted: 8 ERROR, 8 FAIL, 51 INFO]"
|
||||
)
|
||||
|
||||
|
||||
def test_result_properties() -> None:
|
||||
result = LogCompressionResult(
|
||||
compressed="small",
|
||||
original="this is a substantially longer log body",
|
||||
original_line_count=20,
|
||||
compressed_line_count=5,
|
||||
format_detected=LogFormat.GENERIC,
|
||||
compression_ratio=0.25,
|
||||
)
|
||||
assert result.tokens_saved_estimate > 0
|
||||
assert result.lines_omitted == 15
|
||||
def test_omitted_marker_counts_only_dropped_lines_on_native_path() -> None:
|
||||
lines = [f"INFO worker processed item {i}" for i in range(400)]
|
||||
error = "ERROR worker failed: connection reset"
|
||||
lines.insert(200, error)
|
||||
result = LogCompressor(LogCompressorConfig(enable_ccr=False)).compress("\n".join(lines))
|
||||
marker = next(line for line in result.compressed.splitlines() if "lines omitted:" in line)
|
||||
counts = {
|
||||
level: int(count) for count, level in re.findall(r"(\d+) (ERROR|FAIL|WARN|INFO)", marker)
|
||||
}
|
||||
|
||||
assert error in result.compressed.splitlines()
|
||||
assert counts == {"INFO": result.lines_omitted}
|
||||
assert marker == f"[{result.lines_omitted} lines omitted: {result.lines_omitted} INFO]"
|
||||
assert result.stats["errors"] == 1
|
||||
assert result.stats["info"] == 400
|
||||
|
||||
|
||||
@pytest.mark.parametrize("prefix", ["DEBUG ", "TRACE ", ""])
|
||||
def test_non_severity_omissions_keep_count_notice_on_native_path(prefix: str) -> None:
|
||||
error = "ERROR retained failure"
|
||||
original = "\n".join([f"{prefix}progress step {i}" for i in range(100)] + [error])
|
||||
result = LogCompressor(LogCompressorConfig(enable_ccr=False)).compress(original)
|
||||
marker = re.search(r"(?m)^\[(\d+) lines omitted[^\]]*\]$", result.compressed)
|
||||
assert marker is not None
|
||||
assert int(marker[1]) == result.lines_omitted
|
||||
assert not re.search(r"\d+ (ERROR|FAIL|WARN|INFO)\b", marker[0])
|
||||
assert marker[0] == f"[{result.lines_omitted} lines omitted]"
|
||||
assert error in result.compressed.splitlines()
|
||||
assert result.stats["errors"] == 1
|
||||
assert result.stats["total"] == 101
|
||||
|
||||
Reference in New Issue
Block a user