fix(smart-crusher): keep the row the question names (#4713)

## Description

Ask "Who opened issue #4475?" over a 100-issue `gh issue list` result,
and SmartCrusher cuts row #4475. `prioritize_indices` fills the budget
with errors, outliers and first/last rows, then fills by position, so
rows matching a query anchor (ID, UUID, email, hostname, quoted string)
come last. `plan_top_n` already keeps anchor matches unconditionally,
and the docs list them as preserved. This makes `smart_sample`,
`cluster_sample` and `time_series` do the same.

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)

## Changes Made

- `apply_query_signals` returns the anchor matches; `prioritize_indices`
keeps them with the critical set.
- If an anchor matches more rows than the whole budget, it isn't
selective, so those rows compete as before. Like errors, matches can
push the result over `max_items`, by at most `max_items` rows.
- BM25 matches are unchanged. Anchor matching is the existing
`item_matches_anchors`, unchanged here.
- The `planning.rs` call sites only pass the new set; the extra lines
are rustfmt.

## Testing

- [x] New tests: errors fill the budget, no error rows, anchor matches
more rows than the budget, end to end through `crush_array`. Each fails
without the fix; `cargo mutants --in-diff`: 10/10 caught.
- [x] `cargo test --workspace`, `cargo fmt --check`, `cargo clippy
--workspace -D warnings` (1.95.0), `make test-parity` (smart_crusher
17/17). Fork CI: green.

### Test Output

```
$ cargo test -p headroom-core --lib smart_crusher
test transforms::smart_crusher::orchestration::tests::prioritize_keeps_query_match_when_errors_fill_budget ... ok
test transforms::smart_crusher::orchestration::tests::prioritize_keeps_query_match_without_error_rows ... ok
test transforms::smart_crusher::orchestration::tests::prioritize_ignores_anchor_matching_more_than_budget ... ok
test transforms::smart_crusher::crusher::tests::query_named_row_survives_lossy_crush ... ok
test result: ok. 354 passed; 0 failed; 0 ignored; 0 measured; 689 filtered out
```

## Real Behavior Proof

- Environment: macOS arm64, `compress()` from the README (0.43.1 vs this
branch built from source), gpt-oss-20b local, temperature 0.
- Exact command / steps: 12 random issues, "Who opened issue #N, and is
it open or closed?" Scripts and answers:
https://github.com/cherylaurelia/headroom-demo/tree/main/bench
- Observed result: 0.43.1 kept 2 of the 12 asked rows (2/12 correct);
this branch keeps 12/12 (10/12 correct; the 2 misses had the row in
context). About 90 more prompt tokens.
- Not tested: rows named only by title words can still be cut.

Through `headroom proxy`, Claude Code also needs
`fix/toolsearch-query-note` (its relevance query is "Tool loaded."):
this fix alone 2/12, both 12/12.

## Runtime Rollout Safety

- Rollout-managed feature(s): n/a
- Minimum rollout channel: n/a
- Stable/default behavior changed: yes, one more kept row when the
question names it
- Kill switch / disable path: n/a
- Unsafe override required: no
- Qualification impact: none
- Rollback path: revert

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] I did **not** edit `CHANGELOG.md`

Written with Claude Code, which also ran the tests and measurements
above.
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4713"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude <noreply@anthropic.com>
This commit is contained in:
cheryl
2026-11-09 14:54:45 -06:00
committed by GitHub
co-authored by Claude
parent 0511bcd331
commit e6e2ebb678
3 changed files with 157 additions and 17 deletions
@@ -2206,6 +2206,35 @@ mod tests {
assert_eq!(recovered, Value::Array(rows));
}
#[test]
fn query_named_row_survives_lossy_crush() {
// An issue list where every 4th title is a bug report: the error
// rows alone fill the lossy budget, and the row the question names
// by number (index 81, past the first/last anchors) must be kept
// rather than cut by position.
let c = SmartCrusher::without_compaction(SmartCrusherConfig::default());
let items: Vec<Value> = (0..100)
.map(|i| {
let title = if i % 4 == 0 {
format!("[BUG] request failed in module {}", i % 7)
} else {
format!("Add option for module {}", i % 7)
};
json!({
"number": if i == 81 { 4475 } else { 4300 + 2 * i },
"title": title,
"state": if i % 3 == 0 { "OPEN" } else { "CLOSED" },
})
})
.collect();
let result = c.crush_array(&items, "Who opened issue #4475?", 1.0);
assert!(
result.items.iter().any(|it| it["number"] == json!(4475)),
"issue 4475 was dropped: {}",
result.strategy_info
);
}
#[test]
fn crusher_with_custom_scorer() {
use crate::relevance::BM25Scorer;
@@ -143,7 +143,10 @@ pub fn fill_remaining_slots(
/// 3. **Already under budget?** Return as-is.
/// 4. **Otherwise**: keep ALL critical items (errors + structural
/// outliers + numeric anomalies — non-negotiable per Python's
/// "quality guarantee"). Then add first-3 + last-2 if room. Then
/// "quality guarantee") and the items that exactly match a query
/// anchor (`query_matches`: an ID, UUID, email, hostname or quoted
/// string from the user's question), unless the anchors match more
/// than `effective_max` items. Then add first-3 + last-2 if room. Then
/// fill remaining with non-critical kept indices in ascending order.
///
/// May return MORE than `effective_max` items when critical items
@@ -156,6 +159,7 @@ pub fn prioritize_indices(
n: usize,
analysis: Option<&ArrayAnalysis>,
effective_max: usize,
query_matches: &BTreeSet<usize>,
) -> BTreeSet<usize> {
// Dedup pass.
let mut current = if config.dedup_identical_items {
@@ -195,6 +199,15 @@ pub fn prioritize_indices(
prioritized.extend(&anomaly_indices);
prioritized.extend(&learned_indices);
// The rows the question names, kept unconditionally as `plan_top_n`
// already does. An anchor that matches more rows than the whole budget
// is not selective (a shared hostname, a common number) and pinning all
// of them would defeat the crush, so those rows compete as ordinary
// kept indices instead.
if query_matches.len() <= effective_max {
prioritized.extend(query_matches.intersection(&current));
}
// First 3 / last 2 anchors if we have room.
let mut remaining = effective_max.saturating_sub(prioritized.len());
if remaining > 0 {
@@ -411,11 +424,23 @@ mod tests {
// ---------- prioritize_indices ----------
fn prioritize(kept: &BTreeSet<usize>, items: &[Value], max: usize) -> BTreeSet<usize> {
prioritize_indices(
&cfg(),
kept,
items,
items.len(),
None,
max,
&BTreeSet::new(),
)
}
#[test]
fn prioritize_under_budget_passthrough_after_dedup() {
let items: Vec<Value> = (0..5).map(|i| json!({"id": i})).collect();
let kept = idx_set(&[0, 1, 2]);
let result = prioritize_indices(&cfg(), &kept, &items, items.len(), None, 10);
let result = prioritize(&kept, &items, 10);
// 3 items < max 10 → fill kicks in; we get 5 (all items).
assert_eq!(result.len(), 5);
}
@@ -428,7 +453,7 @@ mod tests {
json!({"name": "bob"}),
];
let kept = idx_set(&[0, 1, 2]);
let result = prioritize_indices(&cfg(), &kept, &items, items.len(), None, 10);
let result = prioritize(&kept, &items, 10);
// Dedup collapses 0+1 to 0; fill stays put because n=3 already covered.
assert_eq!(result, idx_set(&[0, 2]));
}
@@ -441,19 +466,77 @@ mod tests {
.collect();
items.push(json!({"id": 30, "msg": "FATAL: out of memory"}));
let kept: BTreeSet<usize> = (0..items.len()).collect();
let result = prioritize_indices(&cfg(), &kept, &items, items.len(), None, 10);
let result = prioritize(&kept, &items, 10);
assert!(
result.contains(&30),
"error item must survive prioritization"
);
}
#[test]
fn prioritize_keeps_query_match_when_errors_fill_budget() {
// 39 error-keyword rows fill the budget of 10; the row the question
// names (index 25) has no error keyword and must still survive.
let items: Vec<Value> = (0..40)
.map(|i| {
let outcome = if i == 25 { "ok" } else { "failed" };
json!({"number": 4400 + i, "title": format!("parser {outcome} on case {i}")})
})
.collect();
let kept: BTreeSet<usize> = (0..items.len()).collect();
let result = prioritize_indices(
&cfg(),
&kept,
&items,
items.len(),
None,
10,
&idx_set(&[25]),
);
assert!(
result.contains(&25),
"query-matched row must survive prioritization"
);
}
#[test]
fn prioritize_keeps_query_match_without_error_rows() {
// No critical rows at all: the budget fills by position, and the
// matched row in the middle must still be kept.
let items: Vec<Value> = (0..40)
.map(|i| json!({"id": 9000 + i, "v": "ok"}))
.collect();
let kept: BTreeSet<usize> = (0..items.len()).collect();
let result = prioritize_indices(
&cfg(),
&kept,
&items,
items.len(),
None,
10,
&idx_set(&[20]),
);
assert!(result.contains(&20), "got {result:?}");
assert!(result.len() <= 10, "got {} items", result.len());
}
#[test]
fn prioritize_ignores_anchor_matching_more_than_budget() {
// An anchor that matches 20 rows against a budget of 10 isn't
// selective: it must not force all 20 into the result.
let items: Vec<Value> = (0..30).map(|i| json!({"id": i, "v": i})).collect();
let kept: BTreeSet<usize> = (0..items.len()).collect();
let matches: BTreeSet<usize> = (5..25).collect();
let result = prioritize_indices(&cfg(), &kept, &items, items.len(), None, 10, &matches);
assert!(result.len() <= 10, "got {} items", result.len());
}
#[test]
fn prioritize_includes_first_3_and_last_2_when_room() {
// No errors / outliers / anomalies → first 3 + last 2 anchors fill.
let items: Vec<Value> = (0..30).map(|i| json!({"id": i, "v": i})).collect();
let kept: BTreeSet<usize> = (5..15).collect();
let result = prioritize_indices(&cfg(), &kept, &items, items.len(), None, 10);
let result = prioritize(&kept, &items, 10);
// With no critical items and budget 10, dedup is a no-op (all
// distinct) and fill keeps us at <= 10. We should see at least
// some of items 0..3 OR 28..30 covered through fill.
@@ -217,13 +217,21 @@ impl<'a> SmartCrusherPlanner<'a> {
}
// 5/6. Query-anchor matches + relevance scores.
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
let query_matches =
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
// TOIN preserve_fields.
self.apply_preserve_field_matches(items, query_context, preserve_fields, &mut keep);
let final_keep =
prioritize_indices(self.config, &keep, items, n, Some(analysis), max_items);
let final_keep = prioritize_indices(
self.config,
&keep,
items,
n,
Some(analysis),
max_items,
&query_matches,
);
plan.keep_indices = final_keep.into_iter().collect();
plan
}
@@ -406,13 +414,21 @@ impl<'a> SmartCrusherPlanner<'a> {
}
// 4/5. Query signals.
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
let query_matches =
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
// TOIN preserve_fields.
self.apply_preserve_field_matches(items, query_context, preserve_fields, &mut keep);
let final_keep =
prioritize_indices(self.config, &keep, items, n, Some(analysis), max_items);
let final_keep = prioritize_indices(
self.config,
&keep,
items,
n,
Some(analysis),
max_items,
&query_matches,
);
plan.keep_indices = final_keep.into_iter().collect();
plan
}
@@ -458,13 +474,21 @@ impl<'a> SmartCrusherPlanner<'a> {
self.apply_constraints(items, item_strings, &mut keep);
// 4/5. Query signals.
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
let query_matches =
self.apply_query_signals(items, query_context, item_strings, &mut keep, false);
// TOIN preserve_fields.
self.apply_preserve_field_matches(items, query_context, preserve_fields, &mut keep);
let final_keep =
prioritize_indices(self.config, &keep, items, n, Some(analysis), max_items);
let final_keep = prioritize_indices(
self.config,
&keep,
items,
n,
Some(analysis),
max_items,
&query_matches,
);
plan.keep_indices = final_keep.into_iter().collect();
plan
}
@@ -474,7 +498,8 @@ impl<'a> SmartCrusherPlanner<'a> {
/// Apply query-anchor matches (deterministic) + relevance scoring
/// (probabilistic). When `keep_existing_only` is true (top_n's
/// "additive only" mode), only items not already in keep are added.
/// When false, all matches are added.
/// When false, all matches are added. Returns the indices added by the
/// deterministic anchor match, which `prioritize_indices` keeps.
fn apply_query_signals(
&self,
items: &[Value],
@@ -482,9 +507,10 @@ impl<'a> SmartCrusherPlanner<'a> {
item_strings: Option<&[String]>,
keep: &mut BTreeSet<usize>,
keep_existing_only: bool,
) {
) -> BTreeSet<usize> {
let mut anchor_matches = BTreeSet::new();
if query_context.is_empty() {
return;
return anchor_matches;
}
// Deterministic anchor match.
@@ -495,6 +521,7 @@ impl<'a> SmartCrusherPlanner<'a> {
}
if item_matches_anchors(item, &anchors) {
keep.insert(i);
anchor_matches.insert(i);
}
}
@@ -519,6 +546,7 @@ impl<'a> SmartCrusherPlanner<'a> {
keep.insert(i);
}
}
anchor_matches
}
fn apply_preserve_field_matches(