fix(telemetry): give empty-item compressions a stable tool signature (#4396)

## Description

`ToolSignature.from_items([])` hashed a fresh `uuid4()`, so each
empty-items compression created a new entry in
`TelemetryCollector._tool_stats` (unbounded while `HEADROOM_TELEMETRY`
is on). Use a fixed hash: empty outputs carry no structure, so they
share one bucket.

Fixes #4392

## Type of Change

- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [ ] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)

## Changes Made

- `headroom/telemetry/models.py`: empty-items hash is
`sha256(b"empty")[:24]` instead of a per-call uuid4 hash (still 24
chars, still not the literal `"empty"`).
- `tests/test_telemetry.py`: `TestEmptyItemsSignature` (stable hash; 50
empty compressions give one tool stat).

## Testing

- [x] Unit tests pass (`pytest`) (telemetry file only, see Not tested)
- [x] Linting passes (`ruff check .`) (changed files only)
- [ ] Type checking passes (`mypy headroom`)
- [x] New tests added for new functionality
- [ ] Manual testing performed

### Test Output

```text
# source change reverted, new tests only
FAILED tests/test_telemetry.py::TestEmptyItemsSignature::test_empty_signature_is_stable
FAILED tests/test_telemetry.py::TestEmptyItemsSignature::test_repeated_empty_compressions_share_one_tool_stat
E   AssertionError: assert 50 == 1
2 failed, 43 deselected

# with the fix
tests/test_telemetry.py ..   2 passed, 43 deselected
tests/test_telemetry.py      45 passed

ruff check headroom/telemetry/models.py tests/test_telemetry.py  -> All checks passed!
ruff format --check (same files)                                 -> 2 files already formatted
```

## Real Behavior Proof

- Environment: Python 3.12.15, Ubuntu on WSL2, clone at upstream/main
e995eb9f.
- Exact command / steps: `PYTHONPATH=. python -P -m pytest -q -p
no:cacheprovider tests/test_telemetry.py`, once with only the models.py
change reverted, once with the fix.
- Observed result: 50 tool stats before, 1 after (see above).
- Not tested: mypy; the full test suite. Neighbouring files
test_toin_fixes.py, test_toin_observation_only.py,
test_critical_fixes.py, test_critical_gaps.py, test_proxy_ccr.py show 6
failed / 19 errors both with and without this change on my machine
(CancelledError in test_proxy_ccr, determinism tests in
test_toin_observation_only); I did not investigate them. TOIN behaviour
for empty lists via smart_crusher.py:971 was not exercised. Whether the
collector is bounded elsewhere was not checked.

## Runtime Rollout Safety

- Rollout-managed feature(s): none
- Minimum rollout channel: n/a
- Stable/default behavior changed: no (only affects telemetry
bookkeeping when telemetry is enabled)
- Kill switch / disable path: HEADROOM_TELEMETRY off
- Unsafe override required: no
- Qualification impact: none expected
- Rollback path: revert this commit

## Review Readiness

- [x] I have performed a self-review
- [x] This PR is ready for human review

## Checklist

- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [ ] I have made corresponding changes to the documentation (N/A)
- [x] My changes generate no new warnings
- [x] I have added tests that prove my fix is effective or that my
feature works
- [ ] New and existing unit tests pass locally with my changes (see Not
tested)
- [x] I did **not** edit `CHANGELOG.md`

## Additional Notes

All empty results now share one signature instead of one per call;
per-tool separation for empty outputs was never possible since the
signature has no fields.

Produced with AI assistance (Claude Code) operated by me (HetCreep); the
reproduction was run on my machine on a clone of the repository.

🤖 Generated with [Claude Code](https://claude.com/claude-code)
<!-- devin-review-badge-begin -->

---

<a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4396"
target="_blank"><picture><source media="(prefers-color-scheme: dark)"
srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img
src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4"
alt="Devin Review"></picture></a>
<!-- devin-review-badge-end -->

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
HetCreep
2026-10-24 08:43:54 -07:00
committed by GitHub
co-authored by Claude Opus 5.5 JerrettDavis
parent 755293dd68
commit a79def4dd1
2 changed files with 29 additions and 7 deletions
+5 -7
View File
@@ -214,13 +214,11 @@ class ToolSignature:
def from_items(cls, items: list[dict[str, Any]]) -> ToolSignature:
"""Create signature from sample items."""
if not items:
# HIGH FIX: Generate unique hash for empty outputs to prevent
# different tools' empty responses from colliding into one pattern.
# Use a random component to ensure uniqueness across tool types.
import uuid
# MEDIUM FIX #15: Use 24 chars (96 bits) instead of 16 (64 bits) to reduce collision risk
empty_hash = hashlib.sha256(f"empty:{uuid.uuid4()}".encode()).hexdigest()[:24]
# Fixed hash (not random): a per-call uuid made every empty result a new
# signature, so collector._tool_stats grew without bound. Empty outputs carry
# no structure to distinguish, so they share one bucket. Not the literal
# "empty", to stay distinct from any caller-supplied string key.
empty_hash = hashlib.sha256(b"empty").hexdigest()[:24]
return cls(
structure_hash=empty_hash,
field_count=0,
+24
View File
@@ -502,6 +502,30 @@ class TestTelemetryCollector:
assert len(collector._events) <= 5
class TestEmptyItemsSignature:
"""Empty-item compressions must not create a new tool signature each time."""
def test_empty_signature_is_stable(self):
assert (
ToolSignature.from_items([]).structure_hash
== ToolSignature.from_items([]).structure_hash
)
def test_repeated_empty_compressions_share_one_tool_stat(self):
collector = TelemetryCollector()
for _ in range(50):
collector.record_compression(
items=[],
original_count=0,
compressed_count=0,
original_tokens=10,
compressed_tokens=10,
strategy="none",
)
assert len(collector.get_all_tool_stats()) == 1
class TestTelemetryPersistence:
"""Test telemetry persistence to disk."""