mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-23 02:49:17 +08:00
## Description Adds an estimated CO₂ Saved card to the proxy dashboard and `/stats` JSON response, surfacing carbon footprint reduction alongside the existing token and cost savings. Motivation: Honey (Green-PT/honey-for-devs) ships a CO₂ tracking feature ported from EcoLogits. Headroom already has all the data needed — tokens saved per model — to make the same estimate. This PR wires that up. Methodology: EcoLogits v0.8 energy-per-token constants (Wh/token per model tier) × IEA 2023 global average carbon intensity (0.4 kg CO₂/kWh). Conservative bias throughout — we underestimate rather than overestimate. The methodology string and a bias note are exposed in the API response and dashboard tooltip so users understand the estimate's limits. --- ## Type of Change - [ ] Bug fix - [x] New feature (non-breaking addition) - [ ] Breaking change - [ ] Documentation update - [x] Performance improvement - [ ] Refactoring --- ## Changes Made `headroom/proxy/savings_tracker.py` - Added `estimate_co2_saved_mg(tokens_saved, model)` public function with EcoLogits-based energy constants for Claude, GPT-4, GPT-4o, o1/o3, Gemini families; default for unknown models - Added `co2` block to `stats_preview()` return: `co2_saved_mg`, `co2_saved_g`, `methodology`, `note` - Added `model` parameter to `stats_preview()` (default `""`, backward compatible) `headroom/dashboard/templates/dashboard.html` - Added CO₂ Saved stat card in the token stats grid, after the Savings% card - `formatCO2(mg)` Alpine.js helper: auto-scales to mg / g / kg - Tooltip on card shows EcoLogits methodology `tests/test_co2_estimation.py` (new file) - 7 tests (see Testing section) `CHANGELOG.md` - Added entry under `## Unreleased → ### Features` --- ## Testing - [x] New tests added and passing - [x] Backward compatible — `stats_preview()` default `model=""` uses conservative default factor New tests — all pass: ``` tests/test_co2_estimation.py::test_zero_tokens_returns_zero tests/test_co2_estimation.py::test_positive_tokens_returns_positive tests/test_co2_estimation.py::test_claude_sonnet_uses_correct_factor tests/test_co2_estimation.py::test_unknown_model_uses_default_factor tests/test_co2_estimation.py::test_claude_haiku_lower_than_sonnet tests/test_co2_estimation.py::test_large_savings_in_grams tests/test_co2_estimation.py::test_stats_preview_includes_co2 ``` --- ## Real Behavior Proof - Environment: macOS 15.5 arm64, Python 3.12, local editable current PR checkout. - Exact command / steps: ran the seven estimator/API tests, Ruff, the Chromium Playwright UI regression, a local proxy, and queried `/stats`. - Observed result: estimator/API tests and UI rendering pass; `/stats` exposes the CO2 block and the dashboard renders the estimated savings card. - Not tested: every dashboard theme and operator-supplied custom carbon intensity. Environment: - OS: macOS 15.5 arm64 (Apple M3 Pro) - Python: 3.12.0 - headroom-ai: 0.28.0 (local editable install from this branch) - pytest: 9.0.3 Exact commands: ```bash git clone https://github.com/peterlodri-sec/headroom cd headroom git checkout feat/co2-dashboard-card pip install -e ".[dev]" -q python -m pytest tests/test_co2_estimation.py -v ``` Observed result: ``` collected 7 items tests/test_co2_estimation.py::test_zero_tokens_returns_zero PASSED tests/test_co2_estimation.py::test_positive_tokens_returns_positive PASSED tests/test_co2_estimation.py::test_claude_sonnet_uses_correct_factor PASSED tests/test_co2_estimation.py::test_unknown_model_uses_default_factor PASSED tests/test_co2_estimation.py::test_claude_haiku_lower_than_sonnet PASSED tests/test_co2_estimation.py::test_large_savings_in_grams PASSED tests/test_co2_estimation.py::test_stats_preview_includes_co2 PASSED 7 passed in 1.73s ``` Live verification (headroom proxy running locally): ```bash curl -s http://127.0.0.1:8787/stats | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('co2'))" # → {'tokens_saved': 51046, 'co2_saved_mg': 4.084, 'co2_saved_g': 4.1e-06, 'methodology': '...', 'note': '...'} ``` Not tested: - Dashboard visual rendering in all themes (tested dark mode locally) - Operator-supplied custom carbon intensity (not configurable in this PR — could be a follow-up `HEADROOM_CO2_CARBON_INTENSITY_KG_PER_KWH` env var) --- ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review - [x] Tests passing (`python -m pytest tests/test_co2_estimation.py -v` — 7 passed) - [x] `ruff check headroom/proxy/savings_tracker.py` — clean - [x] I did not edit CHANGELOG.md; release notes are generated automatically - [x] Backward compatible — `model=""` default, `co2` block is additive to existing JSON shape --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
68 lines
2.1 KiB
Python
68 lines
2.1 KiB
Python
"""Tests for CO₂ saved estimation in savings_tracker.
|
||
|
||
Validates estimate_co2_saved_mg() and its presence in stats_preview().
|
||
"""
|
||
|
||
from __future__ import annotations
|
||
|
||
from headroom.proxy.savings_tracker import estimate_co2_saved_mg
|
||
|
||
|
||
def test_zero_tokens_returns_zero() -> None:
|
||
assert estimate_co2_saved_mg(0) == 0.0
|
||
assert estimate_co2_saved_mg(-100) == 0.0
|
||
|
||
|
||
def test_positive_tokens_returns_positive() -> None:
|
||
result = estimate_co2_saved_mg(1_000_000)
|
||
assert result > 0
|
||
|
||
|
||
def test_claude_sonnet_uses_correct_factor() -> None:
|
||
# claude-sonnet: 0.0003 Wh/token × 0.4 kg/kWh × 1000 mg/g = 0.12 mg/token
|
||
result = estimate_co2_saved_mg(1000, model="claude-sonnet-4-5")
|
||
assert abs(result - 120.0) < 1.0 # 0.12 mg × 1000 = 120 mg
|
||
|
||
|
||
def test_unknown_model_uses_default_factor() -> None:
|
||
# default: 0.0002 Wh/token × 0.4 = 0.08 mg/token
|
||
result = estimate_co2_saved_mg(1000, model="unknown-model-xyz")
|
||
assert abs(result - 80.0) < 1.0
|
||
|
||
|
||
def test_claude_haiku_lower_than_sonnet() -> None:
|
||
haiku = estimate_co2_saved_mg(1000, model="claude-haiku")
|
||
sonnet = estimate_co2_saved_mg(1000, model="claude-sonnet")
|
||
assert haiku < sonnet
|
||
|
||
|
||
def test_large_savings_in_grams() -> None:
|
||
# 1M tokens at claude-sonnet = 120,000 mg = 120 g
|
||
mg = estimate_co2_saved_mg(1_000_000, model="claude-sonnet")
|
||
g = mg / 1000
|
||
assert abs(g - 120.0) < 1.0
|
||
|
||
|
||
def test_stats_preview_includes_co2() -> None:
|
||
"""stats_preview() must include a co2 block with co2_saved_mg."""
|
||
import pathlib
|
||
import tempfile
|
||
|
||
from headroom.proxy.savings_tracker import SavingsTracker
|
||
|
||
with tempfile.TemporaryDirectory() as tmp:
|
||
tracker = SavingsTracker(path=str(pathlib.Path(tmp) / "savings.json"))
|
||
tracker.record_compression_savings(
|
||
model="claude-sonnet-4-5",
|
||
tokens_saved=10_000,
|
||
)
|
||
preview = tracker.stats_preview(model="claude-sonnet-4-5")
|
||
|
||
assert "co2" in preview
|
||
co2 = preview["co2"]
|
||
assert "co2_saved_mg" in co2
|
||
assert co2["co2_saved_mg"] > 0
|
||
assert "co2_saved_g" in co2
|
||
assert co2["co2_saved_g"] == round(co2["co2_saved_mg"] / 1000, 6)
|
||
assert "methodology" in co2
|