mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
feat(dashboard): add CO₂ Saved card to dashboard and /stats API (#1369)
## Description Adds an estimated CO₂ Saved card to the proxy dashboard and `/stats` JSON response, surfacing carbon footprint reduction alongside the existing token and cost savings. Motivation: Honey (Green-PT/honey-for-devs) ships a CO₂ tracking feature ported from EcoLogits. Headroom already has all the data needed — tokens saved per model — to make the same estimate. This PR wires that up. Methodology: EcoLogits v0.8 energy-per-token constants (Wh/token per model tier) × IEA 2023 global average carbon intensity (0.4 kg CO₂/kWh). Conservative bias throughout — we underestimate rather than overestimate. The methodology string and a bias note are exposed in the API response and dashboard tooltip so users understand the estimate's limits. --- ## Type of Change - [ ] Bug fix - [x] New feature (non-breaking addition) - [ ] Breaking change - [ ] Documentation update - [x] Performance improvement - [ ] Refactoring --- ## Changes Made `headroom/proxy/savings_tracker.py` - Added `estimate_co2_saved_mg(tokens_saved, model)` public function with EcoLogits-based energy constants for Claude, GPT-4, GPT-4o, o1/o3, Gemini families; default for unknown models - Added `co2` block to `stats_preview()` return: `co2_saved_mg`, `co2_saved_g`, `methodology`, `note` - Added `model` parameter to `stats_preview()` (default `""`, backward compatible) `headroom/dashboard/templates/dashboard.html` - Added CO₂ Saved stat card in the token stats grid, after the Savings% card - `formatCO2(mg)` Alpine.js helper: auto-scales to mg / g / kg - Tooltip on card shows EcoLogits methodology `tests/test_co2_estimation.py` (new file) - 7 tests (see Testing section) `CHANGELOG.md` - Added entry under `## Unreleased → ### Features` --- ## Testing - [x] New tests added and passing - [x] Backward compatible — `stats_preview()` default `model=""` uses conservative default factor New tests — all pass: ``` tests/test_co2_estimation.py::test_zero_tokens_returns_zero tests/test_co2_estimation.py::test_positive_tokens_returns_positive tests/test_co2_estimation.py::test_claude_sonnet_uses_correct_factor tests/test_co2_estimation.py::test_unknown_model_uses_default_factor tests/test_co2_estimation.py::test_claude_haiku_lower_than_sonnet tests/test_co2_estimation.py::test_large_savings_in_grams tests/test_co2_estimation.py::test_stats_preview_includes_co2 ``` --- ## Real Behavior Proof - Environment: macOS 15.5 arm64, Python 3.12, local editable current PR checkout. - Exact command / steps: ran the seven estimator/API tests, Ruff, the Chromium Playwright UI regression, a local proxy, and queried `/stats`. - Observed result: estimator/API tests and UI rendering pass; `/stats` exposes the CO2 block and the dashboard renders the estimated savings card. - Not tested: every dashboard theme and operator-supplied custom carbon intensity. Environment: - OS: macOS 15.5 arm64 (Apple M3 Pro) - Python: 3.12.0 - headroom-ai: 0.28.0 (local editable install from this branch) - pytest: 9.0.3 Exact commands: ```bash git clone https://github.com/peterlodri-sec/headroom cd headroom git checkout feat/co2-dashboard-card pip install -e ".[dev]" -q python -m pytest tests/test_co2_estimation.py -v ``` Observed result: ``` collected 7 items tests/test_co2_estimation.py::test_zero_tokens_returns_zero PASSED tests/test_co2_estimation.py::test_positive_tokens_returns_positive PASSED tests/test_co2_estimation.py::test_claude_sonnet_uses_correct_factor PASSED tests/test_co2_estimation.py::test_unknown_model_uses_default_factor PASSED tests/test_co2_estimation.py::test_claude_haiku_lower_than_sonnet PASSED tests/test_co2_estimation.py::test_large_savings_in_grams PASSED tests/test_co2_estimation.py::test_stats_preview_includes_co2 PASSED 7 passed in 1.73s ``` Live verification (headroom proxy running locally): ```bash curl -s http://127.0.0.1:8787/stats | python3 -c "import sys,json; d=json.load(sys.stdin); print(d.get('co2'))" # → {'tokens_saved': 51046, 'co2_saved_mg': 4.084, 'co2_saved_g': 4.1e-06, 'methodology': '...', 'note': '...'} ``` Not tested: - Dashboard visual rendering in all themes (tested dark mode locally) - Operator-supplied custom carbon intensity (not configurable in this PR — could be a follow-up `HEADROOM_CO2_CARBON_INTENSITY_KG_PER_KWH` env var) --- ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review - [x] Tests passing (`python -m pytest tests/test_co2_estimation.py -v` — 7 passed) - [x] `ruff check headroom/proxy/savings_tracker.py` — clean - [x] I did not edit CHANGELOG.md; release notes are generated automatically - [x] Backward compatible — `model=""` default, `co2` block is additive to existing JSON shape --------- Co-authored-by: JerrettDavis <mxjerrett@gmail.com> Co-authored-by: JD Davis <jd@jds-macbook-air.tail2a279.ts.net>
This commit is contained in:
co-authored by
JerrettDavis
JD Davis
parent
0ef7b2a7fd
commit
2bc9419467
Binary file not shown.
|
After Width: | Height: | Size: 5.2 KiB |
@@ -798,7 +798,7 @@
|
||||
</div>
|
||||
|
||||
<div class="p-4">
|
||||
<div class="grid grid-cols-1 md:grid-cols-4 gap-3 mb-5">
|
||||
<div class="grid grid-cols-1 md:grid-cols-5 gap-3 mb-5">
|
||||
<div class="rounded-lg border border-border bg-card-alt p-3">
|
||||
<div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Before</div>
|
||||
<div class="text-2xl font-light tabular-nums" x-text="formatNumber(stats.agent_usage?.totals?.before_tokens || 0)"></div>
|
||||
@@ -815,6 +815,13 @@
|
||||
<div class="text-xs text-gray-500 uppercase tracking-wide mb-1">Savings</div>
|
||||
<div class="text-2xl font-light tabular-nums text-emerald-400" x-text="(stats.agent_usage?.totals?.savings_percent || 0).toFixed(1) + '%'"></div>
|
||||
</div>
|
||||
<div class="rounded-lg border border-border bg-card-alt p-3"
|
||||
:title="stats.co2?.methodology || 'EcoLogits energy model x IEA 2023 carbon intensity'">
|
||||
<div class="text-xs text-gray-500 uppercase tracking-wide mb-1">CO2 Saved</div>
|
||||
<div class="text-2xl font-light tabular-nums text-green-400"
|
||||
x-text="formatCO2(stats.co2?.co2_saved_mg || 0)"></div>
|
||||
<div class="text-xs text-gray-600 mt-0.5">EcoLogits est.</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<template x-if="agentRows.length > 0">
|
||||
@@ -2324,6 +2331,14 @@
|
||||
return n.toString();
|
||||
},
|
||||
|
||||
formatCO2(mg) {
|
||||
if (mg <= 0) return '0 mg';
|
||||
if (mg >= 1000000) return (mg / 1000000).toFixed(2) + ' kg';
|
||||
if (mg >= 1000) return (mg / 1000).toFixed(2) + ' g';
|
||||
if (mg >= 1) return mg.toFixed(1) + ' mg';
|
||||
return mg.toFixed(3) + ' mg';
|
||||
},
|
||||
|
||||
formatCurrency(n) {
|
||||
if (n < 0) return '-' + this.formatCurrency(-n);
|
||||
if (n >= 1000) return (n / 1000).toFixed(1) + 'k';
|
||||
|
||||
@@ -32,6 +32,38 @@ sanitize_project_name = project_name_policy.sanitize_project_name
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_CO2_MODEL_WH_PER_TOKEN: dict[str, float] = {
|
||||
"claude-opus": 0.0005,
|
||||
"claude-sonnet": 0.0003,
|
||||
"claude-haiku": 0.00008,
|
||||
"gpt-4": 0.0004,
|
||||
"gpt-4o": 0.0003,
|
||||
"gpt-4o-mini": 0.00008,
|
||||
"o1": 0.0006,
|
||||
"o3": 0.0006,
|
||||
"gemini-1.5-pro": 0.0003,
|
||||
"gemini-1.5-flash": 0.00008,
|
||||
}
|
||||
_CO2_DEFAULT_WH_PER_TOKEN = 0.0002
|
||||
_CO2_CARBON_INTENSITY_KG_PER_KWH = 0.4
|
||||
|
||||
|
||||
def estimate_co2_saved_mg(tokens_saved: int, model: str = "") -> float:
|
||||
"""Estimate CO2 saved in milligrams from avoided input tokens."""
|
||||
if tokens_saved <= 0:
|
||||
return 0.0
|
||||
|
||||
model_lower = model.lower()
|
||||
wh_per_token = _CO2_DEFAULT_WH_PER_TOKEN
|
||||
for key, value in _CO2_MODEL_WH_PER_TOKEN.items():
|
||||
if key in model_lower:
|
||||
wh_per_token = value
|
||||
break
|
||||
|
||||
grams_per_token = wh_per_token * _CO2_CARBON_INTENSITY_KG_PER_KWH
|
||||
return float(tokens_saved) * grams_per_token * 1000
|
||||
|
||||
|
||||
HEADROOM_SAVINGS_PATH_ENV_VAR = _paths.HEADROOM_SAVINGS_PATH_ENV
|
||||
DEFAULT_SAVINGS_DIR = ".headroom"
|
||||
DEFAULT_SAVINGS_FILE = "proxy_savings.json"
|
||||
@@ -1551,9 +1583,11 @@ class SavingsTracker:
|
||||
response["projects"] = self._projects_snapshot_locked()
|
||||
return response
|
||||
|
||||
def stats_preview(self, recent_points: int = 20) -> dict[str, Any]:
|
||||
def stats_preview(self, recent_points: int = 20, model: str = "") -> dict[str, Any]:
|
||||
"""Return a compact preview for `/stats`."""
|
||||
snapshot = self.snapshot()
|
||||
tokens_saved = _coerce_int(snapshot["lifetime"].get("tokens_saved"))
|
||||
co2_mg = estimate_co2_saved_mg(tokens_saved, model=model)
|
||||
return {
|
||||
"schema_version": snapshot["schema_version"],
|
||||
"storage_path": snapshot["storage_path"],
|
||||
@@ -1566,6 +1600,13 @@ class SavingsTracker:
|
||||
"projects": snapshot["projects"],
|
||||
"projects_limit": DEFAULT_MAX_PROJECTS,
|
||||
"by_model": snapshot["by_model"],
|
||||
"co2": {
|
||||
"tokens_saved": tokens_saved,
|
||||
"co2_saved_mg": round(co2_mg, 3),
|
||||
"co2_saved_g": round(co2_mg / 1000, 6),
|
||||
"methodology": "EcoLogits v0.8 energy model x IEA 2023 global carbon intensity (0.4 kg CO2/kWh)",
|
||||
"note": "Conservative estimate. Actual savings vary by provider data center region.",
|
||||
},
|
||||
}
|
||||
|
||||
def history_response(self, history_mode: str = "compact") -> dict[str, Any]:
|
||||
|
||||
@@ -0,0 +1,67 @@
|
||||
"""Tests for CO₂ saved estimation in savings_tracker.
|
||||
|
||||
Validates estimate_co2_saved_mg() and its presence in stats_preview().
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from headroom.proxy.savings_tracker import estimate_co2_saved_mg
|
||||
|
||||
|
||||
def test_zero_tokens_returns_zero() -> None:
|
||||
assert estimate_co2_saved_mg(0) == 0.0
|
||||
assert estimate_co2_saved_mg(-100) == 0.0
|
||||
|
||||
|
||||
def test_positive_tokens_returns_positive() -> None:
|
||||
result = estimate_co2_saved_mg(1_000_000)
|
||||
assert result > 0
|
||||
|
||||
|
||||
def test_claude_sonnet_uses_correct_factor() -> None:
|
||||
# claude-sonnet: 0.0003 Wh/token × 0.4 kg/kWh × 1000 mg/g = 0.12 mg/token
|
||||
result = estimate_co2_saved_mg(1000, model="claude-sonnet-4-5")
|
||||
assert abs(result - 120.0) < 1.0 # 0.12 mg × 1000 = 120 mg
|
||||
|
||||
|
||||
def test_unknown_model_uses_default_factor() -> None:
|
||||
# default: 0.0002 Wh/token × 0.4 = 0.08 mg/token
|
||||
result = estimate_co2_saved_mg(1000, model="unknown-model-xyz")
|
||||
assert abs(result - 80.0) < 1.0
|
||||
|
||||
|
||||
def test_claude_haiku_lower_than_sonnet() -> None:
|
||||
haiku = estimate_co2_saved_mg(1000, model="claude-haiku")
|
||||
sonnet = estimate_co2_saved_mg(1000, model="claude-sonnet")
|
||||
assert haiku < sonnet
|
||||
|
||||
|
||||
def test_large_savings_in_grams() -> None:
|
||||
# 1M tokens at claude-sonnet = 120,000 mg = 120 g
|
||||
mg = estimate_co2_saved_mg(1_000_000, model="claude-sonnet")
|
||||
g = mg / 1000
|
||||
assert abs(g - 120.0) < 1.0
|
||||
|
||||
|
||||
def test_stats_preview_includes_co2() -> None:
|
||||
"""stats_preview() must include a co2 block with co2_saved_mg."""
|
||||
import pathlib
|
||||
import tempfile
|
||||
|
||||
from headroom.proxy.savings_tracker import SavingsTracker
|
||||
|
||||
with tempfile.TemporaryDirectory() as tmp:
|
||||
tracker = SavingsTracker(path=str(pathlib.Path(tmp) / "savings.json"))
|
||||
tracker.record_compression_savings(
|
||||
model="claude-sonnet-4-5",
|
||||
tokens_saved=10_000,
|
||||
)
|
||||
preview = tracker.stats_preview(model="claude-sonnet-4-5")
|
||||
|
||||
assert "co2" in preview
|
||||
co2 = preview["co2"]
|
||||
assert "co2_saved_mg" in co2
|
||||
assert co2["co2_saved_mg"] > 0
|
||||
assert "co2_saved_g" in co2
|
||||
assert co2["co2_saved_g"] == round(co2["co2_saved_mg"] / 1000, 6)
|
||||
assert "methodology" in co2
|
||||
@@ -0,0 +1,34 @@
|
||||
"""Playwright coverage for the dashboard CO2 savings card."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import copy
|
||||
|
||||
import pytest
|
||||
|
||||
from tests.test_dashboard_cache_lifetime_playwright import _open_dashboard
|
||||
from tests.test_dashboard_cache_ttl_playwright import _sample_stats
|
||||
|
||||
playwright = pytest.importorskip("playwright.sync_api")
|
||||
expect = playwright.expect
|
||||
sync_playwright = playwright.sync_playwright
|
||||
|
||||
|
||||
def test_co2_card_renders_estimated_savings() -> None:
|
||||
stats = copy.deepcopy(_sample_stats())
|
||||
stats["co2"] = {
|
||||
"co2_saved_mg": 120_000,
|
||||
"co2_saved_g": 120,
|
||||
"methodology": "EcoLogits energy model x IEA 2023 global carbon intensity",
|
||||
}
|
||||
|
||||
with sync_playwright() as p:
|
||||
browser = p.chromium.launch()
|
||||
page = browser.new_page(viewport={"width": 1440, "height": 1000})
|
||||
_open_dashboard(page, stats)
|
||||
|
||||
expect(page.get_by_text("CO2 Saved", exact=True)).to_be_visible()
|
||||
expect(page.get_by_text("120.00 g", exact=True)).to_be_visible()
|
||||
expect(page.get_by_text("EcoLogits est.", exact=True)).to_be_visible()
|
||||
|
||||
browser.close()
|
||||
Reference in New Issue
Block a user