Commit Graph
11 Commits
Author SHA1 Message Date
dependabot[bot]andJD Davis e42f74ebc9 deps: qualify latest compatible docs and E2E tools (#4580)
# Refresh docs and wrap CLIs to current compatible stable releases

The user explicitly requires dependencies on the newest secure,
compatible versions, with compatibility repairs as needed. This is
authorized dependency maintenance; no claim is made that every prior
version broke Headroom. All11 original group entries remain. Registry
latest-stable snapshot on2026-11-03: Fumadocs core/UI16.17.10,
MDX15.4.10, Twoslash4.0.4, TypeScript integration5.6.0;
@ai-sdk/openai4.0.108, ai7.0.162, openai7.47.0, PostCSS8.5.32;
Codex0.166.0 and OpenClaw2026.10.10. Docs SDK dependencies remain
development-only examples, not a product SDK upgrade.

Also update existing owned overrides to latest compatible patches MCP
SDK1.32.2, Zod4.6.7 and minimatch11.0.4. Undici8.12.3 and npm12.2.1 are
still registry latest stable. Refresh
mainfe69f381168035ea6527bb7b908d11eeeb01838f. Preserve native/platform
lock metadata; only correct added libc tags appear on existing Ubjs
variants, no platform tags lost. E2E README/Dockerfile engine comments
and exception reason reflect actual updated versions.

Actual relationship: MDX15.4.10 peers on core^16.17.8; current core
satisfies that requirement. The Twoslash7.1 integration is exercised by
docs/source.config.ts during generation; its native compiler packages
are legitimate dependencies. Official upstream TS7.1 migration:
https://github.com/fuma-nama/fumadocs/commit/2fced9230e9de9523eb9abce36586a18890ad59c
. No claim that upstream custom-highlighter fixes or new SDK APIs are
needed by current Headroom examples. Upstream097b5cc only changes their
development compiler pins, not a downstream runtime repair.

## Real behavior proof

Setup: actual Linux Docker node:24.22.2-bookworm, npm12.2.1 matching the
updated harness tool pin, fresh npm ci with lifecycle scripts enabled.
OpenClaw2026.10.10 supports Node>=24.16<25 or>=26.1; Node24.22.2
qualifies. npm12 blocks unreviewed dependency lifecycle scripts even
with ignore-scripts=false. The exact reviewed allowScripts identities
and local strict-allow-scripts=true settings preserve required lifecycle
execution and reject future unreviewed scripts. No wildcard, blanket
bypass or ignore flag is used for fresh installations. Lock-only
resolution used ignore-scripts, followed by fresh normal installs.

Public reproduction in a disposable Linux Node24.22.2/npm12.2.1
environment:

```sh
cd docs
npm ci
npm run types:check
node node_modules/typescript/bin/tsc --ignoreConfig --allowJs --checkJs --noEmit --skipLibCheck --module nodenext --target es2022 scripts/verify-sdk-wire.mjs
node scripts/verify-sdk-wire.mjs
npm run build
npm audit --package-lock-only --json
# Separate terminal, then stop after HTTP checks:
npm start -- --hostname 127.0.0.1 --port 32180
curl --fail http://127.0.0.1:32180/
curl --fail http://127.0.0.1:32180/docs
curl --fail 'http://127.0.0.1:32180/api/search?query=proxy'
curl --fail http://127.0.0.1:32180/llms.txt
curl --fail http://127.0.0.1:32180/sitemap.xml
cd ../e2e/wrap/tools
npm ci
npm test
node node_modules/@openai/codex/bin/codex.js --version
node node_modules/@openai/codex/bin/codex.js --help
node node_modules/openclaw/openclaw.mjs --version
npm audit --package-lock-only --json
```

Observed latest candidate: docs fresh install/postinstall, types,
independent SDK proof typecheck and production build all pass, raw
audit0. Actual Linux production server returns200 HTML for / and
/docs,200 JSON search results containing proxy,200 llms text and sitemap
XML. The portable committed SDK proof performs actual HTTP to
ephemeral127.0.0.1 with explicit fake keys: native OpenAI chat and
ai.generateText with default OpenAI Responses provider; checks
authorization/path/input and parsed text/usage. It does not rely on MDX
noErrors:true, make a provider request, or exercise Headroom adapters.
No live-provider interoperability claim.

E2E tools fresh install, all3 dependency regressions pass (Ajv URI
rejection, minimatch bounded expansion and Express malformed body
process health). Actual native CLI versions are Codex0.166.0 and
OpenClaw2026.10.10. Latest OpenClaw loads the actual source-built4384
Headroom plugin after normal production installation: exact
package/lock/dist/hook-shim artifacts copied into privately owned
/opt/headroom-plugin and npm ci --omit=dev, then plugins list --json.
Result: headroom statusloaded, source
/opt/headroom-plugin/dist/index.js, headroom_retrieve tool, no error,
diagnostics[]. Isolated /state app config,
autoStart:false,gatewayProviderIds:[],routeCodexViaProxy:false. Codex
help succeeds with isolated /state/codex. No auth, gateway, message or
inference action; fresh containers inherit no provider credentials.

Important installation boundary: Windows bind-mounted paths appear777
and newest OpenClaw rejects them as world-writable. Normal private
installation permissions resolve that. The pre-fix private copy of
package.json, manifest, dist/ and hook-shim/ without node_modules failed
because the original build really imported external headroom-ai. PR4384
separately repairs the supported copied layouts by bundling the SDK and
removing the runtime dependency. Both repaired layouts now pass actual
OpenClaw2026.10.10 loading without any SDK/node_modules/npm
installation: dist as root loads /opt/headroom-plugin/index.js; fallback
root package+dist+hook-shim loads /opt/headroom-plugin/dist/index.js.
Both register headroom_retrieve, status loaded, no errors,
diagnostics[]. An explicit require.resolve check proves no external
headroom-ai is available. Captured bundle SHA256:
d634d5ad4ce5a0998c1d0ea056220eb32e7c5be5d00f30e0c4cff503efc533a5. No
Python-wheel artifact is claimed.

## Security and trust

Docs raw audit0. E2E raw audit35 package entries (34moderate,1high,
mostly inherited), NOT zero. Existing security.yml checker on the latest
lock/audit passes precisely8 accepted bundled advisory/path/version
entries, no stale or unexpected findings, expiry2026-11-15 unchanged.
The exception reason is updated to current CLI/minimatch versions;
acceptance entries and deadline are unchanged.

Rechecked registry: npm12.2.1 is still latest and its published bundle
still contains undici6.29.0 (five accepted advisories),
ip-address10.7.2, glob13.0.6 and minimatch10.2.5. Normal overrides do
not rewrite these inBundle copies. They require a newer upstream npm
bundle; unsupported vendoring/file surgery is not used. Current root
Undici/minimatch paths are updated/fixed and are NOT excepted. npm
release source: https://registry.npmjs.org/npm/latest ; see existing
npm-audit-exceptions.json for exact advisory IDs and paths. Residual
accepted known vulnerabilities remain despite selecting latest releases.
Do not claim the entire graph is vulnerability-free.

Install/network surface: npm registry tarballs, native optional
Codex/OpenClaw filesystem/TypeScript binaries and lifecycle scripts;
Fumadocs MDX root postinstall, esbuild/protobufjs/OpenClaw scripts in
the installed graph. Docs Next build may fetch Google Fonts. OpenClaw
catalog loading executes bundled plugin code but no provider action was
requested. Docker now pins latest npm12.2.1 as the GLOBAL build/install
tool as well as the existing OpenClaw runtime override.

Limits: no full multi-stage wheel/Aider/gateway wrap suite,
inference/streaming or all-platform validation; no screenshot QA or
correctness guarantee for every illustrative snippet. TypeScript7.1
custom type-table functionality not used by content is not claimed as
tested. Current selected text SDK wire/API and actual installed plugin
loader behaviors are exercised directly.

Evidence:
.codex/dep4580-latest-{registry,security-versions,lock,docs-linux,docs-audit,docs-runtime,tools-linux,tools-audit,host-plugin-installed}.*.
Earlier dep4580-host-plugin/private logs preserve the expected
permission/missing-dependency failures, not hidden retries.
Install/tests/versions all pass; final tools command exit1 is solely the
expected raw npm audit findings. All runtime containers/processes
stopped after proof.

Locally qualified dependency tree:
`6a08e837aa7f112ac5602508597b32e54d5582ee`.


## Required npm12 compatibility repair

The global Docker npm pin is updated from11.16.0 to latest12.2.1. A
fresh baseline npm12 install passed dependency tests while BLOCKING all4
package lifecycle identities: passing tests alone was insufficient.
Exact reviewed approvals now cover tools `@google/genai@2.27.0` (echo
no-op preinstall), `esbuild@0.28.2` (native binary validation/fallback
acquisition), `protobufjs@7.6.6` (dependency-version-scheme warning) and
`openclaw@2026.10.10` (runtime/package-manager checks; package inventory
cleanup and native-prebuild maintenance). Docs approves only
`esbuild@0.28.2`; its root fumadocs postinstall runs normally. Each
local .npmrc sets strict-allow-scripts=true, so a new unreviewed
identity fails instead of silently losing its lifecycle. Approval keys
are exact package/version identities, not globs; no blanket scripts
bypass.

Fresh private Linux npm12 installs with --foreground-scripts positively
log every approved pre/postinstall command and no blocked or unreviewed
script. Tools installation, all3 regressions, Codex0.166.0 help/version,
OpenClaw2026.10.10 version pass. The fresh npm12 raw audit still has35
entries and the unchanged8 exact exceptions pass until2026-11-15. Docs
installation/root MDX generation, types, independent SDK type/wire
proof, production build and audit0 pass. Foreground lifecycle evidence:
dep4580-npm12-{tools,docs}-approved.log. Fresh audit:
dep4580-npm12-tools-audit.json. Historical npm12 RED/default-blocking
evidence remains dep4580-npm12-tools-ci.log. Linux is qualified;
Windows's earlier baseline used npm10 and is not represented as npm12
coverage. HTTP route proof above used the same latest versions built
under npm11 before the build-tool-only change; npm12's fresh build and
API proof are separately green.

The npm12 global executable contains the same upstream bundled
dependency versions as the overridden npm12 runtime package; selecting
latest does not remove the documented upstream residual vulnerabilities.
Existing exception deadline is not extended.

Exact Docker prefix-mode installation also passes with npm12.2.1: COPY
manifests plus .npmrc, npm ci --prefix /opt/wrap-tools --no-fund
--no-audit --strict-allow-scripts (foreground logging added only for
proof), npm test --prefix /opt/wrap-tools, both native CLI versions.
All4 lifecycle identities positively execute. Log:
dep4580-npm12-docker-prefix.log. All task containers/processes have
exited.


## Current-main integration and fresh qualification

The current head retains main’s separately reviewed Anthropic SDK 1.4.0
update alongside all eleven updates in this group. Resolving the docs
conflict changed only the Anthropic package entry and lock root
metadata; no existing lockfile package was removed. The update also
includes npm 12.2.1, exact reviewed `allowScripts` entries, project
strict policies, and explicit strict enforcement in the Docker-prefix
install.

Fresh Linux Node 24.22.2/npm 12.2.1 qualification on dependency tree
`6a08e837a` passed docs generation/type-check, OpenAI/AI SDK and
Anthropic SDK loopback requests, the production docs build, the actual
`/opt/wrap-tools` prefix install, all three wrap-tool regression tests,
and Codex/OpenClaw version/help smoke checks. Docs audit remains clean;
the tools exception gate passes the same eight bundled advisory/path
pairs, with the raw 35 affected entries disclosed above. A harmless
local-package probe verified an unreviewed lifecycle script fails before
execution, then runs only after explicit approval of its exact source.
These are credentialless compatibility checks, not live inference or a
claim of zero security advisories.


The current head `c27516acc10f120b797c6853da94067fad4e8bf8` merges main
without changing docs, wrap-tool files, the security workflow, or the
Anthropic SDK verification script from that qualified dependency tree
(direct git comparison). Local dependency/runtime results above
therefore cover identical inputs. Fresh hosted qualification on the new
head is pending for the added main-side native changes; no new local
native qualification is claimed.

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: JD Davis <mxjerrett@gmail.com>
2026-11-03 23:39:15 -06:00
Adib Mohsin 911eb85a44 new docs UI + ts doc coverage 2026-04-12 13:15:58 +06:00
chopratejas d291710176 Add MkDocs GitHub Pages documentation site
- Add mkdocs.yml with Material theme (indigo, professional)
- Add docs/index.md landing page with quick install
- Add GitHub Actions workflow for auto-deployment
- Remove old docs/README.md (replaced by index.md)
2026-01-30 20:33:29 -08:00
chopratejas 83c0334ccd docs: update documentation for IntelligentContext TOIN + CCR integration
Updates across multiple docs to reflect the new message-level compression
with TOIN + CCR integration:

- docs/ccr.md: Add CCR-enabled components table, message-level CCR section
- docs/ARCHITECTURE.md: Expand Transform 6 with TOIN + CCR integration details
- docs/configuration.md: Add CCR integration config and marker format
- docs/proxy.md: Add CCR integration note for context management
- docs/README.md: Update to reference IntelligentContextManager as default

Also adds examples/test_intelligent_context_toin_ccr.py for scale testing
the TOIN + CCR integration with real API calls.
2026-01-27 16:08:36 -08:00
Tejas Chopra 2fd9552102 Add image token compression with trained ML router
Introduces automatic image compression for LLM requests, reducing token
usage by 40-90% while maintaining answer accuracy.

Key features:
- Trained MiniLM classifier (93.7% accuracy) hosted on HuggingFace
- SigLIP-based image analysis for content-aware routing
- Provider-specific compression:
  - OpenAI: detail="low" parameter
  - Anthropic: PIL resize to 512px
  - Google: PIL resize to 768px (tile-optimized)
- Four compression techniques: full_low, preserve, crop, transcode
- Integration in both Headroom proxy and SDK (ContentRouter)

New files:
- headroom/image/ module with ImageCompressor API
- docs/image-compression.md user documentation
- tests/test_image_compressor.py (51 tests)

Model: chopratejas/technique-router on HuggingFace (~128MB)
2026-01-25 22:40:43 -08:00
Claude Code BotandClaude Sonnet 4.5 3ac26207b8 docs(deployment): add macOS LaunchAgent deployment guide and templates
Add comprehensive macOS deployment support for running headroom proxy as a
persistent background service using LaunchAgent. This enables automatic startup,
crash recovery, and proper lifecycle management for local development environments.

Files added:
- examples/deployment/macos-launchagent/com.headroom.proxy.plist.template
- examples/deployment/macos-launchagent/install.sh (shellcheck-clean)
- examples/deployment/macos-launchagent/uninstall.sh (shellcheck-clean)
- examples/deployment/macos-launchagent/shell-integration.sh (bash + zsh)
- examples/deployment/macos-launchagent/README.md
- docs/macos-deployment.md

Key features:
- Configurable port via HEADROOM_PROXY_PORT environment variable (default: 8787)
- Automated installation and uninstallation scripts
- Shell integration supporting both bash and zsh
- Comprehensive documentation with troubleshooting guide
- All shell scripts are shellcheck-clean (zero errors, warnings, or info messages)

Files modified:
- .gitignore: Added CLAUDE.md to prevent committing local config
- docs/README.md: Added Deployment & Operations section with navigation entry

AI review: Pending (will be run by pre-commit hook)

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
2026-01-19 13:34:52 -08:00
chopratejas aaaca9fa6a Add Agno integration documentation
- Update README.md with Agno quickstart and framework integration
- Create comprehensive docs/agno.md guide covering:
  - Model wrapping with HeadroomAgnoModel
  - Observability hooks (HeadroomPreHook, HeadroomPostHook)
  - Multi-provider auto-detection
  - Real-world examples and configuration reference
- Update docs/README.md index with Agno link
2026-01-16 16:17:14 -08:00
chopratejas 31aa72c885 Add universal compression module with ML-based content detection
- Add headroom.compression module with UniversalCompressor
- ML-based content detection using Magika (JSON, code, logs, text)
- Structure-preserving compression via handler protocol
- JSON handler: preserves keys, brackets, high-entropy values (UUIDs)
- Code handler: preserves imports, signatures, types (tree-sitter AST)
- Entropy-based preservation for identifiers and hashes
- CCR integration for reversible compression
- Comprehensive test suite with LLM eval tests
- Add docs/compression.md with full API documentation
2026-01-15 15:26:14 -08:00
chopratejas bb041047c8 Add seamless LangChain integration
- Add HeadroomChatModel wrapper with auto provider detection (OpenAI, Anthropic, Google)
- Add HeadroomChatMessageHistory for automatic conversation compression
- Add HeadroomDocumentCompressor for retriever integration
- Add wrap_tools_with_headroom() for agent tool output compression
- Add async support (ainvoke, astream)
- Add LangSmith integration for observability
- Restructure integrations package into nested langchain/ and mcp/ subpackages
- Fix Pydantic v2 deprecation warning
- Add comprehensive docs/langchain.md guide with real-world examples
- Update README with LangChain quickstart and framework integrations

Bump version to 0.2.3
2026-01-14 16:03:34 -08:00
chopratejas d724f14022 v0.2.2: Add CCR Response Handler, Context Tracker, and restructure docs
Features:
- CCR Response Handler: Automatically intercepts and handles headroom_retrieve tool calls
- CCR Context Tracker: Multi-turn awareness with proactive expansion of relevant compressed content
- New CCR demo script showing before/after flow

Documentation:
- Restructured README from 885 lines to 190 lines for better DevEx
- Split detailed docs into focused guides: ccr.md, sdk.md, configuration.md,
  text-compression.md, llmlingua.md, metrics.md, errors.md
- Updated docs/README.md index with all new documentation

Tests:
- Added comprehensive tests for Response Handler (32 tests)
- Added comprehensive tests for Context Tracker (32 tests)
- All 977 tests passing
2026-01-14 13:03:41 -08:00
chopratejas 175746cc26 Prepare for OSS release v0.2.0
This commit prepares Headroom for public open source release with
comprehensive documentation, licensing, and community infrastructure.

License & Legal:
- Add Apache 2.0 LICENSE file
- Add NOTICE file with third-party attributions
- Add SECURITY.md for vulnerability reporting

Community:
- Add CONTRIBUTING.md with contribution guidelines
- Add CODE_OF_CONDUCT.md (Contributor Covenant)
- Add GitHub issue templates (bug report, feature request)
- Add pull request template

Documentation:
- Update README.md with compelling value proposition
- Add docs/getting-started.md
- Add docs/proxy.md for proxy server documentation
- Add docs/transforms.md for transform reference
- Add docs/api.md for API reference
- Add examples/README.md

Package Infrastructure:
- Add headroom/py.typed for PEP 561 compliance
- Add headroom/cli.py for CLI entry point
- Add .github/workflows/ci.yml for CI pipeline
- Add .github/workflows/publish.yml for PyPI publishing
- Update pyproject.toml with proper metadata

New Features:
- Add multi-provider support (Google, Cohere, LiteLLM, OpenAI-compatible)
- Add universal tokenizer registry with multiple backends
- Add model registry with pricing and context limits
- Add production proxy server with caching and rate limiting

Code Quality:
- Fix 83 lint issues via ruff auto-fix
- Fix version consistency (benchmarks 0.1.0 → 0.2.0)
- Add skip decorators for optional dependency tests
2026-01-07 11:36:44 -08:00