mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 18:41:07 +08:00
feat/https-compose-caddy
19
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
5df3d946ea |
deps: bump pyo3 from 0.29.3 to 0.30.1 (#4811)
Bumps [pyo3](https://github.com/pyo3/pyo3) from 0.29.3 to 0.30.1. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pyo3/pyo3/releases">pyo3's releases</a>.</em></p> <blockquote> <p>PyO3 0.30.1 is a re-release of PyO3 0.30.0 after investments in CI security unfortunately led to a failed crates.io publish. Rather than delete the failed tag, this new tag comes after the CI issues have (hopefully) been corrected.</p> <p>Thanks to <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a> <a href="https://github.com/Tpt"><code>@Tpt</code></a> and <a href="https://github.com/viicos"><code>@viicos</code></a> for the investigation and fixes.</p> <h2>PyO3 0.30.0</h2> <p>This release brings confirmed testing against Python 3.15.0 final. Support for Python 3.8 has been dropped. The minimum supported Rust version has also been increased to 1.85.</p> <p>The largest sets of improvements come to PyO3 0.29.0's <code>experimental-inspect</code> feature, which has received substantial feedback since 0.29.0 and subsequently has seen many refinements to improve the quality and coverage to the generated stubs. Thank you to everyone who has helped to continue to drive this feature towards completion.</p> <p>New safe wrappers for <code>PyFrozenDict</code> (3.15+) and <code>PyContext</code> have been added.</p> <p>PyO3 now contains meson build files, which should make it possible for projects using meson-python to more straightforwardly use Rust and PyO3 for additional components.</p> <p>Additionally, a number of small optimizations have landed, including:</p> <ul> <li>simplifications to core implementations of <code>PyAnyMethods</code>,</li> <li>reduced overhead from PyO3's internal "reference pool" used to dispose of <code>Py<T></code> values dropped when not attached to the interpreter,</li> <li>optimizations to <code>num-bigint</code> conversions, and</li> <li>better generated code from <code>IntoPyObject</code> and <code>IntoPyObjectRef</code> derives.</li> </ul> <p>There are also many other incremental improvements, bug fixes and smaller features; full detail can be found in the <a href="https://pyo3.rs/v0.30.0/changelog.html">CHANGELOG</a>.</p> <p>Please consult the <a href="https://pyo3.rs/v0.30.0/migration.html">migration guide</a> for help upgrading.</p> <p>Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following contributors' commits are included in this release:</p> <p><a href="https://github.com/adamgerhant-ai"><code>@adamgerhant-ai</code></a> <a href="https://github.com/alex"><code>@alex</code></a> <a href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a> <a href="https://github.com/charliermarsh"><code>@charliermarsh</code></a> <a href="https://github.com/chirizxc"><code>@chirizxc</code></a> <a href="https://github.com/clin1234"><code>@clin1234</code></a> <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a> <a href="https://github.com/dependabot"><code>@dependabot</code></a>[bot] <a href="https://github.com/dylanpulver"><code>@dylanpulver</code></a> <a href="https://github.com/elbaro"><code>@elbaro</code></a> <a href="https://github.com/exg"><code>@exg</code></a> <a href="https://github.com/florentinl"><code>@florentinl</code></a> <a href="https://github.com/FranciscoTGouveia"><code>@FranciscoTGouveia</code></a> <a href="https://github.com/Icxolu"><code>@Icxolu</code></a> <a href="https://github.com/imfeh2"><code>@imfeh2</code></a> <a href="https://github.com/IvanIsCoding"><code>@IvanIsCoding</code></a> <a href="https://github.com/JMak-Security"><code>@JMak-Security</code></a> <a href="https://github.com/jonasdedden"><code>@jonasdedden</code></a> <a href="https://github.com/jorenham"><code>@jorenham</code></a> <a href="https://github.com/konstin"><code>@konstin</code></a> <a href="https://github.com/latent-9"><code>@latent-9</code></a> <a href="https://github.com/MatthieuDartiailh"><code>@MatthieuDartiailh</code></a> <a href="https://github.com/mattip"><code>@mattip</code></a> <a href="https://github.com/mejrs"><code>@mejrs</code></a> <a href="https://github.com/msimacek"><code>@msimacek</code></a></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PyO3/pyo3/blob/v0.30.1/CHANGELOG.md">pyo3's changelog</a>.</em></p> <blockquote> <h2>[0.30.1] - 2026-10-14</h2> <h3>Packaging</h3> <ul> <li>Support PyPy 3.12. <a href="https://redirect.github.com/PyO3/pyo3/pull/6448">#6448</a></li> </ul> <h2>[0.30.0] - 2026-10-13</h2> <p>NB PyO3 0.30.0 failed to push to crates.io due to a CI issue, so 0.30.1 was prepared and pushed with minimal other changes other than confirming support for PyPy 3.12.</p> <h3>Packaging</h3> <ul> <li>Drop support for Python 3.8. <a href="https://redirect.github.com/PyO3/pyo3/pull/6128">#6128</a></li> <li>Bump MSRV to 1.85. <a href="https://redirect.github.com/PyO3/pyo3/pull/6459">#6459</a></li> <li>Support <code>meson-python</code> as a build backend. <a href="https://redirect.github.com/PyO3/pyo3/pull/6483">#6483</a></li> <li>Bump minimum supported GraalPy version to 25.4. <a href="https://redirect.github.com/PyO3/pyo3/pull/6491">#6491</a></li> <li>Support Python 3.15.0 final. <a href="https://redirect.github.com/PyO3/pyo3/pull/6517">#6517</a></li> </ul> <h3>Added</h3> <ul> <li>Add support for <code>PyFrozenDict</code> on 3.15+ <a href="https://redirect.github.com/PyO3/pyo3/pull/6174">#6174</a></li> <li>Add <code>PyContext</code> type. <a href="https://redirect.github.com/PyO3/pyo3/pull/6227">#6227</a></li> <li><code>experimental-inspect</code>: classes which extend their base's instance layout are now decorated with <code>disjoint_base</code> in the generated stubs, as per PEP 800. <a href="https://redirect.github.com/PyO3/pyo3/pull/6362">#6362</a></li> <li>Add methods <code>PyBuffer::as_slice_ptr</code> and <code>PyBuffer::as_fortran_slice_ptr</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6382">#6382</a></li> <li>Implement <code>Copy</code> for <code>PyVisit</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6412">#6412</a></li> <li>Add non-poisoning variant of <code>PyMutex</code> <a href="https://redirect.github.com/PyO3/pyo3/pull/6437">#6437</a></li> <li>Add <code>PyStringMethods::code_point_len</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6455">#6455</a></li> <li>FFI Definitions: <ul> <li>Add FFI definition <code>PyThread_get_thread_ident</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6056">#6056</a></li> <li>Add FFI definitions for PEP 731 (<code>PyInitConfig</code>). <a href="https://redirect.github.com/PyO3/pyo3/pull/6152">#6152</a></li> <li>Add FFI definitions <code>PyContext_AddWatcher</code> and <code>PyContext_ClearWatcher</code> on Python 3.14+. <a href="https://redirect.github.com/PyO3/pyo3/pull/6204">#6204</a></li> <li>Add FFI definitions for PEP 539 ("Thread-Specific Storage" API). <a href="https://redirect.github.com/PyO3/pyo3/pull/6262">#6262</a></li> <li>Enable FFI definitions <code>PyObject_GC_Track</code> and <code>PyObject_GC_UnTrack</code> on PyPy. <a href="https://redirect.github.com/PyO3/pyo3/pull/6299">#6299</a></li> <li>Add FFI definitions <code>PyUnicode_DecodeMBCS</code>, <code>PyUnicode_DecodeMBCSStateful</code>, <code>PyUnicode_DecodeCodePagetateful</code>, <code>PyUnicode_AsMBCSString</code>, <code>PyUnicode_EncodeCodePage</code>, and <code>PyUnicode_Equal</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6349">#6349</a></li> <li>Add FFI definitions <code>PyFloat_Pack{2,4,8}</code> and <code>PyFloat_Unpack{2,4,8}</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6350">#6350</a></li> <li>Add FFI definitions <code>PyFunction_GET_CODE</code>, <code>PyFunction_GET_GLOBALS</code>, <code>PyFunction_GET_MODULE</code>, <code>PyFunction_GET_DEFAULTS</code>, <code>PyFunction_GET_KW_DEFAULTS</code>, <code>PyFunction_GET_CLOSURE</code>, <code>PyFunction_GET_ANNOTATIONS</code>, <code>PyClassMethod_Type</code>, <code>PyStaticMethod_Type</code>, <code>PyClassMethod_New</code>, and <code>PyStaticMethod_New</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6351">#6351</a></li> </ul> </li> </ul> <h3>Changed</h3> <ul> <li>Optimized <code>num-bigint</code> conversions on Python 3.14 and newer. <a href="https://redirect.github.com/PyO3/pyo3/pull/6144">#6144</a></li> <li>Change <code>#[pyclass]</code> to not emit <code>FromPyObject</code> implementation by default (this was previously deprecated). <a href="https://redirect.github.com/PyO3/pyo3/pull/6188">#6188</a></li> <li>Deprecate <code>#[pyclass(skip_from_py_object)]</code> option (as it is now the default). <a href="https://redirect.github.com/PyO3/pyo3/pull/6188">#6188</a></li> <li>Deprecate <code>pyo3_build_config::PythonAbi::from_build_env</code> in favor of <code>pyo3_build_config::PythonAbi::from_stable_abi</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6192">#6192</a></li> <li>Change <code>pyo3_build_config::InterpreterConfig::from_sysconfigdata</code> to no longer select a target ABI based on the content of abi3 or abi3t cargo features (it will now always select a version-specific ABI). <a href="https://redirect.github.com/PyO3/pyo3/pull/6192">#6192</a></li> <li>Optimize PyO3's internal reference pool used for disposal of <code>Py<T></code> values dropped when not attached to the Python interpreter. <a href="https://redirect.github.com/PyO3/pyo3/pull/6200">#6200</a></li> <li><code>experimental-inspect</code>: Emit <code>__all__</code> for complete modules in the type stubs. <a href="https://redirect.github.com/PyO3/pyo3/pull/6242">#6242</a></li> <li>Change <code>PyBytes::new_with_writer</code> to take <code>&mut PyBytesWriter</code> instead of <code>&mut dyn Write</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/6251">#6251</a></li> <li>Change <code>#[pymodule_init]</code> to allow functions which do not take a module argument (for <code>experimental-inspect</code>, such modules are not marked <code>Incomplete</code>). <a href="https://redirect.github.com/PyO3/pyo3/pull/6271">#6271</a></li> <li>Change <code>#[pymodule_init]</code> to optionally allow functions to take a <code>Python<'_></code> marker. <a href="https://redirect.github.com/PyO3/pyo3/pull/6271">#6271</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PyO3/pyo3/commit/f168538672f2c32e32f7c1b702a502ee92de1edf"><code>f168538</code></a> release: 0.30.1</li> <li><a href="https://github.com/PyO3/pyo3/commit/b3c73619cbb6e3c7f3e6a0ad78c85814dbe4d854"><code>b3c7361</code></a> pyo3-introspection sdist: skip maturin license lookup (<a href="https://redirect.github.com/pyo3/pyo3/issues/6528">#6528</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/b564d8cfbacf801583b9c03647e4b575a16d4528"><code>b564d8c</code></a> ci: require <code>Cargo.lock</code> to be up to date during testing (<a href="https://redirect.github.com/pyo3/pyo3/issues/6522">#6522</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/bbd862c5a0177b4aa564c3d0e81091fe73f5bb43"><code>bbd862c</code></a> support PyPy 3.12 (<a href="https://redirect.github.com/pyo3/pyo3/issues/6448">#6448</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/196ca358aa86d59f0957db99fb304dc8d806121b"><code>196ca35</code></a> release: 0.30.0 (<a href="https://redirect.github.com/pyo3/pyo3/issues/6493">#6493</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/d3befaaa14288c4d743eab80369391268eaba721"><code>d3befaa</code></a> remove all functionality deprecated in PyO3 0.28 (<a href="https://redirect.github.com/pyo3/pyo3/issues/6519">#6519</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/3ee4e5658a5e7a3f95c90968646a5916bb8b0d5e"><code>3ee4e56</code></a> <code>experimental-inspect</code>: emit <code>__all__</code> in the generated stubs (<a href="https://redirect.github.com/pyo3/pyo3/issues/6242">#6242</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/071edecda46b6bcf3448e469b8392e289802e445"><code>071edec</code></a> test against 3.15 final (<a href="https://redirect.github.com/pyo3/pyo3/issues/6517">#6517</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/4c68d9cbed8ba67ebc98e19e7669aa340e37c265"><code>4c68d9c</code></a> internal: remove no longer needed <code>non_local_definitions</code> allows (<a href="https://redirect.github.com/pyo3/pyo3/issues/6508">#6508</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/65c8c7da61bd119c88ed7e394d8bef071ee37baa"><code>65c8c7d</code></a> Add example building with <code>meson-python</code> (<a href="https://redirect.github.com/pyo3/pyo3/issues/6483">#6483</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pyo3/pyo3/compare/v0.29.3...v0.30.1">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> <!-- devin-review-badge-begin --> --- <a href="https://app.devin.ai/review/headroomlabs-ai/headroom/pull/4811" target="_blank"><picture><source media="(prefers-color-scheme: dark)" srcset="https://static.devin.ai/assets/gh-devin-review-dark.svg?v=4"><img src="https://static.devin.ai/assets/gh-devin-review-light.svg?v=4" alt="Devin Review"></picture></a> <!-- devin-review-badge-end --> ## Real Behavior Proof - Observed on this PR head: Release `build-wheels (ubuntu-24.04, x86_64-unknown-linux-gnu, 2_28)` job resolved and compiled `pyo3 v0.30.1` via maturin ("Downloaded pyo3 v0.30.1"), abi3 wheel built; entire CI matrix green. - Not tested: installing the built wheel on all target platforms (only the manylinux build matrix exercised). Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
9d98ea5908 |
fix(rust-proxy): forward request paths verbatim and pin the rustls provider (#3853)
## Summary VAPT-readiness findings **03-F2** (Bedrock/Vertex path traversal to a signed request, previously *unverified* — now **confirmed** by the tests below), **03-F3** (`join_upstream_path` base-prefix escape), **03-F1** (rustls provider panic on `wss://`) and part of **03-F9** (lint posture). Rust `headroom-proxy` only; the Python wheel is untouched. **Problem.** `url::Url::set_path` resolves `.`/`..` (also `%2e`) and folds `\\` into `/`. So a client could walk out of a tenant base prefix on the generic forwarder, move a **SigV4-signed** Bedrock request to another resource (`/model/../invoke`), or move an **ADC-signed** Vertex request to another project. Bedrock also re-interpolated the URL-decoded `model_id`, so an inference-profile ARN (contains `/`) became two segments and a hostile id gained segments. **Fix, one generic module rather than per-provider patches.** New `crates/headroom-proxy/src/upstream_path.rs`, used by every upstream URL builder (forwarder, Bedrock invoke + streaming, Vertex, WebSocket upgrade): - `join_request_path` rejects any raw path the URL parser would rewrite and asserts the result still sits under the configured base prefix (400 before anything is signed or sent). - `append_segments` rebuilds provider paths segment by segment from decoded route parameters, percent-encoding `/` so an ARN stays one `%2F` segment, and rejecting `.`, `..`, empty and control-character segments. Side effect: a `bedrock_endpoint` with a path prefix now keeps that prefix. Also: - ring is installed as the **process-wide** rustls `CryptoProvider` in `AppState::new`. The binary links both ring and aws-lc-rs, and rustls panics in `ClientConfig::builder()` without a default. Main already names ring explicitly for the `wss://` relay (#3831), so that path no longer panics there; this makes every present and future TLS client in the process safe and adds the first end-to-end `wss://` relay test (rcgen certificate generated at test time, trusted via `HEADROOM_CA_BUNDLE`). - `#![forbid(unsafe_code)]` on the four crates that have no `unsafe` today; `overflow-checks = true` in the release profile for the proxy package only. ## Reproduction / regression tests `tests/integration_path_traversal.rs` (6) and `tests/integration_ws_tls.rs` (2), mock upstream records the path it receives. Against **origin/main** (test files copied over, test-only crates added): ``` thread 'forwarder_keeps_requests_under_the_base_prefix' (653400) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:103:9: assertion `left == right` failed: /../tenant-b/v1/models must be rejected left: 200 right: 400 thread 'websocket_upgrade_with_traversal_path_is_rejected_before_upgrade' (653402) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:130:5: assertion `left == right` failed: a traversal path must not be upgraded (101) left: 404 right: 400 thread 'vertex_dot_segment_project_is_rejected_unsigned' (653401) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:269:5: assertion `left == right` failed left: 200 right: 400 thread 'bedrock_streaming_dot_segment_model_id_is_rejected_unsigned' (653399) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:246:9: assertion `left == right` failed: /model/../invoke-with-response-stream must be rejected left: 200 right: 400 thread 'bedrock_dot_segment_model_id_is_rejected_unsigned' (653397) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:172:9: assertion `left == right` failed: /model/../invoke must be rejected left: 200 right: 400 thread 'bedrock_encoded_slashes_in_model_id_stay_one_segment' (653398) panicked at crates/headroom-proxy/tests/integration_path_traversal.rs:203:5: traversal reached the upstream: ["/guardrail/GID/version/1/apply/invoke"] test result: FAILED. 0 passed; 6 failed ``` The last line is the important one: on main a crafted `model_id` made the proxy sign and send a request to `/guardrail/GID/version/1/apply/invoke`. `integration_ws_tls.rs` **passes on main as well**: the explicit-ring fix in #3831 already covers the relay, and test builds unify crypto features differently from the release binary, so these two tests guard the behaviour but do not reproduce the original panic. Stated plainly so nobody counts them as fail-before. On this branch: `cargo test -p headroom-proxy` **498 passed, 0 failed**; `cargo clippy -p headroom-proxy --all-targets -- -D warnings` clean; `cargo fmt --all -- --check` clean. ## Real behavior proof The integration tests run the real proxy router in-process against a real local HTTP upstream and a real TLS WebSocket upstream; the traversal assertions are on the path the upstream actually received. **Not tested:** a live Bedrock or Vertex endpoint (no credentials used), the release binary's startup against a live `wss://` provider, and the `overflow-checks` performance cost (no benchmark run; expected to be small on the proxy, which is I/O bound). The Rust proxy ships only in the OSS core image, not in the enterprise wheelhouse. ## Behaviour change / release note Requests whose path contains dot-segments, encoded dot-segments or backslashes, or that would leave the configured base prefix, now get **400** instead of being normalised and forwarded. Bedrock model ids containing `/` (inference-profile ARNs) are now sent as one encoded segment, which is what the Bedrock API expects. ## Decision assumed None. Not done here from the original item: workspace-level clippy `unwrap_used`/`expect_used` warnings (left for a separate, noisier PR). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
|
96dfc7005f |
chore: finish the org and docs-domain migration (#3389)
Two stale identities left over from moves that were only half applied. ## GitHub org `install/state.py` already migrated Docker images to `ghcr.io/headroomlabs-ai/headroom` and carries a `_DEPRECATED_IMAGE_REPO` constant that rewrites the old path on load — so this move was decided long ago. Everything else still said `chopratejas/headroom`. Rewritten across packaging (`pyproject.toml`, `Cargo.toml`), both plugin marketplaces, the agent-hooks manifests, install scripts, CI templates, crate docs and tests. **Two deliberate exclusions:** - **`ghcr.io/chopratejas/headroom` is untouched everywhere.** It is the *input* to `_migrate_deprecated_image()` and to the tests covering it. Rewriting it would silently disable the migration. - **`CHANGELOG.md` keeps its 238 historical references.** GitHub's rename redirect resolves them, and rewriting would bury this diff in noise. **HuggingFace model ids are NOT affected and were never in scope.** `chopratejas/kompress-base`, `kompress-v2-base`, `kompress-finance`, `technique-router`, `technique-router-onnx` and `siglip-image-encoder-onnx` are account-scoped identifiers on a different service. All six were validated against the HF API before this change: | Model | HTTP | |---|---| | `kompress-base` | 200 | | `kompress-v2-base` | 200 | | `technique-router` | 200 | | `technique-router-onnx` | 200 | | `siglip-image-encoder-onnx` | 200 | | `kompress-finance` | **401 (private)** | The 401 is unrelated to this migration, but worth knowing: `kompress_compressor.py:1255` references it, so that path is not reachable by an OSS user without access. **One functional fix, not cosmetic:** `_marketplace_source()` (`headroom/cli/init.py`) returned the old org as its fallback, so plugin installs were pointed at the retired repo. ## Docs domain The site moved to `docs.headroomlabs.ai`, but eleven references still named the vercel.app host — including **`llms.txt` (27 links)**, which is served at `docs.headroomlabs.ai/llms.txt` and read by LLM crawlers, the **dashboard's own Documentation link**, and a `headroom wrap` help message. The docs app's canonical-URL fallbacks were fixed separately in #3388. ## Also folded in `RUST_DEV.md` corrections found while verifying the above: - `make build-wheel` points at `Cargo.toml`, not `pyproject.toml` - the Rust toolchain is pinned, not tracking `stable` - `headroom-simulators` was missing from the workspace listing - the Python `CompressionStore` defaults to **SQLite**, not in-memory, when `HEADROOM_CCR_BACKEND` is unset (verified by execution) — this one matters for multi-worker reasoning And the install scripts pointed at `docs/docker-install.md`, a path that no longer exists; they now point at the live docs page. ## Verification 297 tests pass, including the plugin-manifest, install-state and release-workflow suites that assert on these exact strings. `ruff check` clean. Unrelated lockfile churn (`uv.lock`, both `package-lock.json`) is deliberately excluded. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01HRaEPjQcCSAwD2xbQMNENu --------- Co-authored-by: Claude Opus 5 <noreply@anthropic.com> |
||
|
|
5731be7e68 |
deps: bump axum from 0.7.9 to 0.8.9 (#2966)
Bumps [axum](https://github.com/tokio-rs/axum) from 0.7.9 to 0.8.9. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/tokio-rs/axum/releases">axum's releases</a>.</em></p> <blockquote> <h2>axum-v0.8.9</h2> <ul> <li><strong>added:</strong> <code>WebSocketUpgrade::{requested_protocols, set_selected_protocol}</code> for more flexible subprotocol selection (<a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.80 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>)</li> <li><strong>fixed:</strong> Set connect endpoint on correct field in MethodRouter (<a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>)</li> <li><strong>fixed:</strong> Return specific error message when multipart body limit is exceeded (<a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3597">#3597</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3597">tokio-rs/axum#3597</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3620">#3620</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3620">tokio-rs/axum#3620</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3656">#3656</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3656">tokio-rs/axum#3656</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3611">#3611</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3611">tokio-rs/axum#3611</a></p> <h2>axum v0.8.8</h2> <ul> <li>Clarify documentation for <code>Router::route_layer</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3567">#3567</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3567">tokio-rs/axum#3567</a></p> <h2>axum v0.8.7</h2> <ul> <li>Relax implicit <code>Send</code> / <code>Sync</code> bounds on <code>RouterAsService</code>, <code>RouterIntoService</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>)</li> <li>Make it easier to visually scan for default features (<a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>)</li> <li>Fix some documentation typos</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3550">#3550</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3550">tokio-rs/axum#3550</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3555">#3555</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3555">tokio-rs/axum#3555</a></p> <h2>axum v0.8.5</h2> <ul> <li><strong>fixed:</strong> Reject JSON request bodies with trailing characters after the JSON document (<a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>)</li> <li><strong>added:</strong> Implement <code>OptionalFromRequest</code> for <code>Multipart</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>)</li> <li><strong>added:</strong> Getter methods <code>Location::{status_code, location}</code></li> <li><strong>added:</strong> Support for writing arbitrary binary data into server-sent events (<a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>)]</li> <li><strong>added:</strong> <code>middleware::ResponseAxumBodyLayer</code> for mapping response body to <code>axum::body::Body</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>)</li> <li><strong>added:</strong> <code>impl FusedStream for WebSocket</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>)</li> <li><strong>changed:</strong> The <code>sse</code> module and <code>Sse</code> type no longer depend on the <code>tokio</code> feature (<a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>)</li> <li><strong>changed:</strong> If the location given to one of <code>Redirect</code>s constructors is not a valid header value, instead of panicking on construction, the <code>IntoResponse</code> impl now returns an HTTP 500, just like <code>Json</code> does when serialization fails (<a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>)</li> <li><strong>changed:</strong> Update minimum rust version to 1.78 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>)</li> </ul> <p><a href="https://redirect.github.com/tokio-rs/axum/issues/3154">#3154</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3154">tokio-rs/axum#3154</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3220">#3220</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3220">tokio-rs/axum#3220</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3377">#3377</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3377">tokio-rs/axum#3377</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3412">#3412</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3412">tokio-rs/axum#3412</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3425">#3425</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3425">tokio-rs/axum#3425</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3443">#3443</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3443">tokio-rs/axum#3443</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3453">#3453</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3453">tokio-rs/axum#3453</a> <a href="https://redirect.github.com/tokio-rs/axum/issues/3469">#3469</a>: <a href="https://redirect.github.com/tokio-rs/axum/pull/3469">tokio-rs/axum#3469</a></p> <h2>axum v0.8.4</h2> <ul> <li><strong>added:</strong> <code>Router::reset_fallback</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3320">#3320</a>)</li> <li><strong>added:</strong> <code>WebSocketUpgrade::selected_protocol</code> (<a href="https://redirect.github.com/tokio-rs/axum/issues/3248">#3248</a>)</li> <li><strong>fixed:</strong> Panic location for overlapping method routes (<a href="https://redirect.github.com/tokio-rs/axum/issues/3319">#3319</a>)</li> <li><strong>fixed:</strong> Don't leak a tokio task when using <code>serve</code> without graceful shutdown (<a href="https://redirect.github.com/tokio-rs/axum/issues/3129">#3129</a>)</li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/tokio-rs/axum/commit/c59208c86fded335cd85e388030ad59347b0e5ae"><code>c59208c</code></a> revert axum-core changelog changes</li> <li><a href="https://github.com/tokio-rs/axum/commit/99068f5a4b309d0966777eb6e5a8ce924f204e6d"><code>99068f5</code></a> Revert "Fix <code>IntoResponse</code> for tuples overriding error response codes (<a href="https://redirect.github.com/tokio-rs/axum/issues/3603">#3603</a>)"</li> <li><a href="https://github.com/tokio-rs/axum/commit/23d7098691871ccec71ca17ea31d1d40b036c0d0"><code>23d7098</code></a> Revert "axum-core 0.5.6"</li> <li><a href="https://github.com/tokio-rs/axum/commit/e8a39ad416d1ee4f61249904309691909db2db09"><code>e8a39ad</code></a> axum-macros 0.5.1</li> <li><a href="https://github.com/tokio-rs/axum/commit/6e9a249a4fa45507b1157e570f9b6ec58d71cb86"><code>6e9a249</code></a> axum-extra 0.12.6</li> <li><a href="https://github.com/tokio-rs/axum/commit/0ec9041a1b903778a91a23558e064a83b43674c1"><code>0ec9041</code></a> axum 0.8.9</li> <li><a href="https://github.com/tokio-rs/axum/commit/c3fcebb38f356ccf96da158199d4e920aa8cfda3"><code>c3fcebb</code></a> axum-core 0.5.6</li> <li><a href="https://github.com/tokio-rs/axum/commit/a8790fc29b0db5708cdbcae70597d37c5afe1143"><code>a8790fc</code></a> update release notes</li> <li><a href="https://github.com/tokio-rs/axum/commit/26ba7bb6f21cf8996493481a5275c01152f0aaf9"><code>26ba7bb</code></a> docs: consolidate state management docs in crate root (<a href="https://redirect.github.com/tokio-rs/axum/issues/3683">#3683</a>)</li> <li><a href="https://github.com/tokio-rs/axum/commit/9fc59efc1fa9a11f4157cff1f2d22355f01d7bc0"><code>9fc59ef</code></a> Update to tokio-tungstenite 0.29 (<a href="https://redirect.github.com/tokio-rs/axum/issues/3689">#3689</a>)</li> <li>Additional commits viewable in <a href="https://github.com/tokio-rs/axum/compare/axum-v0.7.9...axum-v0.8.9">compare view</a></li> </ul> </details> <br /> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
2c9eb7c5f1 |
feat(simulators): add provider simulator service (#2014)
## Description Adds a Rust-only `headroom-simulators` workspace crate: a deterministic local upstream simulator service for Headroom proxy and pipeline validation. It supplies configurable stubs plus bottled provider-shaped responses for supported provider/path surfaces without calling real LLMs. ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `crates/headroom-simulators` Rust crate with library and `headroom-simulators` binary. - Added clean domain classification for supported surfaces: Anthropic `/v1/messages`, OpenAI chat/responses/conversations, Bedrock invoke/stream routes, Vertex raw/stream predict, health, and generic fallback. - Added JSON-configured stub matching by method, path, body substring, and JSON pointer. - Added bottled provider-shaped JSON, SSE, and Bedrock EventStream responses for unconfigured requests. - Added a container `Dockerfile` and README for local/GitHub Actions usage. - Added unit and HTTP integration tests for defaults, configured stubs, SSE, Vertex, and Bedrock EventStream behavior. - Added proxy-level simulator-backed E2E tests that run Headroom against the simulator across Anthropic, OpenAI Chat, OpenAI Responses, OpenAI Conversations, Bedrock invoke/converse/streaming, Vertex raw/stream predict, and upstream health. - Added simulator-backed provider error-path E2E coverage for OpenAI 429, Anthropic 529, Bedrock 502, and Vertex 503 responses flowing through Headroom unchanged. - Added Headroom-owned preflight error E2E coverage proving Bedrock missing credentials and invalid Vertex envelopes stop inside the proxy instead of silently falling through to the simulator/provider. - Fixed direct Rust `headroom-core` binaries/tests on Windows so Magika initializes ONNX Runtime via `ort::init_from` from an explicit pip `onnxruntime` library path, with fail-fast fallback only when no safe runtime is discoverable. - Added a Rust CI `simulator-e2e` matrix for `ubuntu-latest`, `macos-latest`, and `windows-latest` that runs `cargo test -p headroom-proxy --test e2e_simulators`. - Gated dynamic Magika `Path`/`PathBuf` imports to Windows and x86_64 macOS so Linux clippy does not see unused dynamic-ORT-only imports. ## Testing - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output cargo fmt --all -- --check # passed cargo clippy --workspace -- -D warnings # passed $env:ORT_DYLIB_PATH=$null cargo test -p headroom-core transforms::magika_detector::tests:: --lib # 17 passed, 0 failed; Magika initialized from discovered pip onnxruntime DLL $env:ORT_DYLIB_PATH=$null cargo test --workspace # passed gitleaks protect --staged --no-banner --redact # no leaks found gitleaks git --log-opts="headroomlabs/main..HEAD" --no-banner --redact # 5 commits scanned; no leaks found ## Real Behavior Proof - **Environment:** Windows PowerShell, Rust toolchain `1.95.0`, clean worktree from `headroomlabs/main` at `9bacf481`. - **Exact simulator command / steps:** - `cargo run -p headroom-simulators -- --listen 127.0.0.1:8789` - Point Headroom proxy upstream at `http://127.0.0.1:8789` for local deterministic provider responses. - Use optional `--config path/to/simulator.json` to bind exact request fixtures. - **Observed simulator result:** - OpenAI chat default returns `chat.completion` shape. - OpenAI Responses stream returns named SSE events. - Vertex raw predict returns Anthropic message shape. - Bedrock stream can return binary `application/vnd.amazon.eventstream` bytes. - Configured stubs override bottled defaults. - **Observed Magika result:** - Direct Rust `headroom-core` tests pass with `ORT_DYLIB_PATH` unset. - Magika discovers the installed pip `onnxruntime.dll`, loads it via `ort::init_from`, and only falls back if no safe runtime is available. - **Not tested:** - No live provider calls; simulator behavior is intentionally offline and deterministic. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable ## Screenshots (if applicable) N/A ## Additional Notes No CHANGELOG entry was added because this introduces a developer/CI simulator crate plus a Windows direct-Rust Magika runtime fix, without changing shipped Python package behavior. The simulator intentionally does not include a lightweight fallback LLM in this slice; unbound inputs receive deterministic bottled responses so tests stay reproducible and offline. |
||
|
|
e448d7ba4d |
deps: bump thiserror from 1.0.69 to 2.0.18 (#1519)
Bumps [thiserror](https://github.com/dtolnay/thiserror) from 1.0.69 to 2.0.18. <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/dtolnay/thiserror/releases">thiserror's releases</a>.</em></p> <blockquote> <h2>2.0.18</h2> <ul> <li>Make compatible with project-level <code>needless_lifetimes = "forbid"</code> (<a href="https://redirect.github.com/dtolnay/thiserror/issues/443">#443</a>, thanks <a href="https://github.com/LucaCappelletti94"><code>@LucaCappelletti94</code></a>)</li> </ul> <h2>2.0.17</h2> <ul> <li>Use differently named __private module per patch release (<a href="https://redirect.github.com/dtolnay/thiserror/issues/434">#434</a>)</li> </ul> <h2>2.0.16</h2> <ul> <li>Add to "no-std" crates.io category (<a href="https://redirect.github.com/dtolnay/thiserror/issues/429">#429</a>)</li> </ul> <h2>2.0.15</h2> <ul> <li>Prevent <code>Error::provide</code> API becoming unavailable from a future new compiler lint (<a href="https://redirect.github.com/dtolnay/thiserror/issues/427">#427</a>)</li> </ul> <h2>2.0.14</h2> <ul> <li>Allow build-script cleanup failure with NFSv3 output directory to be non-fatal (<a href="https://redirect.github.com/dtolnay/thiserror/issues/426">#426</a>)</li> </ul> <h2>2.0.13</h2> <ul> <li>Documentation improvements</li> </ul> <h2>2.0.12</h2> <ul> <li>Prevent elidable_lifetime_names pedantic clippy lint in generated impl (<a href="https://redirect.github.com/dtolnay/thiserror/issues/413">#413</a>)</li> </ul> <h2>2.0.11</h2> <ul> <li>Add feature gate to tests that use std (<a href="https://redirect.github.com/dtolnay/thiserror/issues/409">#409</a>, <a href="https://redirect.github.com/dtolnay/thiserror/issues/410">#410</a>, thanks <a href="https://github.com/Maytha8"><code>@Maytha8</code></a>)</li> </ul> <h2>2.0.10</h2> <ul> <li>Support errors containing a generic type parameter's associated type in a field (<a href="https://redirect.github.com/dtolnay/thiserror/issues/408">#408</a>)</li> </ul> <h2>2.0.9</h2> <ul> <li>Work around <code>missing_inline_in_public_items</code> clippy restriction being triggered in macro-generated code (<a href="https://redirect.github.com/dtolnay/thiserror/issues/404">#404</a>)</li> </ul> <h2>2.0.8</h2> <ul> <li>Improve support for macro-generated <code>derive(Error)</code> call sites (<a href="https://redirect.github.com/dtolnay/thiserror/issues/399">#399</a>)</li> </ul> <h2>2.0.7</h2> <ul> <li>Work around conflict with #[deny(clippy::allow_attributes)] (<a href="https://redirect.github.com/dtolnay/thiserror/issues/397">#397</a>, thanks <a href="https://github.com/zertosh"><code>@zertosh</code></a>)</li> </ul> <h2>2.0.6</h2> <ul> <li>Suppress deprecation warning on generated From impls (<a href="https://redirect.github.com/dtolnay/thiserror/issues/396">#396</a>)</li> </ul> <h2>2.0.5</h2> <ul> <li>Prevent deprecation warning on generated impl for deprecated type (<a href="https://redirect.github.com/dtolnay/thiserror/issues/394">#394</a>)</li> </ul> <h2>2.0.4</h2> <ul> <li>Eliminate needless_lifetimes clippy lint in generated <code>From</code> impls (<a href="https://redirect.github.com/dtolnay/thiserror/issues/391">#391</a>, thanks <a href="https://github.com/matt-phylum"><code>@matt-phylum</code></a>)</li> </ul> <h2>2.0.3</h2> <ul> <li>Support the same Path field being repeated in both Debug and Display representation in error message (<a href="https://redirect.github.com/dtolnay/thiserror/issues/383">#383</a>)</li> <li>Improve error message when a format trait used in error message is not implemented by some field (<a href="https://redirect.github.com/dtolnay/thiserror/issues/384">#384</a>)</li> </ul> <h2>2.0.2</h2> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/dtolnay/thiserror/commit/dc0f6a23a3fb6ae34ef117133ec43650450c4b32"><code>dc0f6a2</code></a> Release 2.0.18</li> <li><a href="https://github.com/dtolnay/thiserror/commit/027529273572e8171262231190c4328e40ec1b36"><code>0275292</code></a> Touch up PR 443</li> <li><a href="https://github.com/dtolnay/thiserror/commit/3c33bc60add478225e6bbd19e9974a633da08545"><code>3c33bc6</code></a> Merge pull request <a href="https://redirect.github.com/dtolnay/thiserror/issues/443">#443</a> from LucaCappelletti94/master</li> <li><a href="https://github.com/dtolnay/thiserror/commit/995939cc2eae15cc23f873f354f8ef3fbfd6a707"><code>995939c</code></a> Reproduce issue 442</li> <li><a href="https://github.com/dtolnay/thiserror/commit/21653d1d33f563f0b8f8881671894a23cc3a2a27"><code>21653d1</code></a> Made clippy lifetime allows conditional</li> <li><a href="https://github.com/dtolnay/thiserror/commit/45e5388009e68c32f20182db21ee5887bc1c3304"><code>45e5388</code></a> Update actions/upload-artifact@v5 -> v6</li> <li><a href="https://github.com/dtolnay/thiserror/commit/386aac126a95a97fae374c05bd2ebb1dd550a361"><code>386aac1</code></a> Update actions/upload-artifact@v4 -> v5</li> <li><a href="https://github.com/dtolnay/thiserror/commit/ec50561375fcd29d700da43a79f4f5d9794e5b4b"><code>ec50561</code></a> Update actions/checkout@v5 -> v6</li> <li><a href="https://github.com/dtolnay/thiserror/commit/247eab5d79e27ad28859afdf8bc600a4242829b7"><code>247eab5</code></a> Update name of empty_enum clippy lint</li> <li><a href="https://github.com/dtolnay/thiserror/commit/91b181f0899fd42f41c210e73822c29eef29dd6d"><code>91b181f</code></a> Raise required compiler to Rust 1.68</li> <li>Additional commits viewable in <a href="https://github.com/dtolnay/thiserror/compare/1.0.69...2.0.18">compare view</a></li> </ul> </details> <br /> [](https://docs.github.com/en/github/managing-security-vulnerabilities/about-dependabot-security-updates#about-compatibility-scores) Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting `@dependabot rebase`. [//]: # (dependabot-automerge-start) [//]: # (dependabot-automerge-end) --- <details> <summary>Dependabot commands and options</summary> <br /> You can trigger Dependabot actions by commenting on this PR: - `@dependabot rebase` will rebase this PR - `@dependabot recreate` will recreate this PR, overwriting any edits that have been made to it - `@dependabot show <dependency name> ignore conditions` will show all of the ignore conditions of the specified dependency - `@dependabot ignore this major version` will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this minor version` will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself) - `@dependabot ignore this dependency` will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself) </details> Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> |
||
|
|
5771a8020e |
fix(deps): remediate dependency CVEs and publish SBOM (#1509)
## Description
Supply-chain hardening: takes the **shipped** dependency surface from
**26 known CVEs to 0**. `pip install headroom-ai[all]` now resolves with
no known vulnerabilities (verified with Anchore syft + grype). Also
publishes a checked-in SBOM package (`sbom/`) so any user — especially
pilots running their own security review — can verify what's inside and
that we track it.
This addresses the Dependabot alerts on `main` (9 high / 4 moderate / 7
low at time of writing).
Closes #
## Type of Change
- [x] Bug fix (non-breaking change that fixes an issue)
- [ ] New feature (non-breaking change that adds functionality)
- [ ] Breaking change (fix or feature that would cause existing
functionality to change)
- [x] Documentation update
- [ ] Performance improvement
- [ ] Code refactoring (no functional changes)
## Changes Made
**Rust**
- `pyo3` 0.24 → 0.29 (GHSA-36hh-v3qg-5jq4 High, GHSA-chgr-c6px-7xpp
Med). Migrated `Python::allow_threads` → `Python::detach` (10 sites) and
added `from_py_object` to the `Clone`-deriving `#[pyclass]` types (both
required by the 0.25+ API).
- `pyo3-log` 0.12 → 0.13; `lru` 0.12 → 0.18 (GHSA-rhfx-m35p-ff5j).
**Python**
- `torch` → 2.12.1, `mem0ai` → 2.x.
- Floor-pinned transitive CVE deps via `[tool.uv]
constraint-dependencies`: `pygments>=2.20.0`,
`pydantic-settings>=2.14.2`, `gitpython>=3.1.50`, `langsmith>=0.9.0`.
- **Removed `benchmark` from the `[all]` aggregate** so the default
install is CVE-free. `lm-eval` is invoked as an external subprocess
(`python -m lm_eval`) and never imported, so it is not a true runtime
dep — it remains available via the opt-in `[benchmark]` extra. See
[Accepted Risks](#additional-notes).
**npm (build/test tooling — never shipped in the
wheel/container/published SDK)**
- `esbuild` override `>=0.28.1` in `sdk/typescript` + `plugins/openclaw`
(GHSA-g7r4-m6w7-qqqr).
- `docs/`: `@anthropic-ai/sdk` → `^0.106.0` (GHSA-p7fg-763f-g4gf),
`postcss` override to force Next.js's bundled copy ≥8.5.10
(GHSA-qx2v-qp2m-jg93); regenerated a stale `bun.lock` that carried a
**Critical** vitest/vite.
**CI**
- Pinned `pypa/gh-action-pypi-publish` `@release/v1` → `@v1.13.0`
(GHSA-vxmw-7h4f-hqxh) in `release.yml` + `publish.yml`.
**SBOM**
- New `sbom/` directory: CycloneDX 1.7 + SPDX 2.3 SBOMs, grype scan
evidence, 330-package license inventory, and a regeneration guide.
## Testing
- [ ] Unit tests pass (`pytest`) — N/A, no Python source changed
(deps/config only)
- [x] Linting passes — `cargo fmt --check` + `cargo clippy` clean on the
changed crate; 0 `.py` files changed so `ruff`/`mypy` scope is
unaffected
- [x] Type checking passes — `cargo check --workspace` (0 errors)
- [ ] New tests added — N/A (dependency bumps; covered by existing
suites)
- [x] Manual testing performed — see Real Behavior Proof
### Test Output
```text
# headroom-ai[all] product surface — the number that matters
$ grype sbom:sbom/headroom-sbom-all-extra.cdx.json
No vulnerabilities found
# full repo scan (universal lock incl. opt-in [benchmark] + dev)
$ grype sbom:sbom/headroom-sbom.cdx.json
NAME INSTALLED TYPE VULNERABILITY SEVERITY
sqlitedict 2.1.0 python GHSA-g4r7-86gm-pgqc High # [benchmark]-only, unpatchable, accepted
nltk 3.9.4 python GHSA-p4gq-832x-fm9v High # [benchmark]-only, unpatchable, accepted
# pyo3 0.29 migration — extension builds + imports + runs
$ cargo check --workspace
Finished `dev` profile [unoptimized + debuginfo] target(s)
$ maturin develop && python -c "from headroom._core import DiffCompressor, SmartCrusher; ..."
extension OK — detach + from_py_object paths exercised
# lru 0.18 — eviction path
$ cargo test -p headroom-proxy --lib drift
14 passed, 213 filtered out
# per-ecosystem npm audits
$ (cd sdk/typescript && npm audit) -> found 0 vulnerabilities
$ (cd plugins/openclaw && npm audit) -> found 0 vulnerabilities
$ (cd docs && npm audit && bun audit) -> found 0 vulnerabilities / No vulnerabilities found
```
## Real Behavior Proof
- Environment: macOS (darwin 25.4.0, arm64), Python 3.12 `.venv`, Rust
1.95 toolchain, syft 1.46.0, grype 0.115.0, bun 1.3.14, maturin 1.13.3.
- Exact command / steps: (1) `uv export --extra all --no-dev
--no-emit-project | syft → grype` for the product surface; (2) `cargo
check --workspace` + `maturin develop` + extension import/compress smoke
test; (3) `cargo test -p headroom-proxy --lib drift`; (4) `cargo fmt
--check` + `cargo clippy -p headroom-py`; (5) `npm audit` in
sdk/openclaw/docs + `bun audit` in docs.
- Observed result: `headroom-ai[all]` resolution scans clean — "No
vulnerabilities found" (179 pkgs); full/prod SBOM shows only the 2
documented accepted CVEs; pyo3 0.29 extension imports and runs (detach +
from_py_object paths exercised); drift tests 14/14 pass; cargo fmt +
clippy clean; all npm/bun audits report 0.
- Not tested: full `pytest` suite (no Python source changed);
release-profile wheel build (used dev-profile `maturin develop` for the
import proof — the extension is semantically identical).
## Review Readiness
- [x] I have performed a self-review
- [x] This PR is ready for human review
## Checklist
- [x] My code follows the project's style guidelines
- [x] I have performed a self-review of my code
- [x] I have commented my code, particularly in hard-to-understand areas
- [x] I have made corresponding changes to the documentation
(`sbom/README.md`)
- [x] My changes generate no new warnings
- [ ] I have added tests that prove my fix is effective — N/A
(dependency bumps; existing suites + scans cover it)
- [x] New and existing unit tests pass locally with my changes
- [ ] I have updated the CHANGELOG.md — N/A (Release Please
auto-generates from the conventional commit)
## Additional Notes
**Accepted risks (the 2 residual CVEs).** Both originate solely from the
EleutherAI `lm-evaluation-harness` under the **opt-in `[benchmark]`
extra**, which Headroom invokes as a subprocess (never imports):
- `sqlitedict` CVE-2024-35515 (High) — pickle deserialization; package
abandoned (last release 2021), **no upstream fix exists**.
- `nltk` CVE-2026-54293 (High) — path traversal in `nltk.data.load()`;
affects ≤3.9.4 (current latest), **no patched release**.
Neither is in `[all]`, the published wheel, or the container. They are
documented in `sbom/README.md` and will be picked up automatically once
upstream ships fixes.
**Release/CHANGELOG:** N/A items above are because this is a
dependency/security PR with no Python source changes; CHANGELOG is
Release-Please-managed via the conventional commit message.
|
||
|
|
3ccdad6c67 |
Pin ORT dylib on Windows; init Python logging (#1010)
## Description On Windows, headroom's Rust core resolves `onnxruntime.dll` at runtime via `ort-load-dynamic`. Without an explicit `ORT_DYLIB_PATH`, the bare DLL search can land on `C:\Windows\System32\onnxruntime.dll`, the Windows ML OS component, and `Session::new()` can deadlock instead of returning an error. Since a hang is not an `Err`, the tiered fallback cannot engage until the proxy-level timeout fires. This PR pins `ORT_DYLIB_PATH` to the pip-installed `onnxruntime` DLL at import time, and wires Rust `tracing` events into Python logging so the proxy log surfaces these failures when they occur. Closes #928 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [x] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Added `headroom/_ort.py` with a Windows-only, idempotent `ensure_ort_dylib_pinned()` resolver that respects an existing `ORT_DYLIB_PATH`. - Call the pin from `headroom/__init__.py` before importing `_core` consumers. - Log the effective ORT dylib path from the content router startup path on Windows. - Enable Rust tracing-to-log compatibility and initialize `pyo3-log` in the `_core` module. - Add timeout diagnostics in the Magika detector with the effective `ORT_DYLIB_PATH`. - Document `ORT_DYLIB_PATH` and `HEADROOM_MAGIKA_INIT_TIMEOUT_SECS`. - Add unit coverage for the resolver behavior. ## Testing - [x] Unit tests pass (`python -m pytest tests/test_transforms/test_ort_dylib.py -q`) - [x] Linting passes (`ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] Formatting passes (`ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py`) - [x] New tests added for new functionality - [x] Manual testing performed ### Test Output ```text $ python -m pytest tests/test_transforms/test_ort_dylib.py -q 7 passed in 0.19s $ ruff check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py All checks passed! $ ruff format --check headroom/_ort.py headroom/__init__.py headroom/transforms/content_router.py tests/test_transforms/test_ort_dylib.py 4 files already formatted $ cargo check -p headroom-py cargo: The term 'cargo' is not recognized as a name of a cmdlet, function, script file, or executable program. ``` ## Real Behavior Proof - Environment: Windows 11 24H2, Python 3.13, RTX 4080 - Exact command / steps: `python -c "import headroom; from headroom._core import detect_content_type as d; print(d(open('headroom/compress.py').read()).content_type)"` - Observed result: `source_code` in 301ms, clean exit, `Magika: ENABLED` in proxy log - Not tested: macOS/Linux manual runtime behavior; `_ort.py` is a no-op outside Windows, and CI covers cross-platform build/test behavior. ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable (N/A: repo uses release-please) ## Additional Notes The branch was rebased onto current `main` and the commit subject was updated to satisfy commitlint. Local Rust verification could not be run on this Windows machine because `cargo` is not installed; GitHub CI should be treated as the Rust build verification for the `pyo3-log` dependency and workspace lockfile changes. --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> |
||
|
|
0dc2e1cb3f |
feat(bedrock): cross-region + Converse compression; bundle proxy binary in images (#999)
## Description The native Bedrock path (Phase D) compresses + signs Anthropic-on-Bedrock requests, but two real-world cases slipped through, and the native binary that powers it was never shipped. This PR closes those gaps as a focused set of give-backs. Aligns with the Rust migration plan (see below). ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [x] New feature (non-breaking change that adds functionality) - [ ] Breaking change - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - **Cross-region inference-profile detection** via a new `bedrock::vendor` module (`canonical_vendor()`), following the design proposed in #953: strip a known geo prefix (`eu.`/`us.`/`apac.`/`global.`) then match the canonical vendor. Geo-prefixed Anthropic profiles (`eu.anthropic.…`) now get live-zone compression instead of being silently skipped; geo-prefixed non-Anthropic vendors stay correctly excluded. - **Converse-body compression (two parts)**: 1. `run_anthropic_compression` no longer bails to passthrough when the body lacks an InvokeModel `anthropic_version` envelope; envelope re-emit stays gated on successful parse. 2. The **live-zone dispatcher now recognizes Bedrock Converse content blocks**. Converse blocks carry no `type` discriminator (the variant is the key: `{"text": …}` vs Anthropic's `{"type":"text","text":…}`), so real Converse user-message text was still passing through uncompressed. A typeless block whose `text` is a JSON string now routes through the same surgical text path. Anthropic blocks always carry `type`, so the Anthropic path is byte-for-byte unchanged; non-text Converse blocks (`{"image":…}`, `{"toolUse":…}`) stay unrecognized and no-op. - **Correct `/converse` upstream routing**: the non-streaming handler resolved the upstream action from a hard-coded `"invoke"`, so `/converse` requests were forwarded to Bedrock's `/invoke` endpoint. It now resolves the action from the inbound path (`extract_invoke_action`), mirroring the streaming handler's `extract_streaming_action`. SigV4 signs the same URL it forwards, so the signature stays consistent. - **`aws-config` `sso` feature**: SSO profiles now resolve through the default credential chain for SigV4 — the credential chain in `docs/bedrock.md` already promised SSO; this makes the code match. - **Ship the `headroom-proxy` binary in published images** (`Dockerfile`): built in the builder stage (`--locked`, with the cargo registry cache mounted at `CARGO_HOME`) and copied into both the debian and distroless runtime images. - **Docs** (`docs/bedrock.md`): document cross-region inference profiles and a "Running the proxy" section. AWS credentials mount at `/home/nonroot/.aws` (the default nonroot image home) where the SDK looks for `~/.aws`, with a note on the root-image alternative. ## Related issues - Closes #976 — ship the `headroom-proxy` binary in published images (this PR implements the exact fix proposed there). - Addresses the **cross-region inference-profile** half of #953 via its proposed `canonical_vendor()` design. Non-Anthropic vendor compression parity (Nova/GLM/MiniMax/ Kimi) is the natural follow-up — `bedrock::vendor` is the shared resolver it can build on. - Extends the native Bedrock InvokeModel compression requested in #734 (the Bedrock slice of #510) to cross-region profiles and Converse bodies. - Partially enables #181 (native, Python-free packaging): the native binary now ships in the images, though full Python-free distribution remains out of scope. ## Alignment with the Rust migration plan Per `docs/spec/022-rust-migration.md`, the migration is **proxy-first**: `headroom-proxy` is the deployable Rust artifact, native routes replace Python passthroughs one at a time (Stage 4 = provider expansion, Bedrock included), and the binary is meant to be "built, tested, and **released together with the Python package**." Two ways this PR advances that: - The binary-in-images change makes the codebase do what the spec already states (ship the artifact) — closing the gap that forced downstreams to build from source. - Hardening the native Bedrock route (cross-region, Converse routing + body compression) is exactly the Stage-4 provider-expansion work, keeping the native path at parity with real traffic so it can be the default rather than a passthrough. ## Testing - [x] Unit tests pass (`cargo test -p headroom-core -p headroom-proxy` — full suites, 0 failures) - [x] Linting passes (`cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings`) - [x] Formatting passes (`cargo fmt -- --check`) - [x] New tests added — `bedrock::vendor` (foundation + inference-profile matching), `extract_invoke_action` + converse upstream URL, and live-zone Converse text-block routing (`block_has_string_text_field`, converse-vs-anthropic dispatch equivalence). - [x] Manual testing performed ### Test Output ```text $ cargo test -p headroom-core -p headroom-proxy # all suites: ok, 0 failed $ cargo clippy -p headroom-core -p headroom-proxy --all-targets -- -D warnings # Finished, no warnings $ cargo fmt -- --check # clean # image validation (local, proxy/code extras): $ docker build --target runtime ... # debian: /usr/local/bin/headroom-proxy, --help OK $ docker build --target runtime-slim ... # distroless: binary links + --help OK ``` ## Real Behavior Proof - Environment: native Bedrock proxy against `bedrock-runtime.eu-west-2`, SSO profile, model `eu.anthropic.claude-haiku-4-5-20251001-v1:0`. - Exact command / steps: POST a large multi-turn Converse body to `/model/eu.anthropic.claude-haiku-4-5-20251001-v1:0/converse`; separately build the `runtime` + `runtime-slim` targets and run `/usr/local/bin/headroom-proxy --help`. - Observed result: before — `bedrock_compression_skipped` (geo-prefixed id not recognized), forwarded uncompressed to the wrong `/invoke` upstream; after — geo-prefixed id recognized, `/converse` forwarded to the `/converse` upstream, live-zone dispatcher compresses the Converse user-message text, measurable token savings. Images contain a runnable `headroom-proxy` in both variants. - Not tested: non-Anthropic vendor compression parity (#953 follow-up); Converse `toolResult` nested-text compression (follow-up — only top-level Converse text blocks compress today). ## Review Readiness - [x] I have performed a self-review - [x] This PR is ready for human review ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [x] I have commented my code, particularly in hard-to-understand areas - [x] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [x] I have added tests that prove my fix is effective - [x] New and existing unit tests pass locally with my changes - [x] I have updated the CHANGELOG.md if applicable ## Additional Notes - An earlier revision flipped the EventStream `Accept` default (`*/*`/absent → passthrough); **dropped** — `*/*` is what most clients (incl. reqwest and the proxy's own metrics tests) send while expecting SSE, so forcing passthrough breaks the standard SSE path. - The binary build adds the native-proxy compile to the image build; happy to gate it behind a build arg if maintainers prefer it opt-in. - Addressed a Copilot review round: corrected the `/converse` upstream routing, the stale `run_anthropic_compression` comment, the Dockerfile cargo cache mount + `--locked`, and the nonroot AWS-credentials docs example. |
||
|
|
4ff7b4426d |
ci: bump pyo3 from 0.22.6 to 0.24.1 in the cargo group across 1 directory (#270)
Bumps the cargo group with 1 update in the / directory: [pyo3](https://github.com/pyo3/pyo3). Updates `pyo3` from 0.22.6 to 0.24.1 <details> <summary>Release notes</summary> <p><em>Sourced from <a href="https://github.com/pyo3/pyo3/releases">pyo3's releases</a>.</em></p> <blockquote> <h2>PyO3 0.24.1</h2> <p>This release is a security fix for the <code>PyString::from_object</code> method, which passed <code>&str</code> data to the Python C API without checking for a terminating nul byte. All historical PyO3 versions are affected, and we recommend you upgrade if you are using <code>PyString::from_object</code>. Thank you to <a href="https://github.com/vthib"><code>@vthib</code></a> for the report and <a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a> for the fix. A RUSTSEC advisory will be published shortly.</p> <p>Aside from the security fix, this release contains a number of other non-breaking additions:</p> <ul> <li>An <code>abi3-py313</code> feature to support compiling with the Python 3.13 stable ABI.</li> <li><code>PyAnyMethods::getattr_opt</code> to get optional attributes without paying the cost of a Python exception when the attribute in question does not exist.</li> <li>Constructor for <code>PyInt::new</code>.</li> <li><code>with_critical_section2</code> for locking two objects at the same time on the free-threaded build.</li> <li>Fix for a PyO3 0.24.0 regression with <code>Option<&str></code> and <code>Option<&T></code> (where <code>T: PyClass</code>) function arguments no longer being permitted</li> </ul> <p>There are also a few other small bug fixes for edge cases, mostly related to compile errors from PyO3's macro code.</p> <p>Thank you to the following contributors for the improvements:</p> <p><a href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a> <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a> <a href="https://github.com/Dr-Emann"><code>@Dr-Emann</code></a> <a href="https://github.com/emmagordon"><code>@emmagordon</code></a> <a href="https://github.com/epontan"><code>@epontan</code></a> <a href="https://github.com/Icxolu"><code>@Icxolu</code></a> <a href="https://github.com/IvanIsCoding"><code>@IvanIsCoding</code></a> <a href="https://github.com/jelmer"><code>@jelmer</code></a> <a href="https://github.com/jonaspleyer"><code>@jonaspleyer</code></a> <a href="https://github.com/ngoldbaum"><code>@ngoldbaum</code></a> <a href="https://github.com/Owen-CH-Leung"><code>@Owen-CH-Leung</code></a> <a href="https://github.com/Tpt"><code>@Tpt</code></a> <a href="https://github.com/Trolldemorted"><code>@Trolldemorted</code></a> <a href="https://github.com/XuehaiPan"><code>@XuehaiPan</code></a></p> <h2>PyO3 0.24.0</h2> <p>This release is an incremental improvement of refinements and optimizations following the new APIs established in PyO3's last few releases.</p> <p>Support for <code>jiff</code> datetime conversions have been added, and also UUID conversions.</p> <p>The <code>FromPyObject</code> derive macro has gained new <code>#[pyo3(default = ...)]</code> and <code>#[pyo3(rename_all = ...)]</code> options, and the <code>IntoPyObject</code> derive macro has gained a new <code>#[pyo3(into_py_with = ...)]</code> option.</p> <p>PyO3 will now pass positional arguments to Python functions using the "vectorcall" protocol in many cases, which should be an optimization over the previous behaviour (of creating a Python tuple of positional arguments).</p> <p>Many methods on iterators of Python collections have been optimized.</p> <p>There are also many other incremental improvements, bug fixes and smaller features.</p> <p>Thank you to everyone who contributed code, documentation, design ideas, bug reports, and feedback. The following contributors' commits are included in this release:</p> <p><a href="https://github.com/0x676e67"><code>@0x676e67</code></a> <a href="https://github.com/alex"><code>@alex</code></a> <a href="https://github.com/arielb1"><code>@arielb1</code></a> <a href="https://github.com/bschoenmaeckers"><code>@bschoenmaeckers</code></a> <a href="https://github.com/davidhewitt"><code>@davidhewitt</code></a></p> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Changelog</summary> <p><em>Sourced from <a href="https://github.com/PyO3/pyo3/blob/main/CHANGELOG.md">pyo3's changelog</a>.</em></p> <blockquote> <h2>[0.24.1] - 2025-03-31</h2> <h3>Added</h3> <ul> <li>Add <code>abi3-py313</code> feature. <a href="https://redirect.github.com/PyO3/pyo3/pull/4969">#4969</a></li> <li>Add <code>PyAnyMethods::getattr_opt</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4978">#4978</a></li> <li>Add <code>PyInt::new</code> constructor for all supported number types (i32, u32, i64, u64, isize, usize). <a href="https://redirect.github.com/PyO3/pyo3/pull/4984">#4984</a></li> <li>Add <code>pyo3::sync::with_critical_section2</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4992">#4992</a></li> <li>Implement <code>PyCallArgs</code> for <code>Borrowed<'_, 'py, PyTuple></code>, <code>&Bound<'py, PyTuple></code>, and <code>&Py<PyTuple></code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/5013">#5013</a></li> </ul> <h3>Fixed</h3> <ul> <li>Fix <code>is_type_of</code> for native types not using same specialized check as <code>is_type_of_bound</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4981">#4981</a></li> <li>Fix <code>Probe</code> class naming issue with <code>#[pymethods]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4988">#4988</a></li> <li>Fix compile failure with required <code>#[pyfunction]</code> arguments taking <code>Option<&str></code> and <code>Option<&T></code> (for <code>#[pyclass]</code> types). <a href="https://redirect.github.com/PyO3/pyo3/pull/5002">#5002</a></li> <li>Fix <code>PyString::from_object</code> causing of bounds reads with <code>encoding</code> and <code>errors</code> parameters which are not nul-terminated. <a href="https://redirect.github.com/PyO3/pyo3/pull/5008">#5008</a></li> <li>Fix compile error when additional options follow after <code>crate</code> for <code>#[pyfunction]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/5015">#5015</a></li> </ul> <h2>[0.24.0] - 2025-03-09</h2> <h3>Packaging</h3> <ul> <li>Add supported CPython/PyPy versions to cargo package metadata. <a href="https://redirect.github.com/PyO3/pyo3/pull/4756">#4756</a></li> <li>Bump <code>target-lexicon</code> dependency to 0.13. <a href="https://redirect.github.com/PyO3/pyo3/pull/4822">#4822</a></li> <li>Add optional <code>jiff</code> dependency to add conversions for <code>jiff</code> datetime types. <a href="https://redirect.github.com/PyO3/pyo3/pull/4823">#4823</a></li> <li>Add optional <code>uuid</code> dependency to add conversions for <code>uuid::Uuid</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4864">#4864</a></li> <li>Bump minimum supported <code>inventory</code> version to 0.3.5. <a href="https://redirect.github.com/PyO3/pyo3/pull/4954">#4954</a></li> </ul> <h3>Added</h3> <ul> <li>Add <code>PyIterator::send</code> method to allow sending values into a python generator. <a href="https://redirect.github.com/PyO3/pyo3/pull/4746">#4746</a></li> <li>Add <code>PyCallArgs</code> trait for passing arguments into the Python calling protocol. This enabled using a faster calling convention for certain types, improving performance. <a href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li> <li>Add <code>#[pyo3(default = ...']</code> option for <code>#[derive(FromPyObject)]</code> to set a default value for extracted fields of named structs. <a href="https://redirect.github.com/PyO3/pyo3/pull/4829">#4829</a></li> <li>Add <code>#[pyo3(into_py_with = ...)]</code> option for <code>#[derive(IntoPyObject, IntoPyObjectRef)]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4850">#4850</a></li> <li>Add FFI definitions <code>PyThreadState_GetFrame</code> and <code>PyFrame_GetBack</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4866">#4866</a></li> <li>Optimize <code>last</code> for <code>BoundListIterator</code>, <code>BoundTupleIterator</code> and <code>BorrowedTupleIterator</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li> <li>Optimize <code>Iterator::count()</code> for <code>PyDict</code>, <code>PyList</code>, <code>PyTuple</code> & <code>PySet</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4878">#4878</a></li> <li>Optimize <code>nth</code>, <code>nth_back</code>, <code>advance_by</code> and <code>advance_back_by</code> for <code>BoundTupleIterator</code> <a href="https://redirect.github.com/PyO3/pyo3/pull/4897">#4897</a></li> <li>Add support for <code>types.GenericAlias</code> as <code>pyo3::types::PyGenericAlias</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4917">#4917</a></li> <li>Add <code>MutextExt</code> trait to help avoid deadlocks with the GIL while locking a <code>std::sync::Mutex</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4934">#4934</a></li> <li>Add <code>#[pyo3(rename_all = "...")]</code> option for <code>#[derive(FromPyObject)]</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4941">#4941</a></li> </ul> <h3>Changed</h3> <ul> <li>Optimize <code>nth</code>, <code>nth_back</code>, <code>advance_by</code> and <code>advance_back_by</code> for <code>BoundListIterator</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4810">#4810</a></li> <li>Use <code>DerefToPyAny</code> in blanket implementations of <code>From<Py<T>></code> and <code>From<Bound<'py, T>></code> for <code>PyObject</code>. <a href="https://redirect.github.com/PyO3/pyo3/pull/4593">#4593</a></li> <li>Map <code>io::ErrorKind::IsADirectory</code>/<code>NotADirectory</code> to the corresponding Python exception on Rust 1.83+. <a href="https://redirect.github.com/PyO3/pyo3/pull/4747">#4747</a></li> <li><code>PyAnyMethods::call</code> and friends now require <code>PyCallArgs</code> for their positional arguments. <a href="https://redirect.github.com/PyO3/pyo3/pull/4768">#4768</a></li> <li>Expose FFI definitions for <code>PyObject_Vectorcall(Method)</code> on the stable abi on 3.12+. <a href="https://redirect.github.com/PyO3/pyo3/pull/4853">#4853</a></li> <li><code>#[pyo3(from_py_with = ...)]</code> now take a path rather than a string literal <a href="https://redirect.github.com/PyO3/pyo3/pull/4860">#4860</a></li> </ul> <!-- raw HTML omitted --> </blockquote> <p>... (truncated)</p> </details> <details> <summary>Commits</summary> <ul> <li><a href="https://github.com/PyO3/pyo3/commit/a213b368bd5bf859c2acb655bfed029e17c3b447"><code>a213b36</code></a> release: 0.24.1 (<a href="https://redirect.github.com/pyo3/pyo3/issues/5021">#5021</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/d85a02d9b11f7c057e3627a0393d5d9b876dbc0a"><code>d85a02d</code></a> split <code>PyFunctionArgument</code> to specialize <code>Option</code> (<a href="https://redirect.github.com/pyo3/pyo3/issues/5002">#5002</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/c37a50a7a33e145f6bb87f40cb89cf85f9e5fac7"><code>c37a50a</code></a> Add example of more complex exceptions (<a href="https://redirect.github.com/pyo3/pyo3/issues/5014">#5014</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/dcacb9bbbc8c130238bd88480fc53074e445b4fc"><code>dcacb9b</code></a> Simplify PyFunctionArgument impl on &Bound<T> (<a href="https://redirect.github.com/pyo3/pyo3/issues/5018">#5018</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/03c31c5c7affdd8805957b5944bd8ca05d1bdec8"><code>03c31c5</code></a> fix <code>#[pyfunction]</code> option parsing (<a href="https://redirect.github.com/pyo3/pyo3/issues/5015">#5015</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/0f49eb14b0358a8fe85c5930db84c5c404f97dd7"><code>0f49eb1</code></a> docs: Remove examples with outdated PyO3 and unmaintained projects (<a href="https://redirect.github.com/pyo3/pyo3/issues/4952">#4952</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/1b00b0d27f1b49d4b4237bc616d99016b06c1bd8"><code>1b00b0d</code></a> implement <code>PyCallArgs</code> for borrowed types (<a href="https://redirect.github.com/pyo3/pyo3/issues/5013">#5013</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/5caaa371dce8fe8a93c64d7a465c3c2c80ce6e2f"><code>5caaa37</code></a> fix: convert to cstrings in PyString::from_object (<a href="https://redirect.github.com/pyo3/pyo3/issues/5008">#5008</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/4aca459fd30441fa006c3eb388c812047f5465ce"><code>4aca459</code></a> docs: guide - add link to tables and traits (<a href="https://redirect.github.com/pyo3/pyo3/issues/5001">#5001</a>)</li> <li><a href="https://github.com/PyO3/pyo3/commit/0452c0ee5299a1af42f9d966ba3d136a79edb15d"><code>0452c0e</code></a> replace quansight-labs/setup-python with actions/setup-python (<a href="https://redirect.github.com/pyo3/pyo3/issues/5007">#5007</a>)</li> <li>Additional commits viewable in <a href="https://github.com/pyo3/pyo3/compare/v0.22.6...v0.24.1">compare view</a></li> </ul> </details> <br /> --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com> |
||
|
|
19eac8e00d |
feat: support Python 3.14+ via pyo3 abi3 stable ABI (#516)
## Description Sets pyo3 params to support python above 3.13 Fixes #(408 ## Type of Change - [x] Bug fix (non-breaking change that fixes an issue) - [ ] New feature (non-breaking change that adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to change) - [ ] Documentation update - [ ] Performance improvement - [ ] Code refactoring (no functional changes) ## Changes Made - Updated Cargo.toml ## Testing Describe the tests you ran to verify your changes: - [ ] Unit tests pass (`pytest`) - [ ] Linting passes (`ruff check .`) - [ ] Type checking passes (`mypy headroom`) - [ ] New tests added for new functionality - [x] Manual testing performed ## Test Output ``` Compiles ``` ## Checklist - [x] My code follows the project's style guidelines - [x] I have performed a self-review of my code - [ ] I have commented my code, particularly in hard-to-understand areas - [ ] I have made corresponding changes to the documentation - [x] My changes generate no new warnings - [ ] I have added tests that prove my fix is effective or that my feature works - [x] New and existing unit tests pass locally with my changes - [ ] I have updated the CHANGELOG.md if applicable Co-authored-by: Tejas Chopra <chopratejas@gmail.com> |
||
|
|
a08a0befa7 |
ci: build the CI test wheel with a fast cargo profile
The release profile (lto=thin, codegen-units=1) is great for the shipped wheel but slow to compile — it was the build-wheel long pole (~3m38s) gating the test shards. Add [profile.ci] (no LTO, codegen-units=256, opt-level=1) and build the CI wheel with --profile ci. Does not affect --release / shipped wheels. |
||
|
|
d73cbd6b0a |
fix(build): shrink Rust extension wheels — strip + thin-LTO + single codegen unit
PyPI rejected the v0.21.37 release publish with:
HTTPError: 400 Bad Request from https://upload.pypi.org/legacy/
Project size too large. Limit for project 'headroom-ai' total size is 10 GB.
PyPI inventory check confirmed: **191 versions × ~213 MB/release =
10.00 GB exactly** — at the cumulative project storage ceiling. Each
recent release ships 12 wheels × ~16-18 MB each.
Post-mortem inspection of a production wheel
(``headroom_ai-0.21.36-cp311-cp311-manylinux_2_28_x86_64.whl``)
showed the binary was ``not stripped``:
.text 18.3 MB (code)
.rodata 11.4 MB (Magika model + ONNX runtime data)
.strtab 4.9 MB (debug strings — strippable)
.eh_frame 1.9 MB (unwind tables)
.symtab 1.5 MB (debug symbols — strippable)
.gcc_except_table 1.2 MB
This commit adds a release profile:
[profile.release]
strip = "symbols"
lto = "thin"
codegen-units = 1
That:
* Strips ``.symtab`` + ``.strtab`` (~6.4 MB direct savings per wheel)
* Enables thin link-time optimization for cross-crate dead-code
elimination (~5-10% ``.text`` savings)
* Single codegen unit for better inlining + DCE at the cost of
~30-50% slower release builds (acceptable for CI)
Deliberately NOT setting ``panic = "abort"``:
* The proxy is a long-lived async process. A panic on one bad
request triggering process abort would disconnect every concurrent
client. Accept the smaller savings; keep unwind behaviour.
Estimated impact
* Per wheel: ~16-18 MB → ~10-11 MB (40% smaller)
* Per release (12 wheels): ~213 MB → ~130 MB
* PyPI capacity: ~30+ more releases before hitting 10 GB again
Verification
* Local build of ``headroom._core`` with new profile:
``.so`` size 29 MB on macOS arm64 (was ~45 MB pre-fix; final wheel
compressed will be smaller on Linux which also benefits from the
``strip`` directive).
* 77 Rust-parity tests pass — extension still functional.
* Single-codegen-unit slows build by ~30-50% but maturin/cibuildwheel
build time was never the bottleneck.
Forward strategy (separate work)
* Submit a PyPI project-size-limit-increase request to unblock the
immediate release.
* Adopt a release-deprecation policy: yank versions older than N
patches per minor; consider dropping Python 3.10 wheels (EOL'd
October 2026) and manylinux_2_28_aarch64 wheels (niche audience,
largest at 18.75 MB).
* Investigate runtime-download for Magika model (~10 MB further
savings) — same pattern Kompress already uses.
|
||
|
|
c10a2195af |
fix(proxy): PR-D4 native Vertex publisher path + ADC bearer auth
Adds a Rust-native Vertex AI publisher route ahead of the LiteLLM
Python converter (which dropped `thinking`, `redacted_thinking`,
`document`, `image`, `server_tool_use`, `mcp_tool_use` block kinds —
the P4-37 / P4-38 bug). After this PR the Vertex `:rawPredict` and
`:streamRawPredict` calls survive byte-equal upstream and benefit
from the live-zone Anthropic dispatcher (PR-B-series) running over
the body — same behaviour as `/v1/messages`.
New module `crates/headroom-proxy/src/vertex/`:
- `mod.rs` — single dispatch handler at the
`/v1beta1/.../models/:model_action` route. Splits the trailing
`:<verb>` segment with `str::rsplit_once(':')` (no regex) and
flips an `attach_sse_tee` flag to dispatch to the streaming or
non-streaming arm. Both verbs share one axum route shape because
matchit can't distinguish two patterns that overlap on a
parameter.
- `envelope.rs` — `VertexEnvelope` parser. Confirms
`anthropic_version` present + `model` field absent (the two
fingerprints of the Vertex envelope vs `/v1/messages`).
- `adc.rs` — `TokenSource` trait + `GcpAdcTokenSource` (production,
`gcp_auth` 0.12) + `StaticTokenSource` (tests). Caches tokens
with a 60s refresh-ahead-of-expiry window. Emits structured
`event = "vertex_adc_token_refreshed"` per refresh.
- `raw_predict.rs` — POST handler + shared `forward_vertex_request`.
Buffers body, parses envelope, runs live-zone Anthropic
compression, fetches ADC bearer, attaches
`Authorization: Bearer <token>` (overwrites client-supplied
Authorization header), forwards. SSE telemetry tee for the
streaming verb reuses PR-C1's `AnthropicStreamState` directly
(Vertex streams plain SSE, unlike Bedrock's binary EventStream).
- `stream_raw_predict.rs` — module-level docs + alias to the
shared dispatcher (the streaming-vs-non-streaming difference is
one boolean flag inside the shared forwarder).
Modifications:
- `proxy.rs::build_app` — registers the single Vertex route.
- `proxy.rs::AppState` — new `vertex_token_source: Arc<dyn TokenSource>`
field. Production constructs `GcpAdcTokenSource` lazily (no GCP
call until first `bearer()`); tests inject `StaticTokenSource`
via the new `AppState::with_token_source` helper.
- `config.rs` — adds `--vertex-region` / `HEADROOM_PROXY_VERTEX_REGION`
(default `us-central1`, observability tag only — the upstream URL
is `--upstream`) and `--vertex-adc-scope` /
`HEADROOM_PROXY_VERTEX_ADC_SCOPE` (default `cloud-platform`).
- `Cargo.toml` (workspace + proxy) — adds `gcp_auth = "0.12"` and
`async-trait = "0.1"`.
- `tests/common/mod.rs` — `start_proxy_with_state` accepts both
config + state customizers; `install_static_token_source` helper
for tests.
`crates/headroom-proxy/tests/integration_vertex_raw_predict.rs` —
all five tests pass:
1. `native_envelope_round_trip_byte_equal` — Vertex-shape body
(with `anthropic_version`, no `model`) round-trips SHA-256
byte-equal upstream.
2. `adc_bearer_token_signed_correctly` — `Authorization: Bearer
<static-test-token>` reaches upstream verbatim and OVERWRITES a
client-supplied Authorization header.
3. `thinking_block_preserved` — request with `thinking` (incl.
signature) + `redacted_thinking` (incl. opaque `data`) blocks
round-trips byte-equal even with `LiveZone` compression mode
enabled. This is the P4-37 / P4-38 teeth.
4. `stream_raw_predict_sse_handled` — `:streamRawPredict` proxies
an Anthropic SSE response (full `message_start` →
`content_block_delta` → `message_stop` sequence) back to the
client without corruption; SSE content-type preserved end-to-end;
bearer attached.
5. (bonus, no-silent-fallback contract)
`adc_failure_returns_5xx_no_silent_forward` — when the token
source returns `Err`, the proxy returns 5xx and never reaches
upstream. Verifies the `event = "vertex_adc_fetch_failed"`
error path.
Workspace: `cargo test --workspace` green; `cargo clippy --workspace
-- -D warnings` clean; `make ci-precheck` passes.
- No silent fallbacks: ADC failure → structured 5xx, never an
unauthenticated forward.
- No hardcodes: every knob (region, ADC scope, upstream URL) is
CLI-flag + env-var configurable.
- No regexes: axum path parameters + `str::rsplit_once` only.
- Comprehensive structured logs: `event` field on every decision
point — `vertex_envelope_parsed`, `vertex_envelope_invalid`,
`vertex_compression_skipped`, `vertex_compression_applied`,
`vertex_adc_token_refreshed`, `vertex_adc_fetch_failed`,
`vertex_streaming_pipeline_active`, `vertex_sse_stream_closed`,
`vertex_forwarded`, `vertex_unknown_verb`, etc.
- Performant: no body clone; ADC token cached + refreshed
ahead-of-expiry, not fetched per request.
- Comprehensive tests: realistic Anthropic block content
(signature payload, redacted_thinking opaque blob) in
`thinking_block_preserved`.
The local `gcloud auth application-default print-access-token`
returns no credentials, so manual validation against a real Vertex
endpoint is not possible in this PR. Follow-up: the user runs
`gcloud auth application-default login` once and exercises a live
Vertex request — should be a no-code-change check.
PR-D1 (Bedrock native) is running concurrently and will land its
own envelope module at `crates/headroom-proxy/src/bedrock/envelope.rs`.
The two envelope modules are intentionally siblings (not a shared
trait) — the shapes differ (Bedrock has a different
`anthropic_version` value, no `model` field, AWS SigV4 instead of
GCP ADC), and a premature shared abstraction would obscure the
provider-specific contracts. Whichever PR merges second rebases
without conflict.
Retires P4-38 (and the Vertex parts of P4-39); marketplace BYOC
pitch (per project memory) gets one more native provider.
|
||
|
|
f2d4fe39cb |
fix(proxy): PR-D1 native Bedrock InvokeModel route + SigV4
Phase D PR-D1 lands the first native Rust path for AWS Bedrock,
replacing the lossy LiteLLM Python converter for Anthropic-on-Bedrock
non-streaming requests. Eliminates part of P4-37 and P4-39.
What landed
-----------
- New crates/headroom-proxy/src/bedrock/ module:
- envelope.rs: parses the {"anthropic_version": "...", ...}
Bedrock body shape; re-emits with anthropic_version preserved
as the first key (relies on serde_json preserve_order).
- sigv4.rs: AWS SigV4 signer wrapping the aws-sigv4 crate.
Forces PayloadChecksumKind::XAmzSha256 so x-amz-content-sha256
is in the canonical request, hashed over the post-compression
body bytes (the bytes that actually hit Bedrock). No silent
fallback: signing failures return 5xx with
event=bedrock_sigv4_failed.
- invoke.rs: POST handler for /model/{model_id}/invoke
(and /converse - same wire shape for anthropic.claude-*).
Detects Anthropic vendor via literal starts_with("anthropic.")
(no regex per project rule), routes Anthropic-shape bodies
through the existing compress_anthropic_request live-zone
dispatcher, then signs and forwards to the configured Bedrock
endpoint.
- Modified:
- proxy.rs: routes /model/:model_id/invoke and
/model/:model_id/converse when enable_bedrock_native is on
(default). Adds bedrock_credentials: Option<Arc<Credentials>>
to AppState.
- config.rs: new flags --bedrock-region (default us-east-1,
env HEADROOM_PROXY_BEDROCK_REGION), --bedrock-endpoint
(operator override for FIPS/VPC/test setups),
--enable-bedrock-native (default true), --aws-profile.
- main.rs: resolves AWS credentials at startup via
aws_config::defaults(BehaviorVersion::latest()). Failure logs
event=bedrock_credentials_unavailable at WARN; the handler
refuses to forward unsigned (event=bedrock_credentials_missing).
- Cargo.toml: workspace deps aws-sigv4, aws-config,
aws-credential-types, aws-smithy-runtime-api.
Tests
-----
8 integration tests under
crates/headroom-proxy/tests/integration_bedrock_invoke.rs:
1. native_envelope_round_trip_byte_equal
2. sigv4_signed_correctly_after_compression - confirms
authorization is SigV4-shape and x-amz-content-sha256
matches sha256(body received by upstream).
3. thinking_block_preserved_through_bedrock
4. redacted_thinking_preserved
5. document_block_preserved
6. tool_result_array_with_image_preserved
7. stop_sequence_null_only_when_present - pins that the proxy
does NOT inject stop_sequence: null (P4-37 hardcode).
8. tool_use_input_byte_equal_preserves_key_order
All eight pass. Full workspace test run is green; clippy + fmt
clean. make ci-precheck (rust + python + commitlint) passes
locally.
Build constraints honoured
--------------------------
- No silent fallbacks: missing creds / signing failures return
5xx with structured event=... log; no path ever forwards
unsigned.
- No hardcodes: region, endpoint, profile, enable-flag all
configurable via CLI + env.
- No regexes: vendor detection is str::starts_with.
- Comprehensive structured logs: event=bedrock_invoke_received,
bedrock_envelope_parsed, bedrock_compression_skipped,
bedrock_credentials_missing, sigv4_signed,
bedrock_invoke_forwarded, etc.
- Performant: body buffered once, passed by &[u8] to signer
(zero-copy), Bytes::clone only for ownership transfer to
reqwest. Sign exactly once per request.
- Elegant: 4 small focused modules mirror handlers/ + sse/.
- Tests use realistic Anthropic block content (real thinking,
redacted_thinking, document, base64 image fixtures).
Streaming (PR-D2) and observability (PR-D3) follow.
|
||
|
|
3f99128236 |
fix(rust): A4 — honor cache_control markers; serde_json arbitrary_precision + raw_value
PR-A4 of the Realignment Phase A lockdown
(REALIGNMENT/03-phase-A-lockdown.md). Eliminates P0-3 (Rust proxy
ignores customer cache_control markers) and P0-5 (numeric precision
lost via serde_json::Value round-trip) at the library level; Phase B
PR-B2 wires the helper into the live-zone block dispatcher.
Cargo.toml — add `arbitrary_precision` and `raw_value` to
`serde_json` workspace features. `arbitrary_precision` keeps `1.0`
from collapsing to `1` and preserves >2^53 integers; `raw_value`
exposes `&RawValue` so PR-B2 can forward unmodified `messages[*]`
entries as exact byte copies.
crates/headroom-core/src/cache_control.rs (new) — `compute_frozen_count`
walks `messages[i].content[*].cache_control` via serde_json
accessors only (no regex) and returns the smallest N such that
`messages[i]` is frozen for every i < N. Markers in `system` or
`tools[*]` log at debug! but never bump the floor (those fields are
unconditionally cache-hot per invariant I2). TTL ordering violations
(5m before 1h, guide §2.19) emit `tracing::warn!` but the function
computes the correct count regardless — the customer's request, not
ours to reject.
crates/headroom-core/src/lib.rs — re-export `compute_frozen_count` at
crate root so the proxy crate has a stable import path.
crates/headroom-proxy/src/compression/anthropic.rs — add
`resolve_frozen_count` thin wrapper that consults the
`cache_control_auto_frozen` config flag. When `disabled`, returns 0
regardless of body content (operator opt-out for benchmarking).
crates/headroom-proxy/src/config.rs — add `CacheControlAutoFrozen`
enum and the matching CLI flag `--cache-control-auto-frozen` /
env var `HEADROOM_PROXY_CACHE_CONTROL_AUTO_FROZEN`. Default is
`enabled`. Documented in the doc comments.
Tests
- crates/headroom-core/src/cache_control.rs (inline): 11 unit tests
covering marker detection, system/tools negative cases, ordering
state machine, defensive (missing fields, non-array messages,
non-object content blocks).
- crates/headroom-core/tests/cache_control.rs: 11 unit + 3 property
tests (monotonic non-decrease as markers are added; system/tools
markers don't change count; empty messages → 0).
- crates/headroom-proxy/tests/integration_cache_control.rs: 8 tests
exercising the proxy wrapper (configurability gate; tracing
capture for the 5m-before-1h warn path).
Acceptance gates: `cargo build --workspace`, `cargo test --workspace`
(33 new tests green), `cargo clippy --workspace -- -D warnings`,
`cargo fmt --all --check` all clean. No new `regex::` imports;
`git grep -n 'regex::' crates/{headroom-core/src/cache_control.rs,
headroom-core/tests/cache_control.rs, headroom-proxy/tests/
integration_cache_control.rs}` empty.
Honors the realignment build constraints: configurable (CLI + env),
no hardcodes (TTL strings live as const), no regex (serde_json
accessor walk), no fallbacks (one impl), structured logging
(debug!/warn! with field/index/ttl/rule context), tests
comprehensive (unit + property + integration + tracing capture).
|
||
|
|
a64716d5d1 |
fix(rust): smart_crusher scaffold review findings — hash truncation, int parse, python-repr matcher
Code review (`/code-review` on commit `d219bee`) caught one critical bug, two important parity gaps, and a few quality nits. Fixed all of them; all 135 unit tests pass; diff_compressor parity harness unaffected (27/27 still matched). # Critical fix — `hash_field_name` truncation length Rust truncated SHA-256 to **16** hex chars; Python uses **8** (per `smart_crusher.py:177`: `hashlib.sha256(...).hexdigest()[:8]`). 16-char hashes would never collide with TOIN's 8-char `preserve_fields`, silently disabling the entire `use_feedback_hints` cache lookup path. Fix: `hex[..8]` instead of `hex[..16]`. Three pinning tests re-verified against actual Python reference output. Doc comment now warns explicitly that the length must match Python or TOIN lookups silently miss. # Important fix — `python_int_parse` mirrors Python's `int()` semantics `statistics.rs::detect_sequential_pattern` previously called `s.parse::<i64>()`. Python's `int()` differs in three ways that affect realistic payloads: - strips ASCII whitespace (Rust's `parse` rejects) - accepts leading `+` (Rust accepts; same) - accepts PEP 515 underscores like `"3_000"` (Rust rejects) A field with `[" 1 ", " 2 ", " 3 ", "4", "5"]` would parse all five in Python (sequential = True) but only one in Rust (`nums.len() < 5` → False). Silent parity break. Fix: new private `python_int_parse` helper that strips whitespace, handles underscore separators, and rejects edge cases Python rejects. Six new tests pin the behavior. # Important fix — `python_repr` for `item_matches_anchors` Python compares anchors via `anchor in str(item).lower()`. We were using `serde_json::to_string(&item).to_lowercase()`, which differs in three ways that affect substring matching: - quote chars (`'` vs `"`) - bool/null literals (`True`/`False`/`None` vs `true`/`false`/`null`) - spacing (`key: value, ...` vs `key:value,...`) Anchor `"none"` would match Python form but not JSON. Inverse for `"null"`. Real divergence. Fix: new private `python_repr` walks `serde_json::Value` and emits Python-equivalent form. Plus enable `serde_json/preserve_order` at workspace level so `Value::Object` preserves JSON parse order (matching Python `dict` since 3.7). # Suggestion fixes - Classifier comment for `[True, False, 1] -> MIXED_ARRAY` now walks both Python and Rust paths step by step. - `ArrayAnalysis::field_stats` doc notes the BTreeMap vs Python-dict order nuance for the analyzer port to resolve. - Added regression tests for "all unparseable strings", "single int among strings", fractional-step sequential, and the email-typo pattern. # Build / test - `cargo build -p headroom-core` clean. - `cargo clippy -p headroom-core -- -D warnings` clean. - 135 unit tests in `headroom-core`, all passing (was 55). - `cargo run -p headroom-parity run` — diff_compressor 27/27 still matched. |
||
|
|
9ce1c01b87 |
feat(rust): tokenizer crate with tiktoken-rs + HuggingFace + estimator
Stage 2 of the Rust port: a `headroom_core::tokenizer` module mirroring the Python `headroom.tokenizers` surface, with three backends behind a single `Tokenizer` trait. Backends, in dispatch order: 1. HuggingFace (`HfTokenizer`) — pure-Rust `tokenizers` crate loading any public `tokenizer.json`. Covers the gap between OpenAI (tiktoken) and the Anthropic/Gemini estimator: Cohere `command-*`, Llama-3.x, Mistral, Qwen, BERT, T5, etc. Construct from bytes or a file path; register against a model-name prefix via `register_hf` for automatic dispatch. No `hf-hub` auto-download yet — keeps networking, auth, and `~/.cache/huggingface` out of core. Longest-prefix wins; lookups are RwLock-protected. 2. Tiktoken (`TiktokenCounter`) — `tiktoken-rs` 0.11 BPE for OpenAI / o-series families. Byte-identical to Python `tiktoken` for ordinary text. Lazy shared `Arc<CoreBPE>` per encoding (o200k_base, cl100k_base, p50k_base, r50k_base). 3. Estimation (`EstimatingCounter`) — `chars / cpt` last-resort fallback. Matches Python's `max(1, int(len(text) / cpt + 0.5))` round-half-up formula (a self-review caught and fixed an earlier `ceil`-based version that diverged in the middle of the range, e.g. 5 chars at 4.0 cpt). Tests: 43 unit tests + 5 proptests; parity 40/40 byte-equal. Bench: criterion baseline on small/medium/large inputs. Workspace MSRV bumped 1.78 → 1.80 for `LazyLock`/`OnceLock`. No proxy wiring. Library-only; production behavior unchanged. |
||
|
|
0414cb70e4 |
feat(rust): scaffold workspace + parity harness (phase-0)
Bootstrap the Rust port of Headroom. Additive only — no existing Python code modified. Ships the workshop, not the widgets. Layout Cargo.toml (workspace) + rust-toolchain.toml crates/headroom-core — transform library, stub only crates/headroom-proxy — axum binary, /healthz only crates/headroom-py — PyO3 cdylib, exposes headroom._core.hello() crates/headroom-parity — Rust-vs-Python oracle harness + parity-run CLI Tooling Makefile: test, test-parity, bench, build-proxy, build-wheel, fmt, lint .github/workflows/rust.yml: test, wheels (linux/mac), audit, parity-nightly deny.toml for cargo-deny Parity corpus tests/parity/recorder.py + scripts/record_fixtures.py 125 recorded fixtures across 5 leaf transforms (ccr, tokenizer, log_compressor, diff_compressor, cache_aligner) Docs RUST_DEV.md — developer setup and workspace reference docs/spec/022-rust-migration.md — migration plan and stage breakdown .gitignore: whitelist scripts/record_fixtures.py; ignore target/ Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |