mirror of
https://github.com/headroomlabs-ai/headroom.git
synced 2026-11-22 13:29:19 +08:00
deps: Bump the cargo-minor-patch group across 1 directory with 12 updates (#4553)
## Purpose Retain the complete selected 12-version Cargo group and qualify its compatibility with Headroom. The AWS release moves its config, credential, Smithy runtime/schema and SDK dependencies together to a new coordinated compatibility line; upstream explicitly warns against upgrading only part of that line. Existing proxy credential resolution and SigV4 signing must continue to work with that coherent set. This is a compatibility-qualified grouped update. Companion updates are not individually claimed to fix a Headroom bug where upstream only added unused APIs. ## Selected versions and project relevance | Package | Selected update | Relevance and qualification | | --- | --- | --- | | tokio | 1.53.3 → 1.54.1 | Proxy HTTP/SSE/WebSocket execution and timers; includes upstream timer-handle regression revert. Local native proxy/SSE/WSS tests exercise runtime. | | aws-config | 1.12.0 → 1.13.0 | Required companion for new AWS schema/runtime compatibility line; production environment credential chain exercised with explicit fake credentials. | | aws-credential-types | 1.3.0 → 1.3.1 | Same coordinated AWS line, production credential conversion and SigV4 signing exercised. | | aws-smithy-runtime-api | 1.19.0 → 1.19.2 | Same coordinated AWS line; timeout finalizer behavior changed upstream. Local signing/wire tests do not establish SSO/STS timeout behavior. | | sha2 | 0.11.0 → 0.11.1 | Existing cache/SigV4-related payload hashing; upstream unsafe-op lint fix. Local signed payload SHA256 verified; no claimed cryptographic vulnerability fixed. | | redis | 1.7.1 → 1.8.0 | Optional multiworker CCR backend; upstream TLS error/secret redaction and subscription correctness fixes. Production backend roundtrip exercised against local RESP fixture; cluster/pubsub are not Headroom backend usage. | | tree-sitter | 0.27.0 → 0.27.1 | Code compressor parser/error recovery fixes. Reconstructed 9 samples × 8 pinned languages compare old/new node kinds, complete row/column spans, ERROR/MISSING: 72/72 identical. | | uuid | 1.28.0 → 1.29.0 | Full-group companion target; new NonNilUuid constructors are unused. Existing request-ID generation exercised through real proxy calls and both response x-request-id values parsed as UUIDv4; no invented bug/security rationale. | | crc32fast | 1.5.2 → 1.5.3 | Full-group companion target; upstream total-length API is unused. Existing Bedrock EventStream checksum/framing path exercised by streaming integration suite. | | rustls-native-certs | 0.8.4 → 0.8.5 | Native trust-store handling; avoids duplicate Unix certificate-file reads. Windows build and WSS fail-closed behavior exercised; Unix store traversal not locally exercised. | | webpki-roots | 1.0.10 → 1.0.11 | Updated bundled CA trust set (CCADB CA removals); WSS verification tests exercise existing explicit TLS configuration. No public CA endpoint contacted. | | cc | 1.8.0 → 1.10.0 | Native C/C++ dependency builds; upstream fixes MSVC archive object paths and cl.exe wrapper handling. Actual clean Windows MSVC native dependency compilation performed. | ## Maintenance and install surface The packages remain maintained by their existing upstream projects: Tokio, AWS/Smithy, RustCrypto, redis-rs, tree-sitter, uuid-rs, srijs/rust-crc32fast, rustls and rust-lang/cc-rs. Release/compare links in the original Dependabot body identify the reviewed releases; AWS authoritative release notes: https://github.com/awslabs/aws-sdk-rust/blob/main/CHANGELOG.md (October 20–30, 2026). UUID comparison confirms its change only adds NonNilUuid constructors; it is not presented as a security fix. No new package source/registry is introduced. Cargo resolves registry downloads at build time; cc executes the native compiler for existing C/C++ dependencies (tree-sitter grammars and other existing native crates). The lock update changes the transitive AWS SDK/schema/runtime family together. Existing ONNX runtime loading/model behavior is not altered by this PR. Redis is feature gated and adds runtime network only when explicitly configured. AWS credentials can normally resolve through environment, profiles, SSO/STS or metadata; qualification uses only explicit fake environment credentials, NUL config files and disabled IMDS. Normal configured proxy upstream traffic and native trust-store reads remain existing runtime surfaces. Pinned qualification compiler: Rust 1.95.0 on Windows/MSVC. Resolved proxy AWS floor is 1.94.1; core/Python/parity graph includes tree-sitter-language floor 1.90. Both already existed in the base lock, so no global rust-version change is made; inherited workspace rust-version=1.80 is not a valid full-workspace source-build promise. Fresh base/head Windows-filtered metadata comparison confirms unchanged floors: core/Python/parity1.90,proxy1.94.1,simulators1.86(ICU). These are dependency-declared effective floors, not measured lower-toolchain builds. No lower-toolchain build was attempted. ## Real behavior proof Windows, Rust1.95.0 x86_64-pc-windows-msvc, isolated worktree, dedicated CARGO_TARGET_DIR, CARGO_BUILD_JOBS=2. No actual AWS/provider credentials, profiles, cloud inference or model downloads used. - `cargo test --locked -p headroom-core -p headroom-proxy` with explicit existing x64 ONNX Runtime DLL (`ORT_DYLIB_PATH`) and `HEADROOM_OFFLINE=1`, `HF_HUB_OFFLINE=1`, `TRANSFORMERS_OFFLINE=1`: **1,659 passed, 0 failed, 3 ignored; exit 0**, across 61 test/doc-test executables. Initial run without the Windows loader path was diagnosed loader-blocked and terminated; its separate failed log is retained. - `cargo test --locked -p headroom-proxy --test integration_bedrock_invoke --test integration_bedrock_streaming --test integration_ws_tls --test integration_sse`: 23passed,0failed (8invoke,11streaming,2SSE,2WSS). - `python G:/git/headroom/.codex/dep4553-runtime.py`: built headroom-proxy.exe on ephemeral ports with compression disabled; passthrough and native Bedrock routes each return local200, both bodies byte equal; AWS environment provider loaded fake credentials and SigV4 Credential prefix/payload SHA256 checked; both generated response x-request-id headers validated UUIDv4. Synthetic wire fixture, not real AWS service emulation. - `python G:/git/headroom/.codex/dep4553-redis-wire.py`: temporary integration test calls production RedisCcrStore using redis1.8.0; local RESP fixture validates PING,miss,UTF8 payload write/read,TTL refresh and replacement; 1passed. Synthetic protocol server; supplemented by `python G:/git/headroom/.codex/dep4553-redis-real.py`: real Redis7Alpine ephemeral loopback Docker service with persistence disabled; production backend2testsPASS,including real TTLrefresh andexpiry. Owned --rmcontainer stopped automatically. TLS/cluster/failover/pubsub not tested. - Standalone canary under `.codex/dep4553-canary`: exact engine versions0.27.0 and0.27.1, same eight pinned grammars,72/72 AST snapshots identical. Historical canary source was absent; reconstructed inputs are recorded, including second distinct valid fixture per language (explicit Cfunctionpointer sample), CRLF,truncated,error,concatenated andUnicode cases. - Isolated Python3.12 env: recorder regression9tests passed; Ruff checks/format passed. Original official fixture generator reproduced KeyError(bash). Minimal3line helper fix raises ValueError, matching existing detector unsupported-language contract; all8pinned grammars remain covered. Official generator then recorded30fixtures. - Generator revealed12preexisting Python-reference fixture differences. Exact main74277d47 source extracted independently, same Python environment +same helper fix:30/30 regenerated semantic fixture objects match PR source, excluding recorded_at only. Preserve original repo fixture bytes; do not mask unrelated Python-reference drift by replacing goldens. Baseline/regenerated files and comparisons retained as artifacts. ## Remaining gates / limitations Initial broad Cargo suite was loader-blocked: source live_zone.rs documents indefinite Windows initialization without ORT_DYLIB_PATH. Controlled termination of exact owned testPID717412 produced authoritative exit1 for original session37711. Distinct corrected configuration uses existing verified x64ORT1.26.0 DLL(API24), explicit ORT_DYLIB_PATH and offline guards; corrected full core+proxy qualification completed exit0,1659passed,0failed,3ignored. This is an environment diagnosis, not a demonstrated dependency regression. Hosted checks must be rerun on the exact final remote SHA; prior shard4 red is not claimed fixed. Canonical fixture rerecord was executed and its baseline-identical semantic drift documented, not silently normalized away. No Linux/macOS build, lower MSRV build, real AWS SSO/STS/IMDS, public CA endpoint, Redis TLS/cluster/failover, production provider inference or model download was tested. No remote push, PR edit/review or merge performed by this qualification task. ## Public reproduction The committed standard-library harness reproduces the full 72-case parser comparison and launches the actual native proxy against credentialless loopback fixtures. It stores both complete AST snapshots and inputs when `--artifacts-dir` is provided. No private `.codex` artifacts are needed. Windows, from a clean checkout with Rust 1.95/MSVC and Python 3.11+: ```powershell cargo build --locked -p headroom-proxy --target-dir target/qualification python scripts/qualify_cargo_dependencies.py all --binary target/qualification/debug/headroom-proxy.exe --artifacts-dir target/qualification-proof --canary-target-dir target/qualification-canary ``` Linux/macOS use the same commands with the binary path `target/qualification/debug/headroom-proxy` (omit `.exe`). Those platforms are reproduction instructions, not platforms locally qualified here. The canary compiles standalone exact engine versions0.27.0/0.27.1 against all eight production grammar pins, so Cargo registry downloads/native compilers may be needed. Runtime checks use fake AWS credentials, platform null config files, disabled IMDS, offline model guards, compression off, and local HTTP endpoints only. The source-comment-required Python fixture rerecord is independently reproducible: ```text python -m pip install tree-sitter==0.25.2 tree-sitter-python==0.25.0 tree-sitter-javascript==0.25.0 tree-sitter-typescript==0.23.2 tree-sitter-go==0.25.0 tree-sitter-rust==0.24.2 tree-sitter-java==0.23.5 tree-sitter-c==0.24.2 tree-sitter-cpp==0.23.4 python scripts/record_code_compressor_fixtures.py ``` Use an isolated development Python environment with the project development dependencies installed. The recorder helper fix in this PR preserves its existing eight-grammar scope; unsupported detector candidates are skipped through the existing ValueError contract. Regeneration currently exposes baseline Python-reference drift as documented above; inspect semantic differences rather than replacing goldens blindly. --------- Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: JerrettDavis <mxjerrett@gmail.com>
This commit is contained in:
co-authored by
JerrettDavis
parent
7e0094d6c8
commit
b9049d703a
@@ -42,6 +42,7 @@ scripts/*
|
||||
!scripts/refresh_tool_hashes.py
|
||||
!scripts/record_kompress_fixtures.py
|
||||
!scripts/record_code_compressor_fixtures.py
|
||||
!scripts/qualify_cargo_dependencies.py
|
||||
|
||||
# Rust / Cargo build artifacts
|
||||
/target/
|
||||
|
||||
Generated
+102
-141
@@ -299,9 +299,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-config"
|
||||
version = "1.12.0"
|
||||
version = "1.13.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b8d7b388a9fc3a6db15a5ec778c38b354eff1364882c94d08e0252f7a47dcaa4"
|
||||
checksum = "939e256cfe0ed68e96d32f194339d88210d5c10685cff0787400158f852c6819"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-runtime",
|
||||
@@ -319,7 +319,7 @@ dependencies = [
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"hex",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"sha1 0.10.6",
|
||||
"time",
|
||||
"tokio",
|
||||
@@ -330,9 +330,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-credential-types"
|
||||
version = "1.3.0"
|
||||
version = "1.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e93964ffdaf57857f544be3666a5f57570bb699e934700f11b49708f61bb556e"
|
||||
checksum = "576e8da35751dc85409ec1cd15999f692c277a1fbe9336780fe6f15e49dd637d"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api",
|
||||
@@ -365,9 +365,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-runtime"
|
||||
version = "1.9.2"
|
||||
version = "1.10.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef47857a1d4488b528f4a5d5715fa7c3300820897824152234d3fa22b1426657"
|
||||
checksum = "9a0cc2144962cc5e358467b16e3d82981b1d424a7c3d34602be0036103814a1a"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-sigv4",
|
||||
@@ -380,8 +380,8 @@ dependencies = [
|
||||
"bytes",
|
||||
"bytes-utils",
|
||||
"fastrand",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"tracing",
|
||||
@@ -390,9 +390,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-sso"
|
||||
version = "1.109.0"
|
||||
version = "1.118.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c3cfe74df5d9ad2fedd691973ad3521ebf4f27a3c68c792556686aedb5519bab"
|
||||
checksum = "b4a89786db8ee6dff2c9302a354ef081354fcb0f527e16ee3cb249a6a5cb6df7"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"aws-credential-types",
|
||||
@@ -408,17 +408,16 @@ dependencies = [
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-ssooidc"
|
||||
version = "1.111.0"
|
||||
version = "1.120.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "81b0ec31ed6191bd11350aae4b2004198f2db21350cb0a20c57e0a92e55dd161"
|
||||
checksum = "521c6a11a25f1244efddf947523f277a51a63f0a7ec228f0f62d826e88106366"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"aws-credential-types",
|
||||
@@ -434,17 +433,16 @@ dependencies = [
|
||||
"aws-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sdk-sts"
|
||||
version = "1.114.0"
|
||||
version = "1.123.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ef45745026107ec30c4ef86bd8ae4b002e7e5f6a86e4225240bdf6b06a0b944a"
|
||||
checksum = "8754d38d29448bd1d4aa4a6312ed010dfa296b3436eb2c551cc6e42fc6a66a38"
|
||||
dependencies = [
|
||||
"arc-swap",
|
||||
"aws-credential-types",
|
||||
@@ -461,17 +459,16 @@ dependencies = [
|
||||
"aws-smithy-xml",
|
||||
"aws-types",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"regex-lite",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-sigv4"
|
||||
version = "1.6.0"
|
||||
version = "1.6.4"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2312577f088c9fbf4206dfdb884cf1de9407b43e1a923cbed5237775116fc24b"
|
||||
checksum = "f385da9b7757ed4c8214797ad663ff534fb622006143530aa80e21f8e06ff85c"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-smithy-http",
|
||||
@@ -481,8 +478,7 @@ dependencies = [
|
||||
"form_urlencoded",
|
||||
"hex",
|
||||
"hmac",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"percent-encoding",
|
||||
"sha2",
|
||||
"time",
|
||||
@@ -491,9 +487,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-async"
|
||||
version = "1.3.0"
|
||||
version = "1.3.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "f02e407fb3b54891734224b9ffac8a71fdd35f542500fa1af95754a6b2beb316"
|
||||
checksum = "de681a9df7ff79823b759ee17061076ec049cd61a8fa28ece4293d89c9f6964e"
|
||||
dependencies = [
|
||||
"futures-util",
|
||||
"pin-project-lite",
|
||||
@@ -502,9 +498,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-http"
|
||||
version = "0.64.0"
|
||||
version = "0.64.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "37843d9add67c3aff5856f409c6dc315d3cdff60f9c0cb5b670dab1e9920306d"
|
||||
checksum = "19af609dd5edf8b056cd0bb001c706c8b3b1458055328b334ec13d7f46f6aaa8"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
@@ -512,26 +508,25 @@ dependencies = [
|
||||
"bytes-utils",
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-http-client"
|
||||
version = "1.4.0"
|
||||
version = "1.5.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ebfd138fac0337cee7516c352757ea73b9f2266e57d0bcb5bc70e9547e45aef1"
|
||||
checksum = "859a81364f8185349c584fb41cd8765a400cf982f447adada590cf008a9f3b85"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"h2",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"hyper",
|
||||
"hyper-rustls",
|
||||
"hyper-util",
|
||||
@@ -547,9 +542,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-json"
|
||||
version = "0.63.0"
|
||||
version = "0.65.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "3dc65a121adb4b33729919fcfa14fa36fb33c1555a8f06bb0e2188dbfdc1d9ef"
|
||||
checksum = "b45f9fc77ffe92c92c341165a01a565bd30c39ecf5aa8c0719b41719009604f9"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-schema",
|
||||
@@ -567,9 +562,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-query"
|
||||
version = "0.62.0"
|
||||
version = "0.64.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "512346c7212ab7436df2d77a16d976a468ae44a418835511d2a69269810aaf62"
|
||||
checksum = "165f6265342a313ad6f7a1ac17a0cdba30c522039d3f41d8d6871ce8a2b34ecd"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-schema",
|
||||
@@ -580,9 +575,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-runtime"
|
||||
version = "1.14.0"
|
||||
version = "1.16.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "b82e438d30e02a825d363bd639a9efaed68a8089d86101054b0081e7e0d3e606"
|
||||
checksum = "4cb80fb19c084da5fca9d797845735e184445e5b958599440d67529bf6ef1b64"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-http",
|
||||
@@ -593,28 +588,25 @@ dependencies = [
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"fastrand",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http-body 0.4.6",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"tokio",
|
||||
"tracing",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-runtime-api"
|
||||
version = "1.19.0"
|
||||
version = "1.19.2"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "c0730c16f91124c6a2abb4932c77e299288b3dd9f967ea2e9ec48cc6731e87a4"
|
||||
checksum = "8c4e85cffe9944b10d576b075432bd8f98dbc081e8681cb37d5b2d87dc27bc84"
|
||||
dependencies = [
|
||||
"aws-smithy-async",
|
||||
"aws-smithy-runtime-api-macros",
|
||||
"aws-smithy-types",
|
||||
"bytes",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"pin-project-lite",
|
||||
"tokio",
|
||||
"tracing",
|
||||
@@ -623,9 +615,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-runtime-api-macros"
|
||||
version = "1.1.0"
|
||||
version = "1.2.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "221eaa237ddf1ca79b60d1372aad77e47f9c0ea5b3ce5099da8c61d027dc77b3"
|
||||
checksum = "ce237f890a2283c9b328955c3b0918d2d5bf66a1cb6ceccd36a0047246fbc6e6"
|
||||
dependencies = [
|
||||
"proc-macro2",
|
||||
"quote",
|
||||
@@ -634,33 +626,30 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-schema"
|
||||
version = "0.2.0"
|
||||
version = "1.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7d56e0a4e53127a632224e43633b0fe045fa9e1e3cfc68b9830f1115e103f910"
|
||||
checksum = "bec8dae1015a6d959a5faf86fbdf77486cfe19e4ccb6ee009d8a46efc45028a4"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-types",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-types"
|
||||
version = "1.8.1"
|
||||
version = "1.9.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "69bb407740a197147da48238ecc94498493c9e85445732360cec180296ca45f1"
|
||||
checksum = "cf183d70774ea2cfcc101ca74a8884e4c76be8b9df65cd303f6a1175c88e8f80"
|
||||
dependencies = [
|
||||
"base64-simd",
|
||||
"bytes",
|
||||
"bytes-utils",
|
||||
"http 0.2.12",
|
||||
"http 1.5.0",
|
||||
"http-body 0.4.6",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"itoa",
|
||||
"num-integer",
|
||||
"pin-project-lite",
|
||||
"pin-utils",
|
||||
"ryu",
|
||||
"serde",
|
||||
"time",
|
||||
@@ -668,9 +657,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-smithy-xml"
|
||||
version = "0.62.0"
|
||||
version = "0.64.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ce84f71c72fee2cbbadde6e7d082f5fb466e3a84733855295fa7aafd1b31b7d8"
|
||||
checksum = "590fdc1a70546b2f7f2061c42eaf173fb6f26d935330d069edc04dbd8f01d924"
|
||||
dependencies = [
|
||||
"aws-smithy-runtime-api",
|
||||
"aws-smithy-schema",
|
||||
@@ -680,9 +669,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "aws-types"
|
||||
version = "1.6.0"
|
||||
version = "1.7.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "209f3a6d82a6e9e5f94abbed94c7a26e1c052341002bf57a5fb5481f625896fc"
|
||||
checksum = "6ba0d970d3947bb064d2ae1d823efa186c1dbde88f88afa8f6549946e5d8deee"
|
||||
dependencies = [
|
||||
"aws-credential-types",
|
||||
"aws-smithy-async",
|
||||
@@ -705,8 +694,8 @@ dependencies = [
|
||||
"bytes",
|
||||
"form_urlencoded",
|
||||
"futures-util",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"hyper",
|
||||
"hyper-util",
|
||||
@@ -738,8 +727,8 @@ checksum = "08c78f31d7b1291f7ee735c1c6780ccde7785daae9a9206026862dab7d8792d1"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"mime",
|
||||
"pin-project-lite",
|
||||
@@ -987,9 +976,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "cc"
|
||||
version = "1.8.0"
|
||||
version = "1.10.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "6651c9ed80effdc7db0ff72512157f901af5e3549e341e24b1dd4887d836d838"
|
||||
checksum = "86e055fa6e3e40ad6187ba25b6855eeb635d5271f8e0f0e1e9a5826b8d75bea0"
|
||||
dependencies = [
|
||||
"find-msvc-tools",
|
||||
"jobserver",
|
||||
@@ -1257,9 +1246,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "crc32fast"
|
||||
version = "1.5.2"
|
||||
version = "1.5.3"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "01a7799fd6b852db0e61728dde9a204c423b44d689dbd432522543614b490e78"
|
||||
checksum = "0ba106e844045ddc02c20768ac4b0b7cb45f2cc2ecbd833ac9f5dacb183c2055"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
]
|
||||
@@ -1710,9 +1699,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "find-msvc-tools"
|
||||
version = "0.1.14"
|
||||
version = "0.1.15"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "aedcfb3409746eddb02b9e19ebda1c3394f759a152e48ee875a0844d1b955484"
|
||||
checksum = "1c66915cc5391097de7a2b0b972e8505aa33c0df401fd3a6811dbdb32881ba48"
|
||||
|
||||
[[package]]
|
||||
name = "flate2"
|
||||
@@ -1850,7 +1839,7 @@ dependencies = [
|
||||
"base64 0.22.1",
|
||||
"bytes",
|
||||
"chrono",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"http-body-util",
|
||||
"hyper",
|
||||
"hyper-rustls",
|
||||
@@ -1937,7 +1926,7 @@ dependencies = [
|
||||
"fnv",
|
||||
"futures-core",
|
||||
"futures-sink",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"indexmap",
|
||||
"slab",
|
||||
"tokio",
|
||||
@@ -2005,7 +1994,7 @@ dependencies = [
|
||||
"fastembed",
|
||||
"flate2",
|
||||
"hf-hub",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"icu_segmenter",
|
||||
"magika",
|
||||
"md-5",
|
||||
@@ -2068,7 +2057,7 @@ dependencies = [
|
||||
"gcp_auth",
|
||||
"headroom-core",
|
||||
"headroom-simulators",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"http-body-util",
|
||||
"humantime",
|
||||
"hyper",
|
||||
@@ -2121,7 +2110,7 @@ dependencies = [
|
||||
"bytes",
|
||||
"clap",
|
||||
"crc32fast",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"http-body-util",
|
||||
"reqwest",
|
||||
"serde",
|
||||
@@ -2158,7 +2147,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "aef3982638978efa195ff11b305f51f1f22f4f0a6cabee7af79b383ebee6a213"
|
||||
dependencies = [
|
||||
"dirs",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"indicatif",
|
||||
"libc",
|
||||
"log",
|
||||
@@ -2180,17 +2169,6 @@ dependencies = [
|
||||
"digest 0.11.3",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http"
|
||||
version = "0.2.12"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "601cbb57e577e2f5ef5be8e7b83f0f63994f25aa94d673e54a92d5c516d101f1"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"fnv",
|
||||
"itoa",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http"
|
||||
version = "1.5.0"
|
||||
@@ -2201,17 +2179,6 @@ dependencies = [
|
||||
"itoa",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http-body"
|
||||
version = "0.4.6"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7ceab25649e9960c0311ea418d17bee82c0dcec1bd053b5f9a66e265a693bed2"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"http 0.2.12",
|
||||
"pin-project-lite",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "http-body"
|
||||
version = "1.0.1"
|
||||
@@ -2219,7 +2186,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1efedce1fb8e6913f23e0c92de8e62cd5b772a67e7b3946df930a62566c93184"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
]
|
||||
|
||||
[[package]]
|
||||
@@ -2230,8 +2197,8 @@ checksum = "23169fe34a5fbcdd3f3862e78fb9b6fccd5f02a6dc6f732547005d45631ce71c"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"futures-core",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"pin-project-lite",
|
||||
]
|
||||
|
||||
@@ -2273,8 +2240,8 @@ dependencies = [
|
||||
"futures-channel",
|
||||
"futures-core",
|
||||
"h2",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"httparse",
|
||||
"httpdate",
|
||||
"itoa",
|
||||
@@ -2290,7 +2257,7 @@ version = "0.27.9"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "33ca68d021ef39cf6463ab54c1d0f5daf03377b70561305bb89a8f83aab66e0f"
|
||||
dependencies = [
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"hyper",
|
||||
"hyper-util",
|
||||
"rustls",
|
||||
@@ -2311,8 +2278,8 @@ dependencies = [
|
||||
"bytes",
|
||||
"futures-channel",
|
||||
"futures-util",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"httparse",
|
||||
"hyper",
|
||||
"ipnet",
|
||||
@@ -3222,12 +3189,6 @@ version = "0.2.17"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
|
||||
|
||||
[[package]]
|
||||
name = "pin-utils"
|
||||
version = "0.1.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "8b870d8c151b6f2fb93e84a13146138f05d02ed11c7e7c54f8826aaaf7c9f184"
|
||||
|
||||
[[package]]
|
||||
name = "pkg-config"
|
||||
version = "0.3.33"
|
||||
@@ -3726,9 +3687,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "redis"
|
||||
version = "1.7.1"
|
||||
version = "1.8.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ed5b98ae99461e02895c3a96a25dbe4d852a3d5e5b815d31d5d9e4b5344d99e9"
|
||||
checksum = "7acd809e80afa754c63f586955699d8990efe1c4b6613183ee5f027a11fbc8b5"
|
||||
dependencies = [
|
||||
"arcstr",
|
||||
"combine",
|
||||
@@ -3806,8 +3767,8 @@ dependencies = [
|
||||
"futures-core",
|
||||
"futures-util",
|
||||
"h2",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"http-body-util",
|
||||
"hyper",
|
||||
"hyper-rustls",
|
||||
@@ -3938,9 +3899,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "rustls-native-certs"
|
||||
version = "0.8.4"
|
||||
version = "0.8.5"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "dab5152771c58876a2146916e53e35057e1a4dfa2b9df0f0305b07f611fdea4d"
|
||||
checksum = "4a073f5dc7469f984c52ad2752b63b0807745133b6de880b7b64c1ac4c48aec4"
|
||||
dependencies = [
|
||||
"openssl-probe",
|
||||
"rustls-pki-types",
|
||||
@@ -4161,9 +4122,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "sha2"
|
||||
version = "0.11.0"
|
||||
version = "0.11.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
|
||||
checksum = "47d7069beb7d6ac7b9acd1039986e73443f24234f41074da099d6f994ac9ad19"
|
||||
dependencies = [
|
||||
"cfg-if",
|
||||
"cpufeatures 0.3.0",
|
||||
@@ -4570,9 +4531,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "tokio"
|
||||
version = "1.53.3"
|
||||
version = "1.54.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "ce3335fa71841cda333a58d7615b03901380ecf09d59b3296d21f8bbac0dde4e"
|
||||
checksum = "b52fd3255258d26fac7264ab5a76b53c13367136dc011d3d6ad4f8ed070396a3"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"libc",
|
||||
@@ -4734,8 +4695,8 @@ dependencies = [
|
||||
"bitflags",
|
||||
"bytes",
|
||||
"futures-util",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"pin-project-lite",
|
||||
"tower",
|
||||
"tower-layer",
|
||||
@@ -4751,8 +4712,8 @@ checksum = "08a05a66a4fdd61cbbe0a1d755ffe0ca6aba159dd4820936a0ff8a8278245b9c"
|
||||
dependencies = [
|
||||
"bitflags",
|
||||
"bytes",
|
||||
"http 1.5.0",
|
||||
"http-body 1.0.1",
|
||||
"http",
|
||||
"http-body",
|
||||
"percent-encoding",
|
||||
"pin-project-lite",
|
||||
"tower",
|
||||
@@ -4861,9 +4822,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "tree-sitter"
|
||||
version = "0.27.0"
|
||||
version = "0.27.1"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "2038684e0058edba0d17302619f62eabce4a8e11c6ac59506996a8d79848851d"
|
||||
checksum = "a5129de2ccc97808c4583d48f1fb44576d015cf241e432800572bda11ef30c3b"
|
||||
dependencies = [
|
||||
"cc",
|
||||
"regex",
|
||||
@@ -4972,7 +4933,7 @@ checksum = "6c01152af293afb9c7c2a57e4b559c5620b421f6d133261c60dd2d0cdb38e6b8"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"data-encoding",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"httparse",
|
||||
"log",
|
||||
"rand 0.9.4",
|
||||
@@ -4988,7 +4949,7 @@ checksum = "e48ac77174b19c110a50ab2128b24215ac9cb40e0e12e093fb602d175c569d22"
|
||||
dependencies = [
|
||||
"bytes",
|
||||
"data-encoding",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"httparse",
|
||||
"log",
|
||||
"rand 0.10.2",
|
||||
@@ -5093,7 +5054,7 @@ source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "e994ba84b0bd1b1b0cf92878b7ef898a5c1760108fe7b6010327e274917a808c"
|
||||
dependencies = [
|
||||
"base64 0.22.1",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"httparse",
|
||||
"log",
|
||||
]
|
||||
@@ -5136,9 +5097,9 @@ checksum = "06abde3611657adf66d383f00b093d7faecc7fa57071cce2578660c9f1010821"
|
||||
|
||||
[[package]]
|
||||
name = "uuid"
|
||||
version = "1.28.0"
|
||||
version = "1.29.0"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "7cc1186384beb7dd8eedea376413fd654937285ea6c9cfbb928dc3043ea4b606"
|
||||
checksum = "4a0e2d11620c50695fe12bbbfde96b3477fe606e7284c22d505c12ce51214514"
|
||||
dependencies = [
|
||||
"getrandom 0.4.3",
|
||||
"js-sys",
|
||||
@@ -5313,9 +5274,9 @@ dependencies = [
|
||||
|
||||
[[package]]
|
||||
name = "webpki-roots"
|
||||
version = "1.0.10"
|
||||
version = "1.0.11"
|
||||
source = "registry+https://github.com/rust-lang/crates.io-index"
|
||||
checksum = "1c1c8e20c8a38fc7d65abfe74eb7280e4aa667e8eec8bc84996d2874810748d9"
|
||||
checksum = "f5f280bbb2b287bc8e7de7e3852945a0ff50d1bbcb5ed42b6c7a568c546df171"
|
||||
dependencies = [
|
||||
"rustls-pki-types",
|
||||
]
|
||||
@@ -5517,7 +5478,7 @@ dependencies = [
|
||||
"base64 0.22.1",
|
||||
"deadpool",
|
||||
"futures",
|
||||
"http 1.5.0",
|
||||
"http",
|
||||
"http-body-util",
|
||||
"hyper",
|
||||
"hyper-util",
|
||||
|
||||
@@ -146,7 +146,7 @@ http = "1"
|
||||
# samples × 8 languages) confirmed 100% identical node-type + line-span
|
||||
# trees at these exact pins. Bumping any pin requires re-running the
|
||||
# canary and re-recording the code_aware_compressor fixtures.
|
||||
tree-sitter = "=0.27.0"
|
||||
tree-sitter = "=0.27.1"
|
||||
tree-sitter-python = "=0.25.0"
|
||||
tree-sitter-javascript = "=0.25.0"
|
||||
tree-sitter-typescript = "=0.23.2"
|
||||
|
||||
@@ -0,0 +1,294 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Reproduce the Cargo parser canary and credentialless native proxy proof.
|
||||
|
||||
Standard-library-only driver. Run from any directory; Cargo uses the repository's
|
||||
pinned toolchain. The parser comparison compiles two temporary standalone crates
|
||||
and uses the eight exact production grammar pins. Runtime proof requires an
|
||||
already-built headroom-proxy binary and never enables model compression.
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import hashlib
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import socket
|
||||
import subprocess
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import urllib.request
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
ROOT = Path(__file__).resolve().parents[1]
|
||||
|
||||
RUST_CANARY = r"""
|
||||
use serde_json::{json, Value};
|
||||
fn visit(n: tree_sitter::Node, out: &mut Vec<Value>) {
|
||||
out.push(json!([
|
||||
n.kind(),
|
||||
n.start_position().row,
|
||||
n.start_position().column,
|
||||
n.end_position().row,
|
||||
n.end_position().column,
|
||||
n.is_error(),
|
||||
n.is_missing()
|
||||
]));
|
||||
let mut c = n.walk();
|
||||
for child in n.children(&mut c) {
|
||||
visit(child, out);
|
||||
}
|
||||
}
|
||||
fn main() {
|
||||
let inputs: Vec<Value> = serde_json::from_str(include_str!("../inputs.json")).unwrap();
|
||||
let mut outputs = vec![];
|
||||
for input in inputs {
|
||||
let l = input["language"].as_str().unwrap();
|
||||
let language: tree_sitter::Language = match l {
|
||||
"python" => tree_sitter_python::LANGUAGE.into(),
|
||||
"javascript" => tree_sitter_javascript::LANGUAGE.into(),
|
||||
"typescript" => tree_sitter_typescript::LANGUAGE_TYPESCRIPT.into(),
|
||||
"go" => tree_sitter_go::LANGUAGE.into(),
|
||||
"rust" => tree_sitter_rust::LANGUAGE.into(),
|
||||
"java" => tree_sitter_java::LANGUAGE.into(),
|
||||
"c" => tree_sitter_c::LANGUAGE.into(),
|
||||
"cpp" => tree_sitter_cpp::LANGUAGE.into(),
|
||||
_ => panic!(),
|
||||
};
|
||||
let mut p = tree_sitter::Parser::new();
|
||||
p.set_language(&language).unwrap();
|
||||
let tree = p.parse(input["source"].as_str().unwrap(), None).unwrap();
|
||||
let mut nodes = vec![];
|
||||
visit(tree.root_node(), &mut nodes);
|
||||
outputs.push(json!({"language":l,"sample":input["sample"],"nodes":nodes}));
|
||||
}
|
||||
println!("{}", serde_json::to_string(&outputs).unwrap());
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def parser_canary(artifacts, target):
|
||||
crate = artifacts / "parser-canary"
|
||||
(crate / "src").mkdir(parents=True, exist_ok=True)
|
||||
fixtures = {}
|
||||
for path in sorted((ROOT / "tests/parity/fixtures/code_aware_compressor").glob("*.json")):
|
||||
fixture = json.loads(path.read_text(encoding="utf-8"))
|
||||
sources = fixtures.setdefault(fixture["output"]["language"], [])
|
||||
if fixture["input"] not in sources:
|
||||
sources.append(fixture["input"])
|
||||
fixtures["c"] = [
|
||||
"struct Point { int x; int y; };\nint main(void) { struct Point p={1,2}; return p.x; }\n",
|
||||
"int apply(int (*fn)(int), int x) { return fn(x); }\nint square(int x) { return x*x; }\nint main(void) { return apply(square, 4); }\n",
|
||||
]
|
||||
inputs = []
|
||||
for language in ("python", "javascript", "typescript", "go", "rust", "java", "c", "cpp"):
|
||||
original, second = fixtures[language][:2]
|
||||
samples = [
|
||||
original,
|
||||
"",
|
||||
original + "\n",
|
||||
second,
|
||||
original.replace("\n", "\r\n"),
|
||||
original[: len(original) // 2],
|
||||
original + "\n} ) ]",
|
||||
original + "\n" + original,
|
||||
original.replace("Processor", "Caf\u00e9")
|
||||
if "Processor" in original
|
||||
else original + "\n\u03bb\u4e2d\U0001f600",
|
||||
]
|
||||
inputs.extend(
|
||||
{"language": language, "sample": i + 1, "source": source}
|
||||
for i, source in enumerate(samples)
|
||||
)
|
||||
(crate / "inputs.json").write_text(
|
||||
json.dumps(inputs, ensure_ascii=False), encoding="utf-8", newline="\n"
|
||||
)
|
||||
(crate / "src/main.rs").write_text(RUST_CANARY, encoding="utf-8", newline="\n")
|
||||
manifest = """[package]
|
||||
name="headroom-parser-qualification"
|
||||
version="0.0.0"
|
||||
edition="2021"
|
||||
[workspace]
|
||||
[dependencies]
|
||||
tree-sitter="=VERSION"
|
||||
serde_json="=1.0.154"
|
||||
tree-sitter-python="=0.25.0"
|
||||
tree-sitter-javascript="=0.25.0"
|
||||
tree-sitter-typescript="=0.23.2"
|
||||
tree-sitter-go="=0.25.0"
|
||||
tree-sitter-rust="=0.24.2"
|
||||
tree-sitter-java="=0.23.5"
|
||||
tree-sitter-c="=0.24.2"
|
||||
tree-sitter-cpp="=0.23.4"
|
||||
"""
|
||||
outputs = []
|
||||
for version in ("0.27.0", "0.27.1"):
|
||||
(crate / "Cargo.toml").write_text(
|
||||
manifest.replace("VERSION", version), encoding="utf-8", newline="\n"
|
||||
)
|
||||
result = subprocess.run(
|
||||
[
|
||||
"cargo",
|
||||
"run",
|
||||
"--quiet",
|
||||
"--manifest-path",
|
||||
str(crate / "Cargo.toml"),
|
||||
"--target-dir",
|
||||
str(target),
|
||||
],
|
||||
cwd=ROOT,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
check=True,
|
||||
)
|
||||
(artifacts / f"parser-{version}.json").write_text(
|
||||
result.stdout, encoding="utf-8", newline="\n"
|
||||
)
|
||||
outputs.append(json.loads(result.stdout))
|
||||
assert len(outputs[0]) == len(outputs[1]) == 72
|
||||
for before, after in zip(*outputs, strict=True):
|
||||
assert before == after, (before["language"], before["sample"])
|
||||
print("AST", before["language"], before["sample"], "PASS", len(before["nodes"]), "nodes")
|
||||
print(
|
||||
"PASS 72/72 reconstructed samples: 8 pinned languages x 9 samples; historical corpus unavailable"
|
||||
)
|
||||
|
||||
|
||||
def native_runtime(binary, artifacts):
|
||||
opener = urllib.request.build_opener(urllib.request.ProxyHandler({}))
|
||||
seen = []
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def do_POST(self):
|
||||
body = b""
|
||||
if self.headers.get("Transfer-Encoding", "").lower() == "chunked":
|
||||
while True:
|
||||
size = int(self.rfile.readline().strip().split(b";")[0], 16)
|
||||
if not size:
|
||||
self.rfile.readline()
|
||||
break
|
||||
body += self.rfile.read(size)
|
||||
self.rfile.read(2)
|
||||
else:
|
||||
body = self.rfile.read(int(self.headers.get("Content-Length", "0")))
|
||||
seen.append((self.path, dict(self.headers), body))
|
||||
answer = b'{"content":[{"type":"text","text":"fixture-ok"}],"usage":{"input_tokens":3,"output_tokens":2}}'
|
||||
self.send_response(200)
|
||||
self.send_header("Content-Type", "application/json")
|
||||
self.send_header("Content-Length", str(len(answer)))
|
||||
self.end_headers()
|
||||
self.wfile.write(answer)
|
||||
|
||||
def log_message(self, *args):
|
||||
pass
|
||||
|
||||
server = http.server.ThreadingHTTPServer(("127.0.0.1", 0), Handler)
|
||||
threading.Thread(target=server.serve_forever, daemon=True).start()
|
||||
with socket.socket() as s:
|
||||
s.bind(("127.0.0.1", 0))
|
||||
port = s.getsockname()[1]
|
||||
base = f"http://127.0.0.1:{server.server_port}"
|
||||
env = {k: v for k, v in os.environ.items() if not k.startswith(("AWS_", "HEADROOM_PROXY_"))}
|
||||
env.update(
|
||||
AWS_ACCESS_KEY_ID="AKIDEXAMPLE",
|
||||
AWS_SECRET_ACCESS_KEY="fake-local-test-secret",
|
||||
AWS_EC2_METADATA_DISABLED="true",
|
||||
AWS_CONFIG_FILE=os.devnull,
|
||||
AWS_SHARED_CREDENTIALS_FILE=os.devnull,
|
||||
AWS_REGION="us-east-1",
|
||||
HEADROOM_OFFLINE="1",
|
||||
HF_HUB_OFFLINE="1",
|
||||
NO_PROXY="127.0.0.1,localhost",
|
||||
TRANSFORMERS_OFFLINE="1",
|
||||
)
|
||||
log = open(artifacts / "runtime-proxy.log", "w", encoding="utf-8", newline="\n")
|
||||
proc = subprocess.Popen(
|
||||
[
|
||||
str(binary),
|
||||
"--listen",
|
||||
f"127.0.0.1:{port}",
|
||||
"--upstream",
|
||||
base,
|
||||
"--enable-bedrock-native",
|
||||
"true",
|
||||
"--bedrock-endpoint",
|
||||
base,
|
||||
],
|
||||
env=env,
|
||||
stdout=log,
|
||||
stderr=subprocess.STDOUT,
|
||||
)
|
||||
try:
|
||||
for _ in range(150):
|
||||
if proc.poll() is not None:
|
||||
raise RuntimeError(f"proxy exited {proc.returncode}")
|
||||
try:
|
||||
with socket.create_connection(("127.0.0.1", port), timeout=0.1):
|
||||
break
|
||||
except OSError:
|
||||
time.sleep(0.1)
|
||||
payload = b'{"anthropic_version":"bedrock-2023-05-31","max_tokens":8,"messages":[{"role":"user","content":"local fixture"}]}'
|
||||
for path in ["/v1/messages", "/model/anthropic.claude-3-haiku-20240307-v1:0/invoke"]:
|
||||
with opener.open(
|
||||
urllib.request.Request(
|
||||
f"http://127.0.0.1:{port}" + path,
|
||||
data=payload,
|
||||
headers={"Content-Type": "application/json"},
|
||||
),
|
||||
timeout=15,
|
||||
) as response:
|
||||
answer = json.load(response)
|
||||
assert answer["content"][0]["text"] == "fixture-ok"
|
||||
rid = uuid.UUID(response.headers["x-request-id"])
|
||||
assert rid.version == 4
|
||||
print(path, response.status, "UUIDv4", str(rid))
|
||||
assert len(seen) == 2, seen
|
||||
assert all(body == payload for _, _, body in seen)
|
||||
headers = {k.lower(): v for k, v in seen[-1][1].items()}
|
||||
assert headers["authorization"].startswith("AWS4-HMAC-SHA256 Credential=AKIDEXAMPLE/")
|
||||
assert headers["x-amz-content-sha256"] == hashlib.sha256(payload).hexdigest()
|
||||
print(
|
||||
"PASS: built binary passthrough + AWS environment provider/SigV4 local wire proof; body byte-equal; payload hash verified. No real AWS/SDK service response emulation."
|
||||
)
|
||||
finally:
|
||||
try:
|
||||
if proc.poll() is None:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=20)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait(timeout=20)
|
||||
finally:
|
||||
try:
|
||||
server.shutdown()
|
||||
finally:
|
||||
try:
|
||||
server.server_close()
|
||||
finally:
|
||||
log.close()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("mode", choices=("canary", "runtime", "all"))
|
||||
parser.add_argument("--binary", type=Path, help="built headroom-proxy executable")
|
||||
parser.add_argument("--artifacts-dir", type=Path)
|
||||
parser.add_argument(
|
||||
"--canary-target-dir", type=Path, default=ROOT / "target/cargo-qualification"
|
||||
)
|
||||
args = parser.parse_args()
|
||||
if args.mode in ("runtime", "all") and not args.binary:
|
||||
parser.error("--binary is required for runtime/all")
|
||||
with tempfile.TemporaryDirectory(prefix="headroom-cargo-proof-") as temporary:
|
||||
artifacts = (args.artifacts_dir or Path(temporary)).resolve()
|
||||
artifacts.mkdir(parents=True, exist_ok=True)
|
||||
if args.mode in ("canary", "all"):
|
||||
parser_canary(artifacts, args.canary_target_dir.resolve())
|
||||
if args.mode in ("runtime", "all"):
|
||||
native_runtime(args.binary.resolve(strict=True), artifacts)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -948,6 +948,9 @@ def install_individual_grammar_parsers() -> None:
|
||||
cache: dict[str, Any] = {}
|
||||
|
||||
def _get_parser(language: str) -> Any:
|
||||
if language not in langs:
|
||||
# detect_language skips unsupported grammars on ValueError.
|
||||
raise ValueError(f"unsupported recorder grammar: {language}")
|
||||
if language not in cache:
|
||||
try:
|
||||
cache[language] = Parser(langs[language])
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""The fixture recorder exposes only its eight pinned grammar wheels."""
|
||||
|
||||
import pytest
|
||||
|
||||
GRAMMARS = ("python", "javascript", "typescript", "go", "rust", "java", "c", "cpp")
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def recorder_parsers(monkeypatch):
|
||||
pytest.importorskip("tree_sitter")
|
||||
for language in GRAMMARS:
|
||||
pytest.importorskip(f"tree_sitter_{language}")
|
||||
import headroom.transforms.code_compressor as compressor
|
||||
from tests.parity.recorder import install_individual_grammar_parsers
|
||||
|
||||
monkeypatch.setattr(compressor, "_get_parser", compressor._get_parser)
|
||||
monkeypatch.setattr(
|
||||
compressor, "_check_tree_sitter_available", compressor._check_tree_sitter_available
|
||||
)
|
||||
install_individual_grammar_parsers()
|
||||
return compressor._get_parser
|
||||
|
||||
|
||||
@pytest.mark.parametrize("language", GRAMMARS)
|
||||
def test_recorder_retains_each_pinned_grammar(recorder_parsers, language):
|
||||
assert recorder_parsers(language).parse(b"").root_node is not None
|
||||
|
||||
|
||||
def test_recorder_unsupported_grammar_uses_detector_skip_contract(recorder_parsers):
|
||||
with pytest.raises(ValueError, match="unsupported recorder grammar: bash"):
|
||||
recorder_parsers("bash")
|
||||
Reference in New Issue
Block a user