mirror of
https://github.com/DeusData/codebase-memory-mcp.git
synced 2026-10-21 05:59:35 +08:00
docs/cli-daemon-behavior
4
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Martin Vogelanddarkverbito
|
706bb2ce81 |
feat(lang): add Chialisp support with a first-party tree-sitter grammar
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:
* comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
* `(include foo.clib)` was rejected because of the dot in the filename;
* `(defconstant NAME <expr>)` accepted only a primitive after the name.
So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.
It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.
Extraction, and the four decisions in it:
* `Constant` is admitted to the cross-file name registry
(cbm_label_is_registry_symbol). Chialisp's condition codes live as
`(defconstant ...)` in a shared .clib and are referenced by name from every
puzzle; labelled but unregistered they are seeded nowhere, and roughly half
of a Chialisp graph would be present as nodes yet never the target of an
edge. All three registry-seeding sites route through that one predicate, so
they cannot diverge. No other language emits `Constant`, so the blast radius
is exactly Chialisp. Pinned by a two-file graph test that goes red when the
label is removed from the predicate.
* `export` / `namespace` are NOT definition heads. `(export foo)` names a
function already defined in the same file, so treating it as a def head
mints a second node for one symbol. They stay in the not-a-call filter.
* A parameter list is a `list` too, but it binds rather than invokes. Without
that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
* `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
recording it would invent a dependency on a file that does not exist.
Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.
Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.
Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.
Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
|
||
|
Martin Vogel
|
032ebf5327 |
fix(vendor): ship arkts upstream MIT byte-identical and register the fork
The vendored LICENSE carried tree-sitter-typescript's MIT text plus an appended derivative notice. That is not what MIT asks of a derivative -- it asks that upstream's notice travel with the code -- and the appended text meant the provenance audit could not byte-verify the file against upstream, so arkts landed as NO-MANIFEST-ENTRY and failed license-gate. The vendored LICENSE is now byte-identical to upstream's, and arkts is registered in the audit's FORKS map, which is defined for exactly this case: a self-maintained fork whose vendored LICENSE must match its upstream. That is a routing entry, not an exemption -- check_upstream still byte-verifies against the real upstream repository, so the fact is checked rather than asserted. Audit now reports arkts as IDENTICAL. Our copyright for the ArkTS additions is not lost, it moves to where our SOURCE is: the grammar.js header (which now states all three copyrights explicitly) and THIRD_PARTY.md, which ships inside the release archives. The edit to scripts/audit-license-provenance.py is a maintainer-only enforcement-surface change, made deliberately and stated here rather than slipped in; a contribution may never supply its own audit entry. No generated parser or source changes -- comments, licence text, manifest prose and one routing entry only. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
0df990aa60 |
Add ArkTS (.ets) language support via first-party tree-sitter-typescript fork
Routing .ets through the plain TypeScript grammar produced no definition node for any @Component struct (0/19 across the OpenHarmony sample corpus), parsed struct members as statement labels and build() as a call expression, and registered ArkUI builtins (Column, Row, ListItem, ...) as user function definitions with 28% fabricated cross-file CALLS edges. This adds a dedicated ArkTS grammar that keeps everything TypeScript already handles and models the ArkTS syntax TS cannot parse. Grammar (tools/tree-sitter-arkts/, generated parser vendored to internal/cbm/vendored/grammars/arkts/, ABI 15 via tree-sitter-cli 0.25.10): first-party derivative of tree-sitter/tree-sitter-typescript v0.23.2 @ 75b3874edb2d (the commit our vendored typescript/tsx come from) on the tree-sitter-javascript v0.23.1 base @ 3a837b6f3658 (the version upstream's package-lock pins). LICENSE carries the upstream MIT text verbatim plus a derivative notice; the external scanner is a symbol-rename trampoline over a byte-identical copy of the already-vendored typescript scanner header. ArkTS additions: decorated struct component declarations with class-shaped bodies; UI-DSL trailing-closure component calls with attribute chains after the closing brace; decorators on function declarations (@Builder, @Extend(T), @Styles); @Extend/@Styles leading-dot attribute-chain bodies; anonymous stateStyles style blocks; import lazy named imports. Wiring: CBM_LANG_ARKTS enum + lang spec (struct_declaration as a class kind labeled Struct, public_field_definition fields), .ets extension, LANG_NAMES/userconfig entries, grammar shim, and TS-family joins across the extractors and the name-resolution guards (parse_es_imports, JS keywords/test-file/func-kind/scope tables, TS heritage/param/receiver/ namespace/type-ref/channel/usage paths, js_ts_family and the weak-method- match suppression). Field definitions now also carry their decorators, so @State/@Prop/@Link members are findable with their roles. Tests: grammar corpus (35 cases), per-language regression + label golden + import net + extension cases, seven ArkTS extraction tests including the phantom-builtin regression (ArkUI builtins must never be definitions), and capability-ledger / call-node-manifest / call-argument-matrix rows. On the 14-file OpenHarmony sample corpus the TS-grammar baseline had 12/14 files with parse errors (231 ERROR nodes); the arkts grammar parses 14/14 clean, and an end-to-end index yields all 19 components as Struct nodes, 68 methods, 119 fields, and zero phantom builtin definitions. Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com> |
||
|
Martin Vogel
|
18fa9979ee |
Replace vmem with mimalloc global allocator, add extraction-phase prescan, fix __init__.py QN collision
Memory management: - Vendor mimalloc v2.1.9 as global allocator (MI_OVERRIDE=1 in prod) - New mem.h/mem.c: RSS-based budget tracking via mi_process_info() - Remove vmem.c/vmem.h (mmap-based budget tracking) - Remove slab tier2 bump allocator (~300 LOC); >64B goes to mimalloc - Slab tier1 pages from malloc (= mimalloc) instead of vmem - Arena blocks from malloc instead of vmem - Budget raised from 35% to 50% RAM (no more untracked C++ heap) Extraction-phase prescan (eliminates disk re-reads): - HTTP call sites: keyword check + URL extraction during extraction - HTTP routes: decorator + source-based extraction during extraction - Config file refs: regex scan during extraction - httplinks: 41.8s → 13ms on Linux kernel (3,212x faster) - configlink: 41.4s → 0.8s on Linux kernel (54x faster) - Linux kernel fast-mode total: 2m38s → 1m18s Bug fixes: - __init__.py Module QN no longer collides with Folder QN - index.ts same fix for JS/TS packages - 13 regression tests for QN collision at FQN + extraction layers - search_graph/search_code default limit raised from 10 to 500k - Resolve all clang-tidy, cppcheck, and clang-format warnings Repo cleanup: - tree-sitter-form, tree-sitter-magma moved to tools/ - .gitignore: build/, node_modules/, graph-ui/dist/, TEST_PLAN.md |