trending/codebase-memory-mcp
1
0
Fork 0
mirror of https://github.com/DeusData/codebase-memory-mcp.git synced 2026-11-09 09:11:00 +08:00
Code Issues Packages Projects Releases Wiki Activity
Files
282fbfcdabfec79c19c36f5bec5f75c22eb36df5
codebase-memory-mcp/THIRD_PARTY.md
T
Martin Vogelanddarkverbito 706bb2ce81 feat(lang): add Chialisp support with a first-party tree-sitter grammar 
Chialisp (.clsp/.clib/.clinc) is the Chia smart-coin language. The only
public grammar, Quexington/tree-sitter-chialisp, cannot parse it: on the five
reference files from chia-blockchain@main it produced 3-34 ERROR nodes each.
Three defects account for all of them, and each desynchronised the rest of the
file rather than losing one node:

  * comments were `/;.*\r\n/` -- CRLF-only, while real files are LF;
  * `(include foo.clib)` was rejected because of the dot in the filename;
  * `(defconstant NAME <expr>)` accepted only a primitive after the name.

So the grammar is written here instead, in tools/tree-sitter-chialisp/, and
files as first-party: its vendored LICENSE is the project's own, byte-identical
to the repository root, carrying no third-party copyright.

It is deliberately GENERIC -- source_file / list / symbol / string / number /
hex / dot / comment, modelling the clvm_tools reader rather than the Chialisp
form vocabulary. `mod`, `defun`, `defconstant` and `include` are ordinary head
symbols; which lists are definitions is decided in extract_defs.c. A dialect
that adds a form (cl-21 -> cl-23 -> cl-26 each added several) therefore needs
no regenerated parser. Acceptance: all five reference files parse with zero
ERROR and zero MISSING nodes; the three defects above are pinned as tests.

Extraction, and the four decisions in it:

  * `Constant` is admitted to the cross-file name registry
    (cbm_label_is_registry_symbol). Chialisp's condition codes live as
    `(defconstant ...)` in a shared .clib and are referenced by name from every
    puzzle; labelled but unregistered they are seeded nowhere, and roughly half
    of a Chialisp graph would be present as nodes yet never the target of an
    edge. All three registry-seeding sites route through that one predicate, so
    they cannot diverge. No other language emits `Constant`, so the blast radius
    is exactly Chialisp. Pinned by a two-file graph test that goes red when the
    label is removed from the predicate.
  * `export` / `namespace` are NOT definition heads. `(export foo)` names a
    function already defined in the same file, so treating it as a def head
    mints a second node for one symbol. They stay in the not-a-call filter.
  * A parameter list is a `list` too, but it binds rather than invokes. Without
    that gate `(defun check_conditions (HEIGHTLOCK conditions) ...)` mints a
    phantom call to HEIGHTLOCK, and real Chialisp is mostly such definitions.
  * `(include *standard-cl-26*)` is a dialect directive, not a file dependency;
    recording it would invent a dependency on a file that does not exist.

Three helpers that the defs, calls and unified walks all need -- the quote-data
gate, comment-skipping named-child access, and the Chialisp def-head set -- live
in helpers.c rather than being copied per translation unit. They previously
existed as byte-identical duplicates that had already drifted: extract_defs.c
skipped comments when reading a def head while its mirror in extract_unified.c
did not, so a comment between a head and its name desynchronised defs from call
scope. Both now use the same accessor.

Ledger: 161 -> 162 grammars across the seven published surfaces, ABI-14 86 -> 87,
first-party 13 -> 14, CBM_LANG_COUNT 164 -> 165.

Distilled from PR #919 by darkverbito, whose extraction logic -- the CLVM
not-a-call vocabulary, the quoted-data gate, the filename-derived `mod` name and
the include/embed import handling -- is kept and is the substance of this change.
The grammar that PR vendored is not used.

Co-authored-by: darkverbito <8049985+darkverbito@users.noreply.github.com>
Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
2026-08-28 21:30:06 +02:00

181 lines
10 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Third-Party Licenses
This project vendors third-party code. We are grateful to the authors and
maintainers of these projects for making their work freely available.
Every vendored component directory carries the upstream `LICENSE`
(or `COPYING` / `NOTICE`) file alongside the sources.
## Tree-sitter Runtime
The tree-sitter C runtime is vendored in `internal/cbm/vendored/ts_runtime/`.
- **Project:** [tree-sitter](https://github.com/tree-sitter/tree-sitter)
- **License:** MIT
- **Copyright:** (c) 2018–2024 Max Brunsfeld
**Local modification** (`internal/cbm/vendored/ts_runtime/src/stack.c`, #913): a
single CBM patch bounds the recursive ambiguity-merge in `stack_node_add_link`
at `CBM_TS_STACK_MERGE_MAX_DEPTH` (512). Deeply nested grammar-ambiguous input
(e.g. Perl `f(f(f(...)))`) otherwise recurses once per level on the native C
stack and overflows it during parsing (SIGSEGV on the ~1 MB Windows thread
stack, and even the 8 MB POSIX stack at extreme depth) before any extractor
runs. Past the cap the ambiguity is left on the GLR stack instead of merged —
still a valid parse, never a wrong one — mirroring the existing
`MAX_LINK_COUNT` bail-out. The change is clearly marked `// CBM patch:` inline.
**On re-vendor (e.g. ts_runtime → 0.26.x): re-apply this bound.**
The shared scanner helpers in `internal/cbm/vendored/common/` (`scanner.h`,
`tag.h`) originate from
[tree-sitter-html](https://github.com/tree-sitter/tree-sitter-html) (MIT,
(c) 2014 Max Brunsfeld) and carry that project's `LICENSE` in
`internal/cbm/vendored/common/`.
The core runtime headers in `internal/cbm/vendored/common/tree_sitter/`
(`alloc.h`, `array.h`, `parser.h`) are part of the tree-sitter C runtime
([tree-sitter](https://github.com/tree-sitter/tree-sitter), MIT,
(c) 2018 Max Brunsfeld) and carry their own `LICENSE` in that directory.
## Tree-sitter Grammars
160 pre-generated parsers are vendored in `internal/cbm/vendored/grammars/<lang>/`
(generated `parser.c` plus `scanner.c` where applicable, compiled statically).
Each grammar is the work of its upstream authors and each grammar directory
contains the upstream `LICENSE` file.
The **canonical provenance record** — upstream repository, pinned commit, and
cross-registry verification status for every grammar — is
[`internal/cbm/vendored/grammars/MANIFEST.md`](internal/cbm/vendored/grammars/MANIFEST.md).
License summary:
- Nearly all grammars are **MIT**-licensed.
- `clojure` ([sogaiu/tree-sitter-clojure](https://github.com/sogaiu/tree-sitter-clojure)) is **CC0-1.0**;
`fennel` is **CC0-1.0**; `jinja2` and `just` are **Apache-2.0**;
`pine` is **ISC** (declared by its upstream).
- The grammars authored in-house for this project (`chialisp`, `cobol`, `form`,
`janet`, `magma`, `protobuf`, `wolfram`) are **MIT** under the project's own
license, (c) DeusData. Each ships the repository's own LICENSE, byte-identical
to the root copy; they carry no third-party copyright because there is no
third party. `chialisp` is a generic s-expression grammar for the Chia
smart-coin language, written for this project because no usable public
grammar exists; its source and corpus tests live in
`tools/tree-sitter-chialisp/`.
- Seven further grammars (`arkts`, `assembly`, `cfml`,
`cfscript`, `dotenv`, `pine`, `qml`) are self-maintained forks that retain
their original upstream authors' licenses — see the manifest for per-grammar
provenance. `arkts` is a first-party derivative of
[tree-sitter/tree-sitter-typescript](https://github.com/tree-sitter/tree-sitter-typescript)
(MIT, (c) 2017 Max Brunsfeld; on the
[tree-sitter-javascript](https://github.com/tree-sitter/tree-sitter-javascript)
base, MIT, (c) 2014 Max Brunsfeld) with (c) 2026 DeusData ArkTS additions;
its grammar source lives in `tools/tree-sitter-arkts/`.
### tree-sitter-plsql
- **Project:** [AndreasMaierDe/tree-sitter-plsql](https://github.com/AndreasMaierDe/tree-sitter-plsql)
- **License:** MIT
- **Copyright:** (c) 2022 AndreasMaierDe
- **Vendored at:** `internal/cbm/vendored/grammars/plsql/`
- **Pinned commit:** `28aebef209be`
- **Notes:** Community-maintained grammar for Oracle PL/SQL; not in the
nvim-treesitter or Helix registries (`community-niche` in the manifest). No
external scanner. One local patch: `parser.c`'s
`#include <tree_sitter/parser.h>` is changed to the quoted form used by every
other vendored grammar, as documented in
`internal/cbm/vendored/grammars/MANIFEST.md`. PL/SQL support was originally
contributed in PR #1033 by Oğuz (@ouzsrcm).
### tree-sitter-objectscript (UDL + routine)
- **Project:** [intersystems/tree-sitter-objectscript](https://github.com/intersystems/tree-sitter-objectscript)
- **License:** MIT
- **Copyright:** (c) 2025 InterSystems Corporation
- **Vendored at:** `internal/cbm/vendored/grammars/objectscript_udl/`, `internal/cbm/vendored/grammars/objectscript_routine/`
- **Pinned commit:** `a7ffcdf`
- **Notes:** InterSystems-maintained grammar for the ObjectScript language (InterSystems IRIS / Caché). Vendor-maintained; not in nvim-treesitter or Helix registries. Each `scanner.c`'s upstream `#include "../../common/scanner.h"` is repointed to a per-directory `objectscript_common.h` copied from upstream `common/scanner.h`; two loop counters in that copy are widened from `uint8_t` to `int` as documented in `internal/cbm/vendored/grammars/MANIFEST.md`.
## Vendored C/C++ Libraries
| Library | Path | License | Project |
|---------|------|---------|---------|
| SQLite 3 | `vendored/sqlite3/` | Public Domain | [sqlite.org](https://www.sqlite.org/) |
| mimalloc | `vendored/mimalloc/` | MIT | [microsoft/mimalloc](https://github.com/microsoft/mimalloc) |
| yyjson | `vendored/yyjson/` | MIT | [ibireme/yyjson](https://github.com/ibireme/yyjson) |
| xxHash | `vendored/xxhash/` | BSD-2-Clause | [Cyan4973/xxHash](https://github.com/Cyan4973/xxHash) |
| TRE | `vendored/tre/` | BSD-2-Clause | [laurikari/tre](https://github.com/laurikari/tre) |
| LZ4 | `internal/cbm/vendored/lz4/` | BSD-2-Clause (library files) | [lz4/lz4](https://github.com/lz4/lz4) |
| Zstandard | `internal/cbm/vendored/zstd/` | BSD-3-Clause (dual BSD / GPLv2 — BSD selected) | [facebook/zstd](https://github.com/facebook/zstd) |
| simplecpp | `internal/cbm/vendored/simplecpp/` | 0BSD | [danmar/simplecpp](https://github.com/danmar/simplecpp) |
| Verstable | `internal/cbm/vendored/verstable/` | MIT | [JacksonAllan/Verstable](https://github.com/JacksonAllan/Verstable) |
| wyhash | `internal/cbm/vendored/wyhash/` | Unlicense (public domain) | [wangyi-fudan/wyhash](https://github.com/wangyi-fudan/wyhash) |
Local modifications to these libraries are documented next to the
vendored sources (currently only SQLite: `vendored/sqlite3/PATCHES.md`,
raising the Unix VFS `MAX_PATHNAME` ceiling from 512 to 4096 to match
CBM's 4 KiB path support). Patches must be reapplied on every upstream
refresh and are covered by `scripts/vendored-checksums.txt`.
The graph-UI HTTP server is a first-party implementation
(`src/ui/httpd.c` + `src/ui/http_server.c`) — no third-party HTTP library
is used.
## Embedded Model Data
Semantic vector search uses static token embeddings derived from the
**nomic-embed-code** model, vendored in `vendored/nomic/`:
- **Model:** [nomic-ai/nomic-embed-code](https://huggingface.co/nomic-ai/nomic-embed-code)
- **License:** Apache License 2.0
- **Copyright:** (c) Nomic AI
See `vendored/nomic/NOTICE` for the exact derivation procedure
(per-token inference + int8 quantization via `scripts/extract_nomic_vectors.py`).
## Hybrid LSP — Reference Language Servers
The Hybrid LSP layer (`internal/cbm/lsp/`) is an original C implementation
written for this project. **It contains no source code from any language
server.** Its type-resolution behavior is structurally inspired by, and
validated for output compatibility against, the published behavior of the
following language servers and language specifications. They are listed here
as acknowledgment; their licenses are noted for reference:
| Language | Reference implementation / specification | Upstream license |
|----------|-------------------------------------------|------------------|
| TypeScript / JavaScript | tsserver ([microsoft/TypeScript](https://github.com/microsoft/TypeScript)), [typescript-go](https://github.com/microsoft/typescript-go) | Apache-2.0 |
| Python | [pyright](https://github.com/microsoft/pyright) | MIT |
| Go | gopls ([golang/tools](https://github.com/golang/tools)) | BSD-3-Clause |
| PHP | PHP language reference + Composer PSR-4 autoloading specification | — |
| C# | Roslyn ([dotnet/roslyn](https://github.com/dotnet/roslyn)) | MIT |
| C / C++ | clangd ([llvm/llvm-project](https://github.com/llvm/llvm-project)) | Apache-2.0 WITH LLVM-exception |
| Java | Java Language Specification; output parity with [Eclipse JDT LS](https://github.com/eclipse-jdtls/eclipse.jdt.ls) | EPL-2.0 (reference only) |
| Kotlin | Kotlin language specification; [fwcd/kotlin-language-server](https://github.com/fwcd/kotlin-language-server) | MIT |
| Rust | [rust-analyzer](https://github.com/rust-lang/rust-analyzer) | MIT OR Apache-2.0 |
### Standard-library type data
The stdlib type registries in `internal/cbm/lsp/generated/` were produced as
follows:
- **Python** (`python_stdlib_data.c`) — generated from
[python/typeshed](https://github.com/python/typeshed) type stubs
(commit `a7912d521e16ff63caf7a8b64b9072542be36777`), **Apache-2.0**,
(c) the typeshed contributors. The generator is `scripts/gen-py-stdlib.py`.
- **Go** (`go_stdlib_data.c`) — generated by introspecting the public API of
the Go standard library ([golang/go](https://github.com/golang/go),
BSD-3-Clause).
- **Java, Kotlin, C#, PHP, C/C++, Rust** — hand-curated from public API
documentation and language specifications; no upstream source code was
extracted or transcribed.
## Embedded Graph UI
Release binaries built with `--with-ui` embed the compiled `graph-ui/`
frontend bundle. Its npm dependencies (React, three.js, @react-three/*,
radix-ui, lucide-react, tailwindcss, and friends) are all under permissive
licenses (MIT / ISC / Apache-2.0 / Zlib); the exact set is recorded in
`graph-ui/package.json` and `graph-ui/package-lock.json`, and the per-package
license texts of the production bundle are appended to the
`THIRD_PARTY_NOTICES.md` shipped inside the `-ui` release archives
(generated by `scripts/gen-ui-licenses.py`).
Reference in New Issue View Git Blame Copy Permalink