mirror of
https://github.com/Crosstalk-Solutions/project-nomad.git
synced 2026-10-02 03:24:51 +08:00
chriscrosstalkandClaude Opus 5
1b67a87254
* feat(translate): offline translation for the Information Library Implements #1272. Ships as a Supply Depot app so users who do not need translation never see it. A path-preserving reverse proxy in front of kiwix-serve. /viewer, /skin/*, /search, /random and /catalog/* are forwarded byte for byte; only text/html under /content/ is rewritten. Preserving the path is the whole design: viewer.js keeps the content iframe in sync with location.hash by comparing contentWindow.location.pathname, so moving the path starts the ping-pong its own comment warns about and forces us to patch Kiwix's JavaScript on every Kiwix bump. Because the path never moves, no Kiwix JS is modified, and Kiwix's header, search, library and random keep working because they are Kiwix. Language is a cookie, so links inside the ZIM need no rewriting either. Bergamot rather than the AI Assistant: measured on the same box and the same passage, 12,818 words/s on CPU against llama3.1:8b at 8 words/s on an RTX 5060. It is also more careful with names, where the LLM translated the product name and a smaller model hallucinated a clause. No GPU, so it works on CPU-only boxes where the LLM path is unusable. Models come from Firefox Remote Settings, the endpoint Firefox itself uses, not the mozilla/firefox-translations-models repo, which was archived on 2026-08-21. MPL-2.0, about 37 MB per direction. Failure handling is deliberate throughout: a failed model fetch is non-fatal, a translation error returns the original article with a comment rather than an error page, and a box with no models still starts and still forwards the library. Translation being unavailable must never make the Information Library unreachable. amd64 only, and pinned as such in the build workflow. Bergamot's intgemm backend is x86 specific and there is no aarch64 wheel, so a multi-arch build would produce an ARM image that pulls cleanly and dies at start. Also adds .gitattributes pinning *.sh to LF. The repo stores LF but Windows checks out CRLF, and a CRLF after a shebang makes the interpreter unresolvable inside the image. CI builds on Linux so this never bit, but it breaks a local image build and would have bitten the existing sidecar script too. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV * fix(translate): vary the cache on the language cookie Found by clicking the language buttons in a browser rather than trusting curl. Kiwix sends ETag and Cache-Control: max-age=3600, and the proxy forwarded both unchanged on a response whose content now varies by cookie. The browser cached the English article for an hour and served it straight back when the reader picked a language, so the feature looked like it simply did not work. A shared cache would have been worse, serving one reader's language to another. Translatable responses now send Vary: Cookie, and their ETag is namespaced by language so revalidation still works per language rather than being turned off. Forwarded paths are untouched, so /viewer, /skin/* and /catalog/* stay byte for byte identical to Kiwix. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV * fix(translate): move off port 8480, which Vaultwarden already uses Caught by seeing the two cards side by side in Supply Depot with the same port. My port survey grepped ui_location for a bare number, and Vaultwarden's is 'https:8480' because it needs an https Open link, so it did not match and 8480 looked free. Moved to 8460, which fits the existing spacing between Meshtastic Web on 8450 and Homebox on 8470. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV * fix(translate): pin the base image to python 3.10 The build fails outright on 3.12: bergamot publishes no wheel for it and pip resolves no further than 0.1.2. The spike used 3.10 for this reason and I had modernised it without checking. Caught by building the image rather than reading it. Moving off this pin means moving off the 2022 PyPI wheel to a build from mozilla/translations, which is the follow-up already noted in the Dockerfile. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV * fix(translate): translate text that lives in a bare div Sotoki-generated StackExchange ZIMs put every question excerpt in a <div class="...excerpt">, so on a questions list the titles translated and the descriptions underneath them did not. Reported against the ham radio ZIM. div could not simply be added to BLOCK_TAGS: the outermost-block rule would claim a top-level layout wrapper and send the whole page as one job. It is now claimed speculatively and abandoned the moment anything block-level opens inside it, so a wrapper yields to the blocks it contains and only a genuine text leaf is translated. Also moves the test harness's reporting block back to the end of the file. New checks had been appended after it, so they executed but their failures were never reported: verified that removing div from the set now fails 3 checks, where before it reported success. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV * fix(translate): decode numeric character references before translating Bergamot's HTML mode decodes named entities (`&` arrives as "&") but not numeric ones. It treats `'` as literal text and escapes the ampersand on the way out, so "If I'm in the wilderness" came back as "Si I&#x27;m dans la nature" and rendered the escape sequence visibly to the user. Confirmed by round-tripping fragments through the model directly. Numeric refs are now decoded before the round trip. `&`, `<` and `>` are deliberately left encoded: decoding those would inject real markup into the fragment and fail the balance check that gates HTML mode, dropping the block to plain text and losing its inline links. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01F8Mgg1vd2eKSs8StZiuyMV --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
79 lines
1.2 KiB
Plaintext
79 lines
1.2 KiB
Plaintext
# Logs
|
|
logs
|
|
*.log
|
|
|
|
# Diagnostic reports (https://nodejs.org/api/report.html)
|
|
report.[0-9]*.[0-9]*.[0-9]*.[0-9]*.json
|
|
|
|
# Compiled binary addons (https://nodejs.org/api/addons.html)
|
|
build/Release
|
|
|
|
# Dependency directories
|
|
node_modules/
|
|
|
|
# Optional npm cache directory
|
|
.npm
|
|
|
|
# dotenv environment variables file
|
|
.env
|
|
.env.*
|
|
!.env.example
|
|
|
|
# Credentials and keys. Never commit these — the installer generates every
|
|
# password and app key locally at install time, so nothing real belongs in the
|
|
# repo. See SECURITY.md.
|
|
*.pem
|
|
*.key
|
|
*.p12
|
|
*.pfx
|
|
*.ppk
|
|
id_rsa
|
|
id_ed25519
|
|
.npmrc
|
|
.netrc
|
|
.htpasswd
|
|
|
|
# Local copies of the deployed compose file, which contain the real generated
|
|
# DB passwords and APP_KEY. The tracked install/management_compose.yaml template
|
|
# is the only compose file that belongs in git.
|
|
compose.yml
|
|
compose.yaml
|
|
docker-compose.yml
|
|
docker-compose.yaml
|
|
management_compose.local.yaml
|
|
|
|
# Build / Dist
|
|
dist
|
|
build
|
|
tmp
|
|
|
|
# macOS Metafiles
|
|
.DS_Store
|
|
|
|
# Fonts
|
|
.ttf
|
|
|
|
# Runtime-generated Files
|
|
server/public
|
|
server/temp
|
|
|
|
# IDE Files
|
|
.vscode
|
|
.idea
|
|
|
|
# Agent Files
|
|
.agents
|
|
.claude
|
|
.codegraph
|
|
.mcp.json
|
|
|
|
# Frontend assets compiled code
|
|
admin/public/assets
|
|
|
|
# Admin specific development files
|
|
admin/storage
|
|
|
|
# Python bytecode (install/nomad-translate helper scripts)
|
|
__pycache__/
|
|
*.pyc
|