Compare commits
2
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 nesquena-hermes
|
a03d718ae0 | ||
|
nesquena-hermesandpix0127
|
5f3d967d3b |
@@ -5,4 +5,3 @@ __pycache__
|
||||
*.pyo
|
||||
tests/
|
||||
.env*
|
||||
/AGENTS.md
|
||||
|
||||
@@ -45,42 +45,8 @@ GID=1000
|
||||
# REQUIRED if you expose the container on anything other than 127.0.0.1.
|
||||
# Without a password, anyone who can reach the port can run commands as
|
||||
# the agent.
|
||||
#
|
||||
# All three compose files forward this into the WebUI container, so
|
||||
# uncommenting it here is sufficient — no compose edit needed:
|
||||
# HERMES_WEBUI_PASSWORD=change-me-to-something-strong
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Additional security / networking / limit knobs — advanced
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# The WebUI process reads more HERMES_WEBUI_* variables than the compose
|
||||
# files forward by default. To use any of these in Docker, uncomment it here
|
||||
# AND add a matching passthrough line under `environment:` in your compose
|
||||
# file (e.g. `- HERMES_WEBUI_SESSION_TTL=${HERMES_WEBUI_SESSION_TTL:-}`).
|
||||
# Full descriptions live in .env.example.
|
||||
#
|
||||
# Auth cookie lifetime in seconds (default 2592000 = 30 days):
|
||||
# HERMES_WEBUI_SESSION_TTL=2592000
|
||||
#
|
||||
# Force the Secure cookie flag when serving HTTPS via a reverse proxy:
|
||||
# HERMES_WEBUI_SECURE=1
|
||||
#
|
||||
# Allowed public origins (scheme required) when behind a reverse proxy:
|
||||
# HERMES_WEBUI_ALLOWED_ORIGINS=https://myapp.example.com
|
||||
#
|
||||
# Trust reverse-proxy forwarded headers (only enable behind a proxy YOU run):
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_HOST=1
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_FOR=1
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_PROTO=1
|
||||
#
|
||||
# Enable the passkey/WebAuthn login surface (default off):
|
||||
# HERMES_WEBUI_PASSKEY=1
|
||||
#
|
||||
# Archive extraction cap (MB) and "download folder as ZIP" caps:
|
||||
# HERMES_WEBUI_MAX_EXTRACTED_MB=2048
|
||||
# HERMES_WEBUI_FOLDER_ZIP_MAX_MB=1024
|
||||
# HERMES_WEBUI_FOLDER_ZIP_MAX_FILES=50000
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Permission handling for bind-mounted .hermes — advanced
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
@@ -121,15 +87,3 @@ GID=1000
|
||||
# bind), ALL THREE containers must mount the SAME host path and run as the
|
||||
# SAME UID/GID. Mismatched UIDs → "Permission denied" → the WebUI crashes
|
||||
# on every HTTP request because it can't read its own auth signing key.
|
||||
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# Gateway API server — multi-container only
|
||||
# ──────────────────────────────────────────────────────────────────────────
|
||||
# The two/three-container compose files forward this into the hermes-agent
|
||||
# container. The agent only starts the gateway API listener (port 8642) when
|
||||
# a usable API_SERVER_KEY is present (>=16 chars) — API_SERVER_ENABLED alone
|
||||
# does nothing. Without a key, the gateway daemon still runs (messaging +
|
||||
# cron) but the WebUI's Tasks/System pill shows "Gateway endpoint not
|
||||
# reachable". Set a long random string here to enable it; the same value is
|
||||
# forwarded to the WebUI as HERMES_WEBUI_GATEWAY_API_KEY automatically:
|
||||
# API_SERVER_KEY=replace-with-a-long-random-string
|
||||
|
||||
-117
@@ -32,120 +32,3 @@
|
||||
|
||||
# Display name for the assistant in the UI (default: Hermes)
|
||||
# HERMES_WEBUI_BOT_NAME=Hermes
|
||||
|
||||
# ── Security & remote access ─────────────────────────────────────────────────
|
||||
|
||||
# Password for the login page. REQUIRED if you bind to anything other than
|
||||
# 127.0.0.1 — without it, anyone who can reach the port can run commands.
|
||||
# HERMES_WEBUI_PASSWORD=change-me-to-something-strong
|
||||
|
||||
# Enable the passkey/WebAuthn login surface (default: off).
|
||||
# Alternative: set `webui_passkey_enabled: true` in the profile's config.yaml.
|
||||
# HERMES_WEBUI_PASSKEY=1
|
||||
|
||||
# Auth cookie lifetime in seconds (default: 2592000 = 30 days).
|
||||
# Clamped to [60, 31536000]. Also settable as session_ttl_seconds in settings.json.
|
||||
# HERMES_WEBUI_SESSION_TTL=2592000
|
||||
|
||||
# Force the Secure flag on auth cookies: 1/true or 0/false.
|
||||
# Unset = auto-detect (direct TLS socket, or a trusted X-Forwarded-Proto).
|
||||
# HERMES_WEBUI_SECURE=1
|
||||
|
||||
# Override the auth cookie name (default: derived internally).
|
||||
# HERMES_WEBUI_COOKIE_NAME=hermes_webui_session
|
||||
|
||||
# Extra allowed origins for CSRF/origin checks when serving behind a reverse
|
||||
# proxy or on a public hostname. Comma-separated; each entry MUST include the
|
||||
# scheme. Entries without a scheme are ignored with a warning.
|
||||
# HERMES_WEBUI_ALLOWED_ORIGINS=https://myapp.example.com:8000
|
||||
|
||||
# Serve HTTPS directly (both must be set; otherwise plain HTTP).
|
||||
# HERMES_WEBUI_TLS_CERT=/path/to/fullchain.pem
|
||||
# HERMES_WEBUI_TLS_KEY=/path/to/privkey.pem
|
||||
|
||||
# Extra sources appended to the Content-Security-Policy (space-separated).
|
||||
# HERMES_WEBUI_CSP_CONNECT_EXTRA=https://api.example.com wss://api.example.com
|
||||
# HERMES_WEBUI_CSP_FRAME_EXTRA=https://embed.example.com
|
||||
|
||||
# ── Reverse-proxy header trust (opt-in; only enable behind a proxy YOU run) ───
|
||||
# By default these forwarded headers are NOT trusted, because any direct client
|
||||
# could forge them. Enable (1/true/yes/on) only when a reverse proxy you control
|
||||
# sets them. TRUST_FORWARDED_HOST: honor X-Forwarded-Host/X-Real-Host in origin
|
||||
# checks. TRUST_FORWARDED_FOR: honor X-Forwarded-For for the client IP.
|
||||
# TRUST_FORWARDED_PROTO: honor X-Forwarded-Proto for HTTPS detection.
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_HOST=1
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_FOR=1
|
||||
# HERMES_WEBUI_TRUST_FORWARDED_PROTO=1
|
||||
|
||||
# ── Trusted-header auth / reverse-proxy SSO (opt-in; OFF by default) ──────────
|
||||
# Delegate authentication to a reverse proxy you run (Authelia, oauth2-proxy,
|
||||
# corporate SSO, …) that authenticates the user and forwards their identity in a
|
||||
# header. When HERMES_WEBUI_TRUSTED_AUTH_HEADER is set, the WebUI treats a request
|
||||
# as that identity — but ONLY when the request's un-spoofable raw socket peer is
|
||||
# an allowlisted proxy (HERMES_WEBUI_TRUSTED_PROXY_CIDRS below). With no header
|
||||
# set, this path is disabled and normal password/passkey auth applies.
|
||||
#
|
||||
# ⚠️ SECURITY: HERMES_WEBUI_TRUSTED_PROXY_CIDRS MUST list the EXACT IP(s) of your
|
||||
# proxy — e.g. 10.0.0.5/32 or 127.0.0.1/32. NEVER use 0.0.0.0/0 or a broad range:
|
||||
# any client whose peer IP falls in the range could forge the identity header and
|
||||
# log in as anyone. If the allowlist is empty/malformed, the header is ignored
|
||||
# (fails closed). Loopback is trusted by default. Set the CIDR to your proxy only.
|
||||
# HERMES_WEBUI_TRUSTED_AUTH_HEADER=Remote-User
|
||||
# HERMES_WEBUI_TRUSTED_PROXY_CIDRS=10.0.0.5/32
|
||||
# Optional: map a forwarded groups header to WebUI profiles (JSON object), and a
|
||||
# logout URL to redirect to after clearing the local session (SSO single-logout).
|
||||
# HERMES_WEBUI_TRUSTED_GROUPS_HEADER=Remote-Groups
|
||||
# HERMES_WEBUI_GROUP_PROFILE_MAP={"admins":"default","team-a":"projecta"}
|
||||
# HERMES_WEBUI_TRUSTED_AUTH_LOGOUT_URL=https://sso.example.com/logout
|
||||
|
||||
# ── Uploads & limits ─────────────────────────────────────────────────────────
|
||||
|
||||
# Where chat attachments are stored (default: <state dir>/attachments).
|
||||
# Attachments are transient agent context, kept out of the workspace by default.
|
||||
# HERMES_WEBUI_ATTACHMENT_DIR=~/.hermes/webui/attachments
|
||||
|
||||
# Cap on total bytes an uploaded archive may expand to (zip-bomb guard).
|
||||
# Default derives from the upload size limit; override in MB.
|
||||
# HERMES_WEBUI_MAX_EXTRACTED_MB=2048
|
||||
|
||||
# Caps for the "download folder as ZIP" feature in the workspace browser.
|
||||
# HERMES_WEBUI_FOLDER_ZIP_MAX_MB=1024
|
||||
# HERMES_WEBUI_FOLDER_ZIP_MAX_FILES=50000
|
||||
|
||||
# Requests slower than this many seconds get a stack-dump diagnostic in the log
|
||||
# (default: 5.0; set 0 to log every request's timing).
|
||||
# HERMES_WEBUI_SLOW_REQUEST_SECONDS=5.0
|
||||
|
||||
# ── Onboarding ───────────────────────────────────────────────────────────────
|
||||
|
||||
# Skip the first-run onboarding wizard entirely.
|
||||
# HERMES_WEBUI_SKIP_ONBOARDING=1
|
||||
|
||||
# ── Advanced integrations ────────────────────────────────────────────────────
|
||||
|
||||
# Script whose stdout prefills new-session context (legacy generic hook; the
|
||||
# WebUI dynamic-recall hook in config.yaml takes precedence when configured).
|
||||
# HERMES_WEBUI_PREFILL_MESSAGES_SCRIPT=/path/to/script
|
||||
# HERMES_WEBUI_PREFILL_MESSAGES_SCRIPT_TIMEOUT=5
|
||||
# HERMES_WEBUI_PREFILL_CONTEXT_MAX_CHARS=12000
|
||||
|
||||
# Extra sources for the notes drawer (see docs for the expected format).
|
||||
# HERMES_WEBUI_EXTERNAL_NOTES_SOURCES=
|
||||
|
||||
# Directory scanned for WebUI plugins (default: ~/.hermes/plugins).
|
||||
# HERMES_WEBUI_PLUGINS_DIR=~/.hermes/plugins
|
||||
|
||||
# API key used when polling the Hermes gateway health endpoint.
|
||||
# HERMES_WEBUI_GATEWAY_API_KEY=
|
||||
|
||||
# Override the Claude Code projects directory scanned by the CLI session
|
||||
# bridge (default: auto-discovered ~/.claude/projects).
|
||||
# HERMES_WEBUI_CLAUDE_PROJECTS_DIR=~/.claude/projects
|
||||
|
||||
# Override the Codex CLI home scanned by the CLI session bridge and model
|
||||
# catalog merge (default: ~/.codex).
|
||||
# CODEX_HOME=~/.codex
|
||||
|
||||
# Path to the LLM wiki root shown in the wiki panel (default: resolved from
|
||||
# HERMES_HOME's wiki env file).
|
||||
# WIKI_PATH=/path/to/wiki
|
||||
|
||||
@@ -1,10 +0,0 @@
|
||||
# Normalize line endings to LF in the repository.
|
||||
* text=auto
|
||||
|
||||
# Shell scripts must stay LF on checkout too. A CRLF shebang (`#!/bin/bash\r`)
|
||||
# breaks the Docker image: the kernel can't find interpreter `/bin/bash\r`, so
|
||||
# local `docker compose up -d --build` on Windows machines with
|
||||
# `core.autocrlf=true` fails with `exec /hermeswebui_init.bash: no such file
|
||||
# or directory`.
|
||||
*.sh text eol=lf
|
||||
*.bash text eol=lf
|
||||
@@ -8,12 +8,6 @@ name: Browser smoke
|
||||
#
|
||||
# No secrets, no credentials: the server boots agent-free and the smoke script
|
||||
# strips every *_API_KEY from the environment before launch.
|
||||
#
|
||||
# Docs-only fast path: `browser-smoke` is a REQUIRED status check in the master
|
||||
# ruleset, and a skipped required check reports as pending forever (auto-merge
|
||||
# would hang). So on a docs/CHANGELOG/README-only change we keep the job running
|
||||
# and reporting green but short-circuit the expensive browser install + smoke.
|
||||
# Detection FAILS SAFE: any uncertainty or any non-doc path -> full smoke runs.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
@@ -22,85 +16,13 @@ on:
|
||||
branches: [master]
|
||||
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
docs_only: ${{ steps.detect.outputs.docs_only }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Detect docs-only change set
|
||||
id: detect
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
# Fail-safe: ALWAYS exits 0 and ALWAYS emits docs_only (default 'false' =
|
||||
# run full smoke), so a detection hiccup can never wedge the required
|
||||
# browser-smoke context into perpetual pending.
|
||||
docs_only="false"
|
||||
emit() { echo "docs_only=$docs_only" >> "$GITHUB_OUTPUT"; }
|
||||
trap emit EXIT
|
||||
set -uo pipefail
|
||||
# STRICT ALLOWLIST — docs iff doc extension (*.md/.markdown/.rst) or an
|
||||
# extensionless doc basename. NOT docs: *.txt (requirements.txt!), anything
|
||||
# under docs/ that isn't a doc extension, and code files merely NAMED
|
||||
# README/CHANGELOG. See tests.yml for the full rationale.
|
||||
is_docs() {
|
||||
f="$1"; low="$(printf '%s' "$f" | tr '[:upper:]' '[:lower:]')"; base="${f##*/}"
|
||||
case "$low" in
|
||||
*.md|*.markdown|*.rst) return 0 ;;
|
||||
esac
|
||||
case "$base" in
|
||||
LICENSE|README|CHANGELOG|NOTICE|AUTHORS|CONTRIBUTING) return 0 ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
if [ -n "${BASE_SHA:-}" ] && [ -n "${HEAD_SHA:-}" ]; then
|
||||
git fetch --no-tags --depth=1 origin "$BASE_SHA" 2>/dev/null || git fetch --no-tags origin master || true
|
||||
RANGE="$BASE_SHA...$HEAD_SHA"
|
||||
else
|
||||
RANGE="${{ github.event.before }}...${{ github.sha }}"
|
||||
fi
|
||||
# --no-renames so a rename reveals BOTH sides (a renamed-away code file is visible).
|
||||
if ! FILES="$(git diff --name-only --no-renames "$RANGE" 2>/dev/null)"; then
|
||||
echo "Could not diff $RANGE — running full smoke (fail-safe)."; exit 0
|
||||
fi
|
||||
if [ -z "$FILES" ]; then
|
||||
echo "Empty diff — running full smoke (fail-safe)."; exit 0
|
||||
fi
|
||||
echo "Changed files:"; echo "$FILES"
|
||||
result="true"
|
||||
while IFS= read -r f; do
|
||||
[ -z "$f" ] && continue
|
||||
if ! is_docs "$f"; then
|
||||
echo "Non-docs path -> full smoke required: $f"
|
||||
result="false"
|
||||
break
|
||||
fi
|
||||
done <<< "$FILES"
|
||||
docs_only="$result"
|
||||
echo "docs_only=$docs_only"
|
||||
|
||||
browser-smoke:
|
||||
needs: changes
|
||||
# Belt-and-suspenders (see tests.yml): run even if `changes` errored/was skipped
|
||||
# so this required context can never wedge; step guards degrade to full smoke.
|
||||
if: ${{ always() }}
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
steps:
|
||||
- name: Docs-only short-circuit
|
||||
if: needs.changes.outputs.docs_only == 'true'
|
||||
run: echo "Docs-only change set — skipping browser smoke (no page-executable change)."
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
|
||||
- name: Set up Python
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
@@ -110,7 +32,6 @@ jobs:
|
||||
**/pyproject.toml
|
||||
|
||||
- name: Install dependencies
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install "pyyaml>=6.0" playwright
|
||||
@@ -118,5 +39,4 @@ jobs:
|
||||
python -m playwright install --with-deps chromium
|
||||
|
||||
- name: Run browser smoke
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: python tests/browser_smoke.py
|
||||
|
||||
@@ -1,88 +0,0 @@
|
||||
name: Conversation lifecycle (informational)
|
||||
|
||||
# Scoped to the code this browser gate actually exercises — the chat
|
||||
# render/streaming surface — so it doesn't spin up playwright+chromium on
|
||||
# docs-only or unrelated changes. If none of these paths change, the run is
|
||||
# skipped entirely (GitHub treats a path-filtered no-match as "not triggered").
|
||||
on:
|
||||
pull_request:
|
||||
branches: [master]
|
||||
paths:
|
||||
- 'static/**'
|
||||
- 'api/**.py'
|
||||
- 'server.py'
|
||||
- 'tests/browser_conversation_lifecycle.py'
|
||||
- 'tests/browser_historical_transcript_hydration.py'
|
||||
- 'requirements*.txt'
|
||||
- 'pyproject.toml'
|
||||
- '.github/workflows/conversation-lifecycle.yml'
|
||||
push:
|
||||
branches: [master]
|
||||
paths:
|
||||
- 'static/**'
|
||||
- 'api/**.py'
|
||||
- 'server.py'
|
||||
- 'tests/browser_conversation_lifecycle.py'
|
||||
- 'tests/browser_historical_transcript_hydration.py'
|
||||
- 'requirements*.txt'
|
||||
- 'pyproject.toml'
|
||||
- '.github/workflows/conversation-lifecycle.yml'
|
||||
|
||||
jobs:
|
||||
live-to-final:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 10
|
||||
continue-on-error: true
|
||||
strategy:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
include:
|
||||
- name: normal
|
||||
script: tests/browser_conversation_lifecycle.py
|
||||
scenario: normal
|
||||
- name: terminal-error
|
||||
script: tests/browser_conversation_lifecycle.py
|
||||
scenario: terminal-error
|
||||
- name: historical-transcript-hydration
|
||||
script: tests/browser_historical_transcript_hydration.py
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
cache: 'pip'
|
||||
cache-dependency-path: |
|
||||
**/requirements*.txt
|
||||
**/pyproject.toml
|
||||
.github/workflows/conversation-lifecycle.yml
|
||||
|
||||
- name: Install browser gate dependencies
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
pip install -r requirements.txt playwright
|
||||
python -m playwright install --with-deps chromium
|
||||
|
||||
- name: Run conversation lifecycle gate
|
||||
env:
|
||||
LIFECYCLE_ARTIFACT_DIR: lifecycle-artifacts
|
||||
HISTORICAL_HYDRATION_ARTIFACT_DIR: lifecycle-artifacts
|
||||
LIFECYCLE_SCENARIO: ${{ matrix.scenario }}
|
||||
run: python ${{ matrix.script }}
|
||||
|
||||
- name: Run settlement frame proof
|
||||
if: matrix.scenario == 'normal'
|
||||
env:
|
||||
LIFECYCLE_ARTIFACT_DIR: lifecycle-artifacts
|
||||
LIFECYCLE_SCENARIO: normal
|
||||
LIFECYCLE_TEST_BITE: settle-worklog-frame-proof
|
||||
run: python tests/browser_conversation_lifecycle.py
|
||||
|
||||
- name: Upload failure artifacts
|
||||
if: failure()
|
||||
uses: actions/upload-artifact@v4
|
||||
with:
|
||||
name: conversation-lifecycle-failure-${{ matrix.name }}
|
||||
path: lifecycle-artifacts
|
||||
if-no-files-found: ignore
|
||||
@@ -87,176 +87,6 @@ jobs:
|
||||
cache-from: type=gha
|
||||
cache-to: type=gha,mode=max
|
||||
|
||||
# Startup-health gate for UID/GID auto-detection (#7027).
|
||||
#
|
||||
# The compose variants above all mount a hermes-home volume, so they never
|
||||
# exercise the plain single-container shape from the issue: a state directory
|
||||
# bind-mounted from a host directory owned by a non-1024 user, /workspace left
|
||||
# as the image's own build-time 1024, and no explicit WANTED_UID/WANTED_GID.
|
||||
# Before the fix the auto-detector read /workspace, remapped to 1024, failed
|
||||
# its own state-dir writability check and restart-looped. Source-level
|
||||
# invariants can't catch that — only actually booting the container can.
|
||||
state-dir-uid:
|
||||
name: State-dir UID detection (#7027)
|
||||
runs-on: ubuntu-latest
|
||||
needs: [compose-config, build-image]
|
||||
timeout-minutes: 20
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
- uses: docker/setup-buildx-action@v3
|
||||
|
||||
- name: Restore Docker image from cache
|
||||
uses: docker/build-push-action@v6
|
||||
with:
|
||||
context: .
|
||||
load: true
|
||||
tags: ghcr.io/nesquena/hermes-webui:latest
|
||||
cache-from: type=gha
|
||||
|
||||
- name: Boot on a host-owned state mount with no explicit IDs
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
CONTAINER="hermes-smoke-uid-${{ github.run_id }}-${{ github.run_attempt }}"
|
||||
HOST_UID=1001
|
||||
STATE_DIR="$(mktemp -d -t hermes-smoke-uidstate-XXXXXX)"
|
||||
sudo chown -R "$HOST_UID:$HOST_UID" "$STATE_DIR"
|
||||
|
||||
cleanup() {
|
||||
local rc=$?
|
||||
echo "::group::Cleanup (rc=$rc)"
|
||||
docker logs "$CONTAINER" 2>&1 | tail -200 || true
|
||||
docker rm -f "$CONTAINER" || true
|
||||
sudo rm -rf "$STATE_DIR" || true
|
||||
echo "::endgroup::"
|
||||
return $rc
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
# Deliberately NOT mounting /workspace: the point of this gate is that
|
||||
# the stock image's own /workspace (owned 1024:1024) must not win over
|
||||
# the host-owned state mount.
|
||||
echo "::group::docker run"
|
||||
docker run -d --name "$CONTAINER" \
|
||||
-p 127.0.0.1:8788:8787 \
|
||||
-e HERMES_WEBUI_STATE_DIR=/app/data \
|
||||
-v "$STATE_DIR":/app/data \
|
||||
ghcr.io/nesquena/hermes-webui:latest
|
||||
echo "::endgroup::"
|
||||
|
||||
# ----- The product gate: the container must reach /health -----
|
||||
echo "::group::Probe /health"
|
||||
attempts=0
|
||||
max_attempts=60
|
||||
until curl --fail --silent --max-time 5 http://127.0.0.1:8788/health > /dev/null; do
|
||||
attempts=$((attempts + 1))
|
||||
if [ "$attempts" -ge "$max_attempts" ]; then
|
||||
echo "❌ WebUI /health never returned 200 after $max_attempts attempts (~5m)"
|
||||
echo " Startup logs follow in the cleanup group below."
|
||||
exit 1
|
||||
fi
|
||||
if [ "$(docker inspect -f '{{.State.Running}}' "$CONTAINER")" != "true" ]; then
|
||||
echo "❌ Container exited before /health came up — this is the #7027 restart loop"
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
echo "✅ /health = 200 after $attempts attempts"
|
||||
echo "::endgroup::"
|
||||
|
||||
echo "::group::Verify the detected identity"
|
||||
LOGS="$(docker logs "$CONTAINER" 2>&1)"
|
||||
|
||||
# The identity must have been detected from the state dir, not /workspace.
|
||||
if ! echo "$LOGS" | grep -qE -- "-- Auto-detected UID: ${HOST_UID} \(from /app/data\)"; then
|
||||
echo "❌ UID was not auto-detected from the state directory (#7027)"
|
||||
echo "$LOGS" | grep -E -- "-- (Auto-detected|WANTED_)" || true
|
||||
exit 1
|
||||
fi
|
||||
if ! echo "$LOGS" | grep -qE -- "-- Auto-detected GID: ${HOST_UID} \(from /app/data\)"; then
|
||||
echo "❌ GID was not auto-detected from the state directory (#7027)"
|
||||
echo "$LOGS" | grep -E -- "-- (Auto-detected|WANTED_)" || true
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# And the runtime user must really carry it.
|
||||
ACTUAL_UID="$(docker exec "$CONTAINER" id -u hermeswebui)"
|
||||
ACTUAL_GID="$(docker exec "$CONTAINER" id -g hermeswebui)"
|
||||
if [ "$ACTUAL_UID" != "$HOST_UID" ] || [ "$ACTUAL_GID" != "$HOST_UID" ]; then
|
||||
echo "❌ hermeswebui runs as ${ACTUAL_UID}:${ACTUAL_GID}, expected ${HOST_UID}:${HOST_UID}"
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ hermeswebui runs as ${ACTUAL_UID}:${ACTUAL_GID}, detected from the state mount"
|
||||
echo "::endgroup::"
|
||||
|
||||
echo "::group::Startup log scan"
|
||||
BAD_PATTERNS='!! ERROR|!! Exiting script|Failed to verify state directory|Permission denied'
|
||||
if echo "$LOGS" | grep -E -i "$BAD_PATTERNS"; then
|
||||
echo "❌ Startup logs contain known-bad pattern (see above)"
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ No known-bad patterns in startup logs"
|
||||
echo "::endgroup::"
|
||||
|
||||
- name: Explicit WANTED_UID=1024 must survive auto-detection
|
||||
run: |
|
||||
set -euo pipefail
|
||||
|
||||
CONTAINER="hermes-smoke-uid1024-${{ github.run_id }}-${{ github.run_attempt }}"
|
||||
STATE_DIR="$(mktemp -d -t hermes-smoke-uid1024-XXXXXX)"
|
||||
HERMES_DIR="$(mktemp -d -t hermes-smoke-uid1024home-XXXXXX)"
|
||||
# Both mounts are owned by a *different* UID than the one requested.
|
||||
# The hermes-home mount is what makes this discriminating: it is a
|
||||
# probe candidate the pre-fix script already read, so if the explicit
|
||||
# 1024 is still treated as "unset" the container remaps to 1001 and
|
||||
# the assertion below fails. World-writable state dir so the only
|
||||
# thing under test is which identity wins, not mount permissions.
|
||||
sudo chown -R 1001:1001 "$STATE_DIR" "$HERMES_DIR"
|
||||
sudo chmod 0777 "$STATE_DIR"
|
||||
|
||||
cleanup() {
|
||||
local rc=$?
|
||||
echo "::group::Cleanup (rc=$rc)"
|
||||
docker logs "$CONTAINER" 2>&1 | tail -100 || true
|
||||
docker rm -f "$CONTAINER" || true
|
||||
sudo rm -rf "$STATE_DIR" "$HERMES_DIR" || true
|
||||
echo "::endgroup::"
|
||||
return $rc
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
docker run -d --name "$CONTAINER" \
|
||||
-p 127.0.0.1:8789:8787 \
|
||||
-e HERMES_WEBUI_STATE_DIR=/app/data \
|
||||
-e WANTED_UID=1024 \
|
||||
-e WANTED_GID=1024 \
|
||||
-v "$STATE_DIR":/app/data \
|
||||
-v "$HERMES_DIR":/home/hermeswebui/.hermes \
|
||||
ghcr.io/nesquena/hermes-webui:latest
|
||||
|
||||
attempts=0
|
||||
max_attempts=60
|
||||
until curl --fail --silent --max-time 5 http://127.0.0.1:8789/health > /dev/null; do
|
||||
attempts=$((attempts + 1))
|
||||
if [ "$attempts" -ge "$max_attempts" ]; then
|
||||
echo "❌ WebUI /health never returned 200 with explicit WANTED_UID=1024"
|
||||
exit 1
|
||||
fi
|
||||
if [ "$(docker inspect -f '{{.State.Running}}' "$CONTAINER")" != "true" ]; then
|
||||
echo "❌ Container exited before /health came up"
|
||||
exit 1
|
||||
fi
|
||||
sleep 5
|
||||
done
|
||||
|
||||
ACTUAL_UID="$(docker exec "$CONTAINER" id -u hermeswebui)"
|
||||
if [ "$ACTUAL_UID" != "1024" ]; then
|
||||
echo "❌ explicit WANTED_UID=1024 was overwritten by auto-detection — got ${ACTUAL_UID} (#7027)"
|
||||
exit 1
|
||||
fi
|
||||
echo "✅ explicit WANTED_UID=1024 preserved (sentinel no longer shadows the valid value)"
|
||||
|
||||
smoke:
|
||||
name: Smoke ${{ matrix.variant }}
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
@@ -1,116 +0,0 @@
|
||||
name: Docs CI
|
||||
|
||||
# Lightweight checks for documentation changes. Intentionally MINIMAL (Nathan,
|
||||
# 2026-07-18): NOT a Markdown style linter — no line length, heading levels, list
|
||||
# markers, or cosmetic rules. It runs only two things, and only when docs change:
|
||||
# 1. critical_markdown_check.py — flags ONLY a rendering break (a link whose
|
||||
# destination token is split across a newline, or an unclosed inline link).
|
||||
# 2. lychee link check — flags broken/dead links (informational, non-blocking).
|
||||
#
|
||||
# Runs on docs paths only; does NOT gate code PRs. This job is NOT a required
|
||||
# status check — it's an informational quality gate; promote it via the ruleset
|
||||
# later if desired.
|
||||
#
|
||||
# SECURITY: changed filenames on a fork PR are UNTRUSTED. They are never
|
||||
# interpolated into `run:` source via ${{ }} (which Bash would re-evaluate,
|
||||
# executing $(...) in a crafted filename). The file list is passed through an
|
||||
# `env:` variable and referenced as "$DOC_FILES", and lychee is handed a FIXED
|
||||
# glob rather than an attacker-controlled explicit list.
|
||||
|
||||
on:
|
||||
pull_request:
|
||||
branches: [master]
|
||||
paths:
|
||||
- '**/*.md'
|
||||
- '**/*.markdown'
|
||||
- '**/*.rst'
|
||||
- 'docs/**'
|
||||
- 'CHANGELOG*'
|
||||
- '**/README*'
|
||||
push:
|
||||
branches: [master]
|
||||
paths:
|
||||
- '**/*.md'
|
||||
- '**/*.markdown'
|
||||
- '**/*.rst'
|
||||
- 'docs/**'
|
||||
- 'CHANGELOG*'
|
||||
- '**/README*'
|
||||
|
||||
# Read-only; no secrets needed. Fork PRs already get a read-only token, but pin it.
|
||||
permissions:
|
||||
contents: read
|
||||
|
||||
jobs:
|
||||
docs-ci:
|
||||
runs-on: ubuntu-latest
|
||||
timeout-minutes: 8
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Resolve changed doc files (NUL-safe, untrusted names)
|
||||
id: docs
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# Compute the changed markdown/doc files and write them to a file, ONE PER
|
||||
# LINE. We never echo untrusted names into another shell; downstream steps
|
||||
# read this file. Fall back to all tracked docs if the diff can't compute.
|
||||
if [ -n "${BASE_SHA:-}" ] && [ -n "${HEAD_SHA:-}" ]; then
|
||||
git fetch --no-tags --depth=1 origin "$BASE_SHA" 2>/dev/null || git fetch --no-tags origin master || true
|
||||
RANGE="$BASE_SHA...$HEAD_SHA"
|
||||
else
|
||||
RANGE="${{ github.event.before }}...${{ github.sha }}"
|
||||
fi
|
||||
: > doc_files.txt
|
||||
if CHANGED="$(git diff --name-only --no-renames --diff-filter=ACMR "$RANGE" 2>/dev/null)"; then
|
||||
printf '%s\n' "$CHANGED" | grep -Ei '\.(md|markdown|rst)$' > doc_files.txt || true
|
||||
fi
|
||||
if [ ! -s doc_files.txt ]; then
|
||||
git ls-files '*.md' '*.markdown' '*.rst' > doc_files.txt 2>/dev/null || true
|
||||
fi
|
||||
# Keep only files that still exist (skip deletions), preserving exact names.
|
||||
: > doc_files_existing.txt
|
||||
while IFS= read -r f; do
|
||||
[ -n "$f" ] && [ -f "$f" ] && printf '%s\n' "$f" >> doc_files_existing.txt
|
||||
done < doc_files.txt
|
||||
mv doc_files_existing.txt doc_files.txt
|
||||
echo "Doc files to check:"; cat doc_files.txt
|
||||
echo "count=$(wc -l < doc_files.txt)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Critical Markdown check (rendering-breaks only, not style)
|
||||
if: steps.docs.outputs.count != '0'
|
||||
run: |
|
||||
set -uo pipefail
|
||||
# Read the NUL-safe-ish file list (one path per line) into an array. Names
|
||||
# are passed as ARGUMENTS to python, never evaluated as shell.
|
||||
mapfile -t FILES < doc_files.txt
|
||||
if [ "${#FILES[@]}" -eq 0 ]; then echo "No doc files."; exit 0; fi
|
||||
python3 scripts/critical_markdown_check.py "${FILES[@]}"
|
||||
|
||||
# Broken-link check. Handed a FIXED glob (not the attacker-controlled list) so
|
||||
# no untrusted filename ever reaches lychee's `eval`. Over-checking docs is
|
||||
# harmless. Informational only: continue-on-error keeps it from blocking.
|
||||
- name: Broken-link check (lychee)
|
||||
uses: lycheeverse/lychee-action@v2
|
||||
with:
|
||||
args: >-
|
||||
--no-progress
|
||||
--max-concurrency 4
|
||||
--accept 200,206,301,302,303,307,308,401,403,429
|
||||
--timeout 20
|
||||
--max-retries 2
|
||||
"**/*.md"
|
||||
"**/*.markdown"
|
||||
"**/*.rst"
|
||||
fail: true
|
||||
continue-on-error: true
|
||||
@@ -4,7 +4,6 @@ on:
|
||||
push:
|
||||
tags:
|
||||
- 'v*'
|
||||
- 'exp-v*'
|
||||
|
||||
jobs:
|
||||
release:
|
||||
@@ -16,22 +15,11 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
|
||||
# Check whether this is a stable or experimental release
|
||||
- name: Detect release channel
|
||||
id: channel
|
||||
run: |
|
||||
if [[ "${{ github.ref }}" == refs/tags/v* ]]; then
|
||||
echo "is_stable=true" >> "$GITHUB_OUTPUT"
|
||||
else
|
||||
echo "is_stable=false" >> "$GITHUB_OUTPUT"
|
||||
fi
|
||||
|
||||
# Create GitHub Release from tag with auto-generated notes
|
||||
- name: Create GitHub Release
|
||||
uses: softprops/action-gh-release@v2
|
||||
with:
|
||||
generate_release_notes: true
|
||||
prerelease: ${{ steps.channel.outputs.is_stable == 'false' }}
|
||||
|
||||
# Set up multi-arch build (QEMU + Buildx)
|
||||
- uses: docker/setup-qemu-action@v3
|
||||
@@ -53,9 +41,7 @@ jobs:
|
||||
images: ghcr.io/${{ github.repository }}
|
||||
tags: |
|
||||
type=match,pattern=v(\d+\.\d+(?:\.\d+)?),group=1
|
||||
type=match,pattern=exp-v(\d+\.\d+(?:\.\d+)?),group=1
|
||||
type=raw,value=latest,enable=${{ steps.channel.outputs.is_stable == 'true' }}
|
||||
type=raw,value=experimental,enable=${{ steps.channel.outputs.is_stable == 'false' }}
|
||||
type=raw,value=latest
|
||||
|
||||
# Build and push multi-arch image (amd64 + arm64)
|
||||
- name: Build and push Docker image
|
||||
|
||||
+11
-143
@@ -6,120 +6,28 @@ on:
|
||||
push:
|
||||
branches: [master]
|
||||
|
||||
# Docs-only fast path: a CHANGELOG/README/docs-only PR does not need the full pytest
|
||||
# matrix or the JS/lint runtime guards — nothing executable changed. But the `test (…)`
|
||||
# shards and `lint` are REQUIRED status checks in the master ruleset, and a *skipped*
|
||||
# required check reports as pending forever (auto-merge would hang). So we keep the jobs
|
||||
# running and reporting green, but SHORT-CIRCUIT their expensive work when every changed
|
||||
# path is documentation. Detection FAILS SAFE: if anything is uncertain (no base to diff,
|
||||
# any non-doc path touched), docs_only=false and the full suite runs.
|
||||
jobs:
|
||||
changes:
|
||||
runs-on: ubuntu-latest
|
||||
outputs:
|
||||
docs_only: ${{ steps.detect.outputs.docs_only }}
|
||||
steps:
|
||||
- uses: actions/checkout@v4
|
||||
with:
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Detect docs-only change set
|
||||
id: detect
|
||||
env:
|
||||
BASE_SHA: ${{ github.event.pull_request.base.sha }}
|
||||
HEAD_SHA: ${{ github.event.pull_request.head.sha }}
|
||||
run: |
|
||||
# Fail-safe: this step ALWAYS exits 0 and ALWAYS emits docs_only (default
|
||||
# 'false' = run the full suite). If the `changes` job ever failed, the
|
||||
# required test/browser jobs (needs: changes) would be SKIPPED and their
|
||||
# required contexts would never report — wedging the PR in perpetual
|
||||
# pending. So the detector can never fail; worst case it over-runs CI.
|
||||
docs_only="false"
|
||||
emit() { echo "docs_only=$docs_only" >> "$GITHUB_OUTPUT"; }
|
||||
trap emit EXIT
|
||||
set -uo pipefail
|
||||
# STRICT ALLOWLIST — a path counts as "docs" ONLY if it is unambiguously
|
||||
# documentation. Everything else runs the full suite (fail-open to safety).
|
||||
# A file is docs iff:
|
||||
# * it has a doc extension: *.md / *.markdown / *.rst, OR
|
||||
# * its basename is an extensionless doc file (LICENSE/README/CHANGELOG/…).
|
||||
# NOT docs (deliberately): *.txt (requirements.txt controls deps!), anything
|
||||
# under docs/ that isn't a doc extension (docs/Makefile, docs/build.py,
|
||||
# docs/*.json), and any code file whose NAME merely contains README/CHANGELOG
|
||||
# (scripts/CHANGELOG_stamp.py, static/README_renderer.js). Extension/type
|
||||
# wins over name, and a non-doc extension is never docs.
|
||||
is_docs() {
|
||||
f="$1"; low="$(printf '%s' "$f" | tr '[:upper:]' '[:lower:]')"; base="${f##*/}"
|
||||
case "$low" in
|
||||
*.md|*.markdown|*.rst) return 0 ;;
|
||||
esac
|
||||
case "$base" in
|
||||
LICENSE|README|CHANGELOG|NOTICE|AUTHORS|CONTRIBUTING) return 0 ;;
|
||||
esac
|
||||
return 1
|
||||
}
|
||||
|
||||
# Resolve the changed-file list. PR event = base..head; push = before..after.
|
||||
if [ -n "${BASE_SHA:-}" ] && [ -n "${HEAD_SHA:-}" ]; then
|
||||
git fetch --no-tags --depth=1 origin "$BASE_SHA" 2>/dev/null || git fetch --no-tags origin master || true
|
||||
RANGE="$BASE_SHA...$HEAD_SHA"
|
||||
else
|
||||
RANGE="${{ github.event.before }}...${{ github.sha }}"
|
||||
fi
|
||||
|
||||
# --no-renames so a rename shows BOTH sides: `git mv src/app.py docs.md` must
|
||||
# reveal the src/app.py deletion (a code change) instead of collapsing to the
|
||||
# docs.md destination and being mis-classified as docs-only.
|
||||
if ! FILES="$(git diff --name-only --no-renames "$RANGE" 2>/dev/null)"; then
|
||||
echo "Could not diff $RANGE — running full suite (fail-safe)."; exit 0
|
||||
fi
|
||||
if [ -z "$FILES" ]; then
|
||||
echo "Empty diff — running full suite (fail-safe)."; exit 0
|
||||
fi
|
||||
|
||||
echo "Changed files:"; echo "$FILES"
|
||||
result="true"
|
||||
while IFS= read -r f; do
|
||||
[ -z "$f" ] && continue
|
||||
if ! is_docs "$f"; then
|
||||
echo "Non-docs path -> full suite required: $f"
|
||||
result="false"
|
||||
break
|
||||
fi
|
||||
done <<< "$FILES"
|
||||
docs_only="$result"
|
||||
echo "docs_only=$docs_only"
|
||||
|
||||
# Forward-looking Python lint gate (ruff). The Python twin of the ESLint runtime
|
||||
# guard. Runs the curated [tool.ruff] ruleset (E9 + F + B) but only on lines this
|
||||
# PR adds/modifies vs the merge-base — so it keeps NEW code clean without demanding
|
||||
# a reformat of the existing tree's cosmetic backlog (#3273). Fast, fails early.
|
||||
lint:
|
||||
needs: changes
|
||||
# `lint` is not currently a required check, but add always() anyway so it can
|
||||
# never wedge if it's promoted to required later — same rationale as the test/
|
||||
# browser-smoke jobs; a failed `changes` degrades to running the full lint.
|
||||
if: ${{ always() }}
|
||||
runs-on: ubuntu-latest
|
||||
steps:
|
||||
- name: Docs-only short-circuit
|
||||
if: needs.changes.outputs.docs_only == 'true'
|
||||
run: echo "Docs-only change set — skipping lint/runtime guards (nothing executable changed)."
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
with:
|
||||
# Need history so the gate can diff against the merge-base with master.
|
||||
fetch-depth: 0
|
||||
|
||||
- name: Set up Python
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.12'
|
||||
|
||||
- name: Install ruff
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: pip install ruff
|
||||
|
||||
- name: Ensure origin/master ref is available for the diff gate
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: git fetch --no-tags --depth=1 origin master || true
|
||||
|
||||
# Whole-tree Python syntax gate. The Python analog of the JS `node --check`
|
||||
@@ -131,15 +39,13 @@ jobs:
|
||||
# file:line, before the matrix suite runs. Whole-tree + unconditional so it
|
||||
# cannot be bypassed by the diff scope. (#4641)
|
||||
- name: Python syntax gate (whole tree — fails fast on any unparseable .py)
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: python3 -m compileall -q api server.py bootstrap.py mcp_server.py tests scripts
|
||||
|
||||
- name: Ruff forward gate (new/changed lines only)
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: python3 scripts/ruff_lint.py --diff origin/master
|
||||
|
||||
- name: Ruff whole-tree report (informational — never blocks)
|
||||
if: needs.changes.outputs.docs_only != 'true' && always()
|
||||
if: always()
|
||||
run: python3 scripts/ruff_lint.py --all
|
||||
|
||||
# Static-JS runtime-error guards. These catch brick-class bugs that throw only
|
||||
@@ -150,31 +56,20 @@ jobs:
|
||||
# scope, catching a function defined nested but called from a sibling
|
||||
# scope (#3696 — ReferenceError: _sessionAttentionState is not defined).
|
||||
- name: Set up Node for ESLint runtime guards
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
uses: actions/setup-node@v4
|
||||
with:
|
||||
node-version: '20'
|
||||
|
||||
- name: Install ESLint
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: npm install --no-save eslint@^10
|
||||
|
||||
- name: ESLint runtime-error gate (no-const-assign / no-import-assign, #3162)
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: npx eslint --no-config-lookup -c eslint.runtime-guard.config.mjs "static/**/*.js"
|
||||
|
||||
- name: Scope / undefined-reference gate (#3696)
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: python3 scripts/scope_undef_gate.py
|
||||
|
||||
test:
|
||||
needs: changes
|
||||
# Belt-and-suspenders: run even if `changes` errored/was skipped. A required job
|
||||
# that gets SKIPPED reports its context as pending forever (wedge). With always(),
|
||||
# the job runs regardless; the per-step `if: … != 'true'` guards then treat a
|
||||
# missing/empty docs_only (which is what a failed `changes` yields) as "run full
|
||||
# CI" — so a detector failure degrades to running everything, never to a wedge.
|
||||
if: ${{ always() }}
|
||||
runs-on: ubuntu-latest
|
||||
strategy:
|
||||
# Don't cancel the other shards/versions when one fails — we want the full
|
||||
@@ -182,25 +77,19 @@ jobs:
|
||||
fail-fast: false
|
||||
matrix:
|
||||
python-version: ['3.11', '3.12', '3.13']
|
||||
# Split the suite across 5 parallel shards per Python version. pytest-shard
|
||||
# Split the suite across 3 parallel shards per Python version. pytest-shard
|
||||
# partitions tests deterministically by test-id hash; the suite was made
|
||||
# shard-safe (no cross-test state leakage) so every shard passes
|
||||
# independently. See docs/agent-memory note on test-suite shard-safety.
|
||||
# NOTE: pytest-shard is 0-indexed — shard ids must be 0..num_shards-1.
|
||||
# Using 1-based ids would crash the out-of-range job AND silently skip
|
||||
# shard 0's tests.
|
||||
shard: [0, 1, 2, 3, 4]
|
||||
shard: [0, 1, 2]
|
||||
|
||||
steps:
|
||||
- name: Docs-only short-circuit
|
||||
if: needs.changes.outputs.docs_only == 'true'
|
||||
run: echo "Docs-only change set — skipping pytest shard ${{ matrix.shard }} (no executable change)."
|
||||
|
||||
- uses: actions/checkout@v4
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
|
||||
- name: Set up Python ${{ matrix.python-version }}
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: ${{ matrix.python-version }}
|
||||
@@ -211,11 +100,9 @@ jobs:
|
||||
**/pyproject.toml
|
||||
|
||||
- name: Install dependencies
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: |
|
||||
python -m pip install --upgrade pip
|
||||
# Office parsers stay optional at runtime; CI installs them explicitly for the workspace preview tests.
|
||||
pip install "pyyaml>=6.0" pytest pytest-timeout pytest-asyncio pytest-shard python-docx openpyxl python-pptx playwright
|
||||
pip install "pyyaml>=6.0" pytest pytest-timeout pytest-asyncio pytest-shard
|
||||
# ruff is installed so tests/test_ruff_forward_lint.py runs its E9/F821
|
||||
# tree-clean assertions in-suite (mirrors how eslint is available for
|
||||
# tests/test_static_js_runtime_lint.py). If install fails the test
|
||||
@@ -227,26 +114,7 @@ jobs:
|
||||
# it so test coverage exists. If mcp install fails (Python 3.13
|
||||
# wheel not yet available, etc.), tests/test_mcp_server.py uses
|
||||
# importorskip and the matrix stays green.
|
||||
pip install "mcp<2" || echo "mcp install failed — test_mcp_server.py will importorskip"
|
||||
pip install mcp || echo "mcp install failed — test_mcp_server.py will importorskip"
|
||||
|
||||
- name: Get Playwright version
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
id: pw-version
|
||||
run: echo "version=$(pip show playwright | grep '^Version:' | cut -d' ' -f2)" >> "$GITHUB_OUTPUT"
|
||||
|
||||
- name: Cache Playwright browsers
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
path: ~/.cache/ms-playwright
|
||||
key: ${{ runner.os }}-playwright-${{ steps.pw-version.outputs.version }}
|
||||
restore-keys: |
|
||||
${{ runner.os }}-playwright-
|
||||
|
||||
- name: Install Playwright browsers
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: python -m playwright install --with-deps chromium
|
||||
|
||||
- name: Run tests (shard ${{ matrix.shard }} of 5)
|
||||
if: needs.changes.outputs.docs_only != 'true'
|
||||
run: pytest tests/ -v --timeout=60 --shard-id=${{ matrix.shard }} --num-shards=5
|
||||
- name: Run tests (shard ${{ matrix.shard }} of 3)
|
||||
run: pytest tests/ -v --timeout=60 --shard-id=${{ matrix.shard }} --num-shards=3
|
||||
|
||||
-14
@@ -28,16 +28,9 @@ copilot-instructions.md
|
||||
# Generated screenshots and transient artifacts
|
||||
screenshot-*.png
|
||||
full-UI.png
|
||||
# Playwright MCP scratch — directory + ad-hoc PNGs the agent drops while
|
||||
# debugging browser sessions. /final-check.png is the convention used by the
|
||||
# local agent loop for a one-off end-of-task verification grab; ignore
|
||||
# either way so they never sneak into a commit.
|
||||
.playwright-mcp/
|
||||
final-check.png
|
||||
|
||||
# Version file written by Docker/CI build — generated, never committed
|
||||
api/_version.py
|
||||
api/_scm_version.py
|
||||
|
||||
# OS files
|
||||
.DS_Store
|
||||
@@ -77,10 +70,3 @@ graphify-out/
|
||||
node_modules/
|
||||
package-lock.json
|
||||
docs/ui-ux/review-shots/
|
||||
|
||||
# Python build metadata (never commit generated egg-info)
|
||||
*.egg-info/
|
||||
|
||||
# Local WebUI runtime data and credentials
|
||||
.webui_password
|
||||
attachments/
|
||||
|
||||
@@ -69,43 +69,6 @@ Follow that checklist's safety rules:
|
||||
the manual verification performed.
|
||||
- For runtime, streaming, recovery, replay, compression, or sidebar metadata
|
||||
changes, name the state layer being mutated and prove the relevant invariant.
|
||||
- For Docker build changes in `docker_init.bash`, mirror directory exclusions
|
||||
in both the `rsync` and `cp -a` paths — `/opt/hermes` may contain subdirectories
|
||||
with restricted permissions (e.g. `.playwright/`).
|
||||
|
||||
## Before you open a PR — the change guidelines
|
||||
|
||||
Read [`docs/GUIDELINES.md`](docs/GUIDELINES.md) in full before non-trivial work. It is the
|
||||
distilled set of habits that get a change merged in one review round instead of several. The
|
||||
compressed form:
|
||||
|
||||
1. **Fix the class, not the instance.** A bug usually has siblings — other call sites, backends,
|
||||
companion endpoints, layouts, exit paths. Find them all and fix the shared chokepoint, or name
|
||||
the ones you left out of scope.
|
||||
2. **Trace one authoritative value end-to-end** (`input → normalize → decision → action → persist →
|
||||
cleanup`); the code that *decides* and the code that *acts* must use the same resolved value.
|
||||
3. **When you can't confirm something, fail closed and say so.** Never take the permissive branch on
|
||||
uncertainty; never report a failure as success. "Unknown" is not "allowed."
|
||||
4. **Enumerate the state-space before editing** — entry point, backend, item count (0/1/many), every
|
||||
lifecycle exit (success/error/cancel/replace/teardown), auth on/off, concurrency, hostile input —
|
||||
and cover each or mark it out of scope. Most redo rounds are one un-considered dimension.
|
||||
5. **Assume inputs and check-then-use gaps are adversarial** — validate at the point of use (hold a
|
||||
handle, don't re-resolve a path), scope caches by complete identity, handle crafted input.
|
||||
6. **A test must fail before your fix and pass after it.** Assert observable behavior, not a source
|
||||
string or a mock of the thing under test; use multiple items if selection is what's being tested.
|
||||
7. **Name the owner of every piece of state and prove it's released on every exit** (success, error,
|
||||
cancel, replace, shrink, teardown) — not just the happy path.
|
||||
8. **Fallbacks/defaults are contracts — extend the mechanism, don't copy it.** Editing N parallel
|
||||
blocks identically means you missed a chokepoint (e.g. new copy goes in the `en` locale only).
|
||||
9. **The diff is the task and nothing else.** Extras go in the PR description, not the diff; run the
|
||||
affected + neighboring tests before opening.
|
||||
10. **A visible control costs attention on every visit** — place it by frequency of use and by where
|
||||
mainstream chat apps put the equivalent, not by where your diff already is; verify with
|
||||
before/after images at desktop and narrow widths.
|
||||
|
||||
Show the work in the PR body: the siblings you found, proof the test failed before the fix, the
|
||||
verification run, before/after images for visible changes, and an explicit list of what you could
|
||||
not verify.
|
||||
|
||||
## Local state and secrets
|
||||
|
||||
|
||||
+6
-52
@@ -7,8 +7,8 @@
|
||||
>
|
||||
> Keep this document updated as architecture changes are made.
|
||||
|
||||
> Current shipped build: `v0.51.792` (July 1, 2026).
|
||||
> Automated coverage: ~11,500 tests via `pytest tests/ --collect-only -q`. CI runs on
|
||||
> Current shipped build: `v0.51.192` (May 31, 2026).
|
||||
> Automated coverage: ~7,150 tests via `pytest tests/ --collect-only -q`. CI runs on
|
||||
> Python 3.11, 3.12, and 3.13 (3 parallel shards each) against every PR, plus a ruff
|
||||
> lint gate, a headless browser smoke test, and a Docker smoke test.
|
||||
>
|
||||
@@ -50,7 +50,7 @@ actions. The topbar remains focused on conversation context and the workspace/fi
|
||||
bootstrap.py One-shot launcher: optional agent install, deps, health wait, browser open.
|
||||
start.sh Thin wrapper around bootstrap.py for shell-based startup.
|
||||
ctl.sh Daemon lifecycle wrapper (start/stop/restart/status/logs) for homelab installs.
|
||||
pyproject.toml Standard build metadata plus the Ruff lint gate; source-checkout launch surface still centers on bootstrap.py / start.sh / ctl.sh.
|
||||
pyproject.toml Tooling config (ruff lint gate). NOT a packaged distribution.
|
||||
Dockerfile python:3.12-slim container image
|
||||
docker-compose.yml Compose config with named volume and optional auth
|
||||
.dockerignore Excludes .git, tests/, .env* from Docker builds
|
||||
@@ -59,7 +59,6 @@ actions. The topbar remains focused on conversation context and the workspace/fi
|
||||
auth.py Optional password authentication, signed cookies, passkeys/WebAuthn
|
||||
config.py Discovery, globals, model detection, reloadable config
|
||||
helpers.py HTTP helpers: j(), bad(), require(), safe_resolve(), security headers
|
||||
goals.py Persistent-goal commands and profile-scoped native GoalManager bridge
|
||||
models.py Session model + CRUD, per-session profile tracking, CLI/state.db bridge
|
||||
profiles.py Profile state management, hermes_cli wrapper
|
||||
onboarding.py First-run onboarding status, real provider config writes, OAuth linking, readiness detection
|
||||
@@ -87,7 +86,7 @@ actions. The topbar remains focused on conversation context and the workspace/fi
|
||||
sw.js Service worker: offline shell cache, version-pinned assets
|
||||
tests/
|
||||
conftest.py Isolated test server/state fixtures
|
||||
~1,150 test files ~11,500 tests collected via pytest (run `pytest --collect-only -q` for exact)
|
||||
~700 test files ~7,150 tests collected via pytest (run `pytest --collect-only -q` for exact)
|
||||
test_regressions.py Permanent regression gate
|
||||
CONTRIBUTING.md Contributor workflow and PR expectations.
|
||||
ROADMAP.md Feature and product roadmap document.
|
||||
@@ -236,24 +235,6 @@ Session is a plain Python class (not a dataclass, not SQLAlchemy):
|
||||
title_from(): takes messages list, finds first user message, returns first 64 chars.
|
||||
Called after run_conversation() completes to set the session title retroactively.
|
||||
|
||||
#### Imported `state.db` sidebar projection
|
||||
|
||||
`api.models.get_cli_sessions()` projects conversations from the active Hermes
|
||||
profile's `state.db` into sidebar-shaped rows. The default projection keeps
|
||||
interactive sources (CLI, TUI, ACP, messaging, and similar user-facing sessions)
|
||||
in a bounded 20-row candidate window. Background sources use independent recovery
|
||||
passes so a high-volume worker source cannot consume that interactive window:
|
||||
|
||||
- Cron: up to `CRON_PROJECT_CHIP_LIMIT` rows.
|
||||
- Webhook: up to `WEBHOOK_PROJECT_CHIP_LIMIT` rows.
|
||||
- Kanban: up to `KANBAN_PROJECT_CHIP_LIMIT` rows.
|
||||
|
||||
Source-specific views still use their dedicated bounds, and the later sidebar
|
||||
visibility stage decides whether recovered background rows are shown. In
|
||||
`all_profiles=True` mode the per-profile source bounds are disabled before rows
|
||||
are merged; cross-profile scoping, visibility, deduplication, and final route
|
||||
limits remain downstream responsibilities.
|
||||
|
||||
### 4.3 SSE Streaming Engine
|
||||
|
||||
This is the most architecturally interesting part. Two endpoints cooperate:
|
||||
@@ -395,29 +376,6 @@ read_file_content(workspace, rel):
|
||||
- Reads as UTF-8 with errors='replace' (binary files show replacement chars)
|
||||
- Returns {path, content, size, lines}
|
||||
|
||||
### 4.8 Persistent Goal Profile Boundary
|
||||
|
||||
`api/goals.py` exposes the WebUI `/goal` command payloads and post-turn evaluation hook.
|
||||
Hermes Agent's native `GoalManager` is the authoritative owner of goal evaluation,
|
||||
continuation decisions, wait barriers, failure counters, contracts, subgoals, and
|
||||
`state.db` persistence.
|
||||
|
||||
For a profile-scoped WebUI session, the bridge delegates only when the Agent exposes
|
||||
both the context-local `set_hermes_home_override()` API and call-time default
|
||||
`SessionDB` path resolution. The bridge probes the resolved default path under the
|
||||
selected context before constructing the native manager, then binds that profile's
|
||||
Hermes home before every native call. The override is reset in a `finally` block after
|
||||
every operation, so concurrent sessions using the same session ID under different
|
||||
profiles cannot cross-read or cross-write goal state. Goal snapshot rollback uses the
|
||||
same scoped native persistence path.
|
||||
|
||||
Older Hermes Agent versions that lack either capability continue through
|
||||
`_LegacyProfileGoalManager`, which pins persistence to the selected profile's explicit
|
||||
`state.db` path. This includes intermediate versions whose context API is present but
|
||||
whose default `SessionDB()` path remains frozen at module import. Keep this fallback
|
||||
compatibility-only: new goal semantics belong in Hermes Agent's native manager rather
|
||||
than a second WebUI implementation.
|
||||
|
||||
---
|
||||
|
||||
## 5. Frontend Architecture: Current State
|
||||
@@ -760,7 +718,7 @@ Current structure:
|
||||
ui.js, workspace.js, sessions.js, messages.js, panels.js, commands.js, boot.js
|
||||
tests/
|
||||
conftest.py Isolated test server/state fixtures
|
||||
~1,150 test files ~11,500 tests collected
|
||||
488 test files 5303 tests collected
|
||||
test_regressions.py Permanent regression gate
|
||||
|
||||
Route extraction to api/routes.py completed in Sprint 11. server.py remains a
|
||||
@@ -860,7 +818,7 @@ Optional password gate for non-SSH-tunnel deployments.
|
||||
|
||||
### Phase I: Test Infrastructure -- COMPLETE
|
||||
|
||||
~11,500 tests across ~1,150 test files + regression gates. The pytest fixture derives
|
||||
5303 tests across 488 test files + regression gates. The pytest fixture derives
|
||||
an isolated port and state directory from the repo path unless
|
||||
`HERMES_WEBUI_TEST_PORT` / `HERMES_WEBUI_TEST_STATE_DIR` pin them explicitly.
|
||||
Production data never touched.
|
||||
@@ -1349,8 +1307,6 @@ Complete list of all HTTP endpoints as of Sprint 1 (v0.3).
|
||||
/api/sessions List of all session compact() dicts, sorted by updated_at
|
||||
/api/list ?session_id=X&path=. -> directory listing for session workspace
|
||||
/api/file ?session_id=X&path=rel -> file content (text, 200KB limit)
|
||||
/share/<token> Public read-only HTML shell for a sanitized shared transcript snapshot
|
||||
/api/share/<token> Public JSON payload for a sanitized shared transcript snapshot
|
||||
/api/chat/stream ?stream_id=X -> SSE stream. Long-lived. Emits token/tool/
|
||||
approval/done/error events.
|
||||
/api/chat/stream/status ?stream_id=X -> {"active": true/false, "stream_id": X}
|
||||
@@ -1371,8 +1327,6 @@ Complete list of all HTTP endpoints as of Sprint 1 (v0.3).
|
||||
-> {"stream_id", "session_id"}. Starts agent daemon thread.
|
||||
/api/chat (fallback, sync) {"session_id", "message", "model"?, "workspace"?}
|
||||
-> blocks until agent finishes. Returns full result.
|
||||
/api/share/create {"session_id"} -> creates or refreshes a public read-only snapshot link
|
||||
/api/share/revoke {"session_id"} -> revokes the current public snapshot link
|
||||
/api/approval/respond {"session_id", "choice": once|session|always|deny}
|
||||
-> {"ok": true, "choice": choice}
|
||||
|
||||
|
||||
+6
-1600
File diff suppressed because it is too large
Load Diff
@@ -8,8 +8,6 @@ Hermes WebUI is intentionally simple to work on: Python on the server, vanilla J
|
||||
|
||||
- Read [`AGENTS.md`](AGENTS.md) if an AI assistant is doing or helping with the
|
||||
change.
|
||||
- Read [`docs/GUIDELINES.md`](docs/GUIDELINES.md) — the change guidelines that get a PR
|
||||
merged in one review round instead of several (what "complete and verified" means here).
|
||||
- Read [`docs/CONTRACTS.md`](docs/CONTRACTS.md) and any linked contract/RFC for
|
||||
the subsystem you will touch.
|
||||
- For UI or UX work, read [`docs/UIUX-GUIDE.md`](docs/UIUX-GUIDE.md)
|
||||
@@ -97,19 +95,6 @@ There is currently no PR template in this repo, so include the important section
|
||||
- Risks / Follow-ups
|
||||
- Model Used
|
||||
|
||||
Two things reviewers look for specifically in Verification, both detailed in
|
||||
[`docs/GUIDELINES.md`](docs/GUIDELINES.md):
|
||||
|
||||
- When the issue pins the bug's shape — a session capture, a script, exact steps, or
|
||||
a fenced JSON block / field-level trigger conditions — bind your test to *that* shape
|
||||
rather than a fixture rebuilt from your reading of it, satisfying every condition it
|
||||
names; otherwise the test can pass while the reported bug goes untouched. Reconstruct
|
||||
the shape yourself only when the issue leaves it unpinned, and then say what you assumed.
|
||||
- For any claim the repo doesn't own (browser behavior, a provider's API, a registry,
|
||||
an OS convention), name who owns the truth and confirm your proof is one they'd accept.
|
||||
In-page automation proves page behavior, not browser behavior; a mock proves your
|
||||
intent, not the provider's contract.
|
||||
|
||||
If the change is user-visible, include screenshots or a short video.
|
||||
|
||||
If the change is release-note-worthy, include concise release-note wording in
|
||||
|
||||
+45
-61
File diff suppressed because one or more lines are too long
-44
@@ -30,50 +30,6 @@ RUN apt-get update -y --fix-missing --no-install-recommends \
|
||||
&& apt-get clean \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# ── SQLite upgrade ──────────────────────────────────────────────────────────
|
||||
# The python:3.12-slim base ships SQLite 3.46.1 (Debian Trixie), which is
|
||||
# vulnerable to the WAL-reset corruption bug discovered March 2026.
|
||||
# https://sqlite.org/wal.html#walresetbug
|
||||
#
|
||||
# Debian has not backported the fix, so we compile from the amalgamation.
|
||||
# Installs to /usr/local/lib (registered in ld.so.conf.d for arm64 priority).
|
||||
# Build tools are purged after compilation to keep the image lean.
|
||||
# Build args are for forward version bumps only (3.54+, etc.).
|
||||
# When bumping SQLITE_VERSION, recompute the SHA-256 from the official
|
||||
# download and update SQLITE_SHA256 accordingly.
|
||||
ARG SQLITE_VERSION=3530000
|
||||
ARG SQLITE_YEAR=2026
|
||||
ARG SQLITE_SHA256=851e9b38192fe2ceaa65e0baa665e7fa06230c3d9bd1a6a9662d02380d73365a
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends gcc make libc6-dev \
|
||||
&& cd /tmp \
|
||||
&& curl -fsSL "https://sqlite.org/${SQLITE_YEAR}/sqlite-autoconf-${SQLITE_VERSION}.tar.gz" \
|
||||
-o sqlite.tar.gz \
|
||||
&& echo "${SQLITE_SHA256} sqlite.tar.gz" | sha256sum -c - \
|
||||
&& tar xzf sqlite.tar.gz \
|
||||
&& cd "sqlite-autoconf-${SQLITE_VERSION}" \
|
||||
&& CPPFLAGS="-DSQLITE_SECURE_DELETE" ./configure --prefix=/usr/local --disable-static --disable-readline \
|
||||
--enable-fts5 --enable-fts4 --enable-rtree \
|
||||
&& make -j"$(nproc)" \
|
||||
&& make install \
|
||||
&& echo "/usr/local/lib" > /etc/ld.so.conf.d/000-usr-local-lib.conf \
|
||||
&& /sbin/ldconfig \
|
||||
&& cd / && rm -rf /tmp/sqlite* \
|
||||
&& apt-get purge -y gcc make libc6-dev \
|
||||
&& apt-get autoremove -y \
|
||||
&& apt-get clean && rm -rf /var/lib/apt/lists/* \
|
||||
&& python3 -c "\
|
||||
import sqlite3; \
|
||||
v = sqlite3.sqlite_version; \
|
||||
assert tuple(int(x) for x in v.split('.')) >= (3, 51, 3), \
|
||||
f'SQLite {v} still vulnerable'; \
|
||||
c = sqlite3.connect(':memory:'); \
|
||||
assert c.execute('PRAGMA secure_delete').fetchone()[0] == 1, \
|
||||
'SQLITE_SECURE_DELETE not compiled in (deleted rows would remain recoverable)'; \
|
||||
c.execute('CREATE VIRTUAL TABLE _fts5_build_check USING fts5(x)'); \
|
||||
c.execute('DROP TABLE _fts5_build_check'); \
|
||||
c.close()"
|
||||
|
||||
# Optional GPU user-space acceleration libraries for users who pass through
|
||||
# host GPU devices. The default image remains CPU-only.
|
||||
ARG INSTALL_GPU_LIBS=0
|
||||
|
||||
@@ -3,7 +3,9 @@
|
||||
[Hermes Agent](https://hermes-agent.nousresearch.com/) is a sophisticated autonomous agent that lives on your server, accessed via a terminal or messaging apps, that remembers what it learns and gets more capable the longer it runs.
|
||||
|
||||
Hermes WebUI is a lightweight, dark-themed web app interface in your browser for [Hermes Agent](https://hermes-agent.nousresearch.com/).
|
||||
Full parity with the CLI experience - everything you can do from a terminal, you can do from this UI. No build step, no framework, no bundler. Just Python and vanilla JS.
|
||||
Full parity with the CLI experience - everything you can do from a terminal,
|
||||
you can do from this UI. No build step, no framework, no bundler. Just Python
|
||||
and vanilla JS.
|
||||
|
||||
Layout: three-panel. Left sidebar for sessions and navigation, center for chat,
|
||||
right for workspace file browsing. Model, profile, and workspace controls live in
|
||||
@@ -11,9 +13,7 @@ the **composer footer** — always visible while composing. A circular context r
|
||||
shows token usage at a glance. All settings and session tools are in the
|
||||
**Hermes Control Center** (launcher at the sidebar bottom).
|
||||
|
||||
Setup Hermes so you can access it natively on every device:
|
||||
|
||||
<img width="1467" height="881" alt="image" src="https://github.com/user-attachments/assets/9a72cdf3-a5b4-45ed-a836-a715ce46287e" />
|
||||
<img width="2448" height="1748" alt="Hermes Web UI — three-panel layout" src="https://github.com/user-attachments/assets/6bf8af4c-209d-441e-8b92-6515d7a0c369" />
|
||||
|
||||
<table>
|
||||
<tr>
|
||||
@@ -47,14 +47,10 @@ This gives you nearly **1:1 parity with Hermes CLI from a convenient web UI** wh
|
||||
|
||||
## Contents
|
||||
|
||||
[<img width="750" alt="image" src="https://github.com/user-attachments/assets/7e9544a7-ba47-4fc7-8142-1d9d16b17065" />
|
||||
](https://get-hermes.ai/setup/)
|
||||
|
||||
- [Why Hermes](#why-hermes) — what it is and how it compares
|
||||
- [Quick start](#quick-start) — clone + `bootstrap.py` / `start.sh` / `ctl.sh`
|
||||
- [Features](#features) — chat, sessions, workspace, voice, profiles, security, themes, panels, mobile
|
||||
- [Configuration & access](#configuration--access) — auto-discovery, overrides, remote/Tailscale/phone, manual launch
|
||||
- [Nix flake/module](#nix-flake-and-nixos-module) — declarative install and service
|
||||
- [Docker](#docker) — single- and multi-container deploys
|
||||
- [Running tests](#running-tests)
|
||||
- [Architecture](#architecture) — backend/frontend layout, state dir
|
||||
@@ -224,7 +220,6 @@ If an AI assistant is helping with install, reinstall, bootstrap, provider setup
|
||||
- Session tags -- add #tag to titles for colored chips and click-to-filter
|
||||
- Grouped by Today / Yesterday / Earlier in the sidebar (collapsible date groups)
|
||||
- Download as Markdown transcript, full JSON export, or import from JSON
|
||||
- Create a public read-only share link for the active conversation from the Control Center; shared pages show a sanitized transcript snapshot without workspace, profile, or live controls
|
||||
- Sessions persist across page reloads and SSH tunnel reconnects
|
||||
- Browser tab title reflects the active session name
|
||||
- CLI session bridge -- CLI sessions from hermes-agent's SQLite store appear in the sidebar with a gold "cli" badge; click to import with full history and reply normally
|
||||
@@ -262,11 +257,8 @@ If an AI assistant is helping with install, reinstall, bootstrap, provider setup
|
||||
### Authentication and security
|
||||
- Optional password auth -- off by default, zero friction for localhost
|
||||
- Enable via `HERMES_WEBUI_PASSWORD` env var or Settings panel
|
||||
- Installed PWAs work best with WebUI's own password. Reverse proxies are supported, but proxy basic auth can block the service-worker update fetches an installed app needs and leave it on a blank screen after an update; see `docs/troubleshooting.md` for recovery steps.
|
||||
- Optional passkeys/WebAuthn -- register from Settings -> System after signing in with a password; the login page only shows passkey sign-in after at least one passkey exists
|
||||
- After registering at least one passkey, Settings -> System can remove the password and keep passkey-only sign-in enabled. Password auth remains the bootstrap/recovery path until you choose to go passwordless; passkeys are same-origin and stored locally in the WebUI state directory
|
||||
- Optional native OIDC login for WebUI sessions -- configure `webui_oidc.issuer`, `client_id`, `allow_claim`, and `allow_values` in `config.yaml`, or set the matching `HERMES_WEBUI_OIDC_*` environment variables. OIDC stays disabled until all four are present, and startup prints a warning if the config is partial.
|
||||
- Native OIDC stores the PKCE/state nonce flow in process memory. That works for the shipped single-process server, and it also works behind a load balancer when callbacks stay sticky to the same WebUI instance. Multi-instance deployments need session affinity, or the callback can land on a different process and fail state validation.
|
||||
- Signed HMAC HTTP-only cookie with 24h TTL
|
||||
- Minimal dark-themed login page at `/login`
|
||||
- Security headers on all responses (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)
|
||||
@@ -289,12 +281,11 @@ If an AI assistant is helping with install, reinstall, bootstrap, provider setup
|
||||
- Token usage display toggle (off by default, also via `/usage` command)
|
||||
- Control Center always opens on the Conversation tab; resets on close
|
||||
- Unsaved changes guard -- discard/save prompt when closing with unpersisted changes
|
||||
- Cron completion alerts -- toast notifications and unread badges scoped to the active profile on the Tasks tab and session sidebar
|
||||
- Cron completion alerts -- toast notifications and unread badge on Tasks tab
|
||||
- Background agent error alerts -- banner when a non-active session encounters an error
|
||||
|
||||
### Slash commands
|
||||
- Type `/` in the composer for autocomplete dropdown
|
||||
- Plain skills match case-insensitive keywords in their name or description; built-in, agent/plugin, and bundle commands keep prefix matching and take precedence over a same-slug skill
|
||||
- Built-in: `/help`, `/clear`, `/compress [focus topic]`, `/compact` (alias), `/model <name>`, `/workspace <name>`, `/new`, `/usage`, `/theme`
|
||||
- Arrow keys navigate, Tab/Enter select, Escape closes
|
||||
- Unrecognized commands pass through to the agent
|
||||
@@ -326,7 +317,7 @@ If an AI assistant is helping with install, reinstall, bootstrap, provider setup
|
||||
|
||||
| Thing | How it finds it |
|
||||
|---|---|
|
||||
| Hermes agent dir | `HERMES_WEBUI_AGENT_DIR`, then known checkout paths, the `hermes` launcher on `PATH`, and finally the installed `run_agent` module exposed by `HERMES_WEBUI_PYTHON` |
|
||||
| Hermes agent dir | `HERMES_WEBUI_AGENT_DIR` env, then `$HERMES_HOME/hermes-agent` (Windows default `%LOCALAPPDATA%\hermes\hermes-agent`, POSIX default `~/.hermes/hermes-agent`), then sibling `../hermes-agent` |
|
||||
| Python executable | Agent venv first, then `.venv` in this repo, then system `python3` |
|
||||
| State directory | `HERMES_WEBUI_STATE_DIR` env, then `$HERMES_HOME/webui` (Windows default `%LOCALAPPDATA%\hermes\webui`, POSIX default `~/.hermes/webui`) |
|
||||
| Default workspace | `HERMES_WEBUI_DEFAULT_WORKSPACE` env, then `~/workspace`, then state dir |
|
||||
@@ -356,7 +347,7 @@ Full list of environment variables:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `HERMES_WEBUI_AGENT_DIR` | auto-discovered | Path to the Hermes Agent source or installed module root |
|
||||
| `HERMES_WEBUI_AGENT_DIR` | auto-discovered | Path to the hermes-agent checkout |
|
||||
| `HERMES_WEBUI_PYTHON` | auto-discovered | Python executable |
|
||||
| `HERMES_WEBUI_HOST` | `127.0.0.1` | Bind address (`0.0.0.0` for all IPv4, `::` for all IPv6, `::1` for IPv6 loopback) |
|
||||
| `HERMES_WEBUI_PORT` | `8787` | Port |
|
||||
@@ -374,74 +365,12 @@ Full list of environment variables:
|
||||
| `HERMES_CONFIG_PATH` | `$HERMES_HOME/config.yaml` | Path to Hermes config file |
|
||||
| `HERMES_WEBUI_SERVER_CWD` | *(unset)* | Working directory for the server process. Defaults to the agent dir; point it at a writable workspace when the agent dir is read-only so fallback relative writes land somewhere writable |
|
||||
| `HERMES_WEBUI_AGENT_CACHE_MAX` | `25` | Max live agent instances kept warm in the in-memory LRU. Each pins a full conversation transcript, so this is the dominant lever on resident memory — lower it on installs with many long sessions to cap RAM (at the cost of more cold reloads) |
|
||||
| `HERMES_WEBUI_SESSIONS_MAX` | `100` | Legacy operator override for the max compact `Session` objects held in the in-memory LRU. Prefer the `webui.sessions_cache_max` key in `config.yaml` (which takes precedence); this env var remains a fallback. Bounds resident memory so long-running installs cannot accumulate every session ever touched and eventually crash (#4765/#2233/#4633). Eviction only ever drops clean, persisted, non-active sessions; an evicted session lazily reloads from its JSON sidecar on next access |
|
||||
| `HERMES_WEBUI_SESSIONS_MAX` | `100` | Max compact `Session` objects held in the in-memory LRU. Lighter than the agent cache; lower it on installs with hundreds of sessions |
|
||||
|
||||
Extension deployments can inspect sanitized, authenticated diagnostics at `GET /api/extensions/status`; see [WebUI Extensions](docs/EXTENSIONS.md#diagnostics).
|
||||
|
||||
---
|
||||
|
||||
### Nix flake and NixOS module
|
||||
|
||||
Hermes WebUI has a Nix flake package and a NixOS service module so you can run it declaratively.
|
||||
|
||||
Install the latest package with:
|
||||
|
||||
```bash
|
||||
nix shell github:nesquena/hermes-webui#default
|
||||
```
|
||||
|
||||
Use this flake input in your system configuration:
|
||||
|
||||
```nix
|
||||
inputs.hermes-webui.url = "github:nesquena/hermes-webui";
|
||||
```
|
||||
|
||||
Then add the module and configure it in `nixosModules`:
|
||||
|
||||
```nix
|
||||
{
|
||||
inputs = {
|
||||
nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
|
||||
hermes-agent.url = "github:NousResearch/hermes-agent";
|
||||
hermes-webui.url = "github:nesquena/hermes-webui";
|
||||
};
|
||||
|
||||
outputs = { self, nixpkgs, hermes-agent, hermes-webui, ... }: {
|
||||
nixosConfigurations.<host> = nixpkgs.lib.nixosSystem {
|
||||
modules = [
|
||||
hermes-agent.nixosModules.default
|
||||
hermes-webui.nixosModules.default
|
||||
({ pkgs, ... }: {
|
||||
services.hermes-agent.enable = true;
|
||||
services.hermes-webui = {
|
||||
enable = true;
|
||||
host = "127.0.0.1";
|
||||
port = 8787;
|
||||
stateDir = "/var/lib/hermes-webui";
|
||||
user = "hermes";
|
||||
group = "hermes";
|
||||
hermesHome = "/var/lib/hermes/.hermes";
|
||||
agent.package = hermes-agent.packages.${pkgs.stdenv.hostPlatform.system}.default;
|
||||
environmentFiles = [ "/run/secrets/hermes-webui.env" ];
|
||||
};
|
||||
})
|
||||
];
|
||||
};
|
||||
};
|
||||
}
|
||||
```
|
||||
|
||||
The module defaults to `127.0.0.1`. Set `host = "0.0.0.0"` and `openFirewall = true` only when you want direct network access, and pair that with auth, for example `HERMES_WEBUI_PASSWORD` via `environmentFiles`.
|
||||
|
||||
The published Hermes Agent package exposes `passthru.hermesVenv`, so the module derives `HERMES_WEBUI_PYTHON` from its interpreter. Bootstrap then locates the installed `run_agent.py` through that interpreter without importing Agent code and exports its parent as `HERMES_WEBUI_AGENT_DIR`. Packages may expose `passthru.hermesAgentDir` as a direct path instead. Use `agent.dir` and `agent.python` as explicit overrides for custom package layouts.
|
||||
|
||||
When WebUI reads shared Hermes Agent state, run the service as a user that can already read that state. For a co-located Hermes Agent service, set `user` and `group` to the agent service account; the module only creates the default `hermes-webui` account and never changes ownership of an existing `hermesHome`.
|
||||
|
||||
The module maps directly onto existing WebUI environment variables, including:
|
||||
`HERMES_WEBUI_HOST`, `HERMES_WEBUI_PORT`, `HERMES_WEBUI_STATE_DIR`, `HERMES_HOME`, `HERMES_WEBUI_AGENT_DIR`, and `HERMES_WEBUI_PYTHON`.
|
||||
|
||||
Set `environmentFiles` for secrets like API keys. Protected WebUI runtime keys from the module are rejected there, so keep host, port, state, and agent wiring in the module options. Keep reverse proxy and TLS configuration in your surrounding deployment module because those details are deployment-specific.
|
||||
|
||||
### Remote access (SSH tunnel, Tailscale, phone)
|
||||
|
||||
The server binds to `127.0.0.1` by default. To reach it from another machine use an SSH tunnel (`ssh -N -L 8787:127.0.0.1:8787 user@host`, which `start.sh` prints for you over SSH), or join your server and phone to a [Tailscale](https://tailscale.com) network and browse to `http://<server-tailscale-ip>:8787` with `HERMES_WEBUI_HOST=0.0.0.0` + `HERMES_WEBUI_PASSWORD` set. Full walkthrough (incl. a community ARM64-Android field report): [`docs/remote-access.md`](docs/remote-access.md).
|
||||
@@ -590,7 +519,7 @@ system/Homebrew interpreter.
|
||||
|
||||
Tests run against an isolated server with a separate state directory.
|
||||
Production data and real cron jobs are never touched. Current snapshot:
|
||||
**~11,500 tests collected** across **~1,150 test files**, run in CI on Python 3.11,
|
||||
**~7,150 tests collected** across **~700 test files**, run in CI on Python 3.11,
|
||||
3.12, and 3.13 (3 parallel shards each).
|
||||
|
||||
---
|
||||
@@ -636,8 +565,8 @@ boot.js Mobile nav, voice input, theme/skin boot, bfcache handler
|
||||
**Tests + packaging**
|
||||
|
||||
```
|
||||
tests/ Pytest suite (~11,500 tests; isolated server/state fixtures)
|
||||
pyproject.toml Standard build metadata plus the Ruff lint gate; checkout launch surface still centers on bootstrap.py / start.sh / ctl.sh
|
||||
tests/ Pytest suite (~7,150 tests; isolated server/state fixtures)
|
||||
pyproject.toml Tooling config (ruff lint gate) — not a packaged distribution
|
||||
Dockerfile python:3.12-slim container image
|
||||
docker-compose.yml Compose with named volume and optional auth
|
||||
.github/workflows/ CI: ruff + sharded pytest, browser smoke, Docker smoke,
|
||||
@@ -692,7 +621,6 @@ The WebUI is still coupled to Hermes Agent internals for runtime execution, prov
|
||||
- [`TESTING.md`](TESTING.md) — manual browser test plan and automated coverage reference
|
||||
- [`DESIGN.md`](DESIGN.md) — design tokens and the calm-console direction
|
||||
- [`docs/UIUX-GUIDE.md`](docs/UIUX-GUIDE.md) — UI/UX principles sourced from the design docs and visual inventories
|
||||
- [`docs/sse-streams.md`](docs/sse-streams.md) — cross-client SSE endpoint reference: session streaming, gateway SSE probe scope, heartbeats, and proxy behavior
|
||||
- [`docs/CONTRACTS.md`](docs/CONTRACTS.md) — project contract/RFC/design index for contributors and agents
|
||||
- [`docs/rfcs/README.md`](docs/rfcs/README.md) — RFC index for larger architecture and durability proposals
|
||||
|
||||
@@ -709,24 +637,24 @@ The WebUI is still coupled to Hermes Agent internals for runtime execution, prov
|
||||
Hermes WebUI is built with help from the open-source community. Every PR — whether merged directly, absorbed into a batch release, or salvaged from a larger proposal — shapes the project, and we're grateful to everyone who has taken the time to contribute.
|
||||
|
||||
<!-- BEGIN GENERATED CONTRIBUTORS -->
|
||||
Over **326 contributors** have shipped code that landed in a release tag. The full, continuously-updated credit roll — including everyone with one or two PRs and the special-thanks roll for design and architectural work — lives in [`CONTRIBUTORS.md`](CONTRIBUTORS.md). A snapshot of the most prolific contributors:
|
||||
Over **288 contributors** have shipped code that landed in a release tag. The full, continuously-updated credit roll — including everyone with one or two PRs and the special-thanks roll for design and architectural work — lives in [`CONTRIBUTORS.md`](CONTRIBUTORS.md). A snapshot of the most prolific contributors:
|
||||
|
||||
### Top contributors (by PR count, including absorbed/batch-released work)
|
||||
|
||||
| # | Contributor | PRs | First → latest release |
|
||||
|---|---|---:|---|
|
||||
| 1 | [@rodboev](https://github.com/rodboev) | 336 | `v0.51.223` → `v0.51.893` |
|
||||
| 2 | [@franksong2702](https://github.com/franksong2702) | 301 | `v0.49.3` → `v0.51.893` |
|
||||
| 1 | [@franksong2702](https://github.com/franksong2702) | 264 | `v0.49.3` → `v0.51.587` |
|
||||
| 2 | [@rodboev](https://github.com/rodboev) | 204 | `v0.51.223` → `v0.51.586` |
|
||||
| 3 | [@Michaelyklam](https://github.com/Michaelyklam) | 157 | `v0.50.240` → `v0.51.198` |
|
||||
| 4 | [@ai-ag2026](https://github.com/ai-ag2026) | 121 | `v0.50.279` → `v0.51.835` |
|
||||
| 5 | [@bergeouss](https://github.com/bergeouss) | 80 | `v0.48.0` → `v0.51.703` |
|
||||
| 4 | [@ai-ag2026](https://github.com/ai-ag2026) | 116 | `v0.50.279` → `v0.51.519` |
|
||||
| 5 | [@bergeouss](https://github.com/bergeouss) | 80 | `v0.48.0` → `v0.51.527` |
|
||||
| 6 | [@AJV20](https://github.com/AJV20) | 57 | `v0.51.93` → `v0.51.346` |
|
||||
| 7 | [@dso2ng](https://github.com/dso2ng) | 43 | `v0.50.227` → `v0.51.578` |
|
||||
| 8 | [@starship-s](https://github.com/starship-s) | 28 | `v0.50.123` → `v0.51.763` |
|
||||
| 9 | [@Sanjays2402](https://github.com/Sanjays2402) | 27 | `v0.50.292` → `v0.51.484` |
|
||||
| 10 | [@allenliang2022](https://github.com/allenliang2022) | 24 | `v0.51.185` → `v0.51.869` |
|
||||
| 8 | [@Sanjays2402](https://github.com/Sanjays2402) | 27 | `v0.50.292` → `v0.51.484` |
|
||||
| 9 | [@starship-s](https://github.com/starship-s) | 25 | `v0.50.123` → `v0.51.547` |
|
||||
| 10 | [@Hinotoi-agent](https://github.com/Hinotoi-agent) | 23 | `v0.50.10` → `v0.51.522` |
|
||||
|
||||
See [`CONTRIBUTORS.md`](CONTRIBUTORS.md) for the full ranked list of all 326 contributors — the 3+ PR tables, the 1–2 PR roll, and the special-thanks notes for design and architectural contributions.
|
||||
See [`CONTRIBUTORS.md`](CONTRIBUTORS.md) for the full ranked list of all 288 contributors — the 3+ PR tables, the 1–2 PR roll, and the special-thanks notes for design and architectural contributions.
|
||||
<!-- END GENERATED CONTRIBUTORS -->
|
||||
|
||||
### Notable contributions
|
||||
|
||||
+63
-143
@@ -2,14 +2,9 @@
|
||||
|
||||
> Web companion to the Hermes Agent CLI. Same workflows, browser-native.
|
||||
>
|
||||
> This roadmap tracks the whole ecosystem: the web app, the native clients that
|
||||
> wrap it (macOS, Windows/Linux, Android, iOS), the extension system, and the
|
||||
> autonomous maintenance framework that runs the project.
|
||||
>
|
||||
> Version, test count, and release history are **derived live**, not stamped here
|
||||
> (they drift every release). For the current numbers:
|
||||
> - Version + history: `git tag --sort=-v:refname | head` + [CHANGELOG.md](./CHANGELOG.md)
|
||||
> - Test count: `pytest tests/ --collect-only -q`
|
||||
> Last updated: v0.51.192 (May 31, 2026) — ~7,150 tests collected. Recent themes: forward-looking ruff lint gate (#3273), Windows upgrade state-stranding hotfix (#2905), per-model context_length default-only guard (#3256), gateway-configured banner + tooltip i18n, and an ongoing concentric PR-triage cadence.
|
||||
> Test source: `pytest tests/ --collect-only -q`
|
||||
> Per-version detail: see [CHANGELOG.md](./CHANGELOG.md)
|
||||
|
||||
---
|
||||
|
||||
@@ -18,21 +13,15 @@
|
||||
| Surface | Status |
|
||||
|---|---|
|
||||
| **Hermes CLI parity** | ✅ Complete — every CLI workflow has a web equivalent |
|
||||
| **Streaming + tool transparency** | ✅ Live tool cards, reasoning cards, approval prompts, clarify, cancel, steer |
|
||||
| **Multi-provider model support** | ✅ Any provider configured in `config.yaml` shows in the picker + live custom-endpoint discovery |
|
||||
| **Sessions + projects + search** | ✅ CRUD, content search, projects, tags, archive, fork, import, batch ops, CLI bridge |
|
||||
| **Mobile + Docker + auth** | ✅ Hamburger nav, slide-overs, PWA install, password auth, GHCR images |
|
||||
| **Auxiliary surfaces** | ✅ Workspace tree + edit + terminal, cron CRUD, skills CRUD, memory write, MCP server UI |
|
||||
| **Visual polish** | ✅ Light / dark / system × 21 skins, Mermaid, KaTeX, syntax highlighting, Transparent Stream |
|
||||
| **Internationalization** | ✅ 15 locales with key-parity guards + CJK/RTL/IME handling |
|
||||
| **Gateway integration** | ✅ Live external sessions (Telegram, Discord, Slack, WeChat/Weixin, Signal, SMS) + cross-channel handoff |
|
||||
| **Extension system** | ✅ Opt-in loader, capability surface (themes / TTS / nav / sidecars), one-click gallery, vetted library repo |
|
||||
| **Native clients** | ✅ macOS (Swift), Windows + Linux + macOS (Rust/Tauri), Android, iOS — each in its own repo |
|
||||
| **Autonomous maintenance** | ✅ Self-running triage / review / release pipeline; generalized publicly as [StewardOS](https://github.com/nesquena/steward-os) |
|
||||
| **Streaming + tool transparency** | ✅ Live tool cards, reasoning cards, approval prompts, cancel |
|
||||
| **Multi-provider model support** | ✅ Any provider configured in `config.yaml` shows in the picker |
|
||||
| **Sessions + projects + search** | ✅ CRUD, content search, projects, tags, archive, fork, import |
|
||||
| **Mobile + Docker + auth** | ✅ Hamburger nav, slide-overs, password auth, GHCR images |
|
||||
| **Auxiliary surfaces** | ✅ Workspace tree + edit, cron CRUD, skills CRUD, memory write, MCP server UI |
|
||||
| **Visual polish** | ✅ Light/dark/system themes × 11 skins (default, ares, mono, slate, poseidon, sisyphus, charizard, sienna, catppuccin, nous, geist-contrast), Mermaid, KaTeX, syntax highlighting |
|
||||
| **Native distribution** | ✅ macOS desktop app (universal arm64+x86_64 DMG, signed) — separate repo |
|
||||
|
||||
Remaining gaps and forward work live in [Forward Work](#forward-work) below. The
|
||||
short version: the original CLI-parity + Claude-parity vision is done, and the
|
||||
work has shifted from "reach parity" to "harden reliability + widen distribution."
|
||||
Remaining gaps and forward work live in [Forward Work](#forward-work) below.
|
||||
|
||||
---
|
||||
|
||||
@@ -40,13 +29,13 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
|
||||
| Layer | Files | Status |
|
||||
|---|---|---|
|
||||
| Python server | `server.py` + `api/` modules | Thin HTTP shell + auth middleware over the `api/` business logic (config, sessions, streaming, profiles, routes, onboarding, workspace, updates, upload, extensions) |
|
||||
| Python server | `server.py` + `api/` modules | Thin HTTP shell + auth middleware over the `api/` business logic (config, sessions, streaming, profiles, routes, onboarding, workspace, updates, upload) |
|
||||
| HTML template | `static/index.html` | Served from disk |
|
||||
| CSS | `static/style.css` | Themes + skins, mobile responsive, KaTeX, table styles |
|
||||
| JavaScript | `static/{ui,sessions,messages,workspace,panels,boot,commands,icons,i18n,login,onboarding,extension_settings}.js` | Vanilla-JS modules served as static files — no bundler |
|
||||
| JavaScript | `static/{ui,sessions,messages,workspace,panels,boot,commands,icons,i18n,login,onboarding}.js` | Vanilla-JS modules served as static files — no bundler |
|
||||
| Service worker | `static/sw.js` | Offline shell cache, version-pinned assets |
|
||||
| Docker | `Dockerfile`, `docker-compose.yml` | `python:3.12-slim`, multi-arch (amd64+arm64), HEALTHCHECK |
|
||||
| CI/CD | `.github/workflows/` | ruff lint + sharded pytest + Playwright browser + Docker smoke on every PR; auto-release + GHCR publish on tag push |
|
||||
| CI/CD | `.github/workflows/` | ruff lint + sharded pytest + browser smoke + Docker smoke on every PR; auto-release + GHCR publish on tag push |
|
||||
| Test isolation | `tests/_pytest_port.py` | Per-worktree port + state-dir derivation, no collisions |
|
||||
|
||||
> Per-file line counts drift every release; see `ARCHITECTURE.md` for the current module map and `git ls-files` for exact sizes.
|
||||
@@ -58,10 +47,8 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
### Chat and streaming
|
||||
- [x] Send messages, get SSE-streaming responses
|
||||
- [x] Composer-scoped model picker (per-conversation model selection)
|
||||
- [x] Rich provider-grouped, searchable model picker everywhere (composer + Preferences default-model)
|
||||
- [x] Multi-provider API support — OpenAI, Anthropic, Google, OpenRouter, xAI, GLM, DeepSeek, Mistral, MiniMax, Kimi, OpenCode, Nous Portal, custom OpenAI-compatible endpoints
|
||||
- [x] Live custom-endpoint model discovery (Ollama, LM Studio, vLLM via `/v1/models`)
|
||||
- [x] Add a self-hosted provider (Ollama / LM Studio) from Settings — no `config.yaml` edit
|
||||
- [x] Free-form OpenRouter model name (autocomplete + custom input)
|
||||
- [x] Tool progress shown inline via live tool cards
|
||||
- [x] Approval card for dangerous commands (Allow once / session / always, Deny)
|
||||
@@ -71,25 +58,21 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] INFLIGHT guard: switch sessions mid-request without losing response
|
||||
- [x] Session restores from localStorage on page load
|
||||
- [x] Reconnect banner if page reloaded mid-stream
|
||||
- [x] SSE auto-reconnect with extended backoff ladder + full-session poll fallback
|
||||
- [x] SSE auto-reconnect with stream replay
|
||||
- [x] Token / cost estimate per message and per session
|
||||
- [x] Context usage indicator (compact ring badge in composer footer)
|
||||
- [x] Auto-compaction handling + `/compact` command
|
||||
- [x] One-click recovery from "context compression exhausted" (focused continuation)
|
||||
- [x] rAF-throttled token rendering (smooth, no DOM thrash)
|
||||
- [x] Cancel / stop button in composer footer
|
||||
- [x] Reasoning effort selector (low / medium / high / xhigh) + `/reasoning`
|
||||
- [x] Pure-text streaming with crash-recovery — partial messages restored from localStorage on reload
|
||||
- [x] Default message mode on a busy turn: Queue / Interrupt / Steer (new installs default to Steer)
|
||||
- [x] Steer a live turn with attachments (files uploaded + scoped to the owning session)
|
||||
- [x] Transparent Stream mode — chronological worklog with per-word fade-in (respects reduced-motion)
|
||||
|
||||
### Conversation controls
|
||||
- [x] Copy message to clipboard (hover icon on each bubble)
|
||||
- [x] Edit last user message and regenerate
|
||||
- [x] Regenerate last response
|
||||
- [x] Clear conversation (truncation-watermarked so cleared history cannot resurrect)
|
||||
- [x] Branch / fork conversation from any message point (turn-boundary aligned on compacted sessions)
|
||||
- [x] Clear conversation (wipe messages, keep session)
|
||||
- [x] Branch / fork conversation from any message point (#465)
|
||||
- [x] Pure-text + tool-call streams both recover
|
||||
|
||||
### Sessions
|
||||
@@ -112,10 +95,7 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Per-session profile tracking
|
||||
- [x] Per-session toolset override (`/toolsets`)
|
||||
- [x] Batch select mode (multi-select, bulk delete / move / archive)
|
||||
- [x] CLI session bridge — read CLI/agent sessions from state.db, surface running ones live, import as WebUI sessions
|
||||
- [x] Fork a read-only cron (scheduled-job) session into an editable chat
|
||||
- [x] Stable sidebar grouping (fork / compaction / subagent clusters don't reshuffle on refresh)
|
||||
- [x] Cross-device recency re-sort (a reactivated conversation bumps to top even in a background tab)
|
||||
- [x] CLI session bridge — read CLI sessions from state.db, import as WebUI sessions
|
||||
|
||||
### Workspace and files
|
||||
- [x] Add workspace with path validation (existing directory, follows symlinks)
|
||||
@@ -134,7 +114,6 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Create / rename / delete files and folders (in current directory)
|
||||
- [x] Drag-drop / click / clipboard paste upload
|
||||
- [x] Archive upload (zip / tar) with extraction
|
||||
- [x] Copy absolute + relative file path
|
||||
- [x] Syntax highlighted code preview (Prism.js, language-aware)
|
||||
- [x] File preview auto-close on directory navigation
|
||||
- [x] Right panel resizable (drag inner edge)
|
||||
@@ -148,7 +127,6 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Create job from UI (name, schedule, prompt, delivery target)
|
||||
- [x] Edit job inline (full create-form parity, including skills)
|
||||
- [x] Skill picker in create + edit forms
|
||||
- [x] No-cron-syntax schedule builder (frequency presets + time / day pickers, inline expression preview)
|
||||
- [x] Cron run history viewer (expandable per job)
|
||||
- [x] Cron completion alerts (toast + badge)
|
||||
- [x] Run-status tracking with live watch mode
|
||||
@@ -171,15 +149,13 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Multi-profile support — create, switch, delete (#28)
|
||||
- [x] Topbar profile picker with gateway-status dots
|
||||
- [x] Profile management panel (full CRUD)
|
||||
- [x] Seamless switching (no server restart, refreshes models / skills / memory / cron / workspace + forces config reload)
|
||||
- [x] Seamless switching (no server restart, refreshes models / skills / memory / cron / workspace)
|
||||
- [x] Profile-local workspace storage
|
||||
- [x] First-run onboarding wizard with provider config (OpenRouter / Anthropic / OpenAI / Custom)
|
||||
- [x] In-app OAuth for Codex and Claude
|
||||
- [x] Concurrent per-profile isolation (context-local home override so parallel workers can't clobber each other)
|
||||
|
||||
### Configuration
|
||||
- [x] Settings panel (default model, default workspace, send key, theme, skin, voice, font size)
|
||||
- [x] Searchable settings
|
||||
- [x] Settings panel (default model, default workspace, send key, theme, voice, font size)
|
||||
- [x] Send key preference (Enter or Ctrl+Enter)
|
||||
- [x] Password authentication (off by default)
|
||||
- [x] Per-session toolset override
|
||||
@@ -201,7 +177,6 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Password auth with signed HMAC HTTP-only cookies (24h TTL)
|
||||
- [x] Security headers (X-Content-Type-Options, X-Frame-Options, Referrer-Policy)
|
||||
- [x] CSRF protection (scheme-aware, port-normalized for reverse proxies)
|
||||
- [x] CORS preflight echoes allowlisted origin only — never wildcard
|
||||
- [x] PBKDF2 password hashing
|
||||
- [x] Rate limiting on auth endpoints
|
||||
- [x] Session ID validation
|
||||
@@ -216,14 +191,12 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Upload path-traversal guard
|
||||
- [x] Credential redaction in API responses
|
||||
- [x] Profile `.env` secret isolation on switch
|
||||
- [x] Open-redirect guard on `?next=` login param (bounded decode, collapses login-loop chains)
|
||||
- [x] Auto-install gate (opt-in via `HERMES_WEBUI_AUTO_INSTALL=1`)
|
||||
|
||||
### Visual / UX
|
||||
- [x] 3 base modes — Light, Dark, System (auto-sync)
|
||||
- [x] 21 skins layered on the base mode — default, ares, catppuccin, charizard, codex, geist-contrast, github, graphite, hepburn, mono, neon, neon-paint, neon-soft, nous, poseidon, sienna, sisyphus, slate, terracotta, verdigris, zeus
|
||||
- [x] 2-axis appearance model (base mode + skin) for community theme contributions
|
||||
- [x] Mermaid diagram rendering (with fit / fullscreen toolbar)
|
||||
- [x] 8 themes — Dark, Light, System (auto-sync), Slate, Solarized, Monokai, Nord, OLED, Sienna
|
||||
- [x] 2-axis appearance model (theme + skin) for community theme contributions
|
||||
- [x] Mermaid diagram rendering
|
||||
- [x] KaTeX math rendering with fence-before-math fix
|
||||
- [x] Syntax highlighting (Prism.js, language-aware, YAML newline preservation)
|
||||
- [x] Markdown image syntax `` and inline MEDIA: tokens render as `<img>`
|
||||
@@ -233,13 +206,11 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Tool card expand / collapse toggle
|
||||
- [x] Collapsible thinking / reasoning cards (Claude extended thinking, o3 reasoning tokens)
|
||||
- [x] Message timestamps (subtle, full date on hover)
|
||||
- [x] Chat header (model, icon, TPS badge, timestamp) scales with text-size preference
|
||||
- [x] Empty composer hides send button (icon-circle with pop-in animation)
|
||||
- [x] Pluggable Lucide SVG icons (no emoji rendering inconsistencies)
|
||||
- [x] Composer-centric controls (v0.50.0 UI overhaul)
|
||||
- [x] Hermes Control Center modal (centralized actions)
|
||||
- [x] Workspace panel state machine (defaults closed, opens for browsing / preview)
|
||||
- [x] Three-panel desktop layout keeps a readable conversation floor when resizing
|
||||
- [x] PWA manifest + service worker (offline shell)
|
||||
- [x] Favicon (SVG + PNG + ICO)
|
||||
- [x] Branded onboarding wizard
|
||||
@@ -257,135 +228,85 @@ work has shifted from "reach parity" to "harden reliability + widen distribution
|
||||
- [x] Container queries on composer
|
||||
- [x] Android Chrome compatibility fixes
|
||||
- [x] PWA installation (manifest + icons + Android support)
|
||||
- [x] Streaming-scroll hardening — off-screen height retention, native scroll-anchoring, no jump-to-top on iOS/Android
|
||||
- [x] Mobile drawer surfaces dashboard link + extension nav-actions
|
||||
|
||||
### Internationalization
|
||||
- [x] 15 locales — English, Italian, Japanese, Russian, Spanish, German, Chinese (zh + zh-Hant), Portuguese, Korean, French, Czech, Turkish, Polish, Vietnamese
|
||||
- [x] Near-full translation coverage across non-English locales — a small English-fallback tail remains on recently added keys (measured ~2% for Czech up to ~15% for Turkish), backfilled by periodic translation passes
|
||||
- [x] 9 locales — English, Japanese, Russian, Spanish, German, Chinese (zh + zh-Hant), Portuguese, Korean, French
|
||||
- [x] Key-parity test ensures every locale has every key
|
||||
- [x] Right-to-left and CJK input (IME composition fixes)
|
||||
|
||||
### Gateway integration
|
||||
- [x] Real-time gateway sessions in sidebar (Telegram, Discord, Slack, WeChat/Weixin, Signal, SMS) via SSE + DB polling
|
||||
- [x] Real-time gateway sessions in sidebar (Telegram, Discord, Slack, Weixin) via SSE + DB polling
|
||||
- [x] Cross-channel handoff dock — composer-docked flyout summarizing the live external session
|
||||
- [x] Transcript-summary card at 10+ rounds
|
||||
- [x] Sidebar dedup keying on per-conversation identity (distinct chats from same platform stay separate)
|
||||
- [x] Gateway session sync skips dup / delete options for external sessions
|
||||
- [x] LLM Gateway routing metadata display — assistant turns and session metadata show the served model/provider, failover path, and model-switch warnings (#732)
|
||||
- [x] Gateway status card in Settings (#1457)
|
||||
- [x] Gateway approval-runs API opt-in (documented)
|
||||
- [x] LLM Gateway routing metadata display — assistant turns and session metadata show the served model/provider, failover path, and model-switch warnings when response metadata includes `used_provider`, `used_model`, or `routing` (#732)
|
||||
|
||||
### MCP integration
|
||||
- [x] MCP server management UI (System Settings → MCP Servers)
|
||||
- [x] Add / edit / delete MCP server entries
|
||||
|
||||
### Extension system
|
||||
- [x] Opt-in extension loader — serve local static assets, inject same-origin CSS/JS into the app shell
|
||||
- [x] One-click install from an in-app gallery (Settings → Extensions) into a WebUI-managed state-dir directory (no restart, no env vars)
|
||||
- [x] Manifest contract (`extensions.json`) for bundled / multi-extension installs
|
||||
- [x] Capability surface — register custom themes (skins), register custom TTS engines, add nav-actions, declare loopback sidecars, embed an external web app in an iframe tab
|
||||
- [x] Per-extension settings schema + owned storage
|
||||
- [x] Consented extension sidecar proxy path (no arbitrary backend route registration)
|
||||
- [x] Status + diagnostics endpoint
|
||||
- [x] Trust model documented — extensions run with full session authority; install only vetted/self-authored
|
||||
- [x] Vetted, versioned library repo with CI safety gates — [hermes-webui/hermes-webui-extensions](https://github.com/hermes-webui/hermes-webui-extensions) ("in the registry == vetted")
|
||||
- [x] Core repo bundles zero extensions — clean client-side extensions are migrated to the library repo, not merged into core
|
||||
|
||||
### Distribution
|
||||
- [x] Docker support (multi-arch amd64 + arm64, HEALTHCHECK, UID/GID auto-detect)
|
||||
- [x] Two-container Docker compose (webui + agent)
|
||||
- [x] GHCR auto-publish on tag push
|
||||
- [x] Subpath mount support (reverse proxy at `/hermes/`)
|
||||
- [x] PWA installable from any browser
|
||||
- [x] Native macOS app — universal Intel + Apple Silicon, signed + notarized DMG, Sparkle 2 auto-update
|
||||
- [x] Native Windows + Linux + macOS desktop app — Rust/Tauri, per-platform installers
|
||||
- [x] Native Android app — Play-ready APK/AAB releases
|
||||
- [x] Native iOS app — iPhone client (connects over Tailscale)
|
||||
|
||||
See [Native clients](#native-clients) for the per-repo detail.
|
||||
|
||||
---
|
||||
|
||||
## Native clients
|
||||
|
||||
The web app is the engine; a small fleet of native shells wrap it so Hermes runs
|
||||
as a real app on every platform. Each lives in its **own repo** with its own
|
||||
versioning and release cadence — none is a fork of the WebUI source.
|
||||
|
||||
| Client | Platforms | Tech | Repo |
|
||||
|---|---|---|---|
|
||||
| **Hermes for Mac** | macOS (Intel + Apple Silicon) | Swift + WKWebView, SSH tunnel, Sparkle 2 auto-update, signed + notarized DMG | [hermes-webui/hermes-swift-mac](https://github.com/hermes-webui/hermes-swift-mac) |
|
||||
| **Hermes Desktop** | Windows + Linux + macOS | Rust / Tauri (WebView2 / WebKitGTK), per-platform installers | [hermes-webui/hermes-desktop-rust](https://github.com/hermes-webui/hermes-desktop-rust) |
|
||||
| **Hermes for Android** | Android | Native Android, Play-ready releases | [hermes-webui/hermes-android](https://github.com/hermes-webui/hermes-android) |
|
||||
| **Hermes for iOS** | iOS (iPhone) | Native Swift, connects over Tailscale (QR / hostname pairing) | [hermes-webui/hermes-swift-ios](https://github.com/hermes-webui/hermes-swift-ios) |
|
||||
|
||||
One server, every client: run the WebUI on one machine and reach it from web,
|
||||
desktop, and phone. Live versions and release notes are on each repo's Releases
|
||||
page — this table intentionally carries no version numbers (they drift per repo).
|
||||
|
||||
---
|
||||
|
||||
## Autonomous project maintenance
|
||||
|
||||
Hermes WebUI is maintained by an autonomous agent system: inbound issues and PRs
|
||||
are triaged, deep-reviewed (multi-model gate + full test suite + browser QA),
|
||||
released, and closed with attribution — largely without manual steering. The
|
||||
project-agnostic distillation of that system is published as an open framework so
|
||||
any maintainer can adopt it:
|
||||
|
||||
- **StewardOS** — [nesquena/steward-os](https://github.com/nesquena/steward-os) ·
|
||||
docs at [nesquena.github.io/steward-os](https://nesquena.github.io/steward-os/).
|
||||
Roles, autonomy bands, the security spine, issue/PR/quality-gate lifecycles,
|
||||
and reusable skills — harness-agnostic (no specific agent runtime required).
|
||||
|
||||
StewardOS is the *generalization*; the WebUI's own operating procedures are the
|
||||
*source*. This is the reason contributor PRs get same-day deep review and the
|
||||
release cadence runs multiple ships per day.
|
||||
- [x] Native macOS app — universal Intel + Apple Silicon, signed + notarized DMG, Sparkle 2 auto-update — see `hermes-webui/hermes-swift-mac` repo
|
||||
|
||||
---
|
||||
|
||||
## Forward work
|
||||
|
||||
Most of the original forward-work list has shipped. What remains is either an
|
||||
open feature request under active consideration or an explicitly-deferred concept.
|
||||
Issue state drifts — re-derive with `gh issue list --repo nesquena/hermes-webui`.
|
||||
|
||||
### Open candidates (feature requests under active consideration)
|
||||
### Confirmed candidates (open feature requests with sprint-candidate or active interest)
|
||||
|
||||
| Theme | Tracking | Why |
|
||||
|---|---|---|
|
||||
| Persistent-host stability | #1458 | Bootstrap fork pattern crashes under launchd / systemd — partial fix shipped (foreground mode); state.db FD leak and HTTP-unhealthy wedge remain |
|
||||
| Free-tier OpenRouter variants visible | #1426 | `:free` tool-support filter currently hides them from the picker |
|
||||
| macOS scroll override regression | #1360 | Auto-scroll sometimes overrides user scroll on the desktop app |
|
||||
| GLM dual-use (main + auxiliary) | #1291 | Currently mutually exclusive; same provider can't serve both surfaces |
|
||||
| Auto-assign session to filtered project | #1468 | When user is filtering by project X, new session should default to project X |
|
||||
| Update banner "What's new?" link | #1512 | Surface release highlights from the update banner |
|
||||
| Sunset legacy `LMSTUDIO_API_KEY` env var | #1502 | Tracking issue — alias stays for one minor cycle, then removed |
|
||||
| Hermes Agent dashboard cross-link | #1459 | Detect a running Hermes Agent and surface link in nav |
|
||||
| Gateway status card in Settings | #1457 | Current gateway-status dots only on profile picker |
|
||||
| Insights — daily token chart + per-model breakdown | #1456 | Existing usage badge is per-message; need rollup view |
|
||||
| Logs tab — view agent / errors / gateway logs | #1455 | Currently requires terminal access to log files |
|
||||
| Model picker collision handling | #1425 | Same-name models from different providers aren't disambiguated in dropdown |
|
||||
| "Reveal in Finder" right-click on workspace | #1424 | macOS desktop app convenience |
|
||||
| Configurable session persistence timing | #1406 | Currently every checkpoint, want operator control |
|
||||
| Silent credential self-heal on 401 | #1401 | Gateway auth.json drift should resolve without user re-auth |
|
||||
| LLM Wiki status panel | #1257 | On / off toggle for Wiki integration |
|
||||
| Lightweight in-app Canvas editing | #1255 | Text canvas for prompt drafting / shared notes |
|
||||
| Provider / Model source-of-truth alignment | #1240 | Reconcile WebUI vs CLI vs Gateway provider resolution |
|
||||
| Built-in SearXNG web search | #1037 | Lightweight search tool with on / off toggle |
|
||||
| Sunset legacy `LMSTUDIO_API_KEY` env var | #1502 | Alias stays for one minor cycle, then removed |
|
||||
| Native MCP server expose | #733 | Hermes WebUI as an MCP server for direct agent integration |
|
||||
| Teams / agents management panel | #719 | Editable names, roles, assignments |
|
||||
| WebUI profile ↔ Hermes runtime model alignment | #749 | Design parity between WebUI profiles and the runtime model |
|
||||
| Add agent / replace model modals | #698 | Dedicated modals for agent + model management |
|
||||
| Subagent session relationship view | #1004 | Show subagent hierarchy in sidebar with expand / collapse |
|
||||
|
||||
### Backlog (deferred, listed for visibility)
|
||||
|
||||
- **Insights / monitoring suite** — data tabs / live integration (#722), monitor dashboard concepts (#721)
|
||||
- **Insights / monitoring suite** — agent heartbeat + alerts (#716), quota / rate-limit display (#706), data tabs (#722), monitor dashboard concepts (#766, #721)
|
||||
- **Native MCP server expose** — Hermes WebUI as an MCP server for direct agent integration (#733)
|
||||
- **Teams / agents management panel** — editable names, roles, assignments (#719)
|
||||
- **Web UI profile model alignment with Hermes runtime** — design parity (#749)
|
||||
- **DOM windowing / message virtualization** — for sessions with hundreds of messages (#734)
|
||||
- **Searchable global tool list** (#697)
|
||||
- **Add agent / replace model modals** (#698)
|
||||
- **Code execution inline cells** — Jupyter-style cell rendering inside chat
|
||||
- **Sharing / public conversation URLs** — requires hosted backend with access control (out of scope for self-host)
|
||||
|
||||
### Intentionally not planned
|
||||
- Full SwiftUI/native rewrite of the frontend — the WebView shells already get ~95% of native benefit at a fraction of the maintenance cost
|
||||
- App Store distribution for the desktop apps — sandboxing breaks the local-server model
|
||||
- Full SwiftUI rewrite of the frontend — the WKWebView shell already gets 95% of native benefit
|
||||
- App Store distribution — sandboxing breaks the local server model
|
||||
- Real-time multi-user collaboration — single-user assumption throughout
|
||||
- Multi-tenant / white-label hosting — the project is single-user / multi-profile, not multi-tenant
|
||||
- Plugin marketplace — Hermes skills cover this surface
|
||||
- Anthropic / Claude proprietary features — Projects AI memory, Claude artifacts sync (not reproducible)
|
||||
|
||||
> Note: a plugin/extension marketplace was once "not planned" — that gap is now
|
||||
> filled by the [extension system](#extension-system) + the vetted library repo,
|
||||
> which cover the customization surface without a heavyweight dependency system.
|
||||
|
||||
---
|
||||
|
||||
## Sprint history
|
||||
|
||||
Per-version detail lives in [CHANGELOG.md](./CHANGELOG.md). The table below is a high-level chronology of major sprint themes; individual PR / fix detail lives in CHANGELOG to keep this file readable.
|
||||
Per-version detail lives in [CHANGELOG.md](./CHANGELOG.md). The table below is a high-level chronology of major sprint themes; individual PR / fix detail moved to CHANGELOG to keep this file readable.
|
||||
|
||||
| Range | Theme | Highlights |
|
||||
|---|---|---|
|
||||
@@ -406,26 +327,25 @@ Per-version detail lives in [CHANGELOG.md](./CHANGELOG.md). The table below is a
|
||||
| Sprint 21 | Mobile responsive + Docker | Hamburger sidebar, mobile nav, slide-over files, Docker support (#21, #7) |
|
||||
| Sprint 22 | Multi-profile support | Profile picker, management panel, seamless switching, per-session tracking (#28) |
|
||||
| Sprint 23 | Agentic transparency | Token / cost display, subagent cards, skill picker in cron, profile-local storage |
|
||||
| Sprint 24 | Web polish | rAF streaming, git detection, collapsible date groups, context ring (#80–#83) |
|
||||
| Sprint 25 | macOS desktop application | Native Swift + WKWebView shell, universal DMG, Sparkle 2 auto-update |
|
||||
| Sprint 24 | Web polish | rAF streaming, git detection, collapsible date groups, context ring (#80, #81, #82, #83) |
|
||||
| Sprint 25 | macOS desktop application | Native Swift + WKWebView shell, universal DMG, Sparkle 2 auto-update — separate repo |
|
||||
| Sprint 26 | Pluggable themes | Light / Slate / Solarized / Monokai / Nord, settings unsaved-changes guard, `/theme` |
|
||||
| Sprint 27 | Theme polish | 30+ hardcoded colors → CSS variables, light theme final polish |
|
||||
| Sprint 28 | Security hardening | Env race fix, random signing key, upload traversal, PBKDF2 |
|
||||
| Sprints 29–32 | Model routing + custom endpoints + reasoning | Model routing by provider prefix, custom endpoint URL fix, OLED theme, top-level reasoning, message_count sync |
|
||||
| Sprint 33 | Approval card + Lucide icons | Approval prompt surfaced, emoji → SVG, login CSP fix, update diagnostics |
|
||||
| Sprint 34 | v0.50.0 UI overhaul | Composer-centric controls, Control Center modal, workspace state machine, collapsible date groups, rAF throttle, context ring |
|
||||
| Sprints 35–37 | Onboarding + i18n | First-run wizard, provider config, Spanish locale, Docker two-container, mobile Profiles button |
|
||||
| Sprints 38–40 | Session + UI polish | Five-bug clean-up, sidebar timestamp, test port isolation |
|
||||
| Sprints 35–37 | Onboarding + i18n + Spanish | First-run wizard, OpenRouter / Anthropic / OpenAI / Custom config, Spanish locale, Docker two-container, mobile Profiles button |
|
||||
| Sprints 38–40 | Session + UI polish + Sprint 40 | Five-bug clean-up + sidebar timestamp + test port isolation |
|
||||
| Sprints 41–42 | Renderer hardening + KaTeX + handoff | Context ring live usage, renderMd link / image / code stash chain, MEDIA: image rendering, gateway handoff foundation |
|
||||
| Sprints 43+ | Continuous contributor sprints | Custom providers, more locales, IME fixes, model-switch toast, approval queue multi-slot, profile polish, font-size CSS, contributor wave |
|
||||
| Ongoing | Ecosystem expansion + reliability | Extension system + library, Rust/Android/iOS native clients, StewardOS, gateway platform expansion (Signal/SMS/WeChat), Transparent Stream, mobile scroll hardening, session-load perf, test-isolation flake elimination |
|
||||
| Sprints 43+ | Continuous contributor sprints | Custom providers, Russian locale, IME fixes, model-switch toast, approval queue multi-slot, profile polish, font-size CSS, contributor wave |
|
||||
|
||||
---
|
||||
|
||||
## Versioning conventions
|
||||
|
||||
- **Patch** (`v0.51.X`) — small batches, contributor PR releases, hotfixes
|
||||
- **Patch** (`v0.50.X`) — small batches, contributor PR releases, hotfixes
|
||||
- **Minor** (`v0.X.0`) — sprint completion, new feature surface, architecture milestone
|
||||
- **Major** (`v1.0.0`) — declared when the feature surface stabilizes and reliability across all clients reaches steady state
|
||||
- **Major** (`v1.0.0`) — declared when CLI parity + Claude parity reach steady state and the feature surface stabilizes
|
||||
|
||||
Per-version detail and contributor attribution live in [CHANGELOG.md](./CHANGELOG.md).
|
||||
|
||||
+1
-1
@@ -2,7 +2,7 @@
|
||||
|
||||
> Forward-looking sprint plan and active queue.
|
||||
>
|
||||
> Current state: v0.51.792 | ~11,500 tests | port 8787
|
||||
> Current state: v0.50.281 | 3995 tests | port 8787
|
||||
> Target A (CLI parity): ✅ Complete
|
||||
> Target B (Claude parity): ~95% — full subagent transparency UI and code-execution cells remain
|
||||
>
|
||||
|
||||
+6
-85
@@ -8,7 +8,7 @@
|
||||
> Prerequisites: SSH tunnel is active on port 8787. Open http://localhost:8787 in browser.
|
||||
> Server health check: curl http://127.0.0.1:8787/health should return {"status":"ok"}.
|
||||
>
|
||||
> Automated coverage: ~11,500 tests collected via `./scripts/test.sh tests/ --collect-only -q`. Tests run on every PR via GitHub Actions on Python 3.11, 3.12, and 3.13 (3 parallel shards each), alongside a ruff lint gate, a headless browser smoke test, and a Docker smoke test. The suite covers the bootstrap/static wizard, real provider config persistence (`config.yaml` + `.env`), the `/api/onboarding/*` backend, the onboarding skip/existing-config guard, CSS regression coverage for thinking/tool card animation, streaming session persistence, mobile layout breakpoints, locale parity across 14 languages, and hundreds of issue/PR-pinned regression tests.
|
||||
> Automated coverage: ~7,150 tests collected via `./scripts/test.sh tests/ --collect-only -q`. Tests run on every PR via GitHub Actions on Python 3.11, 3.12, and 3.13 (3 parallel shards each), alongside a ruff lint gate, a headless browser smoke test, and a Docker smoke test. The suite covers the bootstrap/static wizard, real provider config persistence (`config.yaml` + `.env`), the `/api/onboarding/*` backend, the onboarding skip/existing-config guard, CSS regression coverage for thinking/tool card animation, streaming session persistence, mobile layout breakpoints, locale parity across 11 languages, and hundreds of issue/PR-pinned regression tests.
|
||||
> Run: `./scripts/test.sh`
|
||||
>
|
||||
> Local regression focus: verify that a previously closed workspace panel stays visually closed from first paint through boot completion on desktop refresh; there should be no brief open-then-close flash.
|
||||
@@ -89,67 +89,9 @@ python tests/browser_smoke.py
|
||||
It is intentionally **credential-free**: it strips every `*_API_KEY` from the
|
||||
environment before launching the server, needs no secrets, and does not drive a
|
||||
real model (it verifies the app *loads and initializes* cleanly — the brick class
|
||||
that breaks the page for everyone).
|
||||
|
||||
## Public conversation lifecycle gate
|
||||
|
||||
`tests/browser_conversation_lifecycle.py` adds a public deterministic
|
||||
multi-row lifecycle gate. It drives the real composer and real WebUI server in Chromium,
|
||||
while a localhost-only fixture supplies reasoning, tool, process, and final/error
|
||||
events through the existing Hermes Gateway Runs API. The gate now covers both
|
||||
normal and terminal-error proof-matrix rows, asserting semantic activity during
|
||||
live streaming, after settlement, and after hard reload, including
|
||||
transcript-backed `activity_scene_v1` persistence and zero unexpected browser
|
||||
errors. It uses isolated temporary state and no provider credentials.
|
||||
|
||||
```bash
|
||||
pip install -r requirements.txt playwright
|
||||
python -m playwright install --with-deps chromium
|
||||
|
||||
# Normal-path deterministic conversation lifecycle gate.
|
||||
python tests/browser_conversation_lifecycle.py
|
||||
|
||||
# Terminal-error lifecycle gate (new row in the proof matrix).
|
||||
LIFECYCLE_SCENARIO=terminal-error python tests/browser_conversation_lifecycle.py
|
||||
|
||||
# Historical ID-linked transcript hydration row.
|
||||
python tests/browser_historical_transcript_hydration.py
|
||||
```
|
||||
|
||||
To certify that the gate catches its target failure, the test owns an opt-in
|
||||
mutation that drops the browser's Anchor-scene persistence request. This command
|
||||
must fail at the hard-reload boundary:
|
||||
|
||||
```bash
|
||||
LIFECYCLE_TEST_BITE=drop-anchor-persistence \
|
||||
python tests/browser_conversation_lifecycle.py
|
||||
|
||||
# Terminal-state-specific mutation bite: remove terminal row from persisted scene
|
||||
# so hard reload cannot recover terminal status.
|
||||
LIFECYCLE_SCENARIO=terminal-error \
|
||||
LIFECYCLE_TEST_BITE=drop-terminal-anchor-row \
|
||||
python tests/browser_conversation_lifecycle.py
|
||||
|
||||
# Historical-hydration mutation: corrupt one persisted tool-result link so the
|
||||
# strict Anchor projection must fail instead of claiming the legacy transcript.
|
||||
HISTORICAL_HYDRATION_TEST_BITE=break-tool-link \
|
||||
python tests/browser_historical_transcript_hydration.py
|
||||
```
|
||||
|
||||
The dedicated `Conversation lifecycle (informational)` workflow runs the current
|
||||
proof rows (`normal`, `terminal-error`, and `historical-transcript-hydration`) and
|
||||
stays non-blocking while the public
|
||||
matrix expands to additional behavior rows. The maintainer's private QA harness
|
||||
remains broader; later public slices will add session switching, reconnect/replay,
|
||||
cancellation, compression, and recovery.
|
||||
|
||||
### Streaming reader intent
|
||||
|
||||
While a response is still streaming, scroll upward with a trackpad or wheel to
|
||||
read earlier content, including a small scroll gesture immediately after a live
|
||||
render. Subsequent streamed content must not pull the reader back to the bottom.
|
||||
Use the jump-to-latest control to resume following the live tail; after that,
|
||||
new streamed content should remain visible at the bottom.
|
||||
that breaks the page for everyone). A full chat golden-path E2E (send → stream →
|
||||
render → switch → reload) lives in the maintainer's private QA harness, which has
|
||||
the agent + a mock LLM provider available.
|
||||
|
||||
|
||||
`tests/test_static_js_runtime_lint.py` runs this automatically when eslint is present
|
||||
@@ -615,27 +557,6 @@ EXPECT:
|
||||
(response text)
|
||||
FAIL: No download triggered, file is empty, file is corrupted JSON instead of markdown.
|
||||
|
||||
### T8.2: Create and Revoke a Public Share Link
|
||||
SETUP: A session with at least 2 visible messages (1 user + 1 assistant).
|
||||
STEPS:
|
||||
1. Click the "Hermes" button in the sidebar footer
|
||||
2. In the Conversation section, click "Share"
|
||||
EXPECT:
|
||||
- A public `/share/<token>` page opens in a new tab
|
||||
- The link is copied to the clipboard
|
||||
- The Conversation section meta line shows that a public share is active
|
||||
- The shared page uses the current Hermes theme/skin and shows only a read-only transcript snapshot
|
||||
FAIL: No link opens, the page requires login unexpectedly, or the shared page exposes workspace/profile/live controls.
|
||||
|
||||
STEPS:
|
||||
3. Return to the app and click "Stop sharing"
|
||||
4. Confirm the revoke dialog
|
||||
EXPECT:
|
||||
- A toast confirms the link was revoked
|
||||
- Reopening the old share URL shows an unavailable/not found state
|
||||
- The active-share status disappears from the Conversation section
|
||||
FAIL: The old link still loads the transcript, or the session still appears as publicly shared after revocation.
|
||||
|
||||
---
|
||||
|
||||
## Section 9: Reconnect Banner (Sprint 1 - B4/B5)
|
||||
@@ -2004,8 +1925,8 @@ Bridged CLI sessions:
|
||||
|
||||
---
|
||||
|
||||
*Last updated: v0.51.792, July 1, 2026*
|
||||
*Total automated tests collected: ~11,500 (run `./scripts/test.sh tests/ --collect-only -q` for the exact current count)*
|
||||
*Last updated: v0.51.192, May 31, 2026*
|
||||
*Total automated tests collected: ~7,150 (run `./scripts/test.sh tests/ --collect-only -q` for the exact current count)*
|
||||
*Regression gate: tests/test_regressions.py*
|
||||
*Run: ./scripts/test.sh*
|
||||
*Source: <repo>/*
|
||||
|
||||
@@ -102,12 +102,6 @@ Two ways to ship it:
|
||||
declare it in `HERMES_WEBUI_EXTENSION_STYLESHEET_URLS`. No code changes
|
||||
needed; the skin attribute can be set from your own JS.
|
||||
|
||||
Extensions that register a skin through `window.registerHermesSkin()` may
|
||||
also set `scheme: "light"` or `scheme: "dark"` for light-only or dark-only
|
||||
skins. The saved Theme preference stays unchanged, but WebUI applies the
|
||||
matching effective base class while that skin is selected so System/Light or
|
||||
System/Dark does not mix incompatible base tokens into the skin.
|
||||
|
||||
### Tips
|
||||
|
||||
- **Test both themes.** A skin that pops on Dark can be illegible on Light.
|
||||
@@ -144,43 +138,11 @@ font size. Persists alongside theme and skin.
|
||||
|
||||
---
|
||||
|
||||
## Typography Tokens
|
||||
|
||||
The three font tokens are declared in `static/style.css`:
|
||||
|
||||
- `--font-ui`: user interface chrome, controls, composer chrome, labels, and
|
||||
ordinary non-code UI text
|
||||
- `--font-conversation`: user/assistant message prose
|
||||
- `--font-mono`: code, tool details, logs, identifiers, and terminal text
|
||||
|
||||
By default:
|
||||
|
||||
```css
|
||||
--font-conversation: var(--font-ui);
|
||||
--font-mono: ui-monospace,"SFMono-Regular","SF Mono",Menlo,Consolas,"Liberation Mono",monospace;
|
||||
```
|
||||
|
||||
For CSS-based custom skins and extension stylesheets/scripts, override the three
|
||||
font tokens directly (`--font-ui`, `--font-conversation`, `--font-mono`) on the
|
||||
skin root (`:root[data-skin="..."]`), including any mode-specific variant.
|
||||
`window.registerHermesSkin()` currently accepts only the documented design-token
|
||||
allowlist (no `--font-*` tokens), so custom font families should be supplied via
|
||||
CSS rather than skin registration.
|
||||
|
||||
Leaving `--font-conversation` untouched keeps prose coupled to `--font-ui`;
|
||||
override it only when a different reading face is explicitly required. Avoid
|
||||
hard-coded selector-level font declarations when one of these tokens already
|
||||
expresses the intent.
|
||||
|
||||
---
|
||||
|
||||
## How It Works Internally
|
||||
|
||||
1. **Theme:** `document.documentElement.classList.toggle('dark', isDark)` —
|
||||
light mode removes the class. System mode tracks
|
||||
`matchMedia('(prefers-color-scheme: dark)')`. Extension-registered skins
|
||||
may declare a light/dark `scheme`; when selected, that scheme controls the
|
||||
effective `.dark` class without rewriting the saved Theme preference.
|
||||
`matchMedia('(prefers-color-scheme: dark)')`.
|
||||
2. **Skin:** `document.documentElement.dataset.skin = name` (or remove the
|
||||
attribute for `default`).
|
||||
3. **Font size:** `document.documentElement.dataset.fontSize = size` (or
|
||||
|
||||
+40
-301
@@ -23,7 +23,6 @@ confirmed outage.
|
||||
from __future__ import annotations
|
||||
|
||||
import importlib
|
||||
import inspect
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
@@ -252,155 +251,6 @@ def _gateway_running_pid(gateway_status: Any, pid_path: Path | None) -> int | No
|
||||
return get_running_pid()
|
||||
|
||||
|
||||
def _callable_code_declares_parameter(get_running_pid: Any, name: str) -> bool:
|
||||
"""Return True when a Python callable's own code declares *name*."""
|
||||
target = getattr(get_running_pid, "__func__", get_running_pid)
|
||||
code = getattr(target, "__code__", None)
|
||||
if code is None:
|
||||
return False
|
||||
declared_count = int(getattr(code, "co_argcount", 0)) + int(
|
||||
getattr(code, "co_kwonlyargcount", 0)
|
||||
)
|
||||
return name in code.co_varnames[:declared_count]
|
||||
|
||||
|
||||
def _declared_pid_path_parameter(
|
||||
get_running_pid: Any,
|
||||
) -> tuple[inspect.Parameter, int, list[inspect.Parameter]] | None:
|
||||
try:
|
||||
signature = inspect.signature(get_running_pid, follow_wrapped=False)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
params = list(signature.parameters.values())
|
||||
for idx, param in enumerate(params):
|
||||
if param.kind not in (
|
||||
inspect.Parameter.POSITIONAL_ONLY,
|
||||
inspect.Parameter.POSITIONAL_OR_KEYWORD,
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
):
|
||||
continue
|
||||
name = str(param.name or "").lower()
|
||||
if (
|
||||
"path" in name
|
||||
and "pid" in name
|
||||
and _callable_code_declares_parameter(get_running_pid, param.name)
|
||||
):
|
||||
positional_index = sum(
|
||||
1
|
||||
for earlier in params[:idx]
|
||||
if earlier.kind
|
||||
in (
|
||||
inspect.Parameter.POSITIONAL_ONLY,
|
||||
inspect.Parameter.POSITIONAL_OR_KEYWORD,
|
||||
)
|
||||
)
|
||||
return param, positional_index, params
|
||||
return None
|
||||
|
||||
|
||||
def _positional_only_pid_args(
|
||||
params: list[inspect.Parameter],
|
||||
positional_index: int,
|
||||
pid_path: Path,
|
||||
) -> list[Any] | None:
|
||||
positional_params = [
|
||||
param
|
||||
for param in params
|
||||
if param.kind
|
||||
in (
|
||||
inspect.Parameter.POSITIONAL_ONLY,
|
||||
inspect.Parameter.POSITIONAL_OR_KEYWORD,
|
||||
)
|
||||
]
|
||||
if positional_index >= len(positional_params):
|
||||
return None
|
||||
args: list[Any] = []
|
||||
for param in positional_params[:positional_index]:
|
||||
if param.default is inspect.Parameter.empty:
|
||||
return None
|
||||
args.append(param.default)
|
||||
args.append(pid_path)
|
||||
return args
|
||||
|
||||
|
||||
def _accepts_cleanup_stale_keyword(params: list[inspect.Parameter]) -> bool:
|
||||
return any(
|
||||
param.name == "cleanup_stale"
|
||||
and param.kind
|
||||
in (
|
||||
inspect.Parameter.KEYWORD_ONLY,
|
||||
inspect.Parameter.POSITIONAL_OR_KEYWORD,
|
||||
)
|
||||
for param in params
|
||||
)
|
||||
|
||||
|
||||
def _gateway_running_pid_strict_path(gateway_status: Any, pid_path: Path) -> int | None:
|
||||
"""Read a PID from an explicit path only; never fall back to ambient state."""
|
||||
get_running_pid = gateway_status.get_running_pid
|
||||
pid_path_match = _declared_pid_path_parameter(get_running_pid)
|
||||
if pid_path_match is None:
|
||||
return None
|
||||
pid_path_param, positional_index, params = pid_path_match
|
||||
|
||||
if pid_path_param.kind is inspect.Parameter.KEYWORD_ONLY:
|
||||
kwargs = {pid_path_param.name: pid_path}
|
||||
try:
|
||||
return get_running_pid(**kwargs, cleanup_stale=False)
|
||||
except TypeError:
|
||||
try:
|
||||
return get_running_pid(**kwargs)
|
||||
except TypeError:
|
||||
return None
|
||||
|
||||
if pid_path_param.kind is inspect.Parameter.POSITIONAL_OR_KEYWORD:
|
||||
kwargs = {pid_path_param.name: pid_path}
|
||||
try:
|
||||
return get_running_pid(**kwargs, cleanup_stale=False)
|
||||
except TypeError:
|
||||
try:
|
||||
return get_running_pid(**kwargs)
|
||||
except TypeError:
|
||||
return None
|
||||
|
||||
if pid_path_param.kind is inspect.Parameter.POSITIONAL_ONLY:
|
||||
args = _positional_only_pid_args(params, positional_index, pid_path)
|
||||
if args is None:
|
||||
return None
|
||||
if _accepts_cleanup_stale_keyword(params):
|
||||
try:
|
||||
return get_running_pid(*args, cleanup_stale=False)
|
||||
except TypeError:
|
||||
pass
|
||||
try:
|
||||
return get_running_pid(*args)
|
||||
except TypeError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def get_active_profile_gateway_running_pid(profile: str | None = None) -> int | None:
|
||||
"""Return the confirmed active-profile PID without state fallbacks.
|
||||
|
||||
Update recovery needs process identity, not just a recent ``running`` state:
|
||||
an old gateway can remain alive when the restart CLI fails before executing.
|
||||
Use the same active-profile home targeted by the restart helper rather than
|
||||
the default-root health path. Keep this fail-closed when status is unavailable.
|
||||
"""
|
||||
try:
|
||||
from api.profiles import get_active_hermes_home, get_hermes_home_for_profile
|
||||
|
||||
gateway_status = _gateway_status_module()
|
||||
if profile is None:
|
||||
gateway_home = get_active_hermes_home()
|
||||
else:
|
||||
gateway_home = get_hermes_home_for_profile(profile)
|
||||
gateway_pid_path = Path(gateway_home) / _GATEWAY_PID_FILE
|
||||
return _gateway_running_pid_strict_path(gateway_status, gateway_pid_path)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _runtime_detail_subset(runtime_status: dict[str, Any] | None) -> dict[str, Any]:
|
||||
"""Return only non-sensitive runtime fields for the browser.
|
||||
|
||||
@@ -464,24 +314,7 @@ _REMOTE_PROBE_PATHS: tuple[str, ...] = ("/health/detailed", "/health", "/v1/heal
|
||||
_REMOTE_PROBE_BODY_LIMIT_BYTES: int = 64 * 1024
|
||||
|
||||
_remote_probe_lock = threading.Lock()
|
||||
# Condition wraps the same lock so cache reads/writes and single-flight waits
|
||||
# share one mutex (mirrors the Condition(_lock) idiom in api/session_lifecycle).
|
||||
_remote_probe_cond = threading.Condition(_remote_probe_lock)
|
||||
_remote_probe_cache: dict[str, Any] = {"url": None, "expires_at": 0.0, "result": None}
|
||||
# base_urls currently being probed by a "leader" thread. Latecomers wait on the
|
||||
# Condition for the leader's result instead of stampeding the (possibly dead)
|
||||
# gateway themselves (#5455 dashboard fan-out, #2476).
|
||||
_remote_probe_inflight: set[str] = set()
|
||||
def _remote_probe_wait_budget_s() -> float:
|
||||
"""How long a latecomer waits for the leader before giving up and self-probing.
|
||||
|
||||
The leader can walk every path (each up to ``_REMOTE_PROBE_TIMEOUT_S``), so
|
||||
budget for the full walk plus a small margin; timing out is a safety valve
|
||||
against a hung leader, never the normal path. Computed on each call (not a
|
||||
module-level constant) so a test that monkeypatches the timeout or the path
|
||||
list gets a budget consistent with those values instead of a stale one.
|
||||
"""
|
||||
return _REMOTE_PROBE_TIMEOUT_S * len(_REMOTE_PROBE_PATHS) + 1.0
|
||||
|
||||
|
||||
def _remote_gateway_base_url() -> str | None:
|
||||
@@ -515,26 +348,7 @@ def _remote_gateway_base_url() -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _remote_gateway_api_key() -> str:
|
||||
"""Return the Bearer token for authenticated gateway health probes.
|
||||
|
||||
Mirrors ``api.gateway_chat._gateway_api_key``: WebUI containers in
|
||||
multi-service deployments must present the same key the agent's API server
|
||||
expects on ``/health/detailed`` (#5418).
|
||||
"""
|
||||
return str(
|
||||
os.environ.get("HERMES_WEBUI_GATEWAY_API_KEY")
|
||||
or os.environ.get("API_SERVER_KEY")
|
||||
or ""
|
||||
).strip()
|
||||
|
||||
|
||||
def _http_probe(
|
||||
url: str,
|
||||
timeout_s: float,
|
||||
*,
|
||||
api_key: str | None = None,
|
||||
) -> tuple[bool, int | None, str | None, bytes | None]:
|
||||
def _http_probe(url: str, timeout_s: float) -> tuple[bool, int | None, str | None, bytes | None]:
|
||||
"""GET ``url`` and return (ok, status_code, error_name, body).
|
||||
|
||||
``ok`` is True only for a 2xx response. 5xx and network errors are not OK.
|
||||
@@ -542,10 +356,7 @@ def _http_probe(
|
||||
on this particular path) so the caller can move on to the next path.
|
||||
``body`` is the raw response bytes for 2xx responses, None otherwise.
|
||||
"""
|
||||
headers: dict[str, str] = {}
|
||||
if api_key:
|
||||
headers["Authorization"] = f"Bearer {api_key}"
|
||||
req = urllib_request.Request(url, method="GET", headers=headers)
|
||||
req = urllib_request.Request(url, method="GET")
|
||||
try:
|
||||
with urllib_request.urlopen(req, timeout=timeout_s) as resp: # noqa: S310 - trusted env var URL
|
||||
status = getattr(resp, "status", None) or resp.getcode()
|
||||
@@ -562,37 +373,27 @@ def _http_probe(
|
||||
return (False, None, type(exc).__name__, None)
|
||||
|
||||
|
||||
def _cached_remote_result_locked(base_url: str, current: float) -> dict[str, Any] | None:
|
||||
"""Return a fresh cached probe result for *base_url*, or None if stale/absent.
|
||||
def _probe_remote_gateway(base_url: str, *, now: float | None = None) -> dict[str, Any]:
|
||||
"""Return an agent-health payload dict for a remote gateway base URL.
|
||||
|
||||
Must be called while holding ``_remote_probe_cond``.
|
||||
Result is cached for ``_REMOTE_PROBE_CACHE_TTL_S`` seconds per base_url.
|
||||
"""
|
||||
if (
|
||||
_remote_probe_cache.get("url") == base_url
|
||||
and _remote_probe_cache.get("expires_at", 0.0) > current
|
||||
and _remote_probe_cache.get("result") is not None
|
||||
):
|
||||
return _remote_probe_cache["result"]
|
||||
return None
|
||||
current = time.monotonic() if now is None else now
|
||||
with _remote_probe_lock:
|
||||
if (
|
||||
_remote_probe_cache.get("url") == base_url
|
||||
and _remote_probe_cache.get("expires_at", 0.0) > current
|
||||
and _remote_probe_cache.get("result") is not None
|
||||
):
|
||||
cached = _remote_probe_cache["result"]
|
||||
# Refresh checked_at so the UI shows a current timestamp without
|
||||
# actually re-hitting the gateway.
|
||||
return {**cached, "checked_at": _checked_at()}
|
||||
|
||||
|
||||
def _run_remote_probe(base_url: str) -> dict[str, Any]:
|
||||
"""Walk the remote gateway health paths and build a payload (no caching/locking).
|
||||
|
||||
This is the expensive, network-bound step: each path can block up to
|
||||
``_REMOTE_PROBE_TIMEOUT_S``. It runs OUTSIDE the probe lock so a single
|
||||
"leader" thread does it while latecomers wait for the cached result.
|
||||
"""
|
||||
last_status: int | None = None
|
||||
last_error: str | None = None
|
||||
gateway_api_key = _remote_gateway_api_key()
|
||||
for path in _REMOTE_PROBE_PATHS:
|
||||
probe_key = gateway_api_key if path == "/health/detailed" else None
|
||||
ok, status, err, body = _http_probe(
|
||||
base_url + path,
|
||||
_REMOTE_PROBE_TIMEOUT_S,
|
||||
api_key=probe_key,
|
||||
)
|
||||
ok, status, err, body = _http_probe(base_url + path, _REMOTE_PROBE_TIMEOUT_S)
|
||||
if ok:
|
||||
details: dict[str, Any] = {
|
||||
"state": "alive",
|
||||
@@ -610,108 +411,46 @@ def _run_remote_probe(base_url: str) -> dict[str, Any]:
|
||||
# An over-cap body (len > limit, i.e. the +1 sentinel byte was read)
|
||||
# is treated as "alive but no parseable gateway_state" — we still
|
||||
# report the gateway as up, just without the detailed state.
|
||||
return {
|
||||
payload = {
|
||||
"alive": True,
|
||||
"checked_at": _checked_at(),
|
||||
"details": details,
|
||||
}
|
||||
break
|
||||
# Remember the most informative failure signal we saw.
|
||||
if status is not None:
|
||||
last_status = status
|
||||
if err is not None:
|
||||
last_error = err
|
||||
else:
|
||||
details: dict[str, Any] = {
|
||||
"state": "down",
|
||||
"reason": "remote_gateway_unreachable",
|
||||
"endpoint": base_url,
|
||||
}
|
||||
if last_status is not None:
|
||||
details["status_code"] = last_status
|
||||
if last_error is not None:
|
||||
details["error"] = last_error
|
||||
payload = {
|
||||
"alive": False,
|
||||
"checked_at": _checked_at(),
|
||||
"details": details,
|
||||
}
|
||||
|
||||
details = {
|
||||
"state": "down",
|
||||
"reason": "remote_gateway_unreachable",
|
||||
"endpoint": base_url,
|
||||
}
|
||||
if last_status is not None:
|
||||
details["status_code"] = last_status
|
||||
if last_error is not None:
|
||||
details["error"] = last_error
|
||||
return {
|
||||
"alive": False,
|
||||
"checked_at": _checked_at(),
|
||||
"details": details,
|
||||
}
|
||||
|
||||
|
||||
def _probe_remote_gateway(base_url: str, *, now: float | None = None) -> dict[str, Any]:
|
||||
"""Return an agent-health payload dict for a remote gateway base URL.
|
||||
|
||||
Result is cached for ``_REMOTE_PROBE_CACHE_TTL_S`` seconds per base_url.
|
||||
|
||||
Concurrency (single-flight): when several dashboard panels fan out on a cold
|
||||
cache, only the first "leader" thread runs the ~2s-per-path network probe;
|
||||
latecomers wait on ``_remote_probe_cond`` for the leader's cached result
|
||||
rather than each hammering the (possibly dead) gateway (#5455, #2476). The
|
||||
leader always clears the in-flight marker and wakes waiters — even on error —
|
||||
so waiters can never deadlock.
|
||||
"""
|
||||
current = time.monotonic() if now is None else now
|
||||
with _remote_probe_cond:
|
||||
cached = _cached_remote_result_locked(base_url, current)
|
||||
if cached is not None:
|
||||
# Refresh checked_at so the UI shows a current timestamp without
|
||||
# actually re-hitting the gateway.
|
||||
return {**cached, "checked_at": _checked_at()}
|
||||
|
||||
# A leader is already probing this base_url: wait for its result instead
|
||||
# of starting a duplicate probe.
|
||||
if base_url in _remote_probe_inflight:
|
||||
deadline = current + _remote_probe_wait_budget_s()
|
||||
while base_url in _remote_probe_inflight:
|
||||
remaining = deadline - time.monotonic()
|
||||
if remaining <= 0:
|
||||
break # leader stuck; fall through and probe ourselves
|
||||
_remote_probe_cond.wait(remaining)
|
||||
cached = _cached_remote_result_locked(base_url, time.monotonic())
|
||||
if cached is not None:
|
||||
return {**cached, "checked_at": _checked_at()}
|
||||
# Woke without a usable cached result (leader failed, produced no
|
||||
# cacheable result, or we timed out): become a leader ourselves.
|
||||
|
||||
# Become the leader for this base_url.
|
||||
_remote_probe_inflight.add(base_url)
|
||||
|
||||
payload: dict[str, Any] | None = None
|
||||
try:
|
||||
payload = _run_remote_probe(base_url)
|
||||
finally:
|
||||
with _remote_probe_cond:
|
||||
if payload is not None:
|
||||
_remote_probe_cache["url"] = base_url
|
||||
# Expire from the moment the probe COMPLETES, not from the
|
||||
# leader's entry time: walking every path of a hung gateway
|
||||
# takes len(paths) * timeout (~6s) which exceeds the 5s TTL, so
|
||||
# `current + TTL` would write an already-expired cache line.
|
||||
# Waiters woken right after this would then miss the cache and
|
||||
# each re-probe the dead gateway — collapsing single-flight and
|
||||
# regressing latency to worse-than-serial (#5455, #2476).
|
||||
_remote_probe_cache["expires_at"] = (
|
||||
time.monotonic() + _REMOTE_PROBE_CACHE_TTL_S
|
||||
)
|
||||
_remote_probe_cache["result"] = payload
|
||||
# Always release the in-flight marker and wake waiters, even if the
|
||||
# probe raised — otherwise latecomers would wait out the full budget.
|
||||
_remote_probe_inflight.discard(base_url)
|
||||
_remote_probe_cond.notify_all()
|
||||
# Only reached when _run_remote_probe returned normally (a raise would have
|
||||
# propagated through the finally), so payload is always a dict here. The
|
||||
# assert makes that explicit for the type checker (declared -> dict[str, Any]).
|
||||
assert payload is not None
|
||||
with _remote_probe_lock:
|
||||
_remote_probe_cache["url"] = base_url
|
||||
_remote_probe_cache["expires_at"] = current + _REMOTE_PROBE_CACHE_TTL_S
|
||||
_remote_probe_cache["result"] = payload
|
||||
return payload
|
||||
|
||||
|
||||
def _reset_remote_probe_cache_for_tests() -> None:
|
||||
"""Test hook: clear the in-process remote-probe cache and in-flight state."""
|
||||
with _remote_probe_cond:
|
||||
"""Test hook: clear the in-process remote-probe cache."""
|
||||
with _remote_probe_lock:
|
||||
_remote_probe_cache["url"] = None
|
||||
_remote_probe_cache["expires_at"] = 0.0
|
||||
_remote_probe_cache["result"] = None
|
||||
_remote_probe_inflight.clear()
|
||||
_remote_probe_cond.notify_all()
|
||||
|
||||
|
||||
def build_agent_health_payload() -> dict[str, Any]:
|
||||
|
||||
@@ -1,435 +0,0 @@
|
||||
"""Fail-closed guard for in-process Hermes Agent source revisions.
|
||||
|
||||
Hermes WebUI currently imports ``run_agent.AIAgent`` into its long-lived server
|
||||
process. If the Agent checkout changes while that process is alive, Python may
|
||||
combine already-cached modules with newly-read source. Refuse to reuse that
|
||||
mixed runtime and require a clean WebUI restart instead.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import errno
|
||||
import math
|
||||
import os
|
||||
from pathlib import Path
|
||||
import stat
|
||||
import sys
|
||||
import subprocess
|
||||
import threading
|
||||
import time
|
||||
|
||||
# Retain the discovered path as a diagnostic/test-visible compatibility value;
|
||||
# runtime identity is deliberately captured from the loaded module below.
|
||||
from api.config import (
|
||||
PYTHON_EXE,
|
||||
_AGENT_DIR, # noqa: F401
|
||||
_DEFAULT_STATE_HOME,
|
||||
)
|
||||
from api.subprocess_utils import windows_hide_flags
|
||||
|
||||
_RESTART_REQUIRED_MESSAGE = (
|
||||
"Hermes Agent was updated while Hermes WebUI was running. "
|
||||
"WebUI cannot verify that the Agent update completed safely. "
|
||||
"Check the Agent update outcome and environment first. "
|
||||
"Restart Hermes WebUI manually before retrying this action."
|
||||
)
|
||||
_AGENT_UPDATE_MARKER = ".hermes-update-in-progress"
|
||||
_AGENT_RECOVERY_MARKERS = (".update-incomplete", ".lazy-refresh-incomplete")
|
||||
_AGENT_UPDATE_MAX_AGE_SECONDS = 20 * 60
|
||||
# The update marker holds a PID and a start timestamp (two short numeric lines).
|
||||
# Anything larger is not a legitimate marker; cap the read so a huge or growing
|
||||
# regular file can never exhaust memory on the stale-runtime request path.
|
||||
_AGENT_UPDATE_MARKER_MAX_BYTES = 64 * 1024
|
||||
# O_NOFOLLOW is POSIX; on platforms that lack it the fast os.open() path is not
|
||||
# taken at all (see _MARKER_SAFE_OPEN_AVAILABLE below).
|
||||
# The marker read hardening relies on two POSIX-only open flags to stay both
|
||||
# non-blocking (never hang on a FIFO/device) and symlink-safe. O_NONBLOCK is
|
||||
# Unix-only and O_NOFOLLOW is absent on some platforms; accessing them
|
||||
# unconditionally raises AttributeError on native Windows. Resolve them safely
|
||||
# and only take the os.open() fast path when BOTH are genuinely available —
|
||||
# otherwise the read cannot prove non-blocking + no-follow and must fall back to
|
||||
# an lstat-only classification (see _read_live_agent_update).
|
||||
_O_NOFOLLOW = getattr(os, "O_NOFOLLOW", 0)
|
||||
_O_NONBLOCK = getattr(os, "O_NONBLOCK", 0)
|
||||
_MARKER_SAFE_OPEN_AVAILABLE = bool(getattr(os, "O_NOFOLLOW", 0)) and bool(
|
||||
getattr(os, "O_NONBLOCK", 0)
|
||||
)
|
||||
_HERMES_HOME = Path(_DEFAULT_STATE_HOME)
|
||||
_AGENT_PYTHON = Path(PYTHON_EXE).expanduser() if PYTHON_EXE else None
|
||||
|
||||
|
||||
def _read_agent_revision(
|
||||
agent_dir: Path | None,
|
||||
*,
|
||||
module_path: Path | None = None,
|
||||
) -> str | None:
|
||||
"""Return the loaded Agent checkout HEAD, or ``None`` if it is not tracked."""
|
||||
if agent_dir is None:
|
||||
return None
|
||||
|
||||
if module_path is None:
|
||||
module = sys.modules.get("run_agent")
|
||||
module_file = getattr(module, "__file__", None)
|
||||
if not module_file:
|
||||
return None
|
||||
try:
|
||||
module_path = Path(module_file).resolve()
|
||||
except (OSError, RuntimeError, TypeError):
|
||||
return None
|
||||
|
||||
try:
|
||||
worktree_result = subprocess.run(
|
||||
["git", "-C", str(agent_dir), "rev-parse", "--show-toplevel"],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=2,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if worktree_result.returncode != 0:
|
||||
return None
|
||||
worktree = Path(worktree_result.stdout.strip()).resolve()
|
||||
relative_module = module_path.relative_to(worktree).as_posix()
|
||||
tracked_result = subprocess.run(
|
||||
[
|
||||
"git",
|
||||
"--literal-pathspecs",
|
||||
"-C",
|
||||
str(worktree),
|
||||
"ls-files",
|
||||
"--error-unmatch",
|
||||
"--",
|
||||
relative_module,
|
||||
],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=2,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if tracked_result.returncode != 0:
|
||||
return None
|
||||
revision_result = subprocess.run(
|
||||
["git", "-C", str(worktree), "rev-parse", "--verify", "HEAD"],
|
||||
check=False,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
timeout=2,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired, RuntimeError, ValueError):
|
||||
return None
|
||||
|
||||
revision = revision_result.stdout.strip()
|
||||
return revision if revision_result.returncode == 0 and revision else None
|
||||
|
||||
|
||||
_AGENT_SOURCE_DIR: Path | None = None
|
||||
_AGENT_MODULE_PATH: Path | None = None
|
||||
_AGENT_REVISION: str | None = None
|
||||
_AIAgent = None
|
||||
_RUNTIME_LOCK = threading.Lock()
|
||||
|
||||
|
||||
class AgentRuntimeChangedError(RuntimeError):
|
||||
"""Raised when the loaded Agent runtime no longer matches its source tree."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
message: str,
|
||||
*,
|
||||
agent_update_state: str | None = None,
|
||||
) -> None:
|
||||
super().__init__(message)
|
||||
self.agent_update_state = agent_update_state
|
||||
|
||||
|
||||
def agent_runtime_stale_payload(exc: AgentRuntimeChangedError) -> dict:
|
||||
"""Return the shared retry response for every stale-runtime entry point."""
|
||||
payload = {
|
||||
"error": str(exc),
|
||||
"type": "agent_runtime_stale",
|
||||
"retryable": True,
|
||||
"restart_scheduled": False,
|
||||
}
|
||||
if exc.agent_update_state is not None:
|
||||
payload["agent_update_state"] = exc.agent_update_state
|
||||
return payload
|
||||
|
||||
|
||||
def _pid_is_alive(pid: int) -> bool | None:
|
||||
"""Return PID liveness, or ``None`` when the platform cannot confirm it."""
|
||||
if pid <= 0:
|
||||
return False
|
||||
if pid.bit_length() > 32:
|
||||
return None
|
||||
if sys.platform == "win32":
|
||||
try:
|
||||
import ctypes
|
||||
from ctypes import wintypes
|
||||
|
||||
kernel32 = ctypes.WinDLL("kernel32", use_last_error=True)
|
||||
kernel32.OpenProcess.argtypes = (
|
||||
wintypes.DWORD,
|
||||
wintypes.BOOL,
|
||||
wintypes.DWORD,
|
||||
)
|
||||
kernel32.OpenProcess.restype = wintypes.HANDLE
|
||||
kernel32.GetExitCodeProcess.argtypes = (
|
||||
wintypes.HANDLE,
|
||||
ctypes.POINTER(wintypes.DWORD),
|
||||
)
|
||||
kernel32.GetExitCodeProcess.restype = wintypes.BOOL
|
||||
kernel32.CloseHandle.argtypes = (wintypes.HANDLE,)
|
||||
kernel32.CloseHandle.restype = wintypes.BOOL
|
||||
handle = kernel32.OpenProcess(0x1000, False, pid)
|
||||
if not handle:
|
||||
error = ctypes.get_last_error()
|
||||
if error == 5: # ERROR_ACCESS_DENIED still proves the PID exists.
|
||||
return True
|
||||
if error == 87: # ERROR_INVALID_PARAMETER for a missing PID.
|
||||
return False
|
||||
return None
|
||||
try:
|
||||
exit_code = wintypes.DWORD()
|
||||
if not kernel32.GetExitCodeProcess(handle, ctypes.byref(exit_code)):
|
||||
return None
|
||||
return exit_code.value == 259 # STILL_ACTIVE
|
||||
finally:
|
||||
kernel32.CloseHandle(handle)
|
||||
except (AttributeError, OSError, TypeError, ValueError):
|
||||
return None
|
||||
|
||||
try:
|
||||
os.kill(pid, 0)
|
||||
except ProcessLookupError:
|
||||
return False
|
||||
except PermissionError:
|
||||
return True
|
||||
except (OverflowError, ValueError):
|
||||
return None
|
||||
except OSError as exc:
|
||||
if exc.errno == errno.ESRCH:
|
||||
return False
|
||||
if exc.errno == errno.EPERM:
|
||||
return True
|
||||
return None
|
||||
return True
|
||||
|
||||
|
||||
def _read_live_agent_update(marker: Path) -> str:
|
||||
"""Classify the shared Agent update marker without changing Agent state.
|
||||
|
||||
The marker is attacker-adjacent shared state (any process that can write the
|
||||
Agent home can create it), so the read is hardened: never follow a symlink,
|
||||
never block on a FIFO/device, and never read an unbounded regular file.
|
||||
Anything that is not a small regular file is classified ``unknown`` rather
|
||||
than allowed to hang or exhaust memory on a stale-runtime request path.
|
||||
"""
|
||||
if not _MARKER_SAFE_OPEN_AVAILABLE:
|
||||
# Without both O_NONBLOCK and O_NOFOLLOW we cannot prove the read is
|
||||
# non-blocking and symlink-safe (e.g. native Windows), so never open the
|
||||
# marker: an unverifiable marker fails closed to ``unknown``, and only a
|
||||
# genuinely missing path is ``absent``.
|
||||
try:
|
||||
marker.lstat()
|
||||
except FileNotFoundError:
|
||||
return "absent"
|
||||
except (OSError, ValueError, TypeError):
|
||||
return "unknown"
|
||||
return "unknown"
|
||||
try:
|
||||
fd = os.open(marker, os.O_RDONLY | _O_NONBLOCK | _O_NOFOLLOW)
|
||||
except FileNotFoundError:
|
||||
try:
|
||||
marker.lstat()
|
||||
except FileNotFoundError:
|
||||
return "absent"
|
||||
except OSError:
|
||||
return "unknown"
|
||||
# Path exists to lstat (e.g. a dangling/looping symlink) but O_NOFOLLOW
|
||||
# refused to open it — treat as an unverifiable marker.
|
||||
return "unknown"
|
||||
except (OSError, ValueError, TypeError):
|
||||
# ELOOP (symlink under O_NOFOLLOW), ENXIO/EWOULDBLOCK (FIFO with no
|
||||
# writer under O_NONBLOCK), a non-path marker object, or any other open
|
||||
# failure — fail closed: an unreadable marker is never proof of safety.
|
||||
return "unknown"
|
||||
|
||||
try:
|
||||
try:
|
||||
st = os.fstat(fd)
|
||||
except OSError:
|
||||
return "unknown"
|
||||
if not stat.S_ISREG(st.st_mode):
|
||||
# FIFO, device, directory, socket — never a legitimate marker.
|
||||
return "unknown"
|
||||
if st.st_size > _AGENT_UPDATE_MARKER_MAX_BYTES:
|
||||
return "unknown"
|
||||
try:
|
||||
# Read one byte past the cap so an oversized file that lied about
|
||||
# st_size (or grew mid-read) is still rejected rather than truncated.
|
||||
data = os.read(fd, _AGENT_UPDATE_MARKER_MAX_BYTES + 1)
|
||||
except (OSError, BlockingIOError):
|
||||
return "unknown"
|
||||
finally:
|
||||
try:
|
||||
os.close(fd)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
if len(data) > _AGENT_UPDATE_MARKER_MAX_BYTES:
|
||||
return "unknown"
|
||||
try:
|
||||
raw = data.decode("utf-8")
|
||||
except UnicodeError:
|
||||
return "unknown"
|
||||
|
||||
lines = raw.splitlines()
|
||||
try:
|
||||
pid = int(lines[0].strip())
|
||||
started_at = float(lines[1].strip())
|
||||
except (IndexError, TypeError, ValueError):
|
||||
return "unknown"
|
||||
if pid <= 0 or not math.isfinite(started_at):
|
||||
return "unknown"
|
||||
|
||||
age_seconds = time.time() - started_at
|
||||
if age_seconds < 0:
|
||||
return "unknown"
|
||||
if age_seconds > _AGENT_UPDATE_MAX_AGE_SECONDS:
|
||||
return "stale"
|
||||
alive = _pid_is_alive(pid)
|
||||
if alive is None:
|
||||
return "unknown"
|
||||
return "active" if alive else "stale"
|
||||
|
||||
|
||||
def _marker_presence(marker: Path) -> str:
|
||||
"""Return ``present``, ``absent``, or ``unknown`` for a recovery marker."""
|
||||
try:
|
||||
marker.lstat()
|
||||
except FileNotFoundError:
|
||||
return "absent"
|
||||
except OSError:
|
||||
return "unknown"
|
||||
return "present"
|
||||
|
||||
|
||||
def _agent_install_roots() -> tuple[Path, ...]:
|
||||
"""Return portable roots that can own the Agent's venv recovery markers."""
|
||||
candidates: list[Path] = []
|
||||
if _AGENT_SOURCE_DIR is not None:
|
||||
candidates.append(_AGENT_SOURCE_DIR)
|
||||
if _AGENT_PYTHON is not None:
|
||||
# A venv Python is commonly a symlink to a shared interpreter. Keep the
|
||||
# configured venv path so its installation's recovery markers are read.
|
||||
python_path = _AGENT_PYTHON
|
||||
if python_path.parent.name.lower() in {"bin", "scripts"}:
|
||||
venv_dir = python_path.parent.parent
|
||||
if venv_dir.name.lower() in {"venv", ".venv"}:
|
||||
candidates.append(venv_dir.parent)
|
||||
|
||||
roots: list[Path] = []
|
||||
seen: set[str] = set()
|
||||
for candidate in candidates:
|
||||
key = os.path.normcase(os.path.abspath(str(candidate)))
|
||||
if key not in seen:
|
||||
seen.add(key)
|
||||
roots.append(candidate)
|
||||
return tuple(roots)
|
||||
|
||||
|
||||
def _agent_update_transaction_state() -> str:
|
||||
"""Report marker diagnostics, never proof of successful completion.
|
||||
|
||||
The Agent removes its active marker on failed/interrupted exits too. Neither
|
||||
its absence nor a stale PID proves the checkout or environment is healthy.
|
||||
"""
|
||||
live_state = _read_live_agent_update(_HERMES_HOME / _AGENT_UPDATE_MARKER)
|
||||
if live_state == "unknown":
|
||||
return "unknown"
|
||||
|
||||
recovery_present = False
|
||||
for root in _agent_install_roots():
|
||||
for marker_name in _AGENT_RECOVERY_MARKERS:
|
||||
presence = _marker_presence(root / marker_name)
|
||||
if presence == "unknown":
|
||||
return "unknown"
|
||||
recovery_present = recovery_present or presence == "present"
|
||||
if recovery_present:
|
||||
return "incomplete"
|
||||
return "unverified" if live_state == "absent" else live_state
|
||||
|
||||
|
||||
def _loaded_agent_source_identity() -> tuple[Path, Path] | None:
|
||||
"""Return the source directory and file that supplied ``run_agent``."""
|
||||
module = sys.modules.get("run_agent")
|
||||
module_file = getattr(module, "__file__", None)
|
||||
if not module_file:
|
||||
return None
|
||||
try:
|
||||
module_path = Path(module_file).resolve()
|
||||
return module_path.parent, module_path
|
||||
except (OSError, RuntimeError, TypeError):
|
||||
return None
|
||||
|
||||
|
||||
def _capture_loaded_agent_revision() -> None:
|
||||
"""Bind the guard to the checkout that supplied the loaded Agent module."""
|
||||
global _AGENT_SOURCE_DIR, _AGENT_MODULE_PATH, _AGENT_REVISION
|
||||
|
||||
if _AGENT_REVISION is not None:
|
||||
ensure_agent_runtime_current()
|
||||
return
|
||||
|
||||
identity = _loaded_agent_source_identity()
|
||||
if identity is None:
|
||||
return
|
||||
source_dir, module_path = identity
|
||||
current_revision = _read_agent_revision(source_dir, module_path=module_path)
|
||||
_AGENT_SOURCE_DIR = source_dir
|
||||
_AGENT_MODULE_PATH = module_path
|
||||
_AGENT_REVISION = current_revision
|
||||
|
||||
|
||||
def ensure_agent_runtime_current() -> None:
|
||||
"""Reject a known Git checkout change instead of mixing Python modules."""
|
||||
if _AGENT_REVISION is None:
|
||||
return
|
||||
if (
|
||||
_read_agent_revision(_AGENT_SOURCE_DIR, module_path=_AGENT_MODULE_PATH)
|
||||
!= _AGENT_REVISION
|
||||
):
|
||||
# Automatic restart needs an Agent-owned success receipt bound to this
|
||||
# transaction, final revision and healthy environment, plus an atomic
|
||||
# handoff excluding mutations across replacement. Marker polling and a
|
||||
# final revision read supply neither contract. Keep this path manual.
|
||||
raise AgentRuntimeChangedError(
|
||||
_RESTART_REQUIRED_MESSAGE,
|
||||
agent_update_state=_agent_update_transaction_state(),
|
||||
)
|
||||
|
||||
|
||||
def require_ai_agent_class():
|
||||
"""Import ``AIAgent`` after proving the loaded source revision is current."""
|
||||
ensure_agent_runtime_current()
|
||||
from run_agent import AIAgent # noqa: PLC0415
|
||||
|
||||
_capture_loaded_agent_revision()
|
||||
return AIAgent
|
||||
|
||||
|
||||
def get_ai_agent_class():
|
||||
"""Return ``AIAgent`` while preserving the existing lazy-import retry."""
|
||||
global _AIAgent, _AGENT_REVISION
|
||||
|
||||
with _RUNTIME_LOCK:
|
||||
ensure_agent_runtime_current()
|
||||
if _AIAgent is None:
|
||||
try:
|
||||
agent_class = require_ai_agent_class()
|
||||
except ImportError:
|
||||
return None
|
||||
_AIAgent = agent_class
|
||||
return _AIAgent
|
||||
+28
-193
@@ -6,49 +6,6 @@ from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# state.db paths that already produced the "no 'source' column" warning below.
|
||||
# ``get_cli_sessions(all_profiles=True)`` re-reads every profile DB on every
|
||||
# sidebar poll (behind a 5 s cache), so a single pre-``source`` profile DB would
|
||||
# otherwise re-emit the identical WARNING line every ~15 s for the life of the
|
||||
# process. The condition is a property of the DB file, not of the poll, so it
|
||||
# is reported once per path. Process-lifetime only: a restart warns again,
|
||||
# which is the desired behaviour (the log line is the operator's cue that the
|
||||
# agent still needs upgrading). Plain ``set`` mutation under the GIL is
|
||||
# sufficient here; a duplicate line from two racing first calls is harmless.
|
||||
_SOURCE_COLUMN_WARNED_DB_PATHS: set[str] = set()
|
||||
|
||||
|
||||
def open_state_db_readonly(db_path: Path, log: logging.Logger | None = None) -> sqlite3.Connection:
|
||||
"""Open the live agent ``state.db`` read-only for a pure-read projection.
|
||||
|
||||
Same rationale as the session-listing path (#5455): a write-capable handle
|
||||
on the multi-GB, WAL ``state.db`` while the agent streams into it adds
|
||||
needless checkpoint/lock surface. The read-only ``file:...?mode=ro`` URI
|
||||
avoids that. Falls back to a writable connection (and warns) if the
|
||||
read-only open fails, so callers never lose data on exotic filesystems.
|
||||
|
||||
The caller must ensure ``db_path`` exists — this raises ``FileNotFoundError``
|
||||
for a missing path rather than letting the writable fallback below create an
|
||||
empty, writable ``state.db`` there (a ghost DB in the agent's HOME). The
|
||||
fallback is only for an *existing* DB whose read-only open fails on an exotic
|
||||
filesystem, so a real read never loses data.
|
||||
|
||||
Callers own the returned connection (wrap it in ``contextlib.closing``).
|
||||
"""
|
||||
log = log or logger
|
||||
if not db_path.exists():
|
||||
raise FileNotFoundError(f"agent state.db not found: {db_path}")
|
||||
read_only_uri = f"{db_path.resolve().as_uri()}?mode=ro"
|
||||
try:
|
||||
return sqlite3.connect(read_only_uri, uri=True)
|
||||
except sqlite3.Error as exc:
|
||||
log.warning(
|
||||
"agent state.db read-only open failed for %s; falling back to writable connection: %s",
|
||||
db_path,
|
||||
exc,
|
||||
)
|
||||
return sqlite3.connect(str(db_path))
|
||||
|
||||
|
||||
MESSAGING_SOURCES = {
|
||||
'discord',
|
||||
@@ -58,32 +15,25 @@ MESSAGING_SOURCES = {
|
||||
'slack',
|
||||
'telegram',
|
||||
'weixin',
|
||||
'matrix',
|
||||
'signal',
|
||||
}
|
||||
|
||||
CLI_MIN_UNTITLED_MESSAGE_COUNT = 6
|
||||
CLI_MIN_UNTITLED_USER_MESSAGE_COUNT = 2
|
||||
|
||||
SOURCE_LABELS = {
|
||||
'acp': 'ACP',
|
||||
'api_server': 'API',
|
||||
'cli': 'CLI',
|
||||
'cron': 'Cron',
|
||||
'discord': 'Discord',
|
||||
'email': 'Email',
|
||||
'kanban': 'Kanban',
|
||||
'wecom': 'WeCom',
|
||||
'wecom_callback': 'WeCom Callback',
|
||||
'slack': 'Slack',
|
||||
'telegram': 'Telegram',
|
||||
'tool': 'Tool',
|
||||
'tui': 'TUI',
|
||||
'webhook': 'Webhook',
|
||||
'webui': 'WebUI',
|
||||
'weixin': 'Weixin',
|
||||
'matrix': 'Matrix',
|
||||
'signal': 'Signal',
|
||||
}
|
||||
|
||||
|
||||
@@ -98,21 +48,12 @@ def normalize_agent_session_source(raw_source: str | None) -> dict:
|
||||
|
||||
if raw == 'webui':
|
||||
session_source = 'webui'
|
||||
elif raw in {'acp', 'cli', 'tui'}:
|
||||
# 'acp' (Agent Client Protocol adapter — Zed, external device bridges)
|
||||
# is a local interactive agent client like the CLI/TUI: its sessions
|
||||
# live only in state.db, so classifying it 'other' would leave them
|
||||
# invisible in both sidebar buckets (webui skips the state.db
|
||||
# projection; cli keeps only CLI-classified rows).
|
||||
elif raw in {'cli', 'tui'}:
|
||||
session_source = 'cli'
|
||||
elif raw in MESSAGING_SOURCES:
|
||||
session_source = 'messaging'
|
||||
elif raw == 'cron':
|
||||
session_source = 'cron'
|
||||
elif raw == 'webhook':
|
||||
session_source = 'webhook'
|
||||
elif raw == 'kanban':
|
||||
session_source = 'kanban'
|
||||
elif raw == 'tool':
|
||||
session_source = 'tool'
|
||||
elif raw == 'api_server':
|
||||
@@ -232,33 +173,18 @@ def is_cli_session_row(row: dict) -> bool:
|
||||
source_label = _safe_lower(row.get("source_label"))
|
||||
if "webui" in {source, source_tag, raw_source, source_name, source_label}:
|
||||
return False
|
||||
# 'subagent' is a delegated delegate_task child: view-only, owned by the
|
||||
# runner, never a writable WebUI/CLI session (#5307). Classify it non-CLI so
|
||||
# sidebar rows and every is_cli_session_row() consumer keep it out of the
|
||||
# CLI/writable treatment.
|
||||
non_cli_sources = MESSAGING_SOURCES | {"cron", "webhook", "kanban", "tool", "api", "api_server", "subagent"}
|
||||
non_cli_sources = MESSAGING_SOURCES | {"cron", "tool", "api", "api_server"}
|
||||
if {source, source_tag, raw_source, source_name, source_label} & non_cli_sources:
|
||||
return False
|
||||
if source == "messaging":
|
||||
return False
|
||||
if source == "cli":
|
||||
return True
|
||||
# External-agent imports (Claude Code, Codex, etc.) are read-only sessions
|
||||
# that Hermes discovers on disk and lists alongside CLI/TUI sessions. The
|
||||
# client renderer (static/sessions.js: _isCliSession) files them in the CLI
|
||||
# bucket via the is_cli_session fallthrough, so the server session-count
|
||||
# classifier MUST agree — otherwise the server counts them under
|
||||
# webui_session_count while the client renders them under CLI, and the WebUI
|
||||
# filter shows a non-zero count with an empty list (#5831). These carry a
|
||||
# real title, so they'd otherwise fall through to the conservative
|
||||
# default-title gate below and be misclassified as non-CLI.
|
||||
if source in {"external_agent", "external-agent"}:
|
||||
return True
|
||||
if (
|
||||
source_tag in {"acp", "cli", "tui"}
|
||||
or raw_source in {"acp", "cli", "tui"}
|
||||
or source_name in {"acp", "cli", "tui"}
|
||||
or source_label in {"acp", "cli", "tui"}
|
||||
source_tag in {"cli", "tui"}
|
||||
or raw_source in {"cli", "tui"}
|
||||
or source_name in {"cli", "tui"}
|
||||
or source_label in {"cli", "tui"}
|
||||
):
|
||||
return True
|
||||
|
||||
@@ -281,33 +207,17 @@ def is_cli_session_row_visible(row: dict) -> bool:
|
||||
if not is_cli_session_row(row):
|
||||
return True
|
||||
|
||||
actual_message_count = _as_positive_int(row.get("actual_message_count"))
|
||||
message_count = actual_message_count or _as_positive_int(row.get("message_count"))
|
||||
message_count = _as_positive_int(row.get("actual_message_count") or row.get("message_count"))
|
||||
if message_count <= 0:
|
||||
return False
|
||||
|
||||
if (
|
||||
actual_message_count > 0
|
||||
and _count_user_turns(row) > 0
|
||||
and row.get("ended_at") is None
|
||||
and not row.get("end_reason")
|
||||
):
|
||||
return True
|
||||
|
||||
interactive_sources = {
|
||||
if "tui" in {
|
||||
_normalize_source_name(row.get("source")),
|
||||
_normalize_source_name(row.get("source_tag")),
|
||||
_normalize_source_name(row.get("raw_source")),
|
||||
_normalize_source_name(row.get("source_label")),
|
||||
}
|
||||
if "tui" in interactive_sources:
|
||||
}:
|
||||
return True
|
||||
if "acp" in interactive_sources:
|
||||
# Like TUI rows, user-driven ACP sessions stay visible even when
|
||||
# ended/untitled. Unlike TUI, an ACP connection can record only
|
||||
# assistant/tool/system rows (e.g. a replayed or aborted turn), so
|
||||
# require at least one user turn before surfacing the row.
|
||||
return _count_user_turns(row) > 0
|
||||
|
||||
if _has_cli_lineage(row):
|
||||
return True
|
||||
@@ -530,41 +440,13 @@ def read_importable_agent_session_rows(
|
||||
``exclude_sources=None``. ``include_sources`` is an additional narrowing
|
||||
filter; callers that want an include-only query should explicitly pass
|
||||
``exclude_sources=None`` so the default exclusions do not also apply.
|
||||
|
||||
``limit`` bounds the *recency slice*, not the returned row count. Subagent
|
||||
rows only render as children when their parent row is in the same payload,
|
||||
so subagent ancestors of selected rows are re-added afterwards and the
|
||||
result can exceed ``limit`` by the number of such anchors. Callers must
|
||||
therefore iterate the result rather than assume ``len(rows) <= limit``.
|
||||
|
||||
That recovery is deliberately bounded by the oversampled candidate set
|
||||
(``limit * 8`` newest sessions): it re-uses rows the projection already
|
||||
fetched and never issues an extra query, so an ancestor older than the
|
||||
oversample stays unresolved and its children render top-level, exactly as
|
||||
they did before. Widening that window is a ``candidate_limit`` change, not
|
||||
a change to this walk.
|
||||
"""
|
||||
db_path = Path(db_path)
|
||||
if not db_path.exists():
|
||||
return []
|
||||
|
||||
log = log or logger
|
||||
# Open read-only for this projection/listing path: it is a pure read, and
|
||||
# holding a write-capable handle on the live (multi-GB, WAL) state.db while
|
||||
# the agent streams into it adds needless checkpoint/lock surface (#5455).
|
||||
# The defensive index self-heal below still runs, but through a separate
|
||||
# short-lived writable connection on the rare missing-index path only.
|
||||
read_only_uri = f"{db_path.resolve().as_uri()}?mode=ro"
|
||||
try:
|
||||
conn = sqlite3.connect(read_only_uri, uri=True)
|
||||
except sqlite3.Error as exc:
|
||||
log.warning(
|
||||
"agent session listing read-only open failed for %s; falling back to writable connection: %s",
|
||||
db_path,
|
||||
exc,
|
||||
)
|
||||
conn = sqlite3.connect(str(db_path))
|
||||
with closing(conn):
|
||||
with closing(sqlite3.connect(str(db_path))) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
cur = conn.cursor()
|
||||
|
||||
@@ -575,15 +457,12 @@ def read_importable_agent_session_rows(
|
||||
cur.execute("PRAGMA table_info(messages)")
|
||||
message_cols = {row[1] for row in cur.fetchall()}
|
||||
if 'source' not in session_cols:
|
||||
warned_key = str(db_path.resolve())
|
||||
if warned_key not in _SOURCE_COLUMN_WARNED_DB_PATHS:
|
||||
_SOURCE_COLUMN_WARNED_DB_PATHS.add(warned_key)
|
||||
log.warning(
|
||||
"agent session listing skipped: state.db at %s has no 'source' column "
|
||||
"(older hermes-agent?). Agent sessions unavailable. "
|
||||
"Upgrade hermes-agent to fix this.",
|
||||
db_path,
|
||||
)
|
||||
log.warning(
|
||||
"agent session listing skipped: state.db at %s has no 'source' column "
|
||||
"(older hermes-agent?). Agent sessions unavailable. "
|
||||
"Upgrade hermes-agent to fix this.",
|
||||
db_path,
|
||||
)
|
||||
return []
|
||||
|
||||
parent_expr = _optional_col('parent_session_id', session_cols)
|
||||
@@ -625,21 +504,16 @@ def read_importable_agent_session_rows(
|
||||
messages_index_present = any(str(row[1]) == "idx_messages_session" for row in cur.fetchall())
|
||||
except sqlite3.Error:
|
||||
messages_index_present = False
|
||||
if not messages_index_present:
|
||||
# Self-heal via a separate writable connection so the common
|
||||
# (index-present) path keeps its read-only handle. On a truly
|
||||
# read-only/locked db this fails and we degrade to the
|
||||
# pre-aggregated cron-only path below, exactly as before.
|
||||
try:
|
||||
with closing(sqlite3.connect(str(db_path))) as _heal:
|
||||
_heal.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_messages_session "
|
||||
"ON messages(session_id, timestamp)"
|
||||
)
|
||||
_heal.commit()
|
||||
try:
|
||||
if not messages_index_present:
|
||||
cur.execute(
|
||||
"CREATE INDEX IF NOT EXISTS idx_messages_session "
|
||||
"ON messages(session_id, timestamp)"
|
||||
)
|
||||
conn.commit()
|
||||
messages_index_present = True
|
||||
except sqlite3.Error:
|
||||
pass # read-only db / locked / older schema — degrade gracefully
|
||||
except sqlite3.Error:
|
||||
pass # read-only db / locked / older schema — degrade gracefully
|
||||
|
||||
if use_messages_join:
|
||||
actual_count_expr = f"COUNT(m.{count_col})"
|
||||
@@ -795,43 +669,7 @@ def read_importable_agent_session_rows(
|
||||
projected = [row for row in projected if is_cli_session_row_visible(row)]
|
||||
if limit is None:
|
||||
return projected
|
||||
selected = projected[:max(0, int(limit))]
|
||||
|
||||
# The recency slice is per-row, but subagent rows are only renderable as
|
||||
# children: the sidebar nests a child under its parent solely when that
|
||||
# parent row is present in the same payload. A frozen orchestrator stops
|
||||
# writing while its leaves keep streaming, so the leaves win the recency
|
||||
# race and the parent falls outside the window — leaving the leaves to be
|
||||
# promoted to top-level sidebar rows. Re-add subagent parents that the
|
||||
# oversampled candidate set already projected (no extra query); webui
|
||||
# ancestors are left out because that sidebar bucket already has them.
|
||||
#
|
||||
# Bounded by construction: ``by_id`` only holds the ``limit * 8``
|
||||
# newest candidates, so an ancestor older than that oversample is not
|
||||
# recovered and its children stay top-level — the pre-existing
|
||||
# behaviour, narrowed rather than fixed. Resolving those would need an
|
||||
# unbounded per-row ancestor query on the hot sidebar path; widen
|
||||
# ``candidate_limit`` instead if the window proves too tight.
|
||||
# NOTE: this can return more than ``limit`` rows (see docstring).
|
||||
have = {row.get('id') for row in selected}
|
||||
by_id = {row.get('id'): row for row in projected if row.get('id')}
|
||||
pending = list(selected)
|
||||
while pending:
|
||||
row = pending.pop()
|
||||
if str(row.get('raw_source') or row.get('source') or '').strip().lower() != 'subagent':
|
||||
continue
|
||||
parent_id = row.get('parent_session_id')
|
||||
if not parent_id or parent_id in have:
|
||||
continue
|
||||
parent = by_id.get(parent_id)
|
||||
if parent is None:
|
||||
continue
|
||||
if str(parent.get('raw_source') or parent.get('source') or '').strip().lower() != 'subagent':
|
||||
continue
|
||||
selected.append(parent)
|
||||
have.add(parent_id)
|
||||
pending.append(parent)
|
||||
return selected
|
||||
return projected[:max(0, int(limit))]
|
||||
|
||||
|
||||
|
||||
@@ -882,7 +720,7 @@ def read_session_lineage_report(db_path: Path, session_id: str | None, max_hops:
|
||||
return _empty_lineage_report(sid)
|
||||
|
||||
try:
|
||||
with closing(open_state_db_readonly(db_path)) as conn:
|
||||
with closing(sqlite3.connect(str(db_path))) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
cur = conn.cursor()
|
||||
cur.execute("PRAGMA table_info(sessions)")
|
||||
@@ -1021,7 +859,7 @@ def read_session_lineage_metadata(db_path: Path, session_ids: list[str] | set[st
|
||||
return {}
|
||||
|
||||
try:
|
||||
with closing(open_state_db_readonly(db_path)) as conn:
|
||||
with closing(sqlite3.connect(str(db_path))) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
cur = conn.cursor()
|
||||
cur.execute("PRAGMA table_info(sessions)")
|
||||
@@ -1245,9 +1083,6 @@ def read_session_lineage_metadata(db_path: Path, session_ids: list[str] | set[st
|
||||
parent_root = _continuation_root_id(rows, parent_id)
|
||||
if parent_root:
|
||||
entry['_parent_lineage_root_id'] = parent_root
|
||||
if parent_root not in lineage_tip_cache:
|
||||
lineage_tip_cache[parent_root] = freshest_continuation_tip(parent_root)
|
||||
entry['_parent_lineage_tip_id'] = lineage_tip_cache[parent_root][0]
|
||||
continue
|
||||
|
||||
root_id, segment_count = continuation_root_and_depth(sid)
|
||||
|
||||
+33
-545
@@ -1,7 +1,7 @@
|
||||
"""
|
||||
Hermes Web UI -- optional authentication.
|
||||
Off by default. Enable by setting HERMES_WEBUI_PASSWORD, configuring a
|
||||
password in Settings, registering passkeys, or configuring native OIDC SSO.
|
||||
password in Settings, or registering passkeys and then going passwordless.
|
||||
"""
|
||||
import hashlib
|
||||
import hmac
|
||||
@@ -16,7 +16,7 @@ import threading
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
from api.config import STATE_DIR, get_config, load_settings
|
||||
from api.config import STATE_DIR, load_settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -51,9 +51,7 @@ def _resolve_session_ttl() -> int:
|
||||
PUBLIC_PATHS = frozenset({
|
||||
'/login', '/health', '/favicon.ico', '/sw.js',
|
||||
'/api/auth/login', '/api/auth/status',
|
||||
'/api/auth/oidc/start', '/api/auth/oidc/callback',
|
||||
'/api/auth/passkey/options', '/api/auth/passkey/login',
|
||||
'/share',
|
||||
'/manifest.json', '/manifest.webmanifest',
|
||||
'/session/manifest.json', '/session/manifest.webmanifest',
|
||||
})
|
||||
@@ -101,39 +99,9 @@ def _warn_auth_persistence_failure(prefix: str, artifact: Path, exc: Exception,
|
||||
|
||||
|
||||
_SESSIONS_FILE = STATE_DIR / '.sessions.json'
|
||||
_TRUSTED_AUTH_HEADER_ENV = 'HERMES_WEBUI_TRUSTED_AUTH_HEADER'
|
||||
_TRUSTED_GROUPS_HEADER_ENV = 'HERMES_WEBUI_TRUSTED_GROUPS_HEADER'
|
||||
_TRUSTED_GROUP_PROFILE_MAP_ENV = 'HERMES_WEBUI_GROUP_PROFILE_MAP'
|
||||
_TRUSTED_AUTH_LOGOUT_URL_ENV = 'HERMES_WEBUI_TRUSTED_AUTH_LOGOUT_URL'
|
||||
# Opt-in: also treat '|' as a group separator in the trusted-groups header.
|
||||
# Off by default so an existing deployment whose group NAME legitimately
|
||||
# contains a literal '|' is never silently re-split into two groups (which
|
||||
# could change its profile binding). Set to 1/true/yes/on for identity
|
||||
# providers (some Authentik outpost configs) that emit "admins|developpeur".
|
||||
_TRUSTED_GROUPS_PIPE_SEPARATOR_ENV = 'HERMES_WEBUI_TRUSTED_GROUPS_PIPE_SEPARATOR'
|
||||
_TRUSTED_AUTH_WARNINGS_EMITTED: set[str] = set()
|
||||
|
||||
|
||||
def _warn_trusted_auth_once(key: str, message: str, *args) -> None:
|
||||
if key in _TRUSTED_AUTH_WARNINGS_EMITTED:
|
||||
return
|
||||
_TRUSTED_AUTH_WARNINGS_EMITTED.add(key)
|
||||
logger.warning(message, *args)
|
||||
|
||||
|
||||
def _session_expiry(record) -> float | None:
|
||||
if isinstance(record, dict):
|
||||
expiry = record.get('expiry', record.get('expires_at'))
|
||||
else:
|
||||
expiry = record
|
||||
try:
|
||||
expiry_f = float(expiry)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return expiry_f
|
||||
|
||||
|
||||
def _load_sessions() -> dict[str, float | dict]:
|
||||
def _load_sessions() -> dict[str, float]:
|
||||
"""Load persisted sessions from STATE_DIR, pruning expired entries.
|
||||
|
||||
Returns an empty dict on any read or parse error so startup is never
|
||||
@@ -171,23 +139,11 @@ def _load_sessions() -> dict[str, float | dict]:
|
||||
)
|
||||
return {}
|
||||
now = time.time()
|
||||
sessions: dict[str, float | dict] = {}
|
||||
for token, record in data.items():
|
||||
if not isinstance(token, str) or not token:
|
||||
continue
|
||||
expiry = _session_expiry(record)
|
||||
if expiry is None or expiry <= now:
|
||||
continue
|
||||
if isinstance(record, dict):
|
||||
normalized = dict(record)
|
||||
normalized['expiry'] = expiry
|
||||
sessions[token] = normalized
|
||||
else:
|
||||
sessions[token] = expiry
|
||||
return sessions
|
||||
return {t: exp for t, exp in data.items()
|
||||
if isinstance(t, str) and isinstance(exp, (int, float)) and exp > now}
|
||||
|
||||
|
||||
def _save_sessions(sessions: dict[str, float | dict]) -> None:
|
||||
def _save_sessions(sessions: dict[str, float]) -> None:
|
||||
"""Atomically persist sessions to STATE_DIR/.sessions.json (0600).
|
||||
|
||||
Uses a temp file + os.replace() so a crash mid-write never leaves a
|
||||
@@ -487,93 +443,9 @@ def are_passkeys_enabled() -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def is_oidc_auth_enabled() -> bool:
|
||||
"""True if native OIDC login is configured for WebUI sessions."""
|
||||
try:
|
||||
from api.auth_oidc import is_oidc_enabled
|
||||
|
||||
return is_oidc_enabled()
|
||||
except Exception as exc:
|
||||
logger.debug("Failed to inspect OIDC availability: %s", exc)
|
||||
return False
|
||||
|
||||
|
||||
def get_oidc_startup_warning() -> str | None:
|
||||
"""Return a startup warning when OIDC auth is only partially configured,
|
||||
or when allow_values uses whitespace that is no longer a separator."""
|
||||
try:
|
||||
cfg = get_config()
|
||||
raw = cfg.get("webui_oidc") if isinstance(cfg, dict) else {}
|
||||
if not isinstance(raw, dict):
|
||||
raw = {}
|
||||
except Exception:
|
||||
logger.debug("Failed to read webui_oidc config", exc_info=True)
|
||||
raw = {}
|
||||
|
||||
def pick(name: str, env_name: str) -> str:
|
||||
env_value = os.getenv(env_name)
|
||||
value = env_value if env_value is not None else raw.get(name)
|
||||
return str(value or "").strip()
|
||||
|
||||
issuer = bool(pick("issuer", "HERMES_WEBUI_OIDC_ISSUER"))
|
||||
client_id = bool(pick("client_id", "HERMES_WEBUI_OIDC_CLIENT_ID"))
|
||||
allow_claim = bool(pick("allow_claim", "HERMES_WEBUI_OIDC_ALLOW_CLAIM"))
|
||||
raw_allow_env = os.getenv("HERMES_WEBUI_OIDC_ALLOW_VALUES")
|
||||
raw_allow = raw_allow_env if raw_allow_env is not None else raw.get("allow_values")
|
||||
normalized_allow_values = []
|
||||
allow_values_warning = None
|
||||
try:
|
||||
from api import auth_oidc
|
||||
|
||||
normalized_allow_values = auth_oidc._normalize_allow_values(raw_allow)
|
||||
allow_values_warning = auth_oidc._ALLOW_VALUES_WHITESPACE_WARNING
|
||||
except Exception:
|
||||
logger.debug("Failed to normalize OIDC allow_values", exc_info=True)
|
||||
allow_values = bool(normalized_allow_values)
|
||||
|
||||
if not any((issuer, client_id, allow_claim, allow_values)):
|
||||
return None
|
||||
|
||||
warnings = []
|
||||
|
||||
if not (issuer and client_id and allow_claim and allow_values):
|
||||
missing = []
|
||||
if not issuer:
|
||||
missing.append("issuer")
|
||||
if not client_id:
|
||||
missing.append("client_id")
|
||||
if not allow_claim:
|
||||
missing.append("allow_claim")
|
||||
if not allow_values:
|
||||
missing.append("allow_values")
|
||||
joined = ", ".join(missing)
|
||||
warnings.append(
|
||||
"Native OIDC login is only partially configured; missing "
|
||||
f"{joined}. The WebUI will not enable OIDC auth until all four fields are set."
|
||||
)
|
||||
|
||||
# Detect whitespace-only allow_values scalar that may contain multiple intended values.
|
||||
# Runs unconditionally so the warning reaches startup even when other auth methods
|
||||
# short-circuit is_auth_enabled() before the OIDC branch is evaluated.
|
||||
if (
|
||||
allow_values_warning is not None
|
||||
and raw_allow is not None
|
||||
and not isinstance(raw_allow, (list, tuple, set))
|
||||
and any(any(ch.isspace() for ch in v) for v in normalized_allow_values)
|
||||
):
|
||||
warnings.append(allow_values_warning)
|
||||
|
||||
return "\n".join(warnings) if warnings else None
|
||||
|
||||
|
||||
def is_auth_enabled() -> bool:
|
||||
"""True if password auth, passkeys, OIDC login, or trusted-header auth is configured."""
|
||||
return (
|
||||
is_password_auth_enabled()
|
||||
or are_passkeys_enabled()
|
||||
or is_oidc_auth_enabled()
|
||||
or is_trusted_auth_enabled()
|
||||
)
|
||||
"""True if password auth or passkey-only auth is configured."""
|
||||
return is_password_auth_enabled() or are_passkeys_enabled()
|
||||
|
||||
|
||||
def verify_password(plain: str) -> bool:
|
||||
@@ -607,22 +479,11 @@ def verify_password(plain: str) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def create_session(*, auth_type: str | None = None, username: str | None = None, bound_profile: str | None = None) -> str:
|
||||
def create_session() -> str:
|
||||
"""Create a new auth session. Returns signed cookie value."""
|
||||
token = secrets.token_hex(32)
|
||||
expiry = time.time() + _resolve_session_ttl()
|
||||
record: float | dict
|
||||
if any(value is not None for value in (auth_type, username, bound_profile)):
|
||||
record = {
|
||||
'expiry': expiry,
|
||||
'auth_type': auth_type,
|
||||
'username': username,
|
||||
'bound_profile': bound_profile,
|
||||
}
|
||||
else:
|
||||
record = expiry
|
||||
with _SESSIONS_LOCK:
|
||||
_sessions[token] = record
|
||||
_sessions[token] = time.time() + _resolve_session_ttl()
|
||||
_save_sessions(_sessions)
|
||||
sig = hmac.new(_signing_key(), token.encode(), hashlib.sha256).hexdigest()
|
||||
return f"{token}.{sig}"
|
||||
@@ -632,7 +493,7 @@ def _prune_expired_sessions():
|
||||
"""Remove all expired session entries to prevent unbounded memory growth."""
|
||||
now = time.time()
|
||||
with _SESSIONS_LOCK:
|
||||
expired = [t for t, record in _sessions.items() if (expiry := _session_expiry(record)) is None or now > expiry]
|
||||
expired = [t for t, exp in _sessions.items() if now > exp]
|
||||
if expired:
|
||||
for token in expired:
|
||||
_sessions.pop(token, None)
|
||||
@@ -655,300 +516,14 @@ def verify_session(cookie_value: str) -> bool:
|
||||
if not valid:
|
||||
return False
|
||||
with _SESSIONS_LOCK:
|
||||
expiry = _session_expiry(_sessions.get(token))
|
||||
if expiry is None or time.time() > expiry:
|
||||
expiry = _sessions.get(token)
|
||||
if not expiry or time.time() > expiry:
|
||||
_sessions.pop(token, None)
|
||||
_save_sessions(_sessions)
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _trusted_auth_header_name() -> str | None:
|
||||
name = os.getenv(_TRUSTED_AUTH_HEADER_ENV, '').strip()
|
||||
if not name:
|
||||
return None
|
||||
if not _COOKIE_NAME_RE.match(name):
|
||||
_warn_trusted_auth_once(
|
||||
'trusted-auth-header',
|
||||
'Ignoring invalid %s=%r; trusted-header auth rejects every request',
|
||||
_TRUSTED_AUTH_HEADER_ENV,
|
||||
name,
|
||||
)
|
||||
return None
|
||||
return name
|
||||
|
||||
|
||||
def _trusted_auth_header_configured() -> bool:
|
||||
return bool(os.getenv(_TRUSTED_AUTH_HEADER_ENV, '').strip())
|
||||
|
||||
|
||||
def _trusted_group_profile_map() -> dict[str, str] | None:
|
||||
raw = os.getenv(_TRUSTED_GROUP_PROFILE_MAP_ENV, '').strip()
|
||||
if not raw:
|
||||
return None
|
||||
try:
|
||||
data = json.loads(raw)
|
||||
except json.JSONDecodeError:
|
||||
_warn_trusted_auth_once(
|
||||
'trusted-group-map',
|
||||
'Ignoring invalid %s JSON; trusted-header auth falls back to default profile binding',
|
||||
_TRUSTED_GROUP_PROFILE_MAP_ENV,
|
||||
)
|
||||
return {}
|
||||
if not isinstance(data, dict):
|
||||
_warn_trusted_auth_once(
|
||||
'trusted-group-map-type',
|
||||
'Ignoring non-dict %s; trusted-header auth falls back to default profile binding',
|
||||
_TRUSTED_GROUP_PROFILE_MAP_ENV,
|
||||
)
|
||||
return {}
|
||||
mapping: dict[str, str] = {}
|
||||
for group, profile in data.items():
|
||||
group_name = str(group or '').strip()
|
||||
profile_name = str(profile or '').strip()
|
||||
if not group_name or not profile_name:
|
||||
_warn_trusted_auth_once(
|
||||
'trusted-group-map-entry',
|
||||
'Ignoring invalid entry in %s; trusted-header auth falls back to default profile binding',
|
||||
_TRUSTED_GROUP_PROFILE_MAP_ENV,
|
||||
)
|
||||
continue
|
||||
mapping[group_name] = profile_name
|
||||
return mapping
|
||||
|
||||
|
||||
def _trusted_groups_header_value(handler) -> list[str]:
|
||||
header_name = os.getenv(_TRUSTED_GROUPS_HEADER_ENV, '').strip()
|
||||
if not header_name:
|
||||
return []
|
||||
try:
|
||||
raw = handler.headers.get(header_name, '')
|
||||
except Exception:
|
||||
return []
|
||||
if not raw:
|
||||
return []
|
||||
values = []
|
||||
# Authentik's outpost typically joins multiple group names with a comma or
|
||||
# newline; parse those as separators by default. Some proxy provider /
|
||||
# property-mapping configs instead emit a pipe-separated list (e.g.
|
||||
# "admins|developpeur"), which a comma-only split would treat as one
|
||||
# unmatched group name — silently dropping the session to the unbound
|
||||
# "default" profile despite a legitimate mapped membership. Pipe splitting
|
||||
# is therefore available but OPT-IN (HERMES_WEBUI_TRUSTED_GROUPS_PIPE_SEPARATOR),
|
||||
# because a group NAME can legitimately contain a literal '|' and must not be
|
||||
# re-split by default — doing so unconditionally could change an existing
|
||||
# deployment's profile binding.
|
||||
normalized = str(raw).replace('\n', ',')
|
||||
if str(os.getenv(_TRUSTED_GROUPS_PIPE_SEPARATOR_ENV, '')).strip().lower() in ('1', 'true', 'yes', 'on'):
|
||||
normalized = normalized.replace('|', ',')
|
||||
for part in normalized.split(','):
|
||||
part = part.strip()
|
||||
if part:
|
||||
values.append(part)
|
||||
return values
|
||||
|
||||
|
||||
def _trusted_auth_username(handler) -> str | None:
|
||||
header_name = _trusted_auth_header_name()
|
||||
if not header_name:
|
||||
return None
|
||||
try:
|
||||
raw = handler.headers.get(header_name, '')
|
||||
except Exception:
|
||||
return None
|
||||
username = str(raw or '').strip()
|
||||
return username or None
|
||||
|
||||
|
||||
def _trusted_auth_bound_profile(handler) -> str | None:
|
||||
mapping = _trusted_group_profile_map()
|
||||
if mapping is None:
|
||||
return None
|
||||
groups = set(_trusted_groups_header_value(handler))
|
||||
for group, profile in mapping.items():
|
||||
if group in groups:
|
||||
return profile
|
||||
return 'default'
|
||||
|
||||
|
||||
def _queue_pending_cookie(handler, cookie_header: str) -> None:
|
||||
if not cookie_header:
|
||||
return
|
||||
pending = getattr(handler, '_pending_set_cookies', None)
|
||||
if pending is None:
|
||||
pending = []
|
||||
handler._pending_set_cookies = pending
|
||||
pending.append(cookie_header)
|
||||
|
||||
|
||||
def _auth_cookie_header(cookie_value, handler=None) -> str:
|
||||
cookie = http.cookies.SimpleCookie()
|
||||
name = _resolve_cookie_name()
|
||||
cookie[name] = cookie_value
|
||||
cookie[name]['httponly'] = True
|
||||
cookie[name]['samesite'] = 'Lax'
|
||||
cookie[name]['path'] = '/'
|
||||
cookie[name]['max-age'] = str(_resolve_session_ttl())
|
||||
if _is_secure_context(handler):
|
||||
cookie[name]['secure'] = True
|
||||
return cookie[name].OutputString()
|
||||
|
||||
|
||||
def _clear_auth_cookie_header() -> str:
|
||||
cookie = http.cookies.SimpleCookie()
|
||||
name = _resolve_cookie_name()
|
||||
cookie[name] = ''
|
||||
cookie[name]['httponly'] = True
|
||||
cookie[name]['path'] = '/'
|
||||
cookie[name]['samesite'] = 'Lax'
|
||||
cookie[name]['max-age'] = '0'
|
||||
return cookie[name].OutputString()
|
||||
|
||||
|
||||
def _build_profile_cookie_header(name: str, session_cookie_value: str | None) -> str:
|
||||
from api.helpers import build_profile_cookie
|
||||
|
||||
return build_profile_cookie(name, session_cookie_value=session_cookie_value)
|
||||
|
||||
|
||||
def _request_profile_matches_bound(bound_profile: str | None) -> bool:
|
||||
if not bound_profile:
|
||||
return True
|
||||
try:
|
||||
from api.profiles import get_active_profile_name, _profiles_match
|
||||
|
||||
return _profiles_match(bound_profile, get_active_profile_name())
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def get_session_info(cookie_value: str) -> dict | None:
|
||||
if not verify_session(cookie_value):
|
||||
return None
|
||||
token = _session_token_from_cookie_value(cookie_value)
|
||||
if not token:
|
||||
return None
|
||||
with _SESSIONS_LOCK:
|
||||
record = _sessions.get(token)
|
||||
expiry = _session_expiry(record)
|
||||
if expiry is None:
|
||||
return None
|
||||
info: dict[str, object] = {'token': token, 'expiry': expiry}
|
||||
if isinstance(record, dict):
|
||||
info.update({k: v for k, v in record.items() if k != 'expiry'})
|
||||
if 'bound_profile' not in info and isinstance(info.get('profile'), str):
|
||||
info['bound_profile'] = info.get('profile')
|
||||
info.setdefault('auth_type', None)
|
||||
info.setdefault('username', None)
|
||||
info.setdefault('bound_profile', None)
|
||||
return info
|
||||
|
||||
|
||||
def session_bound_profile(cookie_value: str) -> str | None:
|
||||
info = get_session_info(cookie_value)
|
||||
if not info:
|
||||
return None
|
||||
bound_profile = info.get('bound_profile')
|
||||
bound_profile = str(bound_profile or '').strip()
|
||||
return bound_profile or None
|
||||
|
||||
|
||||
def is_trusted_auth_enabled() -> bool:
|
||||
return _trusted_auth_header_configured()
|
||||
|
||||
|
||||
def get_trusted_auth_logout_url() -> str | None:
|
||||
value = os.getenv(_TRUSTED_AUTH_LOGOUT_URL_ENV, '').strip()
|
||||
return value or None
|
||||
|
||||
|
||||
def _remember_trusted_auth_session(handler, info: dict | None, cookie_value: str | None = None) -> dict | None:
|
||||
handler._trusted_auth_session_reconciled = info
|
||||
if info and info.get('auth_type') == 'trusted':
|
||||
handler._trusted_auth_session_info = info
|
||||
handler._trusted_auth_session_cookie_value = cookie_value
|
||||
return info
|
||||
|
||||
|
||||
def reset_trusted_auth_request_state(handler) -> None:
|
||||
for name in (
|
||||
'_trusted_auth_session_reconciled',
|
||||
'_trusted_auth_session_rejected',
|
||||
'_trusted_auth_session_info',
|
||||
'_trusted_auth_session_cookie_value',
|
||||
# Clear any auth cookie queued by a prior request but not yet flushed.
|
||||
# The handler is reused across HTTP/1.1 keep-alive requests, so a stale
|
||||
# queued Set-Cookie would otherwise cross the request boundary and be
|
||||
# emitted by a later response — e.g. after trusted-identity rotation on
|
||||
# logout it could overwrite a subsequent valid login cookie and 401 the
|
||||
# user. Reset it at the per-request boundary (server.py do_GET/do_POST).
|
||||
'_pending_set_cookies',
|
||||
):
|
||||
try:
|
||||
delattr(handler, name)
|
||||
except AttributeError:
|
||||
pass
|
||||
|
||||
|
||||
def _apply_trusted_session_profile(handler, bound_profile: str | None, cookie_value: str) -> None:
|
||||
if bound_profile is None:
|
||||
return
|
||||
from api.helpers import get_profile_cookie
|
||||
from api.profiles import set_request_profile
|
||||
|
||||
set_request_profile(bound_profile)
|
||||
if get_profile_cookie(handler) != bound_profile:
|
||||
_queue_pending_cookie(handler, _build_profile_cookie_header(bound_profile, cookie_value))
|
||||
|
||||
|
||||
def ensure_trusted_auth_session(handler) -> dict | None:
|
||||
if hasattr(handler, '_trusted_auth_session_reconciled'):
|
||||
return handler._trusted_auth_session_reconciled
|
||||
cookie_value = parse_cookie(handler)
|
||||
info = get_session_info(cookie_value) if cookie_value and verify_session(cookie_value) else None
|
||||
if info and info.get('auth_type') != 'trusted':
|
||||
return _remember_trusted_auth_session(handler, info)
|
||||
if not is_trusted_auth_enabled():
|
||||
if info:
|
||||
invalidate_session(cookie_value)
|
||||
handler._trusted_auth_session_rejected = True
|
||||
return _remember_trusted_auth_session(handler, None)
|
||||
from api.routes import _raw_peer_is_trusted_proxy
|
||||
|
||||
if not _raw_peer_is_trusted_proxy(handler):
|
||||
if info:
|
||||
invalidate_session(cookie_value)
|
||||
handler._trusted_auth_session_rejected = True
|
||||
return _remember_trusted_auth_session(handler, None)
|
||||
username = _trusted_auth_username(handler)
|
||||
if not username:
|
||||
if info:
|
||||
invalidate_session(cookie_value)
|
||||
handler._trusted_auth_session_rejected = True
|
||||
return _remember_trusted_auth_session(handler, None)
|
||||
bound_profile = _trusted_auth_bound_profile(handler)
|
||||
if info and info.get('username') == username and info.get('bound_profile') == bound_profile:
|
||||
_apply_trusted_session_profile(handler, bound_profile, cookie_value)
|
||||
return _remember_trusted_auth_session(handler, info, cookie_value)
|
||||
if info:
|
||||
invalidate_session(cookie_value)
|
||||
cookie_value = create_session(
|
||||
auth_type='trusted',
|
||||
username=username,
|
||||
bound_profile=bound_profile,
|
||||
)
|
||||
_queue_pending_cookie(handler, _auth_cookie_header(cookie_value, handler))
|
||||
_apply_trusted_session_profile(handler, bound_profile, cookie_value)
|
||||
info = get_session_info(cookie_value)
|
||||
return _remember_trusted_auth_session(handler, info, cookie_value)
|
||||
|
||||
|
||||
def trusted_session_allows_active_profile(info: dict | None) -> bool:
|
||||
if not info:
|
||||
return True
|
||||
return _request_profile_matches_bound(str(info.get('bound_profile') or '') or None)
|
||||
|
||||
|
||||
def _session_token_from_cookie_value(cookie_value: str) -> str | None:
|
||||
"""Return the raw server-side session token from a signed cookie value."""
|
||||
if not cookie_value or '.' not in cookie_value:
|
||||
@@ -1051,90 +626,17 @@ def parse_cookie(handler) -> str | None:
|
||||
return morsel.value if morsel else None
|
||||
|
||||
|
||||
def _safe_login_inner_next(query: str | None) -> str:
|
||||
"""#5578: extract a SAFE, non-login inner redirect from a login page's query.
|
||||
|
||||
When an expired-auth bounce lands back on the login page (which already
|
||||
carries its own `next` in the query), we want to preserve a legitimate inner
|
||||
destination X across the redirect to the real login route — but only if X is
|
||||
itself safe (path-absolute, not protocol-relative/backslash, no control
|
||||
chars) AND not login-shaped / not itself carrying a nested next param.
|
||||
Anything else collapses to '' (no inner redirect), which kills the
|
||||
self-referential chain. Mirrors _safe_login_redirect_path().
|
||||
"""
|
||||
import urllib.parse as _u
|
||||
raw = _u.parse_qs(query or "").get("next", [""])[0]
|
||||
path = str(raw or "").strip()
|
||||
if not path or path[0] != "/" or path[1:2] in {"/", "\\"}:
|
||||
return ""
|
||||
if re.search(r"[\x00-\x1f\x7f\s]", path) or len(path) > 2048:
|
||||
return ""
|
||||
# Collapse only login-route chains — decode a few levels so a nested
|
||||
# `/session/login%3Fnext%3D...` (encoded `?`) is still recognized by its
|
||||
# leading PATH — but preserve a legitimate non-login inner path that merely
|
||||
# carries its own `next=` query key (e.g. `/admin?next=/real/path`).
|
||||
_probe = path
|
||||
for _ in range(8):
|
||||
_p = _probe.split("?", 1)[0].split("#", 1)[0].split("&", 1)[0].rstrip("/")
|
||||
if _p == "/login" or _p.endswith("/login"):
|
||||
return ""
|
||||
_decoded = _u.unquote(_probe)
|
||||
if _decoded == _probe:
|
||||
break
|
||||
_probe = _decoded
|
||||
else:
|
||||
# Still decoding at the cap (pathologically deep encoding) → fail closed.
|
||||
_p = _probe.split("?", 1)[0].split("#", 1)[0].split("&", 1)[0].rstrip("/")
|
||||
if _p == "/login" or _p.endswith("/login"):
|
||||
return ""
|
||||
return ""
|
||||
return path
|
||||
|
||||
|
||||
def check_auth(handler, parsed) -> bool:
|
||||
"""Check if request is authorized. Returns True if OK.
|
||||
If not authorized, sends 401 (API) or 302 redirect (page) and returns False."""
|
||||
if not is_auth_enabled():
|
||||
return True
|
||||
# Public paths don't require auth
|
||||
if (
|
||||
parsed.path in PUBLIC_PATHS
|
||||
or parsed.path.startswith('/share/')
|
||||
or (
|
||||
parsed.path.startswith('/api/share/')
|
||||
and parsed.path not in {'/api/share/create', '/api/share/revoke'}
|
||||
)
|
||||
or parsed.path.startswith('/static/')
|
||||
or parsed.path.startswith('/session/static/')
|
||||
):
|
||||
if parsed.path in PUBLIC_PATHS or parsed.path.startswith('/static/') or parsed.path.startswith('/session/static/'):
|
||||
return True
|
||||
# Check session cookie
|
||||
cookie_val = parse_cookie(handler)
|
||||
has_session = bool(cookie_val and verify_session(cookie_val))
|
||||
if parsed.path == '/api/auth/logout':
|
||||
if has_session:
|
||||
return True
|
||||
body = b'{"error":"Authentication required"}'
|
||||
handler.send_response(401)
|
||||
handler.send_header('Content-Type', 'application/json')
|
||||
handler.send_header('Content-Length', str(len(body)))
|
||||
handler.end_headers()
|
||||
handler.wfile.write(body)
|
||||
return False
|
||||
session_info = ensure_trusted_auth_session(handler)
|
||||
if session_info:
|
||||
if not trusted_session_allows_active_profile(session_info):
|
||||
if parsed.path.startswith('/api/'):
|
||||
body = b'{"error":"Profile access forbidden"}'
|
||||
handler.send_response(403)
|
||||
handler.send_header('Content-Type', 'application/json')
|
||||
else:
|
||||
body = b'Profile access forbidden'
|
||||
handler.send_response(403)
|
||||
handler.send_header('Content-Type', 'text/plain; charset=utf-8')
|
||||
handler.send_header('Content-Length', str(len(body)))
|
||||
handler.end_headers()
|
||||
handler.wfile.write(body)
|
||||
return False
|
||||
if cookie_val and verify_session(cookie_val):
|
||||
return True
|
||||
# Not authorized
|
||||
if parsed.path.startswith('/api/'):
|
||||
@@ -1166,35 +668,6 @@ def check_auth(handler, parsed) -> bool:
|
||||
# the full original URL (the browser auto-decodes once).
|
||||
# (Opus pre-release advisor finding for v0.50.258.)
|
||||
import urllib.parse as _urlparse
|
||||
# #5578: if the page being redirected is ALREADY login-shaped, do NOT
|
||||
# wrap its full `path?query` into a fresh `next=` — that query already
|
||||
# carries a `next=`, so quoting the whole thing nests the login URL into
|
||||
# itself and re-encodes it on every expired-auth bounce, exploding the
|
||||
# URL until the tab breaks. This guard runs in check_auth() (BEFORE
|
||||
# route handling), the actual source of the server-side loop.
|
||||
#
|
||||
# The login page is served ONLY at the public `/login` route (see
|
||||
# PUBLIC_PATHS + the routes.py `/login` handler); the app's client route
|
||||
# `/session/login` is NOT public, so a bare relative `login` from
|
||||
# `/session/login` resolves to `/session/login` again and re-triggers
|
||||
# check_auth() — an infinite redirect. Resolve to the real login route
|
||||
# with `../login`, which lands on `/login` from a `/session/*` scope and
|
||||
# on `<mount>/login` under a subpath mount (verified via urljoin). Carry
|
||||
# through only a validated, non-login inner `next` so a legitimate
|
||||
# post-login destination still survives a bounce that happened to land
|
||||
# on the login page.
|
||||
_login_path = (parsed.path or '/').rstrip('/')
|
||||
if _login_path == '/login' or _login_path.endswith('/login'):
|
||||
# /login itself is public → check_auth never redirects it; this only
|
||||
# fires for the non-public client login route (e.g. /session/login).
|
||||
_target = '../login' if '/' in _login_path.lstrip('/') else 'login'
|
||||
_inner = _safe_login_inner_next(parsed.query)
|
||||
if _inner:
|
||||
_target += '?next=' + _urlparse.quote(_inner, safe='/')
|
||||
handler.send_header('Location', _target)
|
||||
handler.send_header('Content-Length', '0')
|
||||
handler.end_headers()
|
||||
return False
|
||||
_path_with_query = parsed.path or '/'
|
||||
if parsed.query:
|
||||
_path_with_query += '?' + parsed.query
|
||||
@@ -1255,9 +728,24 @@ def _is_secure_context(handler=None) -> bool:
|
||||
|
||||
def set_auth_cookie(handler, cookie_value) -> None:
|
||||
"""Set the auth cookie on the response."""
|
||||
handler.send_header('Set-Cookie', _auth_cookie_header(cookie_value, handler))
|
||||
cookie = http.cookies.SimpleCookie()
|
||||
name = _resolve_cookie_name()
|
||||
cookie[name] = cookie_value
|
||||
cookie[name]['httponly'] = True
|
||||
cookie[name]['samesite'] = 'Lax'
|
||||
cookie[name]['path'] = '/'
|
||||
cookie[name]['max-age'] = str(_resolve_session_ttl())
|
||||
if _is_secure_context(handler):
|
||||
cookie[name]['secure'] = True
|
||||
handler.send_header('Set-Cookie', cookie[name].OutputString())
|
||||
|
||||
|
||||
def clear_auth_cookie(handler) -> None:
|
||||
"""Clear the auth cookie on the response."""
|
||||
handler.send_header('Set-Cookie', _clear_auth_cookie_header())
|
||||
cookie = http.cookies.SimpleCookie()
|
||||
name = _resolve_cookie_name()
|
||||
cookie[name] = ''
|
||||
cookie[name]['httponly'] = True
|
||||
cookie[name]['path'] = '/'
|
||||
cookie[name]['max-age'] = '0'
|
||||
handler.send_header('Set-Cookie', cookie[name].OutputString())
|
||||
|
||||
@@ -1,728 +0,0 @@
|
||||
import copy
|
||||
import base64
|
||||
import hashlib
|
||||
import ipaddress
|
||||
import json
|
||||
import logging
|
||||
import math
|
||||
import os
|
||||
import secrets
|
||||
import socket
|
||||
import threading
|
||||
import time
|
||||
import urllib.error
|
||||
import urllib.parse
|
||||
import urllib.request
|
||||
from typing import Any
|
||||
|
||||
from cryptography.exceptions import InvalidSignature
|
||||
from cryptography.hazmat.primitives import hashes
|
||||
from cryptography.hazmat.primitives.asymmetric import ec, padding, rsa, utils
|
||||
|
||||
from api.config import get_config
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_DEFAULT_SCOPES = ("openid", "profile", "email")
|
||||
_PENDING_TTL_SECONDS = 600
|
||||
_MAX_PENDING_FLOWS = 128
|
||||
_CLOCK_SKEW_SECONDS = 60
|
||||
_CACHE_TTL_SECONDS = 300
|
||||
|
||||
_pending_lock = threading.Lock()
|
||||
_pending_flows: dict[str, dict[str, Any]] = {}
|
||||
|
||||
_discovery_lock = threading.Lock()
|
||||
_discovery_cache: dict[str, tuple[float, dict[str, Any]]] = {}
|
||||
|
||||
_jwks_lock = threading.Lock()
|
||||
_jwks_cache: dict[str, tuple[float, dict[str, Any]]] = {}
|
||||
|
||||
_warned_allow_values: set[str] = set()
|
||||
|
||||
_ALLOW_VALUES_WHITESPACE_WARNING = (
|
||||
"webui_oidc.allow_values (HERMES_WEBUI_OIDC_ALLOW_VALUES) has one or more entries "
|
||||
"with internal whitespace; whitespace is not a value separator, so a value like "
|
||||
'"alice@example.com bob@example.com" is treated as a single entry. '
|
||||
'Use a comma-delimited scalar (e.g. "value1,value2") or a YAML array. '
|
||||
"If this is one intentional multi-word group, it is already correct and no action is needed."
|
||||
)
|
||||
|
||||
|
||||
class _NoRedirect(urllib.request.HTTPRedirectHandler):
|
||||
def redirect_request(self, *args, **kwargs):
|
||||
return None
|
||||
|
||||
|
||||
class OIDCConfigError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class OIDCAuthError(Exception):
|
||||
def __init__(self, message: str, *, status_code: int = 401):
|
||||
super().__init__(message)
|
||||
self.status_code = status_code
|
||||
|
||||
|
||||
def is_oidc_enabled() -> bool:
|
||||
cfg = _resolve_oidc_config()
|
||||
return bool(
|
||||
cfg.get("issuer")
|
||||
and cfg.get("client_id")
|
||||
and cfg.get("allow_claim")
|
||||
and cfg.get("allow_values")
|
||||
)
|
||||
|
||||
|
||||
def build_authorization_redirect(
|
||||
request_base_url: str,
|
||||
next_path: str | None = None,
|
||||
) -> str:
|
||||
cfg = _require_oidc_config()
|
||||
discovery = _get_discovery_document(cfg["issuer"])
|
||||
authorization_endpoint = str(discovery.get("authorization_endpoint") or "").strip()
|
||||
if not authorization_endpoint:
|
||||
raise OIDCConfigError("OIDC discovery document is missing authorization_endpoint")
|
||||
redirect_uri = _resolve_redirect_uri(cfg, request_base_url)
|
||||
state = secrets.token_urlsafe(24)
|
||||
nonce = secrets.token_urlsafe(24)
|
||||
verifier = secrets.token_urlsafe(48)
|
||||
challenge = _b64u(hashlib.sha256(verifier.encode("ascii")).digest())
|
||||
_store_pending_flow(
|
||||
state,
|
||||
{
|
||||
"created_at": time.time(),
|
||||
"nonce": nonce,
|
||||
"code_verifier": verifier,
|
||||
"next_path": _safe_next_path(next_path),
|
||||
},
|
||||
)
|
||||
params = {
|
||||
"response_type": "code",
|
||||
"client_id": cfg["client_id"],
|
||||
"redirect_uri": redirect_uri,
|
||||
"scope": " ".join(cfg["scopes"]),
|
||||
"state": state,
|
||||
"nonce": nonce,
|
||||
"code_challenge": challenge,
|
||||
"code_challenge_method": "S256",
|
||||
}
|
||||
return authorization_endpoint + "?" + urllib.parse.urlencode(params)
|
||||
|
||||
|
||||
def complete_authorization_code_flow(
|
||||
request_base_url: str,
|
||||
state: str,
|
||||
code: str,
|
||||
) -> dict[str, Any]:
|
||||
cfg = _require_oidc_config()
|
||||
pending = _consume_pending_flow(state)
|
||||
if pending is None:
|
||||
raise OIDCAuthError("Invalid OIDC state", status_code=401)
|
||||
discovery = _get_discovery_document(cfg["issuer"])
|
||||
discovery_issuer = str(discovery.get("issuer") or "").strip()
|
||||
if discovery_issuer and discovery_issuer != cfg["issuer"]:
|
||||
raise OIDCAuthError("OIDC discovery issuer did not match the configured issuer", status_code=502)
|
||||
token_endpoint = str(discovery.get("token_endpoint") or "").strip()
|
||||
if not token_endpoint:
|
||||
raise OIDCConfigError("OIDC discovery document is missing token_endpoint")
|
||||
redirect_uri = _resolve_redirect_uri(cfg, request_base_url)
|
||||
token_response = _post_form_json(
|
||||
token_endpoint,
|
||||
{
|
||||
"grant_type": "authorization_code",
|
||||
"client_id": cfg["client_id"],
|
||||
"code": code,
|
||||
"code_verifier": pending["code_verifier"],
|
||||
"redirect_uri": redirect_uri,
|
||||
**({"client_secret": cfg["client_secret"]} if cfg.get("client_secret") else {}),
|
||||
},
|
||||
)
|
||||
id_token = str(token_response.get("id_token") or "").strip()
|
||||
if not id_token:
|
||||
raise OIDCAuthError("OIDC token response did not include an id_token", status_code=502)
|
||||
claims = _validate_id_token(
|
||||
id_token,
|
||||
client_id=cfg["client_id"],
|
||||
issuer=cfg["issuer"],
|
||||
nonce=pending["nonce"],
|
||||
jwks_uri=str(discovery.get("jwks_uri") or "").strip(),
|
||||
)
|
||||
_enforce_allowlist(
|
||||
claims,
|
||||
allow_claim=cfg.get("allow_claim"),
|
||||
allow_values=cfg.get("allow_values") or [],
|
||||
)
|
||||
return {
|
||||
"next_path": pending["next_path"],
|
||||
"subject": str(claims.get("sub") or ""),
|
||||
"email": str(claims.get("email") or ""),
|
||||
"claims": claims,
|
||||
}
|
||||
|
||||
|
||||
def _resolve_oidc_config() -> dict[str, Any]:
|
||||
raw = {}
|
||||
try:
|
||||
cfg = get_config()
|
||||
value = cfg.get("webui_oidc") if isinstance(cfg, dict) else None
|
||||
if isinstance(value, dict):
|
||||
raw.update(value)
|
||||
except Exception:
|
||||
logger.debug("Failed to read webui_oidc config", exc_info=True)
|
||||
|
||||
def pick(name: str, env_name: str) -> Any:
|
||||
env_value = os.getenv(env_name)
|
||||
return env_value if env_value is not None else raw.get(name)
|
||||
|
||||
scopes = _normalize_scopes(pick("scopes", "HERMES_WEBUI_OIDC_SCOPES"))
|
||||
raw_allow = pick("allow_values", "HERMES_WEBUI_OIDC_ALLOW_VALUES")
|
||||
allow_values = _normalize_allow_values(raw_allow)
|
||||
if (
|
||||
raw_allow is not None
|
||||
and not isinstance(raw_allow, (list, tuple, set))
|
||||
and any(any(ch.isspace() for ch in v) for v in allow_values)
|
||||
):
|
||||
key = str(raw_allow)
|
||||
if key not in _warned_allow_values:
|
||||
_warned_allow_values.add(key)
|
||||
logger.warning(_ALLOW_VALUES_WHITESPACE_WARNING)
|
||||
return {
|
||||
"issuer": str(pick("issuer", "HERMES_WEBUI_OIDC_ISSUER") or "").strip(),
|
||||
"client_id": str(pick("client_id", "HERMES_WEBUI_OIDC_CLIENT_ID") or "").strip(),
|
||||
"client_secret": str(pick("client_secret", "HERMES_WEBUI_OIDC_CLIENT_SECRET") or "").strip(),
|
||||
"redirect_uri": str(pick("redirect_uri", "HERMES_WEBUI_OIDC_REDIRECT_URI") or "").strip(),
|
||||
"scopes": scopes,
|
||||
"allow_claim": str(pick("allow_claim", "HERMES_WEBUI_OIDC_ALLOW_CLAIM") or "").strip(),
|
||||
"allow_values": allow_values,
|
||||
}
|
||||
|
||||
|
||||
def _require_oidc_config() -> dict[str, Any]:
|
||||
cfg = _resolve_oidc_config()
|
||||
if not cfg.get("issuer") or not cfg.get("client_id"):
|
||||
raise OIDCConfigError("Native OIDC login is not configured")
|
||||
if not cfg.get("allow_claim") or not cfg.get("allow_values"):
|
||||
raise OIDCConfigError(
|
||||
"Native OIDC login requires webui_oidc.allow_claim and allow_values"
|
||||
)
|
||||
return cfg
|
||||
|
||||
|
||||
def _normalize_scopes(raw: Any) -> list[str]:
|
||||
items = _normalize_text_list(raw)
|
||||
if not items:
|
||||
return list(_DEFAULT_SCOPES)
|
||||
if "openid" not in items:
|
||||
items.insert(0, "openid")
|
||||
deduped = []
|
||||
seen = set()
|
||||
for item in items:
|
||||
if item not in seen:
|
||||
seen.add(item)
|
||||
deduped.append(item)
|
||||
return deduped
|
||||
|
||||
|
||||
def _normalize_allow_values(raw: Any) -> list[str]:
|
||||
"""Normalize allowlist values, splitting only on commas/newlines.
|
||||
|
||||
Unlike ``_normalize_text_list`` (which also splits on whitespace), this
|
||||
preserves multi-word values such as OIDC group names containing spaces
|
||||
(e.g. ``"Hermes Users"`` stays as one entry).
|
||||
|
||||
RFC 6749 §3.3 requires space-delimited scope strings, so
|
||||
``_normalize_scopes`` must keep using ``_normalize_text_list`` -- this
|
||||
function is for allow-values only.
|
||||
"""
|
||||
if raw is None:
|
||||
return []
|
||||
if isinstance(raw, (list, tuple, set)):
|
||||
return [value for value in (str(item).strip() for item in raw) if value]
|
||||
text = str(raw).replace("\n", ",")
|
||||
return [part.strip() for part in text.split(",") if part.strip()]
|
||||
|
||||
|
||||
def _normalize_text_list(raw: Any) -> list[str]:
|
||||
if raw is None:
|
||||
return []
|
||||
if isinstance(raw, (list, tuple, set)):
|
||||
values = [str(item).strip() for item in raw]
|
||||
else:
|
||||
text = str(raw).replace("\n", ",")
|
||||
values = []
|
||||
for comma_part in text.split(","):
|
||||
values.extend(piece.strip() for piece in comma_part.split() if piece.strip())
|
||||
return [value for value in values if value]
|
||||
|
||||
|
||||
def _safe_next_path(raw_path: str | None) -> str:
|
||||
path = str(raw_path or "").strip()
|
||||
if not path:
|
||||
return "/"
|
||||
if path[0] != "/":
|
||||
return "/"
|
||||
if path[1:2] in {"/", "\\"}:
|
||||
return "/"
|
||||
if any(ord(ch) < 32 or ord(ch) == 127 or ch.isspace() for ch in path):
|
||||
return "/"
|
||||
return path
|
||||
|
||||
|
||||
def _resolve_redirect_uri(cfg: dict[str, Any], request_base_url: str) -> str:
|
||||
explicit = str(cfg.get("redirect_uri") or "").strip()
|
||||
if explicit:
|
||||
return explicit
|
||||
return request_base_url.rstrip("/") + "/api/auth/oidc/callback"
|
||||
|
||||
|
||||
def _store_pending_flow(state: str, payload: dict[str, Any]) -> None:
|
||||
now = time.time()
|
||||
with _pending_lock:
|
||||
_prune_pending_flows(now)
|
||||
_trim_pending_flows()
|
||||
_pending_flows[state] = payload
|
||||
|
||||
|
||||
def _consume_pending_flow(state: str) -> dict[str, Any] | None:
|
||||
now = time.time()
|
||||
with _pending_lock:
|
||||
_prune_pending_flows(now)
|
||||
payload = _pending_flows.pop(state, None)
|
||||
return payload
|
||||
|
||||
|
||||
def _prune_pending_flows(now: float) -> None:
|
||||
expired = [
|
||||
state
|
||||
for state, payload in _pending_flows.items()
|
||||
if now - float(payload.get("created_at") or 0) > _PENDING_TTL_SECONDS
|
||||
]
|
||||
for state in expired:
|
||||
_pending_flows.pop(state, None)
|
||||
|
||||
|
||||
def _trim_pending_flows() -> None:
|
||||
overflow = len(_pending_flows) - _MAX_PENDING_FLOWS + 1
|
||||
if overflow <= 0:
|
||||
return
|
||||
oldest = sorted(
|
||||
_pending_flows,
|
||||
key=lambda state: float(_pending_flows[state].get("created_at") or 0),
|
||||
)
|
||||
for state in oldest[:overflow]:
|
||||
_pending_flows.pop(state, None)
|
||||
|
||||
|
||||
def _get_discovery_document(issuer: str) -> dict[str, Any]:
|
||||
discovery_url = _discovery_url_for_issuer(issuer)
|
||||
cached = _cache_get(_discovery_lock, _discovery_cache, discovery_url)
|
||||
if cached is not None:
|
||||
return cached
|
||||
data = _fetch_json(discovery_url)
|
||||
if not isinstance(data, dict):
|
||||
raise OIDCAuthError("OIDC discovery response was not a JSON object", status_code=502)
|
||||
_cache_put(_discovery_lock, _discovery_cache, discovery_url, data)
|
||||
return data
|
||||
|
||||
|
||||
def _discovery_url_for_issuer(issuer: str) -> str:
|
||||
if issuer.endswith("/.well-known/openid-configuration"):
|
||||
return issuer
|
||||
return issuer.rstrip("/") + "/.well-known/openid-configuration"
|
||||
|
||||
|
||||
def _get_jwks_document(jwks_uri: str, *, force_refresh: bool = False) -> dict[str, Any]:
|
||||
if not jwks_uri:
|
||||
raise OIDCConfigError("OIDC discovery document is missing jwks_uri")
|
||||
if force_refresh:
|
||||
with _jwks_lock:
|
||||
_jwks_cache.pop(jwks_uri, None)
|
||||
else:
|
||||
cached = _cache_get(_jwks_lock, _jwks_cache, jwks_uri)
|
||||
if cached is not None:
|
||||
return cached
|
||||
data = _fetch_json(jwks_uri)
|
||||
if not isinstance(data, dict):
|
||||
raise OIDCAuthError("OIDC JWKS response was not a JSON object", status_code=502)
|
||||
_cache_put(_jwks_lock, _jwks_cache, jwks_uri, data)
|
||||
return data
|
||||
|
||||
|
||||
def _cache_get(
|
||||
lock: threading.Lock,
|
||||
cache: dict[str, tuple[float, dict[str, Any]]],
|
||||
key: str,
|
||||
) -> dict[str, Any] | None:
|
||||
now = time.time()
|
||||
with lock:
|
||||
entry = cache.get(key)
|
||||
if entry is None:
|
||||
return None
|
||||
expires_at, value = entry
|
||||
if expires_at <= now:
|
||||
cache.pop(key, None)
|
||||
return None
|
||||
return copy.deepcopy(value)
|
||||
|
||||
|
||||
def _cache_put(
|
||||
lock: threading.Lock,
|
||||
cache: dict[str, tuple[float, dict[str, Any]]],
|
||||
key: str,
|
||||
value: dict[str, Any],
|
||||
) -> None:
|
||||
with lock:
|
||||
cache[key] = (time.time() + _CACHE_TTL_SECONDS, copy.deepcopy(value))
|
||||
|
||||
|
||||
def _fetch_json(url: str) -> dict[str, Any]:
|
||||
_validate_outbound_oidc_url(url)
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
headers={"Accept": "application/json"},
|
||||
)
|
||||
try:
|
||||
with _oidc_opener().open(req, timeout=10) as resp:
|
||||
payload = json.loads(
|
||||
resp.read().decode("utf-8"),
|
||||
parse_constant=_reject_non_finite_json_constant,
|
||||
)
|
||||
except (urllib.error.HTTPError, urllib.error.URLError) as exc:
|
||||
raise OIDCAuthError(f"Failed to reach OIDC endpoint: {url}", status_code=502) from exc
|
||||
except (json.JSONDecodeError, ValueError) as exc:
|
||||
raise OIDCAuthError(f"OIDC endpoint returned invalid JSON: {url}", status_code=502) from exc
|
||||
return payload if isinstance(payload, dict) else {}
|
||||
|
||||
|
||||
def _post_form_json(url: str, form_data: dict[str, Any]) -> dict[str, Any]:
|
||||
_validate_outbound_oidc_url(url)
|
||||
body = urllib.parse.urlencode(form_data).encode("utf-8")
|
||||
req = urllib.request.Request(
|
||||
url,
|
||||
data=body,
|
||||
headers={
|
||||
"Accept": "application/json",
|
||||
"Content-Type": "application/x-www-form-urlencoded",
|
||||
},
|
||||
method="POST",
|
||||
)
|
||||
try:
|
||||
with _oidc_opener().open(req, timeout=10) as resp:
|
||||
payload = json.loads(
|
||||
resp.read().decode("utf-8"),
|
||||
parse_constant=_reject_non_finite_json_constant,
|
||||
)
|
||||
except (urllib.error.HTTPError, urllib.error.URLError) as exc:
|
||||
raise OIDCAuthError("Failed to exchange the OIDC authorization code", status_code=502) from exc
|
||||
except (json.JSONDecodeError, ValueError) as exc:
|
||||
raise OIDCAuthError("OIDC token endpoint returned invalid JSON", status_code=502) from exc
|
||||
return payload if isinstance(payload, dict) else {}
|
||||
|
||||
|
||||
def _oidc_opener() -> urllib.request.OpenerDirector:
|
||||
return urllib.request.build_opener(_NoRedirect)
|
||||
|
||||
|
||||
def _validate_outbound_oidc_url(url: str) -> None:
|
||||
parsed = urllib.parse.urlparse(url)
|
||||
if parsed.scheme != "https":
|
||||
raise OIDCAuthError("OIDC endpoint URLs must use https", status_code=502)
|
||||
if parsed.username or parsed.password:
|
||||
raise OIDCAuthError("OIDC endpoint URLs must not contain credentials", status_code=502)
|
||||
hostname = str(parsed.hostname or "").strip()
|
||||
if not hostname:
|
||||
raise OIDCAuthError("OIDC endpoint URL was missing a hostname", status_code=502)
|
||||
if _is_disallowed_oidc_host(hostname):
|
||||
raise OIDCAuthError(
|
||||
"OIDC endpoint URLs must not target private or local addresses",
|
||||
status_code=502,
|
||||
)
|
||||
|
||||
|
||||
def _is_disallowed_oidc_host(hostname: str) -> bool:
|
||||
literal_ip = _parse_ip_address(hostname)
|
||||
if literal_ip is not None:
|
||||
return _is_disallowed_oidc_ip(literal_ip)
|
||||
try:
|
||||
infos = socket.getaddrinfo(hostname, 443, type=socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
return False
|
||||
for info in infos:
|
||||
sockaddr = info[4]
|
||||
address = _parse_ip_address(sockaddr[0] if sockaddr else "")
|
||||
if address is not None and _is_disallowed_oidc_ip(address):
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _parse_ip_address(value: str):
|
||||
try:
|
||||
return ipaddress.ip_address(value)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _is_disallowed_oidc_ip(address) -> bool:
|
||||
candidate = getattr(address, "ipv4_mapped", None) or address
|
||||
return (
|
||||
candidate.is_loopback
|
||||
or candidate.is_private
|
||||
or candidate.is_link_local
|
||||
or candidate.is_multicast
|
||||
or candidate.is_unspecified
|
||||
or candidate.is_reserved
|
||||
)
|
||||
|
||||
|
||||
def _reject_non_finite_json_constant(value: str):
|
||||
raise ValueError(f"OIDC JSON contained unsupported constant: {value}")
|
||||
|
||||
|
||||
def _validate_id_token(
|
||||
token: str,
|
||||
*,
|
||||
client_id: str,
|
||||
issuer: str,
|
||||
nonce: str,
|
||||
jwks_uri: str,
|
||||
) -> dict[str, Any]:
|
||||
header, claims, signed, signature = _parse_jwt(token)
|
||||
alg = str(header.get("alg") or "").strip()
|
||||
if not alg or alg == "none":
|
||||
raise OIDCAuthError("OIDC id_token uses an unsupported signing algorithm")
|
||||
jwks = _get_jwks_document(jwks_uri)
|
||||
try:
|
||||
public_key = _select_public_key(jwks, header)
|
||||
except OIDCAuthError as exc:
|
||||
if "did not contain the signing key" not in str(exc):
|
||||
raise
|
||||
jwks = _get_jwks_document(jwks_uri, force_refresh=True)
|
||||
public_key = _select_public_key(jwks, header)
|
||||
_verify_jwt_signature(public_key, alg, signed, signature)
|
||||
_validate_registered_claims(claims, client_id=client_id, issuer=issuer, nonce=nonce)
|
||||
if not str(claims.get("sub") or "").strip():
|
||||
raise OIDCAuthError("OIDC id_token did not include a subject")
|
||||
return claims
|
||||
|
||||
|
||||
def _parse_jwt(token: str) -> tuple[dict[str, Any], dict[str, Any], bytes, bytes]:
|
||||
parts = token.split(".")
|
||||
if len(parts) != 3:
|
||||
raise OIDCAuthError("OIDC id_token was not a JWT")
|
||||
header_b64, payload_b64, signature_b64 = parts
|
||||
try:
|
||||
header = json.loads(
|
||||
_b64u_decode(header_b64),
|
||||
parse_constant=_reject_non_finite_json_constant,
|
||||
)
|
||||
claims = json.loads(
|
||||
_b64u_decode(payload_b64),
|
||||
parse_constant=_reject_non_finite_json_constant,
|
||||
)
|
||||
signature = _b64u_decode_bytes(signature_b64)
|
||||
except Exception as exc:
|
||||
raise OIDCAuthError("OIDC id_token could not be decoded") from exc
|
||||
if not isinstance(header, dict) or not isinstance(claims, dict):
|
||||
raise OIDCAuthError("OIDC id_token payload was malformed")
|
||||
signed = f"{header_b64}.{payload_b64}".encode("ascii")
|
||||
return header, claims, signed, signature
|
||||
|
||||
|
||||
def _select_public_key(jwks: dict[str, Any], header: dict[str, Any]):
|
||||
keys = jwks.get("keys")
|
||||
if not isinstance(keys, list) or not keys:
|
||||
raise OIDCAuthError("OIDC JWKS did not contain any signing keys", status_code=502)
|
||||
kid = str(header.get("kid") or "").strip()
|
||||
alg = str(header.get("alg") or "").strip()
|
||||
matches = []
|
||||
for key in keys:
|
||||
if not isinstance(key, dict):
|
||||
continue
|
||||
if key.get("use") not in (None, "sig"):
|
||||
continue
|
||||
if kid and str(key.get("kid") or "").strip() != kid:
|
||||
continue
|
||||
if key.get("alg") not in (None, alg):
|
||||
continue
|
||||
if not _jwk_matches_alg_family(key, alg):
|
||||
continue
|
||||
matches.append(key)
|
||||
if not matches:
|
||||
raise OIDCAuthError("OIDC JWKS did not contain the signing key for this id_token", status_code=502)
|
||||
return _jwk_to_public_key(matches[0])
|
||||
|
||||
|
||||
def _jwk_matches_alg_family(jwk: dict[str, Any], alg: str) -> bool:
|
||||
kty = str(jwk.get("kty") or "").strip()
|
||||
if alg.startswith("RS"):
|
||||
return kty == "RSA"
|
||||
if alg.startswith("ES"):
|
||||
return kty == "EC" and str(jwk.get("crv") or "").strip() == _ec_curve_for_alg(alg)
|
||||
return True
|
||||
|
||||
|
||||
def _ec_curve_for_alg(alg: str) -> str:
|
||||
return {
|
||||
"ES256": "P-256",
|
||||
"ES384": "P-384",
|
||||
"ES512": "P-521",
|
||||
}.get(alg, "")
|
||||
|
||||
|
||||
def _jwk_to_public_key(jwk: dict[str, Any]):
|
||||
kty = str(jwk.get("kty") or "").strip()
|
||||
if kty == "RSA":
|
||||
n = _int_from_b64u(jwk.get("n"))
|
||||
e = _int_from_b64u(jwk.get("e"))
|
||||
return rsa.RSAPublicNumbers(e, n).public_key()
|
||||
if kty == "EC":
|
||||
crv = str(jwk.get("crv") or "").strip()
|
||||
curve = {
|
||||
"P-256": ec.SECP256R1(),
|
||||
"P-384": ec.SECP384R1(),
|
||||
"P-521": ec.SECP521R1(),
|
||||
}.get(crv)
|
||||
if curve is None:
|
||||
raise OIDCAuthError(f"Unsupported OIDC EC curve: {crv}", status_code=502)
|
||||
x = _int_from_b64u(jwk.get("x"))
|
||||
y = _int_from_b64u(jwk.get("y"))
|
||||
return ec.EllipticCurvePublicNumbers(x, y, curve).public_key()
|
||||
raise OIDCAuthError(f"Unsupported OIDC key type: {kty}", status_code=502)
|
||||
|
||||
|
||||
def _verify_jwt_signature(public_key, alg: str, signed: bytes, signature: bytes) -> None:
|
||||
try:
|
||||
if alg == "RS256":
|
||||
public_key.verify(signature, signed, padding.PKCS1v15(), hashes.SHA256())
|
||||
return
|
||||
if alg == "RS384":
|
||||
public_key.verify(signature, signed, padding.PKCS1v15(), hashes.SHA384())
|
||||
return
|
||||
if alg == "RS512":
|
||||
public_key.verify(signature, signed, padding.PKCS1v15(), hashes.SHA512())
|
||||
return
|
||||
if alg == "ES256":
|
||||
public_key.verify(_jose_ecdsa_signature_to_der(signature, 32), signed, ec.ECDSA(hashes.SHA256()))
|
||||
return
|
||||
if alg == "ES384":
|
||||
public_key.verify(_jose_ecdsa_signature_to_der(signature, 48), signed, ec.ECDSA(hashes.SHA384()))
|
||||
return
|
||||
if alg == "ES512":
|
||||
public_key.verify(_jose_ecdsa_signature_to_der(signature, 66), signed, ec.ECDSA(hashes.SHA512()))
|
||||
return
|
||||
except InvalidSignature as exc:
|
||||
raise OIDCAuthError("OIDC id_token signature verification failed") from exc
|
||||
raise OIDCAuthError(f"Unsupported OIDC signing algorithm: {alg}", status_code=502)
|
||||
|
||||
|
||||
def _jose_ecdsa_signature_to_der(signature: bytes, part_size: int) -> bytes:
|
||||
if len(signature) != part_size * 2:
|
||||
raise OIDCAuthError("OIDC id_token ECDSA signature was malformed")
|
||||
r = int.from_bytes(signature[:part_size], "big")
|
||||
s = int.from_bytes(signature[part_size:], "big")
|
||||
return utils.encode_dss_signature(r, s)
|
||||
|
||||
|
||||
def _validate_registered_claims(
|
||||
claims: dict[str, Any],
|
||||
*,
|
||||
client_id: str,
|
||||
issuer: str,
|
||||
nonce: str,
|
||||
) -> None:
|
||||
now = time.time()
|
||||
if str(claims.get("iss") or "").strip() != issuer:
|
||||
raise OIDCAuthError("OIDC id_token issuer did not match the configured issuer")
|
||||
aud = claims.get("aud")
|
||||
if isinstance(aud, list):
|
||||
audiences = [str(item) for item in aud]
|
||||
elif aud is None:
|
||||
audiences = []
|
||||
else:
|
||||
audiences = [str(aud)]
|
||||
if client_id not in audiences:
|
||||
raise OIDCAuthError("OIDC id_token audience did not include this client")
|
||||
if len(audiences) > 1 and str(claims.get("azp") or "").strip() not in {"", client_id}:
|
||||
raise OIDCAuthError("OIDC id_token azp did not match this client")
|
||||
exp = _coerce_numeric_claim(claims, "exp")
|
||||
if exp is None or exp < now - _CLOCK_SKEW_SECONDS:
|
||||
raise OIDCAuthError("OIDC id_token has expired")
|
||||
nbf = _coerce_numeric_claim(claims, "nbf")
|
||||
if nbf is not None and nbf > now + _CLOCK_SKEW_SECONDS:
|
||||
raise OIDCAuthError("OIDC id_token is not valid yet")
|
||||
iat = _coerce_numeric_claim(claims, "iat")
|
||||
if iat is not None and iat > now + _CLOCK_SKEW_SECONDS:
|
||||
raise OIDCAuthError("OIDC id_token has an invalid issued-at time")
|
||||
if str(claims.get("nonce") or "").strip() != nonce:
|
||||
raise OIDCAuthError("OIDC id_token nonce did not match the login request")
|
||||
|
||||
|
||||
def _coerce_numeric_claim(claims: dict[str, Any], name: str) -> float | None:
|
||||
value = claims.get(name)
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
number = float(value)
|
||||
except (TypeError, ValueError) as exc:
|
||||
raise OIDCAuthError(f"OIDC id_token claim {name} was not numeric") from exc
|
||||
if not math.isfinite(number):
|
||||
raise OIDCAuthError(f"OIDC id_token claim {name} was not numeric")
|
||||
return number
|
||||
|
||||
|
||||
def _enforce_allowlist(
|
||||
claims: dict[str, Any],
|
||||
*,
|
||||
allow_claim: str,
|
||||
allow_values: list[str],
|
||||
) -> None:
|
||||
if not allow_claim:
|
||||
return
|
||||
claim_value = _get_claim_path(claims, allow_claim)
|
||||
if claim_value is None:
|
||||
raise OIDCAuthError("OIDC identity is not allowed", status_code=403)
|
||||
actual_values = _claim_values(claim_value)
|
||||
if allow_values:
|
||||
if not any(value in actual_values for value in allow_values):
|
||||
raise OIDCAuthError("OIDC identity is not allowed", status_code=403)
|
||||
return
|
||||
if not actual_values:
|
||||
raise OIDCAuthError("OIDC identity is not allowed", status_code=403)
|
||||
|
||||
|
||||
def _get_claim_path(claims: dict[str, Any], dotted_key: str) -> Any:
|
||||
current: Any = claims
|
||||
for part in dotted_key.split("."):
|
||||
if not isinstance(current, dict) or part not in current:
|
||||
return None
|
||||
current = current[part]
|
||||
return current
|
||||
|
||||
|
||||
def _claim_values(value: Any) -> set[str]:
|
||||
if isinstance(value, (list, tuple, set)):
|
||||
return {str(item) for item in value if str(item).strip()}
|
||||
if isinstance(value, dict):
|
||||
return {str(item) for item in value.values() if str(item).strip()}
|
||||
text = str(value or "").strip()
|
||||
return {text} if text else set()
|
||||
|
||||
|
||||
def _b64u(data: bytes) -> str:
|
||||
return base64.urlsafe_b64encode(data).rstrip(b"=").decode("ascii")
|
||||
|
||||
|
||||
def _b64u_decode(data: str) -> str:
|
||||
return _b64u_decode_bytes(data).decode("utf-8")
|
||||
|
||||
|
||||
def _b64u_decode_bytes(data: str) -> bytes:
|
||||
padded = data + "=" * ((4 - len(data) % 4) % 4)
|
||||
return base64.urlsafe_b64decode(padded.encode("ascii"))
|
||||
|
||||
|
||||
def _int_from_b64u(data: Any) -> int:
|
||||
if not data:
|
||||
raise OIDCAuthError("OIDC JWKS key was missing a required parameter", status_code=502)
|
||||
return int.from_bytes(_b64u_decode_bytes(str(data)), "big")
|
||||
+83
-627
@@ -48,36 +48,15 @@ import time
|
||||
import uuid
|
||||
from typing import Any, Optional
|
||||
|
||||
from api.process_event_utils import (
|
||||
ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
claim_async_delegation_delivery,
|
||||
complete_async_delegation_delivery,
|
||||
completion_delivery_id,
|
||||
release_async_delegation_delivery,
|
||||
requeue_async_delegation_event,
|
||||
schedule_async_delegation_claim_retry,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_DRAIN_THREAD: Optional[threading.Thread] = None
|
||||
_DRAIN_STOP = threading.Event()
|
||||
_PROCESS_RECOVERY_DONE = False
|
||||
_PROCESS_CHECKPOINT_RECOVERED = False
|
||||
_PROCESS_RECOVERY_LOCK = threading.Lock()
|
||||
|
||||
_REAPER_THREAD: Optional[threading.Thread] = None
|
||||
_REAPER_STOP = threading.Event()
|
||||
_REAPER_INTERVAL_SECS = 60.0
|
||||
|
||||
# Serializes the check-then-start of the module's daemon threads
|
||||
# (``start_drain_thread`` / ``start_session_channel_reaper``). Without it two
|
||||
# concurrent callers can both observe ``is_alive() == False`` and each spawn a
|
||||
# thread; the loser's thread is never referenced by the module global and runs
|
||||
# forever, un-joinable. A dedicated lock (not the purpose-bound
|
||||
# ``SESSION_CHANNELS_LOCK`` / ``_EMIT_COALESCE_LOCK``) keeps this narrow.
|
||||
_THREAD_LIFECYCLE_LOCK = threading.Lock()
|
||||
|
||||
# T3: per-session coalesce gate for the public bg_task_complete SSE emit.
|
||||
# The server-side wakeup path remains immediate; only the browser-observation
|
||||
# frame is throttled so a burst of background task completions does not flood an
|
||||
@@ -271,77 +250,6 @@ def subscribe_to_session_channel(
|
||||
return ch, q
|
||||
|
||||
|
||||
# Bounded window a cancelling worker may stay lifecycle-busy before an entry
|
||||
# with no live SSE channel is treated as an orphan. Matches the unwind ceiling
|
||||
# used by the chat-start successor guard in ``api.routes``.
|
||||
_ACTIVE_RUN_CANCEL_UNWIND_SECONDS = 180.0
|
||||
|
||||
|
||||
def _active_run_ids_for_session(
|
||||
session_id: str,
|
||||
*,
|
||||
attachable_only: bool,
|
||||
) -> list[str]:
|
||||
"""Return this session's run stream ids under the requested semantics.
|
||||
|
||||
``attachable_only`` selects browser-recovery semantics and drops every
|
||||
``phase="cancelling"`` row: cancellation is already terminal for the client
|
||||
even while the worker unwinds. Busy checks pass ``False`` so a freshly
|
||||
cancelled run still blocks a successor for its bounded unwind window.
|
||||
|
||||
A cancelling row past that window with no live ``STREAMS`` channel is an
|
||||
orphan: it is dropped from ``ACTIVE_RUNS`` and its stream-owner entry is
|
||||
released, so a wedged worker cannot suppress background wakeups forever.
|
||||
"""
|
||||
from api import config as _cfg
|
||||
|
||||
sid = str(session_id or "").strip()
|
||||
if not sid:
|
||||
return []
|
||||
try:
|
||||
with _cfg.STREAMS_LOCK:
|
||||
live_stream_ids = set((_cfg.STREAMS or {}).keys())
|
||||
now = time.time()
|
||||
matches: list[str] = []
|
||||
stale_keys: list[str] = []
|
||||
with _cfg.ACTIVE_RUNS_LOCK:
|
||||
for run_key, meta in list((_cfg.ACTIVE_RUNS or {}).items()):
|
||||
if not isinstance(meta, dict) or meta.get("session_id") != sid:
|
||||
continue
|
||||
stream_id = str(meta.get("stream_id") or run_key or "").strip()
|
||||
if not stream_id:
|
||||
continue
|
||||
cancelling = not _cfg.active_run_is_attachable(meta)
|
||||
stale_cancel = (
|
||||
cancelling
|
||||
and _cfg.active_run_cancel_is_stale(
|
||||
meta,
|
||||
grace_seconds=_ACTIVE_RUN_CANCEL_UNWIND_SECONDS,
|
||||
now=now,
|
||||
)
|
||||
and run_key not in live_stream_ids
|
||||
and stream_id not in live_stream_ids
|
||||
)
|
||||
if stale_cancel:
|
||||
stale_keys.append(run_key)
|
||||
continue
|
||||
if attachable_only and cancelling:
|
||||
continue
|
||||
matches.append(stream_id)
|
||||
for run_key in stale_keys:
|
||||
(_cfg.ACTIVE_RUNS or {}).pop(run_key, None)
|
||||
for run_key in stale_keys:
|
||||
_cfg.unregister_stream_owner(run_key)
|
||||
return matches
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"ACTIVE_RUNS lookup failed for %s",
|
||||
sid,
|
||||
exc_info=True,
|
||||
)
|
||||
return []
|
||||
|
||||
|
||||
def active_stream_id_for_session(session_id: str) -> Optional[str]:
|
||||
"""Return the stream_id of the live run for *session_id*, or None.
|
||||
|
||||
@@ -359,79 +267,25 @@ def active_stream_id_for_session(session_id: str) -> Optional[str]:
|
||||
|
||||
Keys on ACTIVE_RUNS (worker-lifecycle registry) — the same source
|
||||
``_session_has_active_turn`` / ``_emit_to_session_streams`` already trust
|
||||
to map a stream back to its owning session — but returns only rows that
|
||||
are still ATTACHABLE. A ``phase="cancelling"`` row stays lifecycle-busy
|
||||
while its worker unwinds, yet the client already reached a terminal state
|
||||
for that run, so replaying ``server_turn_started`` for it makes the tab
|
||||
attach, receive the terminal event, resubscribe, and loop forever.
|
||||
to map a stream back to its owning session. Returns the first matching
|
||||
stream_id (a session has at most one live run; cancel/reconnect can
|
||||
briefly hold two — either is a valid attach target, the frontend dedupes
|
||||
by stream_id).
|
||||
"""
|
||||
matches = _active_run_ids_for_session(session_id, attachable_only=True)
|
||||
return matches[0] if matches else None
|
||||
from api import config as _cfg
|
||||
|
||||
|
||||
def persisted_message_count_for_session(session_id: str) -> Optional[int]:
|
||||
"""Cheap, metadata-only persisted ``message_count`` for *session_id*, or None.
|
||||
|
||||
Companion to ``active_stream_id_for_session`` for the per-session SSE
|
||||
on-subscribe self-heal. ``active_stream_id_for_session`` recovers a turn
|
||||
that is live RIGHT NOW (replay ``server_turn_started``). But a
|
||||
SERVER-initiated turn (self-wake / cron / restart hook) can start AND
|
||||
finish entirely inside an SSE gap: the fire-and-forget
|
||||
``server_turn_started`` reached no subscriber AND the run already cleared
|
||||
from ``ACTIVE_RUNS`` by the time the tab reconnects, so the replay finds
|
||||
nothing (returns None) and the tab's transcript stays stale until a hard
|
||||
refresh — the reported visible-tab defect. To detect that case the handler
|
||||
compares the freshly-(re)subscribed tab's last-known count against this
|
||||
persisted count; a server that is AHEAD means a turn landed during the gap.
|
||||
|
||||
Reads via ``metadata_only=True`` so it never parses the full transcript
|
||||
(this runs on every per-session SSE (re)connect). The persisted count is
|
||||
written by ``Session.save`` as ``meta['message_count'] = len(messages)`` —
|
||||
the SAME basis the frontend's ``S.session.message_count`` is built from —
|
||||
so the comparison is apples-to-apples. Returns None when the count is
|
||||
unknown (legacy sidecars without a persisted count); the caller treats
|
||||
None as "cannot tell, do nothing", never as a trigger.
|
||||
"""
|
||||
try:
|
||||
from api.models import get_session
|
||||
|
||||
s = get_session(session_id, metadata_only=True)
|
||||
count = getattr(s, "_metadata_message_count", None)
|
||||
if count is None:
|
||||
msgs = getattr(s, "messages", None)
|
||||
count = len(msgs) if isinstance(msgs, list) and msgs else None
|
||||
return int(count) if count is not None else None
|
||||
with _cfg.ACTIVE_RUNS_LOCK:
|
||||
for _stream_id, meta in (_cfg.ACTIVE_RUNS or {}).items():
|
||||
if isinstance(meta, dict) and meta.get("session_id") == session_id:
|
||||
return str(_stream_id)
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"persisted_message_count_for_session lookup failed for %s",
|
||||
"active_stream_id_for_session lookup failed for %s",
|
||||
session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def should_emit_session_updated(
|
||||
subscriber_known_count: Optional[int],
|
||||
persisted_count: Optional[int],
|
||||
) -> bool:
|
||||
"""Gate for the per-session SSE "finished during the gap" self-heal emit.
|
||||
|
||||
Single source of truth shared by the SSE handler and its tests so the two
|
||||
cannot drift (the handler MUST call this, not inline the comparison).
|
||||
Emit a ``session-updated`` frame ONLY when:
|
||||
* the (re)subscribing tab reported a last-known count (``?known_count``),
|
||||
AND
|
||||
* the persisted server-side count is known, AND
|
||||
* the server is STRICTLY ahead (a turn landed during the gap).
|
||||
A missing known count (tab didn't report), an unknown persisted count
|
||||
(legacy sidecar), or an equal/behind count all return False — never a
|
||||
spurious reload.
|
||||
"""
|
||||
if subscriber_known_count is None:
|
||||
return False
|
||||
if persisted_count is None:
|
||||
return False
|
||||
return persisted_count > subscriber_known_count
|
||||
return None
|
||||
|
||||
|
||||
def _reaper_loop() -> None:
|
||||
@@ -458,32 +312,6 @@ def _reaper_loop() -> None:
|
||||
for sid in collected:
|
||||
_LAST_EMIT_TS.pop(sid, None)
|
||||
logger.debug("SessionChannel reaper collected: %s", collected)
|
||||
# Sweep the per-session completion-dedup map by DELIVERY lifecycle,
|
||||
# not channel collection. ``BG_TASK_COMPLETE_EVENTS_SEEN`` gains a
|
||||
# ``session_id -> set[process_id]`` entry the first time a bg task
|
||||
# completes for a session — in ``_process_one``, whether or not any
|
||||
# tab/SSE channel ever existed — and is otherwise never deleted, so it
|
||||
# grows unbounded. Coupling the prune to channel collection (an
|
||||
# earlier version of this fix) missed the dominant case: a headless
|
||||
# completion (task fires, tab closed or never opened) has no channel
|
||||
# to collect. Instead, once a completion has been drained (its
|
||||
# ``session_id`` removed from ``PENDING_BG_TASK_COMPLETIONS``), the
|
||||
# short ``_move_to_finished`` dedup window is closed and the entry is
|
||||
# pure leak — so sweep every delivered (not-pending) session here,
|
||||
# every tick. The registry's own per-``process_id``
|
||||
# ``_completion_consumed`` gate remains the primary idempotency
|
||||
# backstop, so sweeping a delivered session's set can never resurrect
|
||||
# an already-delivered completion (even in the tiny window between
|
||||
# this module's ``SEEN.add`` and ``PENDING.add`` in ``_process_one``).
|
||||
from api import config as _cfg
|
||||
|
||||
with _cfg.BG_TASK_COMPLETE_EVENTS_SEEN_LOCK:
|
||||
for sid in [
|
||||
s
|
||||
for s in _cfg.BG_TASK_COMPLETE_EVENTS_SEEN
|
||||
if s not in _cfg.PENDING_BG_TASK_COMPLETIONS
|
||||
]:
|
||||
_cfg.BG_TASK_COMPLETE_EVENTS_SEEN.pop(sid, None)
|
||||
except Exception:
|
||||
logger.warning("SessionChannel reaper iteration failed", exc_info=True)
|
||||
# Wait but wake up promptly on stop.
|
||||
@@ -494,17 +322,16 @@ def _reaper_loop() -> None:
|
||||
def start_session_channel_reaper() -> bool:
|
||||
"""Start the SessionChannel reaper thread. Idempotent; returns True on first start."""
|
||||
global _REAPER_THREAD
|
||||
with _THREAD_LIFECYCLE_LOCK:
|
||||
if _REAPER_THREAD is not None and _REAPER_THREAD.is_alive():
|
||||
return False
|
||||
_REAPER_STOP.clear()
|
||||
_REAPER_THREAD = threading.Thread(
|
||||
target=_reaper_loop,
|
||||
name="hermes-webui-session-channel-reaper",
|
||||
daemon=True,
|
||||
)
|
||||
_REAPER_THREAD.start()
|
||||
return True
|
||||
if _REAPER_THREAD is not None and _REAPER_THREAD.is_alive():
|
||||
return False
|
||||
_REAPER_STOP.clear()
|
||||
_REAPER_THREAD = threading.Thread(
|
||||
target=_reaper_loop,
|
||||
name="hermes-webui-session-channel-reaper",
|
||||
daemon=True,
|
||||
)
|
||||
_REAPER_THREAD.start()
|
||||
return True
|
||||
|
||||
|
||||
def stop_session_channel_reaper(timeout: float = 2.0) -> None:
|
||||
@@ -614,9 +441,9 @@ def _build_payload(evt: dict, session_id: str) -> dict:
|
||||
``(session_id, event_id)`` to dedupe across reconnects.
|
||||
"""
|
||||
# ProcessRegistry completion events use the field name ``session_id`` for
|
||||
# the process id. Async delegation completions carry ``delegation_id``
|
||||
# instead; expose either stable delivery id as payload ``task_id``.
|
||||
process_id = completion_delivery_id(evt)
|
||||
# the process id. Alias it locally before exposing it as payload ``task_id``
|
||||
# to avoid confusing that wire-format name with the WebUI session id.
|
||||
process_id = str(evt.get("session_id") or "")
|
||||
payload: dict[str, Any] = {
|
||||
"session_id": str(session_id),
|
||||
"task_id": process_id,
|
||||
@@ -895,12 +722,7 @@ def _mark_registry_completion_consumed(process_id: str) -> None:
|
||||
# registry, pre-Option-1 spawns) it is a PURE PASS-THROUGH — it never
|
||||
# suppresses a legitimate Option Z wakeup on uncertainty (Option Z must keep
|
||||
# working).
|
||||
_ENV_IMMUNE_OWNER_ATTRS = (
|
||||
"origin_ui_session_id", # modern hermes-agent exact browser-tab return address
|
||||
"spawn_session_id",
|
||||
"owner_session_id",
|
||||
"turn_session_id",
|
||||
)
|
||||
_ENV_IMMUNE_OWNER_ATTRS = ("spawn_session_id", "owner_session_id", "turn_session_id")
|
||||
|
||||
|
||||
def _env_immune_spawn_owner(proc_session) -> str:
|
||||
@@ -950,248 +772,6 @@ def _resolve_wakeup_target(
|
||||
return owner
|
||||
|
||||
|
||||
def _requeue_async_delegation_event(
|
||||
process_registry,
|
||||
evt: dict,
|
||||
*,
|
||||
claim=None,
|
||||
delay: float = 0.5,
|
||||
) -> bool:
|
||||
"""Retry without enqueueing new work once drain shutdown has started."""
|
||||
completion_queue = getattr(process_registry, "completion_queue", None)
|
||||
return requeue_async_delegation_event(
|
||||
evt,
|
||||
completion_queue,
|
||||
delay=delay,
|
||||
stop_event=_DRAIN_STOP,
|
||||
durable=(bool(getattr(claim, "durable", False)) if claim is not None else None),
|
||||
)
|
||||
|
||||
|
||||
def _retry_unclaimed_async_delegation_event(
|
||||
process_registry,
|
||||
evt: dict,
|
||||
*,
|
||||
keep_legacy_retrying: bool = False,
|
||||
) -> None:
|
||||
"""Retry from durable state, or make a bounded legacy routing pass.
|
||||
|
||||
``keep_legacy_retrying`` is reserved for a completion whose target session
|
||||
is known but currently busy. That wait state is neither an unroutable event
|
||||
nor a failed delivery attempt, so compatibility-mode delivery must keep
|
||||
backing off until the session becomes idle.
|
||||
"""
|
||||
completion_queue = getattr(process_registry, "completion_queue", None)
|
||||
if schedule_async_delegation_claim_retry(
|
||||
evt,
|
||||
completion_queue,
|
||||
delay=ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
):
|
||||
return
|
||||
if not keep_legacy_retrying and evt.get("_webui_routing_retry_attempted"):
|
||||
return
|
||||
retry_evt = dict(evt)
|
||||
if not keep_legacy_retrying:
|
||||
retry_evt["_webui_routing_retry_attempted"] = True
|
||||
_requeue_async_delegation_event(
|
||||
process_registry,
|
||||
retry_evt,
|
||||
delay=ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
)
|
||||
|
||||
|
||||
def _record_async_delegation_accepted(
|
||||
evt: dict,
|
||||
*,
|
||||
session_id: str,
|
||||
claim,
|
||||
) -> None:
|
||||
"""ACK durable delivery and publish live-view state after turn acceptance."""
|
||||
|
||||
complete_async_delegation_delivery(evt, claim)
|
||||
payload = _build_payload(evt, session_id)
|
||||
try:
|
||||
_emit_bg_task_complete_events_coalesced(session_id, payload)
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"async delegation live-view emit failed for session %s",
|
||||
session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
def _start_async_delegation_wakeup_turn(
|
||||
session_id: str,
|
||||
wakeup_prompt: str,
|
||||
*,
|
||||
delegation_id: str,
|
||||
evt: dict,
|
||||
claim,
|
||||
process_registry,
|
||||
) -> None:
|
||||
"""Start one autonomous delegation turn and ACK only after acceptance."""
|
||||
|
||||
def _runner() -> None:
|
||||
try:
|
||||
from api.routes import start_session_turn
|
||||
|
||||
resp = start_session_turn(
|
||||
session_id,
|
||||
wakeup_prompt,
|
||||
source="process_wakeup",
|
||||
)
|
||||
raw_status = (resp or {}).get("_status")
|
||||
if raw_status is None:
|
||||
status = 200 if (resp or {}).get("stream_id") else 500
|
||||
else:
|
||||
status = int(raw_status)
|
||||
if 200 <= status < 300:
|
||||
_record_async_delegation_accepted(
|
||||
evt,
|
||||
session_id=session_id,
|
||||
claim=claim,
|
||||
)
|
||||
logger.info(
|
||||
"async delegation wakeup turn accepted for session %s "
|
||||
"(stream_id=%s)",
|
||||
session_id,
|
||||
(resp or {}).get("stream_id"),
|
||||
)
|
||||
return
|
||||
|
||||
release_async_delegation_delivery(evt, claim)
|
||||
_retry_unclaimed_async_delegation_event(
|
||||
process_registry, evt, keep_legacy_retrying=True
|
||||
)
|
||||
if status == 409 and (resp or {}).get("error") == "process_wakeup_paused":
|
||||
logger.info(
|
||||
"async delegation wakeup paused for session %s; delivery remains retryable",
|
||||
session_id,
|
||||
)
|
||||
else:
|
||||
logger.debug(
|
||||
"async delegation wakeup not accepted for session %s: "
|
||||
"status=%s err=%r; durable retry scheduled",
|
||||
session_id,
|
||||
status,
|
||||
(resp or {}).get("error"),
|
||||
)
|
||||
except Exception:
|
||||
release_async_delegation_delivery(evt, claim)
|
||||
_retry_unclaimed_async_delegation_event(
|
||||
process_registry, evt, keep_legacy_retrying=True
|
||||
)
|
||||
logger.warning(
|
||||
"async delegation wakeup turn failed for session %s; durable retry scheduled",
|
||||
session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
threading.Thread(
|
||||
target=_runner,
|
||||
name=f"hermes-webui-delegation-wakeup-{str(session_id)[:8]}",
|
||||
daemon=True,
|
||||
).start()
|
||||
|
||||
|
||||
def _process_async_delegation_event(
|
||||
evt: dict,
|
||||
*,
|
||||
session_id: str,
|
||||
delegation_id: str,
|
||||
process_registry,
|
||||
) -> None:
|
||||
"""Claim and route one async completion without private registry markers."""
|
||||
|
||||
# A durable claim has a finite attempt budget. A busy foreground turn is
|
||||
# not a delivery attempt, so leave the record unclaimed and let the shared
|
||||
# restore sweep retry after the session can accept a wakeup.
|
||||
if _session_has_active_turn(session_id):
|
||||
_retry_unclaimed_async_delegation_event(
|
||||
process_registry,
|
||||
evt,
|
||||
keep_legacy_retrying=True,
|
||||
)
|
||||
return
|
||||
|
||||
try:
|
||||
claim = claim_async_delegation_delivery(evt, "webui-background")
|
||||
except Exception:
|
||||
_requeue_async_delegation_event(process_registry, evt)
|
||||
return
|
||||
if claim is None:
|
||||
completion_queue = getattr(process_registry, "completion_queue", None)
|
||||
schedule_async_delegation_claim_retry(evt, completion_queue)
|
||||
return
|
||||
|
||||
try:
|
||||
wakeup_prompt_raw = format_wakeup_prompt(evt)
|
||||
wakeup_prompt = wakeup_prompt_raw.strip() if wakeup_prompt_raw else ""
|
||||
if not wakeup_prompt:
|
||||
raise RuntimeError("async delegation completion could not be formatted")
|
||||
except Exception:
|
||||
# A formatting failure is a permanent, non-retryable defect (a malformed
|
||||
# event will never format correctly), so it must stay on the bounded
|
||||
# one-shot path — never keep_legacy_retrying, or it would loop forever.
|
||||
release_async_delegation_delivery(evt, claim)
|
||||
_retry_unclaimed_async_delegation_event(process_registry, evt)
|
||||
logger.warning(
|
||||
"async delegation completion could not be formatted for session %s",
|
||||
session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return
|
||||
|
||||
try:
|
||||
_start_async_delegation_wakeup_turn(
|
||||
session_id,
|
||||
wakeup_prompt,
|
||||
delegation_id=delegation_id,
|
||||
evt=evt,
|
||||
claim=claim,
|
||||
process_registry=process_registry,
|
||||
)
|
||||
except Exception:
|
||||
# A post-format dispatch failure against a resolved target is transient
|
||||
# (the target may accept on a later pass), so keep the legacy completion
|
||||
# retrying rather than dropping it after one bounded attempt.
|
||||
release_async_delegation_delivery(evt, claim)
|
||||
_retry_unclaimed_async_delegation_event(
|
||||
process_registry, evt, keep_legacy_retrying=True
|
||||
)
|
||||
logger.warning(
|
||||
"server-side async delegation dispatch failed for session %s",
|
||||
session_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
|
||||
def _resolve_completion_target(
|
||||
*,
|
||||
session_key_resolved_sid: str,
|
||||
origin_ui_session_id: str,
|
||||
) -> str:
|
||||
"""Return the WebUI session that owns a detached completion event.
|
||||
|
||||
Modern Hermes Agent events carry ``origin_ui_session_id`` as an exact,
|
||||
immutable return address captured from the commissioning browser turn.
|
||||
It is authoritative over the mutable/legacy session-key index. Older
|
||||
Agent events omit it and retain the existing session-key fallback.
|
||||
"""
|
||||
resolved = str(session_key_resolved_sid or "")
|
||||
owner = str(origin_ui_session_id or "")
|
||||
if not owner:
|
||||
return resolved
|
||||
if resolved and resolved != owner:
|
||||
logger.error(
|
||||
"cross-session completion route BLOCKED: session_key resolved to %r "
|
||||
"but exact origin_ui_session_id is %r; routing to the exact owner",
|
||||
resolved,
|
||||
owner,
|
||||
)
|
||||
return owner
|
||||
|
||||
|
||||
def _process_one(evt: dict) -> None:
|
||||
"""Route a single completion_queue event to the matching WebUI session."""
|
||||
from api import config as _cfg
|
||||
@@ -1207,59 +787,55 @@ def _process_one(evt: dict) -> None:
|
||||
except Exception:
|
||||
_process_registry = None
|
||||
|
||||
process_id = completion_delivery_id(evt)
|
||||
process_id = str(evt.get("session_id") or "")
|
||||
session_key = str(evt.get("session_key") or "")
|
||||
origin_ui_session_id = str(evt.get("origin_ui_session_id") or "")
|
||||
# Root-cause fix (t_0f447014): the notify_on_complete completion event
|
||||
# enqueued by ProcessRegistry._move_to_finished() historically carried NO
|
||||
# "session_key" field — only the watch_match enqueue included one. Without
|
||||
# it the old `evt.get("session_key") or process_id` fell back to the
|
||||
# process id ("proc_xxxx"), which is never a PROCESS_SESSION_INDEX key.
|
||||
# Recover spawn-time routing metadata from the process registry.
|
||||
if process_id and (not session_key or not origin_ui_session_id):
|
||||
# enqueued by ProcessRegistry._move_to_finished() carries NO "session_key"
|
||||
# field — only the watch_match enqueue includes one. Without it the old
|
||||
# `evt.get("session_key") or process_id` fell back to the process id
|
||||
# ("proc_xxxx"), which is never a PROCESS_SESSION_INDEX key (only
|
||||
# webui_session_id -> webui_session_id is registered at chat-start), so
|
||||
# every wakeup was silently dropped here and the frontend never POSTed an
|
||||
# ack. Recover the spawn-time session_key from the process registry's
|
||||
# ProcessSession: the terminal tool captured it synchronously at spawn
|
||||
# (while the turn's env was active), so it survives the turn-end env
|
||||
# restore and is the WebUI session_id for WebUI-spawned processes.
|
||||
if not session_key and process_id:
|
||||
try:
|
||||
if _process_registry is not None:
|
||||
_ps = _process_registry.get(process_id)
|
||||
if _ps is not None:
|
||||
if not session_key and getattr(_ps, "session_key", ""):
|
||||
session_key = str(_ps.session_key)
|
||||
if not origin_ui_session_id:
|
||||
origin_ui_session_id = (
|
||||
str(getattr(_ps, "origin_ui_session_id", "") or "")
|
||||
or str(getattr(_ps, "spawn_session_id", "") or "")
|
||||
)
|
||||
if _ps is not None and getattr(_ps, "session_key", ""):
|
||||
session_key = str(_ps.session_key)
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"session ownership recovery from process registry failed for %r",
|
||||
"session_key recovery from process registry failed for %r",
|
||||
process_id,
|
||||
exc_info=True,
|
||||
)
|
||||
if not session_key and not origin_ui_session_id:
|
||||
if not session_key:
|
||||
logger.debug(
|
||||
"process_complete drop: no recoverable session_key or exact UI owner "
|
||||
"for process_id=%r",
|
||||
"process_complete drop: no recoverable session_key for process_id=%r",
|
||||
process_id,
|
||||
)
|
||||
if evt.get("type") == "async_delegation":
|
||||
_retry_unclaimed_async_delegation_event(_process_registry, evt)
|
||||
return
|
||||
session_id = ""
|
||||
if session_key:
|
||||
with _cfg.PROCESS_SESSION_INDEX_LOCK:
|
||||
session_id = _cfg.PROCESS_SESSION_INDEX.get(session_key) or ""
|
||||
if not session_id and not origin_ui_session_id:
|
||||
with _cfg.PROCESS_SESSION_INDEX_LOCK:
|
||||
session_id = _cfg.PROCESS_SESSION_INDEX.get(session_key)
|
||||
if not session_id:
|
||||
# No mapping — could be a cron/gateway process that uses the same
|
||||
# registry but a non-WebUI session_key. Durable delegation events stay
|
||||
# pending and are retried because their WebUI ownership mapping can be
|
||||
# registered shortly after process restore.
|
||||
# registry but a non-WebUI session_key. Ignore.
|
||||
logger.debug("process_complete drop: no session mapping for key=%r", session_key)
|
||||
if evt.get("type") == "async_delegation":
|
||||
_retry_unclaimed_async_delegation_event(_process_registry, evt)
|
||||
return
|
||||
# ── xsession wakeup misroute defense-in-depth (Option 3) ──────────────
|
||||
# First retain the process-registry spawn-owner cross-check for legacy
|
||||
# terminal events. Then apply origin_ui_session_id as the final authority;
|
||||
# that exact owner is shared by process and async-delegation completions.
|
||||
# session_id above came from PROCESS_SESSION_INDEX.get(session_key), and
|
||||
# session_key was captured by the terminal tool from the (historically
|
||||
# racy) process-global env at spawn. Option 1 binds the per-turn identity
|
||||
# to a contextvar so that capture is no longer racy — but as an INDEPENDENT
|
||||
# safety net, cross-check the resolved target against the env-immune
|
||||
# spawn owner (when the core ProcessSession exposes one). On a positive
|
||||
# mismatch this re-routes the wakeup (and the live-view emit + dedupe
|
||||
# markers below) to the TRUE owner instead of waking the wrong session.
|
||||
# Pure pass-through when no env-immune owner is available (today's core,
|
||||
# cron/CLI procs, pre-Option-1 spawns) — never suppresses a valid wakeup.
|
||||
try:
|
||||
_ps_xs = _process_registry.get(process_id) if (_process_registry is not None and process_id) else None
|
||||
except Exception:
|
||||
@@ -1269,35 +845,6 @@ def _process_one(evt: dict) -> None:
|
||||
session_key_resolved_sid=session_id,
|
||||
proc_session=_ps_xs,
|
||||
)
|
||||
# origin_ui_session_id is the exact, immutable return address; it is the
|
||||
# FINAL routing authority over the (mutable) session-key/Option-3 result.
|
||||
session_id = _resolve_completion_target(
|
||||
session_key_resolved_sid=session_id,
|
||||
origin_ui_session_id=origin_ui_session_id,
|
||||
)
|
||||
if not session_id:
|
||||
logger.debug("process_complete drop: completion target resolved empty")
|
||||
# An async delegation event that resolves empty here must NOT silently
|
||||
# return: no durable claim was taken, so the core row stays pending and
|
||||
# the restart-restore sweep would re-deliver it forever. Route it into
|
||||
# the bounded retry instead (arms a durable retry / one best-effort
|
||||
# legacy requeue), so it stays retryable without a spurious ACK.
|
||||
if evt.get("type") == "async_delegation":
|
||||
_retry_unclaimed_async_delegation_event(_process_registry, evt)
|
||||
return
|
||||
# ── THE SEAM: async delegations take the durable-claim delivery path,
|
||||
# routed to the origin-resolved session. The claim/complete/release
|
||||
# lifecycle (keyed on the immutable delegation_id) is the sole dedupe +
|
||||
# restart-safety authority for async events, so they early-return before
|
||||
# the terminal-process idempotency/emit/Option-Z machinery below. ──
|
||||
if evt.get("type") == "async_delegation":
|
||||
_process_async_delegation_event(
|
||||
evt,
|
||||
session_id=session_id,
|
||||
delegation_id=process_id,
|
||||
process_registry=_process_registry,
|
||||
)
|
||||
return
|
||||
# ── Idempotency vs the REAL merged upstream #2279 (shared dedupe key) ──
|
||||
# The real merged #2279 next-turn drain
|
||||
# (api/streaming._drain_webui_process_notifications) dedupes ONLY via
|
||||
@@ -1409,7 +956,7 @@ def _process_one(evt: dict) -> None:
|
||||
)
|
||||
|
||||
|
||||
def record_deferred_wakeup(session_id: str, process_id: str, wakeup_prompt: str) -> bool:
|
||||
def record_deferred_wakeup(session_id: str, process_id: str, wakeup_prompt: str) -> None:
|
||||
"""Persist a deferred process-completion wakeup for later redelivery.
|
||||
|
||||
Called from ``_process_one`` when a completion arrives while a turn is
|
||||
@@ -1421,11 +968,10 @@ def record_deferred_wakeup(session_id: str, process_id: str, wakeup_prompt: str)
|
||||
|
||||
Idempotent per process_id: if the same process_id is already queued for
|
||||
this session (kill_process racing the reader thread), it is not appended
|
||||
twice. Returns whether the prompt is safely queued; never raises into the
|
||||
drain loop.
|
||||
twice. Best-effort — never raises into the drain loop.
|
||||
"""
|
||||
if not session_id or not wakeup_prompt:
|
||||
return False
|
||||
return
|
||||
from api import config as _cfg
|
||||
|
||||
try:
|
||||
@@ -1434,16 +980,14 @@ def record_deferred_wakeup(session_id: str, process_id: str, wakeup_prompt: str)
|
||||
if process_id and any(
|
||||
e.get("process_id") == process_id for e in entries
|
||||
):
|
||||
return True
|
||||
return
|
||||
entries.append(
|
||||
{"process_id": process_id, "wakeup_prompt": wakeup_prompt}
|
||||
)
|
||||
return True
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"record_deferred_wakeup failed for session %s", session_id, exc_info=True
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def claim_deferred_wakeups(session_id: str) -> list[dict]:
|
||||
@@ -1578,7 +1122,16 @@ def _session_has_active_turn(session_id: str) -> bool:
|
||||
``_start_chat_stream_for_session``'s own active-stream guard — i.e. the
|
||||
same lock /api/chat/start uses is the authoritative race backstop.
|
||||
"""
|
||||
return bool(_active_run_ids_for_session(session_id, attachable_only=False))
|
||||
from api import config as _cfg
|
||||
|
||||
try:
|
||||
with _cfg.ACTIVE_RUNS_LOCK:
|
||||
for _stream_id, meta in (_cfg.ACTIVE_RUNS or {}).items():
|
||||
if isinstance(meta, dict) and meta.get("session_id") == session_id:
|
||||
return True
|
||||
except Exception:
|
||||
logger.debug("ACTIVE_RUNS active-turn check failed", exc_info=True)
|
||||
return False
|
||||
|
||||
|
||||
def _start_server_side_wakeup_turn(
|
||||
@@ -1621,12 +1174,7 @@ def _start_server_side_wakeup_turn(
|
||||
session_id, wakeup_prompt, source="process_wakeup"
|
||||
)
|
||||
status = int((resp or {}).get("_status", 200) or 200)
|
||||
if status == 409 and (resp or {}).get("error") == "process_wakeup_paused":
|
||||
logger.info(
|
||||
"server-side wakeup suppressed for session %s: provider credential state is paused",
|
||||
session_id,
|
||||
)
|
||||
elif status == 409:
|
||||
if status == 409:
|
||||
# Raced an active turn (e.g. a human /api/chat/start, or a
|
||||
# sibling deferred-wakeup thread). Re-defer this prompt so it
|
||||
# is delivered by the winning turn's teardown / next-turn drain
|
||||
@@ -1677,28 +1225,10 @@ def _drain_loop() -> None:
|
||||
return
|
||||
logger.info("bg_task_complete drain thread started")
|
||||
while not _DRAIN_STOP.is_set():
|
||||
# Read the queue defensively: a rebuilt/partially-initialized registry
|
||||
# may not expose ``completion_queue`` (mirrors streaming.py's
|
||||
# ``getattr(process_registry, 'completion_queue', None)`` guard). Direct
|
||||
# attribute access here would raise AttributeError, which the old broad
|
||||
# ``except Exception: continue`` swallowed silently and re-tried with no
|
||||
# backoff — a 100%-CPU tight loop. Back off on the stop event instead.
|
||||
q = getattr(process_registry, "completion_queue", None)
|
||||
if q is None:
|
||||
_DRAIN_STOP.wait(1.0)
|
||||
continue
|
||||
try:
|
||||
evt = q.get(timeout=1.0)
|
||||
except queue.Empty:
|
||||
# Nothing to drain this second — re-check the stop flag and loop.
|
||||
continue
|
||||
evt = process_registry.completion_queue.get(timeout=1.0)
|
||||
except Exception:
|
||||
# Unexpected queue failure: log it (not silent) and back off on the
|
||||
# stop event so a persistent error can't spin the thread hot.
|
||||
logger.warning(
|
||||
"bg_task_complete drain queue read failed", exc_info=True
|
||||
)
|
||||
_DRAIN_STOP.wait(1.0)
|
||||
# queue.Empty or transient — re-check stop flag and continue.
|
||||
continue
|
||||
if not isinstance(evt, dict):
|
||||
continue
|
||||
@@ -1708,58 +1238,6 @@ def _drain_loop() -> None:
|
||||
logger.warning("bg_task_complete event handling failed", exc_info=True)
|
||||
|
||||
|
||||
def recover_processes_for_webui(process_registry=None, get_session_fn=None) -> int:
|
||||
"""Recover core background processes and restore WebUI routing metadata.
|
||||
|
||||
The core gateway performs this during gateway startup, but this WebUI host
|
||||
previously started only the queue drain. That left checkpointed processes
|
||||
invisible after a WebUI restart.
|
||||
"""
|
||||
global _PROCESS_CHECKPOINT_RECOVERED, _PROCESS_RECOVERY_DONE
|
||||
if process_registry is None:
|
||||
try:
|
||||
from tools.process_registry import process_registry
|
||||
except ImportError:
|
||||
# Hermes Agent is optional in isolated WebUI/test environments.
|
||||
# The drain loop already treats a missing registry as unavailable;
|
||||
# startup recovery must preserve that fail-soft contract.
|
||||
logger.debug("process recovery unavailable: Hermes Agent is not installed")
|
||||
return 0
|
||||
if get_session_fn is None:
|
||||
from api.models import get_session as get_session_fn
|
||||
|
||||
with _PROCESS_RECOVERY_LOCK:
|
||||
if _PROCESS_RECOVERY_DONE:
|
||||
return 0
|
||||
|
||||
recovered = 0
|
||||
if not _PROCESS_CHECKPOINT_RECOVERED:
|
||||
recovered = process_registry.recover_from_checkpoint()
|
||||
_PROCESS_CHECKPOINT_RECOVERED = True
|
||||
|
||||
for row in process_registry.list_sessions():
|
||||
process_id = str(row.get("session_id") or "")
|
||||
if not process_id:
|
||||
continue
|
||||
try:
|
||||
proc_session = process_registry.get(process_id)
|
||||
session_key = str(getattr(proc_session, "session_key", "") or "")
|
||||
if not session_key or get_session_fn(session_key, metadata_only=True) is None:
|
||||
continue
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"Could not resolve recovered WebUI process %r",
|
||||
process_id,
|
||||
exc_info=True,
|
||||
)
|
||||
continue
|
||||
register_process_session(session_key, session_key)
|
||||
_PROCESS_RECOVERY_DONE = True
|
||||
if recovered:
|
||||
logger.info("Recovered %d background process(es) for WebUI", recovered)
|
||||
return recovered
|
||||
|
||||
|
||||
def register_process_session(session_key: str, session_id: str) -> None:
|
||||
"""Bind a process-registry session_key to a WebUI session_id.
|
||||
|
||||
@@ -1785,41 +1263,19 @@ def unregister_process_session(session_key: str) -> None:
|
||||
_cfg.PROCESS_SESSION_INDEX.pop(str(session_key), None)
|
||||
|
||||
|
||||
def forget_bg_task_completion_dedup(session_id: str) -> None:
|
||||
"""Drop a session's ``BG_TASK_COMPLETE_EVENTS_SEEN`` entry.
|
||||
|
||||
Called on session deletion so a session deleted while a completion is still
|
||||
pending (undelivered) — which the reaper's delivery-gated sweep deliberately
|
||||
keeps — can't leak its dedup set forever. Safe for unknown ids (no-op).
|
||||
"""
|
||||
if not session_id:
|
||||
return
|
||||
from api import config as _cfg
|
||||
|
||||
with _cfg.BG_TASK_COMPLETE_EVENTS_SEEN_LOCK:
|
||||
_cfg.BG_TASK_COMPLETE_EVENTS_SEEN.pop(str(session_id), None)
|
||||
|
||||
|
||||
def start_drain_thread() -> bool:
|
||||
"""Start the background drain thread idempotently. Returns True on first start."""
|
||||
global _DRAIN_THREAD
|
||||
with _THREAD_LIFECYCLE_LOCK:
|
||||
if _DRAIN_THREAD is not None and _DRAIN_THREAD.is_alive():
|
||||
return False
|
||||
try:
|
||||
recover_processes_for_webui()
|
||||
except Exception:
|
||||
# Recovery is best-effort. A corrupt checkpoint or transient I/O
|
||||
# error must not disable notifications for newly spawned tasks.
|
||||
logger.warning("background process recovery failed", exc_info=True)
|
||||
_DRAIN_STOP.clear()
|
||||
_DRAIN_THREAD = threading.Thread(
|
||||
target=_drain_loop,
|
||||
name="hermes-webui-bg-task-complete-drain",
|
||||
daemon=True,
|
||||
)
|
||||
_DRAIN_THREAD.start()
|
||||
return True
|
||||
if _DRAIN_THREAD is not None and _DRAIN_THREAD.is_alive():
|
||||
return False
|
||||
_DRAIN_STOP.clear()
|
||||
_DRAIN_THREAD = threading.Thread(
|
||||
target=_drain_loop,
|
||||
name="hermes-webui-bg-task-complete-drain",
|
||||
daemon=True,
|
||||
)
|
||||
_DRAIN_THREAD.start()
|
||||
return True
|
||||
|
||||
|
||||
def stop_drain_thread(timeout: float = 2.0) -> None:
|
||||
|
||||
+3
-8
@@ -86,16 +86,11 @@ def clear_pending(session_key: str) -> int:
|
||||
def _with_timeout_metadata(data: dict) -> dict:
|
||||
item = dict(data or {})
|
||||
requested_at = float(item.get("requested_at") or time.time())
|
||||
raw_timeout = item.get("timeout_seconds")
|
||||
timeout_seconds = int(raw_timeout) if raw_timeout is not None else DEFAULT_TIMEOUT_SECONDS
|
||||
timeout_seconds = int(item.get("timeout_seconds") or DEFAULT_TIMEOUT_SECONDS)
|
||||
expires_at = float(item.get("expires_at") or requested_at + timeout_seconds)
|
||||
item["requested_at"] = requested_at
|
||||
item["timeout_seconds"] = timeout_seconds
|
||||
if timeout_seconds <= 0:
|
||||
# Unlimited: no expiry. The frontend renders no countdown and the card
|
||||
# waits until the user answers (or the run is cancelled).
|
||||
item["expires_at"] = 0
|
||||
return item
|
||||
item["expires_at"] = float(item.get("expires_at") or requested_at + timeout_seconds)
|
||||
item["expires_at"] = expires_at
|
||||
return item
|
||||
|
||||
|
||||
|
||||
+7
-37
@@ -385,50 +385,20 @@ def _run_credits_command() -> str:
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def _load_config_for_moa_resolution() -> dict:
|
||||
from hermes_cli.config import load_config
|
||||
|
||||
cfg = load_config()
|
||||
return cfg if isinstance(cfg, dict) else {}
|
||||
|
||||
|
||||
def resolve_moa_config(preset: str | None = None) -> dict:
|
||||
def resolve_moa_config() -> dict:
|
||||
try:
|
||||
from hermes_cli.moa_config import moa_usage, normalize_moa_config
|
||||
except ImportError as exc:
|
||||
raise RuntimeError("MoA runtime unavailable (hermes-agent not installed or too old)") from exc
|
||||
try:
|
||||
from hermes_cli.moa_config import resolve_moa_preset
|
||||
except ImportError:
|
||||
resolve_moa_preset = None
|
||||
|
||||
try:
|
||||
cfg = _load_config_for_moa_resolution()
|
||||
moa_raw = cfg.get("moa") if isinstance(cfg, dict) else {}
|
||||
moa_cfg = normalize_moa_config(moa_raw)
|
||||
from hermes_cli.config import load_config
|
||||
cfg = load_config()
|
||||
moa_cfg = normalize_moa_config(cfg.get("moa") if isinstance(cfg, dict) else {})
|
||||
except Exception:
|
||||
moa_raw = {}
|
||||
moa_cfg = normalize_moa_config({})
|
||||
|
||||
preset_name = str(preset or moa_cfg.get("default_preset") or "default").strip()
|
||||
if preset_name not in (moa_cfg.get("presets") or {}):
|
||||
preset_name = str(moa_cfg.get("default_preset") or "default")
|
||||
|
||||
selected = {}
|
||||
if resolve_moa_preset is not None:
|
||||
try:
|
||||
selected = resolve_moa_preset(moa_raw, preset_name)
|
||||
if not isinstance(selected, dict):
|
||||
selected = {}
|
||||
except Exception:
|
||||
selected = {}
|
||||
preset_name = str(moa_cfg.get("default_preset") or "default")
|
||||
|
||||
resolved = dict(moa_cfg)
|
||||
resolved.update(selected)
|
||||
resolved["preset"] = preset_name
|
||||
resolved["usage"] = moa_usage()
|
||||
return resolved
|
||||
moa_cfg["preset"] = moa_cfg["default_preset"]
|
||||
moa_cfg["usage"] = moa_usage()
|
||||
return moa_cfg
|
||||
|
||||
|
||||
def execute_plugin_command(command: str) -> str:
|
||||
|
||||
@@ -1,111 +0,0 @@
|
||||
"""Compression-exhausted recovery metadata and intent helpers."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
|
||||
COMPRESSION_RECOVERY_TERMINAL_STATE = "compression_exhausted"
|
||||
COMPRESSION_RECOVERY_ACTION_START_FOCUSED = "start_focused_continuation"
|
||||
|
||||
|
||||
_GENERIC_CONTINUATION_INTENTS = frozenset(
|
||||
{
|
||||
"continue",
|
||||
"continue please",
|
||||
"go on",
|
||||
"keep going",
|
||||
"resume",
|
||||
"proceed",
|
||||
"carry on",
|
||||
"继续",
|
||||
"继续吧",
|
||||
"接着",
|
||||
"接着做",
|
||||
"继续做",
|
||||
"继续执行",
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _positive_int(value: Any) -> int:
|
||||
try:
|
||||
parsed = int(value or 0)
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
return parsed if parsed > 0 else 0
|
||||
|
||||
|
||||
def _normalize_intent_text(text: str) -> str:
|
||||
raw = str(text or "").strip().lower()
|
||||
if not raw:
|
||||
return ""
|
||||
# Keep CJK letters while dropping punctuation/emoji/noise around short intents.
|
||||
return re.sub(r"[\W_]+", " ", raw, flags=re.UNICODE).strip()
|
||||
|
||||
|
||||
def is_generic_continuation_intent(text: str) -> bool:
|
||||
"""Return True only for short, content-free continuation requests."""
|
||||
|
||||
normalized = _normalize_intent_text(text)
|
||||
if normalized in _GENERIC_CONTINUATION_INTENTS:
|
||||
return True
|
||||
# A single repeated word such as "continue continue" is still generic, but
|
||||
# longer prompts like "continue by summarizing file X" are substantive.
|
||||
parts = normalized.split()
|
||||
return bool(parts) and len(parts) <= 2 and all(part in _GENERIC_CONTINUATION_INTENTS for part in parts)
|
||||
|
||||
|
||||
def build_compression_recovery_payload(session, *, message: str = "", details: str = "") -> dict:
|
||||
"""Build the durable UI/route payload for a compression-exhausted turn."""
|
||||
|
||||
source_sid = str(getattr(session, "session_id", "") or "")
|
||||
last_prompt_tokens = _positive_int(getattr(session, "last_prompt_tokens", None))
|
||||
threshold_tokens = _positive_int(getattr(session, "threshold_tokens", None))
|
||||
context_length = _positive_int(getattr(session, "context_length", None))
|
||||
payload = {
|
||||
"terminal_state": COMPRESSION_RECOVERY_TERMINAL_STATE,
|
||||
"recommended_action": COMPRESSION_RECOVERY_ACTION_START_FOCUSED,
|
||||
"source_session_id": source_sid,
|
||||
"created_at": time.time(),
|
||||
"title": "Context compression exhausted",
|
||||
"summary": (
|
||||
"This run could not safely shrink the conversation enough to continue in place. "
|
||||
"Start a focused continuation, then describe the next narrow task."
|
||||
),
|
||||
"action_label": "Start focused continuation",
|
||||
"message": str(message or "").strip(),
|
||||
"details": str(details or "").strip()[:1200],
|
||||
"last_prompt_tokens": last_prompt_tokens,
|
||||
"threshold_tokens": threshold_tokens,
|
||||
"context_length": context_length,
|
||||
}
|
||||
return payload
|
||||
|
||||
|
||||
def stamp_compression_exhausted_recovery(session, *, message: str = "", details: str = "") -> dict:
|
||||
"""Persist recovery metadata on a session and return the message payload."""
|
||||
|
||||
payload = build_compression_recovery_payload(session, message=message, details=details)
|
||||
session.recommended_recovery_action = payload["recommended_action"]
|
||||
session.compression_recovery = payload
|
||||
return payload
|
||||
|
||||
|
||||
def compression_recovery_payload_for_session(session) -> dict | None:
|
||||
payload = getattr(session, "compression_recovery", None)
|
||||
if not isinstance(payload, dict):
|
||||
return None
|
||||
if payload.get("terminal_state") != COMPRESSION_RECOVERY_TERMINAL_STATE:
|
||||
return None
|
||||
action = str(payload.get("recommended_action") or getattr(session, "recommended_recovery_action", "") or "")
|
||||
if action != COMPRESSION_RECOVERY_ACTION_START_FOCUSED:
|
||||
return None
|
||||
return payload
|
||||
|
||||
|
||||
def clear_compression_recovery(session) -> None:
|
||||
session.recommended_recovery_action = None
|
||||
session.compression_recovery = {}
|
||||
+481
-3023
File diff suppressed because it is too large
Load Diff
@@ -1,301 +0,0 @@
|
||||
"""Crash-visibility hardening for the Hermes WebUI server (issue #4633).
|
||||
|
||||
Background
|
||||
----------
|
||||
``server.py`` runs a ``ThreadingHTTPServer`` with ``daemon_threads = True``.
|
||||
Under that model a *silent* process death is possible in ways that leave the
|
||||
log ending mid-request with no traceback, no shutdown-audit line, and no core
|
||||
dump:
|
||||
|
||||
* A **native** crash (segfault / abort in a C extension, e.g. sqlite, ssl,
|
||||
a provider SDK) terminates the interpreter with no Python-level traceback.
|
||||
Without ``faulthandler`` enabled, nothing is written.
|
||||
* An **uncaught Python exception in a daemon request/SSE/long-poll thread**
|
||||
is reported through ``threading.excepthook``. The stock hook prints to
|
||||
``sys.stderr`` with no context, and if that stream is gone the report is
|
||||
lost — so a single handler-thread exception can vanish silently.
|
||||
* An uncaught exception on the **main thread** goes through ``sys.excepthook``.
|
||||
|
||||
The paired *root cause* (a memory issue that eventually triggers the death)
|
||||
is being fixed separately in #4765. This module is purely about
|
||||
**diagnostics**: when the process dies we want a recorded reason, and one
|
||||
handler-thread exception must never disappear without a log line.
|
||||
|
||||
Design constraints
|
||||
------------------
|
||||
* Standard library only; no new dependencies.
|
||||
* Every hook is defensive — a logging hook must NEVER raise (a raising hook
|
||||
would itself become a new silent-failure mode).
|
||||
* Diagnostics are written **directly to the stderr stream** (which the
|
||||
bootstrap redirects into the WebUI log file) *and* mirrored through the
|
||||
``logging`` module. The WebUI configures no logging handlers, so an
|
||||
``INFO``/``ERROR`` record would otherwise be swallowed by logging's
|
||||
``lastResort`` filter (WARNING+ only) and never reach the log — the exact
|
||||
failure mode of #4633. The direct write guarantees the line lands.
|
||||
* ``install_crash_visibility()`` is idempotent and safe to call at import or
|
||||
startup time.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import atexit
|
||||
import faulthandler
|
||||
import logging
|
||||
import os
|
||||
import signal
|
||||
import sys
|
||||
import threading
|
||||
import traceback
|
||||
from typing import Optional, TextIO
|
||||
|
||||
__all__ = [
|
||||
"install_crash_visibility",
|
||||
"is_installed",
|
||||
"thread_excepthook",
|
||||
"main_excepthook",
|
||||
]
|
||||
|
||||
_LOGGER = logging.getLogger("server")
|
||||
|
||||
# Guard so repeated calls (e.g. import + explicit startup call, or a test that
|
||||
# re-imports) don't stack duplicate hooks / atexit registrations.
|
||||
_INSTALLED = False
|
||||
_INSTALL_LOCK = threading.Lock()
|
||||
|
||||
# Strong reference to the stream faulthandler writes into. faulthandler keeps
|
||||
# the file descriptor, but holding the object prevents premature GC of a
|
||||
# wrapper and lets tests introspect what we enabled against.
|
||||
_FAULT_STREAM: Optional[TextIO] = None
|
||||
|
||||
# Remember prior hooks so we can chain (and so tests can restore them).
|
||||
_PREV_THREAD_HOOK = None
|
||||
_PREV_SYS_HOOK = None
|
||||
|
||||
|
||||
def is_installed() -> bool:
|
||||
"""Return True once :func:`install_crash_visibility` has run."""
|
||||
return _INSTALLED
|
||||
|
||||
|
||||
def _resolve_stream(stream: Optional[TextIO]) -> TextIO:
|
||||
"""Pick a real, writable stream, defaulting to stderr."""
|
||||
if stream is not None:
|
||||
return stream
|
||||
err = sys.stderr
|
||||
if err is not None:
|
||||
return err
|
||||
# As a last resort, wrap fd 2 directly so we can still emit something.
|
||||
return os.fdopen(os.dup(2), "w", closefd=True)
|
||||
|
||||
|
||||
def _direct_write(text: str) -> None:
|
||||
"""Write a diagnostic line straight to the fault stream. Never raises."""
|
||||
try:
|
||||
stream = _FAULT_STREAM if _FAULT_STREAM is not None else sys.stderr
|
||||
if stream is None:
|
||||
return
|
||||
stream.write(text if text.endswith("\n") else text + "\n")
|
||||
try:
|
||||
stream.flush()
|
||||
except Exception:
|
||||
pass
|
||||
except Exception:
|
||||
# Absolutely nothing a diagnostic writer may raise should propagate.
|
||||
pass
|
||||
|
||||
|
||||
def _emit(level: int, message: str, *, exc_info=None) -> None:
|
||||
"""Emit a diagnostic via logging AND directly to the fault stream.
|
||||
|
||||
Never raises. The direct write is what guarantees visibility in the WebUI
|
||||
log (which configures no logging handlers); the logging call is what lets
|
||||
tests/caplog and any future handler capture the same record.
|
||||
"""
|
||||
text = message
|
||||
if exc_info is not None:
|
||||
try:
|
||||
etype, evalue, etb = exc_info
|
||||
tb_text = "".join(traceback.format_exception(etype, evalue, etb))
|
||||
text = f"{message}\n{tb_text}".rstrip()
|
||||
except Exception:
|
||||
text = message
|
||||
_direct_write(text)
|
||||
try:
|
||||
_LOGGER.log(level, message, exc_info=exc_info)
|
||||
except Exception:
|
||||
# If the logging subsystem itself blows up, the direct write above
|
||||
# already captured the diagnostic — swallow so we never re-crash here.
|
||||
pass
|
||||
|
||||
|
||||
def thread_excepthook(args) -> None:
|
||||
"""threading.excepthook: log any uncaught exception in a daemon/handler thread.
|
||||
|
||||
``args`` is a ``threading.ExceptHookArgs`` (exc_type, exc_value,
|
||||
exc_traceback, thread). A normal thread shutdown may pass exc_type=None.
|
||||
"""
|
||||
try:
|
||||
exc_type = getattr(args, "exc_type", None)
|
||||
exc_value = getattr(args, "exc_value", None)
|
||||
exc_tb = getattr(args, "exc_traceback", None)
|
||||
thread = getattr(args, "thread", None)
|
||||
|
||||
# SystemExit inside a thread is an intentional stop, not a crash.
|
||||
if exc_type is None or issubclass(exc_type, SystemExit):
|
||||
return
|
||||
|
||||
thread_name = getattr(thread, "name", None) or "unknown"
|
||||
thread_ident = getattr(thread, "ident", None)
|
||||
daemon = getattr(thread, "daemon", None)
|
||||
_emit(
|
||||
logging.ERROR,
|
||||
"[crash-visibility] uncaught exception in thread "
|
||||
"name=%s ident=%s daemon=%s type=%s"
|
||||
% (thread_name, thread_ident, daemon, getattr(exc_type, "__name__", exc_type)),
|
||||
exc_info=(exc_type, exc_value, exc_tb),
|
||||
)
|
||||
except Exception:
|
||||
# A hook that raises would re-introduce the silent-death class of bug.
|
||||
try:
|
||||
_direct_write("[crash-visibility] thread_excepthook failed to log an exception")
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
# Chain to any previously-installed hook, but only if it isn't the
|
||||
# stock one (which just prints to stderr again — we already did better).
|
||||
prev = _PREV_THREAD_HOOK
|
||||
try:
|
||||
if prev is not None and prev is not threading.__excepthook__:
|
||||
prev(args)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def main_excepthook(exc_type, exc_value, exc_tb) -> None:
|
||||
"""sys.excepthook: log any uncaught exception on the main thread."""
|
||||
try:
|
||||
is_keyboard_interrupt = bool(
|
||||
isinstance(exc_type, type) and issubclass(exc_type, KeyboardInterrupt)
|
||||
)
|
||||
if is_keyboard_interrupt:
|
||||
# Preserve default Ctrl-C behaviour: no scary traceback dump.
|
||||
prev = _PREV_SYS_HOOK or sys.__excepthook__
|
||||
try:
|
||||
prev(exc_type, exc_value, exc_tb)
|
||||
except Exception:
|
||||
pass
|
||||
return
|
||||
_emit(
|
||||
logging.CRITICAL,
|
||||
"[crash-visibility] uncaught exception on main thread type=%s"
|
||||
% (getattr(exc_type, "__name__", exc_type),),
|
||||
exc_info=(exc_type, exc_value, exc_tb),
|
||||
)
|
||||
except Exception:
|
||||
try:
|
||||
_direct_write("[crash-visibility] main_excepthook failed to log an exception")
|
||||
except Exception:
|
||||
pass
|
||||
finally:
|
||||
prev = _PREV_SYS_HOOK
|
||||
try:
|
||||
if prev is not None and prev is not sys.__excepthook__:
|
||||
prev(exc_type, exc_value, exc_tb)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _exit_audit() -> None:
|
||||
"""atexit hook: record that the process is exiting.
|
||||
|
||||
A plain interpreter exit (return from main, sys.exit, or the tail of a
|
||||
fatal error that still unwinds atexit) leaves a breadcrumb so we can tell a
|
||||
*clean* shutdown apart from a truly silent death — if the log ends with no
|
||||
exit-audit line at all, the process was killed without unwinding
|
||||
(OOM kill / SIGKILL / native abort), which itself narrows the diagnosis.
|
||||
"""
|
||||
try:
|
||||
_emit(
|
||||
logging.INFO,
|
||||
"[crash-visibility] process exit pid=%s thread=%s"
|
||||
% (os.getpid(), threading.current_thread().name),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def install_crash_visibility(
|
||||
*,
|
||||
stream: Optional[TextIO] = None,
|
||||
enable_faulthandler: bool = True,
|
||||
register_sigusr1_dump: bool = True,
|
||||
) -> bool:
|
||||
"""Install faulthandler + excepthooks + exit audit. Idempotent.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
stream:
|
||||
Where diagnostics are written. Defaults to ``sys.stderr`` (redirected
|
||||
into the WebUI log file by the bootstrap).
|
||||
enable_faulthandler:
|
||||
Enable ``faulthandler`` so a native segfault/abort dumps a C-level
|
||||
traceback of every thread instead of vanishing.
|
||||
register_sigusr1_dump:
|
||||
On POSIX, register ``SIGUSR1`` to dump all thread stacks on demand —
|
||||
invaluable for diagnosing a *hang* (send ``kill -USR1 <pid>``) as
|
||||
opposed to a crash. Ignored where SIGUSR1 is unavailable (Windows).
|
||||
|
||||
Returns True if it installed on this call, False if it was already active.
|
||||
"""
|
||||
global _INSTALLED, _FAULT_STREAM, _PREV_THREAD_HOOK, _PREV_SYS_HOOK
|
||||
|
||||
with _INSTALL_LOCK:
|
||||
if _INSTALLED:
|
||||
return False
|
||||
|
||||
_FAULT_STREAM = _resolve_stream(stream)
|
||||
|
||||
# 1) faulthandler: native crash → C-level traceback of all threads.
|
||||
if enable_faulthandler:
|
||||
try:
|
||||
faulthandler.enable(file=_FAULT_STREAM, all_threads=True)
|
||||
except Exception:
|
||||
# Some frozen/embedded builds restrict faulthandler; degrade
|
||||
# gracefully rather than blocking startup.
|
||||
_direct_write("[crash-visibility] faulthandler.enable() failed")
|
||||
|
||||
# 1b) On-demand all-thread stack dump for diagnosing hangs.
|
||||
if register_sigusr1_dump:
|
||||
sigusr1 = getattr(signal, "SIGUSR1", None)
|
||||
if sigusr1 is not None:
|
||||
try:
|
||||
# A diagnostic signal must not inherit SIGUSR1's default
|
||||
# terminate action after the dump. Chaining here made a
|
||||
# hang probe kill the server and trigger service restarts.
|
||||
faulthandler.register(
|
||||
sigusr1, file=_FAULT_STREAM, all_threads=True, chain=False
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
# 2) threading.excepthook: daemon/handler-thread crash → logged.
|
||||
try:
|
||||
_PREV_THREAD_HOOK = threading.excepthook
|
||||
threading.excepthook = thread_excepthook
|
||||
except Exception:
|
||||
_direct_write("[crash-visibility] failed to install threading.excepthook")
|
||||
|
||||
# 3) sys.excepthook: main-thread crash → logged.
|
||||
try:
|
||||
_PREV_SYS_HOOK = sys.excepthook
|
||||
sys.excepthook = main_excepthook
|
||||
except Exception:
|
||||
_direct_write("[crash-visibility] failed to install sys.excepthook")
|
||||
|
||||
# 4) atexit exit audit: breadcrumb on clean/unwound exit.
|
||||
try:
|
||||
atexit.register(_exit_audit)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
_INSTALLED = True
|
||||
return True
|
||||
@@ -1,241 +0,0 @@
|
||||
"""Extension sidecar proxy authentication (token-v1).
|
||||
|
||||
A loopback-sidecar extension runs a stdlib HTTP server on 127.0.0.1:<port>. The
|
||||
browser reaches it ONLY through core's consent-gated proxy
|
||||
(``/api/extensions/{id}/sidecar/*``). But the loopback port is reachable by any
|
||||
local process, and the proxy strips every inbound credential before forwarding
|
||||
(see ``_extension_sidecar_proxy_request_headers`` in ``api/routes.py``), so the
|
||||
sidecar cannot tell a proxied request from a direct one.
|
||||
|
||||
This module mints a per-extension shared secret that core injects on every
|
||||
forwarded request (header ``X-Hermes-Sidecar-Token``) and the sidecar validates.
|
||||
It converts "anyone who can send a loopback TCP packet" (other-UID users, host
|
||||
containers, sandboxed network-only processes) into "processes that can read the
|
||||
user's state dir" — the same protection level core's own signing key already has
|
||||
(``.pbkdf2_key`` / ``.signing_key`` are 0600 files in the same directory).
|
||||
|
||||
SCOPE (be honest — this is documented in the contract too): the token does NOT
|
||||
defend against arbitrary same-UID code. A same-user process can read the token,
|
||||
read core's signing key, or just run the sidecar's underlying tool directly. No
|
||||
mechanism available here (token, HMAC, nonce, UDS with 0600) changes that.
|
||||
|
||||
Design notes (from the reviewed design doc, §9.2):
|
||||
* Per-extension token file under ``STATE_DIR/sidecar-auth/<ext-id>.token``.
|
||||
* Unlike ``auth._load_key``, we NEVER return an in-memory-only token: if the
|
||||
file cannot be persisted+re-read, we report "unavailable" so core fails
|
||||
closed (503) rather than injecting a secret no sidecar can ever read.
|
||||
* The token is re-read from disk per request (fingerprint-cached on
|
||||
inode/mtime/ctime/size) so rotation / deletion takes effect with no restart,
|
||||
and a stale cached token can't keep validating after the file changes.
|
||||
* Cross-process mint writes a unique temporary file, then publishes it with
|
||||
an atomic ``os.link`` create-or-fail operation. Concurrent first-mints
|
||||
converge on a single winning file; losers re-read the winner rather than
|
||||
clobbering it.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Dict, Optional, Tuple
|
||||
|
||||
# Keep this grammar identical to ``_EXTENSION_ID_RE`` in ``api/extensions.py`` —
|
||||
# a narrower pattern here silently 503s legally-named extensions (uppercase,
|
||||
# dots, up to 128 chars). The pattern is filesystem-safe as a filename: no path
|
||||
# separators, cannot start with a dot, so no dotfile / ".."-prefix tricks.
|
||||
_EXT_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$")
|
||||
# Injected/validated tokens are exactly secrets.token_urlsafe output: url-safe
|
||||
# base64 alphabet. Reject anything else (bounds length, and stops a malformed
|
||||
# token file's contents from reaching a header where a ValueError could echo it).
|
||||
_TOKEN_RE = re.compile(r"^[A-Za-z0-9_-]{16,256}$")
|
||||
|
||||
_TOKEN_DIR_NAME = "sidecar-auth"
|
||||
_TOKEN_BYTES = 32 # secrets.token_urlsafe(32) -> ~43 url-safe chars
|
||||
|
||||
# Per-request read cache: ext_id -> (token, fingerprint). Guarded by _LOCK.
|
||||
_LOCK = threading.Lock()
|
||||
_CACHE: Dict[str, Tuple[str, Tuple]] = {}
|
||||
|
||||
|
||||
def _token_dir() -> Path:
|
||||
# Resolve STATE_DIR dynamically (mirrors ``_extension_state_dir`` in
|
||||
# api/extensions.py) so tests and relocated installs see the current dir
|
||||
# rather than the value cached at import time.
|
||||
from api.config import STATE_DIR
|
||||
return STATE_DIR / _TOKEN_DIR_NAME
|
||||
|
||||
|
||||
def _token_path(ext_id: str) -> Path:
|
||||
return _token_dir() / f"{ext_id}.token"
|
||||
|
||||
|
||||
def _valid_ext_id(ext_id: object) -> bool:
|
||||
# Use fullmatch (not match) so a trailing newline can't slip through: in
|
||||
# Python's ``re`` a ``$`` anchor also matches just before a final ``\n``,
|
||||
# so ``match`` would accept ``"templates\n"`` and _token_path would build a
|
||||
# filename containing a literal newline. fullmatch anchors truly to the end.
|
||||
# Validate the exact value that _token_path() uses. Trimming only for the
|
||||
# check would accept one ID but construct a filename from a different one.
|
||||
return isinstance(ext_id, str) and bool(_EXT_ID_RE.fullmatch(ext_id))
|
||||
|
||||
|
||||
def _fingerprint(path: Path) -> Optional[Tuple]:
|
||||
try:
|
||||
st = path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return (st.st_ino, st.st_dev, st.st_mtime_ns, st.st_ctime_ns, st.st_size)
|
||||
|
||||
|
||||
def _read_token_file(path: Path) -> Optional[str]:
|
||||
"""Return the token text iff it is present and well-formed, else None
|
||||
(fail closed). Only a valid token-v1 shape is ever returned, so malformed
|
||||
file contents can never reach a header. Thin wrapper over the stable reader."""
|
||||
tok, _fp = _stable_read(path)
|
||||
return tok
|
||||
|
||||
|
||||
def _stable_read(path: Path) -> Tuple[Optional[str], Optional[Tuple]]:
|
||||
"""Read the token together with a fingerprint that is verified to bracket the
|
||||
read (fingerprint before AND after; retry on any mid-read change). Guarantees
|
||||
the returned (token, fingerprint) pair reflects a single consistent on-disk
|
||||
state — never new content under an old stat key, and never a value that
|
||||
changed underneath us. Returns (None, None) when absent/malformed/racing."""
|
||||
for _ in range(4): # bounded retry; a file being rotated settles fast
|
||||
fp_before = _fingerprint(path)
|
||||
if fp_before is None:
|
||||
return (None, None)
|
||||
try:
|
||||
raw = path.read_text(encoding="utf-8")
|
||||
except (OSError, UnicodeDecodeError):
|
||||
return (None, None)
|
||||
fp_after = _fingerprint(path)
|
||||
if fp_after != fp_before:
|
||||
continue # changed mid-read — retry for a consistent snapshot
|
||||
tok = raw.strip()
|
||||
if not tok or not _TOKEN_RE.match(tok):
|
||||
return (None, None)
|
||||
return (tok, fp_before)
|
||||
return (None, None) # never settled — fail closed
|
||||
|
||||
|
||||
def ensure_token(ext_id: str) -> Optional[str]:
|
||||
"""Get-or-create the per-extension token, returning it ONLY if it is durably
|
||||
persisted and re-readable from disk. Returns None if it can't be persisted
|
||||
(caller must then treat the sidecar proxy as unavailable / 503) — we never
|
||||
hand back an ephemeral token a sidecar could never read.
|
||||
|
||||
Idempotent + atomic across processes: the token is written to a unique temp
|
||||
file, then published with an atomic hard-link create-or-fail operation, so
|
||||
the final path is only ever visible fully written and is never clobbered.
|
||||
One concurrent writer wins; every loser reads that persisted winner.
|
||||
"""
|
||||
if not _valid_ext_id(ext_id):
|
||||
return None
|
||||
path = _token_path(ext_id)
|
||||
|
||||
existing = _read_token_file(path)
|
||||
if existing is not None:
|
||||
return existing
|
||||
|
||||
with _LOCK:
|
||||
existing = _read_token_file(path)
|
||||
if existing is not None:
|
||||
return existing
|
||||
token = secrets.token_urlsafe(_TOKEN_BYTES)
|
||||
tmp: Optional[Path] = None
|
||||
try:
|
||||
d = _token_dir()
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
try:
|
||||
os.chmod(d, 0o700)
|
||||
except OSError:
|
||||
pass # best-effort dir hardening; file perms are the real guard
|
||||
# Write to a unique temp, fsync, then publish with atomic
|
||||
# create-or-fail (os.link, the repo's TOCTOU-safe idiom — see
|
||||
# session_recovery.py:627). Unlike os.replace, link does NOT clobber:
|
||||
# if another process already published, ours fails, we drop our temp,
|
||||
# and everyone reads the single winning file. This closes both the
|
||||
# empty-file-exposure race AND the cross-process clobber race.
|
||||
tmp = path.with_name(f".{path.name}.{os.getpid()}.{threading.get_ident()}.tmp")
|
||||
fd = os.open(str(tmp), os.O_WRONLY | os.O_CREAT | os.O_TRUNC, 0o600)
|
||||
try:
|
||||
os.write(fd, token.encode("utf-8"))
|
||||
os.fsync(fd)
|
||||
finally:
|
||||
os.close(fd)
|
||||
try:
|
||||
os.link(str(tmp), str(path))
|
||||
try:
|
||||
os.chmod(path, 0o600)
|
||||
except OSError:
|
||||
pass
|
||||
except FileExistsError:
|
||||
pass # another writer won; we return the winner via _stable_read below
|
||||
finally:
|
||||
try:
|
||||
tmp.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
tmp = None
|
||||
except OSError:
|
||||
# Persistence failed → report unavailable. Do NOT return the
|
||||
# in-memory token (the _load_key trap: core would inject a secret the
|
||||
# sidecar can never read → permanent 401 that looks like a mismatch).
|
||||
if tmp is not None:
|
||||
try:
|
||||
tmp.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
return None
|
||||
# Return whatever is durably on disk now — under a concurrent mint this
|
||||
# may be another writer's token, which is correct (all readers converge
|
||||
# on the winning file). Only a persisted, well-formed token is returned.
|
||||
persisted, fp = _stable_read(path)
|
||||
if persisted is not None and fp is not None:
|
||||
_CACHE[ext_id] = (persisted, fp)
|
||||
return persisted
|
||||
|
||||
|
||||
def current_token(ext_id: str) -> Optional[str]:
|
||||
"""Return the current on-disk token for validation/injection, re-reading the
|
||||
file when its fingerprint changed since last read — so rotation + deletion
|
||||
take effect immediately and a stale cached token stops validating. Does NOT
|
||||
mint. Returns None when no token exists (fail closed)."""
|
||||
if not _valid_ext_id(ext_id):
|
||||
return None
|
||||
path = _token_path(ext_id)
|
||||
fp = _fingerprint(path)
|
||||
if fp is None:
|
||||
with _LOCK:
|
||||
_CACHE.pop(ext_id, None)
|
||||
return None
|
||||
with _LOCK:
|
||||
cached = _CACHE.get(ext_id)
|
||||
if cached is not None and cached[1] == fp:
|
||||
return cached[0]
|
||||
tok, tok_fp = _stable_read(path) # bracketed read: content matches tok_fp
|
||||
with _LOCK:
|
||||
if tok is None or tok_fp is None:
|
||||
_CACHE.pop(ext_id, None)
|
||||
else:
|
||||
_CACHE[ext_id] = (tok, tok_fp)
|
||||
return tok
|
||||
|
||||
|
||||
def reset_token(ext_id: str) -> Optional[str]:
|
||||
"""Rotate: delete then re-mint. Returns the new token or None on failure.
|
||||
(Recovery entry point; also usable by a future 'reset token' action.)"""
|
||||
if not _valid_ext_id(ext_id):
|
||||
return None
|
||||
with _LOCK:
|
||||
_CACHE.pop(ext_id, None)
|
||||
try:
|
||||
_token_path(ext_id).unlink()
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
except OSError:
|
||||
return None
|
||||
return ensure_token(ext_id)
|
||||
+30
-718
@@ -6,13 +6,10 @@ It is disabled by default and never executes or fetches third-party URLs.
|
||||
"""
|
||||
|
||||
import html
|
||||
import http.client
|
||||
import json
|
||||
import math
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import socket
|
||||
import threading
|
||||
from pathlib import Path
|
||||
from typing import Any, Dict, List, Optional, Set, Tuple
|
||||
@@ -21,11 +18,7 @@ import hashlib
|
||||
import io
|
||||
import time
|
||||
import zipfile
|
||||
from urllib.request import (
|
||||
HTTPRedirectHandler,
|
||||
HTTPSHandler,
|
||||
build_opener,
|
||||
)
|
||||
from urllib.request import HTTPRedirectHandler, build_opener, urlopen
|
||||
|
||||
from api.helpers import _security_headers, j
|
||||
|
||||
@@ -66,14 +59,6 @@ class ExtensionInstallError(Exception):
|
||||
self.status = status
|
||||
|
||||
|
||||
class ExtensionSidecarProxyError(Exception):
|
||||
"""Sanitized sidecar proxy error safe to return to the browser."""
|
||||
|
||||
def __init__(self, message: str, status: int = 400):
|
||||
super().__init__(message)
|
||||
self.status = status
|
||||
|
||||
|
||||
EXTENSION_ROUTE_PREFIX = "/extensions/"
|
||||
_EXTENSION_DIR_ENV = "HERMES_WEBUI_EXTENSION_DIR"
|
||||
_EXTENSION_SCRIPT_URLS_ENV = "HERMES_WEBUI_EXTENSION_SCRIPT_URLS"
|
||||
@@ -86,12 +71,9 @@ _LOOPBACK_SIDECAR_HOSTS = {"127.0.0.1", "localhost", "::1"}
|
||||
_EXTENSION_STATE_FILENAME = "extension-overrides.json"
|
||||
_MAX_EXTENSION_STATE_BYTES = 32 * 1024
|
||||
_MAX_DISABLED_EXTENSION_IDS = 512
|
||||
_MAX_SIDECAR_PROXY_CONSENTS = 512
|
||||
_EXTENSION_ID_RE = re.compile(r"^[A-Za-z0-9][A-Za-z0-9._-]{0,127}$")
|
||||
_EXTENSION_SETTINGS_KEY_RE = re.compile(r"^[A-Za-z][A-Za-z0-9._-]{0,63}$")
|
||||
_EXTENSION_STATE_WARNING_SOURCE = "extension_state"
|
||||
_EXTENSION_STATE_LOCK = threading.Lock()
|
||||
_EXTENSION_SETTING_TYPES = {"boolean", "string", "number", "integer", "enum"}
|
||||
|
||||
_GALLERY_INSTALL_STATE_FILENAME = "extension-install-manifest.json"
|
||||
_MAX_INSTALL_MANIFEST_BYTES = 128 * 1024
|
||||
@@ -114,80 +96,9 @@ class _AllowlistRedirectHandler(HTTPRedirectHandler):
|
||||
return super().redirect_request(req, fp, code, msg, headers, newurl)
|
||||
|
||||
|
||||
def _connect_ipv4_first(address, timeout=socket._GLOBAL_DEFAULT_TIMEOUT, source_address=None):
|
||||
"""Connect to *address*, preferring IPv4 to avoid dead-route stalls.
|
||||
|
||||
Some networks advertise IPv6 routes (router advertisements, tunnel brokers)
|
||||
that do not actually reach the public internet. ``socket.create_connection``
|
||||
follows the OS address ordering, which typically tries IPv6 first; each dead
|
||||
address stalls for the full TCP timeout (~75–130 s) before falling back to
|
||||
IPv4. This makes gallery installs and registry fetches appear to hang until
|
||||
the browser or curl timeout kills them.
|
||||
|
||||
We iterate families in IPv4→IPv6 order so the common case (IPv4 works, IPv6
|
||||
is a broken route) completes instantly. IPv6 is still tried as a fallback so
|
||||
IPv6-only networks are unaffected.
|
||||
|
||||
Signature matches ``socket.create_connection`` including the
|
||||
``_GLOBAL_DEFAULT_TIMEOUT`` sentinel so callers that rely on the stdlib
|
||||
default-timeout behaviour are not broken.
|
||||
"""
|
||||
host, port = address
|
||||
if timeout is socket._GLOBAL_DEFAULT_TIMEOUT:
|
||||
timeout = socket.getdefaulttimeout()
|
||||
last_error: OSError | None = None
|
||||
for family in (socket.AF_INET, socket.AF_INET6):
|
||||
try:
|
||||
infos = socket.getaddrinfo(host, port, family, socket.SOCK_STREAM)
|
||||
except socket.gaierror:
|
||||
continue
|
||||
for _fam, socktype, proto, _canon, sockaddr in infos:
|
||||
sock = None
|
||||
try:
|
||||
sock = socket.socket(_fam, socktype, proto)
|
||||
if timeout is not None:
|
||||
sock.settimeout(timeout)
|
||||
if source_address is not None:
|
||||
sock.bind(source_address)
|
||||
sock.connect(sockaddr)
|
||||
return sock
|
||||
except OSError as exc:
|
||||
last_error = exc
|
||||
if sock is not None:
|
||||
sock.close()
|
||||
if last_error is not None:
|
||||
raise last_error
|
||||
raise OSError(f"Could not connect to {host}:{port}")
|
||||
|
||||
|
||||
class _IPv4FirstHTTPSConnection(http.client.HTTPSConnection):
|
||||
"""HTTPS connection that resolves IPv4 addresses before IPv6.
|
||||
|
||||
Overrides the class-level ``_create_connection`` that
|
||||
``HTTPConnection.connect()`` calls — the stdlib extension point designed for
|
||||
exactly this kind of per-class customisation (added in Python 3.9,
|
||||
bpo-37830). No global state is mutated, so the change is thread-safe under
|
||||
``ThreadingHTTPServer``.
|
||||
"""
|
||||
|
||||
_create_connection = staticmethod(_connect_ipv4_first)
|
||||
|
||||
|
||||
class _IPv4FirstHTTPSHandler(HTTPSHandler):
|
||||
"""HTTPS handler that builds IPv4-first connections for gallery downloads."""
|
||||
|
||||
def https_open(self, req):
|
||||
return self.do_open(_IPv4FirstHTTPSConnection, req)
|
||||
|
||||
|
||||
def _build_gallery_opener():
|
||||
"""Opener with IPv4-first downloads and redirect-host allowlist."""
|
||||
return build_opener(_IPv4FirstHTTPSHandler, _AllowlistRedirectHandler)
|
||||
|
||||
|
||||
def _safe_download(url: str, max_bytes: int, timeout: int = 30) -> bytes:
|
||||
"""Download from an allowlisted host, rejecting cross-host redirects."""
|
||||
opener = _build_gallery_opener()
|
||||
opener = build_opener(_AllowlistRedirectHandler)
|
||||
resp = opener.open(url, timeout=timeout)
|
||||
try:
|
||||
return resp.read(max_bytes + 1)
|
||||
@@ -337,7 +248,7 @@ def _extension_state_file() -> Path:
|
||||
|
||||
|
||||
def _empty_extension_state() -> Dict[str, Any]:
|
||||
return {"version": 1, "disabled_extensions": [], "sidecar_proxy_consents": {}}
|
||||
return {"version": 1, "disabled_extensions": []}
|
||||
|
||||
|
||||
def _load_extension_state(diagnostics: Optional[Dict[str, Any]] = None) -> Dict[str, Any]:
|
||||
@@ -387,44 +298,11 @@ def _load_extension_state(diagnostics: Optional[Dict[str, Any]] = None) -> Dict[
|
||||
diagnostics, "extension_state_truncated", _EXTENSION_STATE_WARNING_SOURCE
|
||||
)
|
||||
break
|
||||
consents_raw = parsed.get("sidecar_proxy_consents", {})
|
||||
consents: Dict[str, str] = {}
|
||||
consents_invalid = False
|
||||
if consents_raw is not None:
|
||||
if not isinstance(consents_raw, dict):
|
||||
invalid = True
|
||||
consents_invalid = True
|
||||
else:
|
||||
for raw_ext_id, raw_origin in consents_raw.items():
|
||||
if not _valid_extension_id(raw_ext_id):
|
||||
invalid = True
|
||||
consents_invalid = True
|
||||
continue
|
||||
origin = _normalize_loopback_sidecar_origin(raw_origin)
|
||||
if origin is None:
|
||||
invalid = True
|
||||
consents_invalid = True
|
||||
continue
|
||||
ext_id = str(raw_ext_id).strip()
|
||||
if ext_id in consents:
|
||||
continue
|
||||
consents[ext_id] = origin
|
||||
if len(consents) >= _MAX_SIDECAR_PROXY_CONSENTS:
|
||||
_add_diagnostic_warning(
|
||||
diagnostics, "extension_state_truncated", _EXTENSION_STATE_WARNING_SOURCE
|
||||
)
|
||||
break
|
||||
if consents_invalid:
|
||||
consents = {}
|
||||
if invalid:
|
||||
_add_diagnostic_warning(
|
||||
diagnostics, "extension_state_invalid_entries", _EXTENSION_STATE_WARNING_SOURCE
|
||||
)
|
||||
return {
|
||||
"version": 1,
|
||||
"disabled_extensions": disabled,
|
||||
"sidecar_proxy_consents": consents,
|
||||
}
|
||||
return {"version": 1, "disabled_extensions": disabled}
|
||||
|
||||
|
||||
def _write_extension_state(state: Dict[str, Any]) -> None:
|
||||
@@ -443,26 +321,7 @@ def _write_extension_state(state: Dict[str, Any]) -> None:
|
||||
disabled.append(ext_id)
|
||||
if len(disabled) >= _MAX_DISABLED_EXTENSION_IDS:
|
||||
break
|
||||
consents_raw = state.get("sidecar_proxy_consents", {})
|
||||
consents: Dict[str, str] = {}
|
||||
if isinstance(consents_raw, dict):
|
||||
for raw_ext_id, raw_origin in consents_raw.items():
|
||||
if not _valid_extension_id(raw_ext_id):
|
||||
continue
|
||||
origin = _normalize_loopback_sidecar_origin(raw_origin)
|
||||
if origin is None:
|
||||
continue
|
||||
ext_id = str(raw_ext_id).strip()
|
||||
if ext_id in consents:
|
||||
continue
|
||||
consents[ext_id] = origin
|
||||
if len(consents) >= _MAX_SIDECAR_PROXY_CONSENTS:
|
||||
break
|
||||
payload = {
|
||||
"version": 1,
|
||||
"disabled_extensions": disabled,
|
||||
"sidecar_proxy_consents": consents,
|
||||
}
|
||||
payload = {"version": 1, "disabled_extensions": disabled}
|
||||
target = _extension_state_file()
|
||||
target.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = target.with_name(f".{target.name}.{os.getpid()}.{threading.get_ident()}.tmp")
|
||||
@@ -662,122 +521,6 @@ def _manifest_entry_text(entry: Dict[str, object], key: str) -> str:
|
||||
return ""
|
||||
return value.strip()
|
||||
|
||||
def _manifest_entry_storage_owned(entry: Dict[str, object]) -> bool:
|
||||
permissions = entry.get("permissions")
|
||||
if not isinstance(permissions, dict):
|
||||
return False
|
||||
storage = permissions.get("storage")
|
||||
return isinstance(storage, dict) and storage.get("owned") is True
|
||||
|
||||
def _settings_text(value: object, *, max_len: int = 160) -> str:
|
||||
if not isinstance(value, str):
|
||||
return ""
|
||||
text = value.strip()
|
||||
return text[:max_len]
|
||||
|
||||
def _normalize_enum_options(options: object) -> Optional[List[Dict[str, str]]]:
|
||||
if not isinstance(options, list) or not options:
|
||||
return None
|
||||
normalized: List[Dict[str, str]] = []
|
||||
seen: Set[str] = set()
|
||||
for option in options:
|
||||
if isinstance(option, str):
|
||||
value = option.strip()
|
||||
label = value
|
||||
elif isinstance(option, dict):
|
||||
raw_value = option.get("value")
|
||||
if not isinstance(raw_value, str):
|
||||
return None
|
||||
value = raw_value.strip()
|
||||
label = _settings_text(option.get("label")) or value
|
||||
else:
|
||||
return None
|
||||
if not value or value in seen:
|
||||
return None
|
||||
seen.add(value)
|
||||
normalized.append({"value": value, "label": label})
|
||||
return normalized
|
||||
|
||||
_SETTINGS_DEFAULT_MISSING = object()
|
||||
|
||||
def _normalize_settings_default(field_type: str, raw_default: object, options: Optional[List[Dict[str, str]]] = None) -> Tuple[bool, object]:
|
||||
if field_type == "boolean":
|
||||
if raw_default is _SETTINGS_DEFAULT_MISSING:
|
||||
return True, False
|
||||
return (True, raw_default) if isinstance(raw_default, bool) else (False, None)
|
||||
if field_type == "string":
|
||||
if raw_default is _SETTINGS_DEFAULT_MISSING:
|
||||
return True, ""
|
||||
return (True, raw_default) if isinstance(raw_default, str) else (False, None)
|
||||
if field_type == "number":
|
||||
if raw_default is _SETTINGS_DEFAULT_MISSING:
|
||||
return True, 0
|
||||
return (
|
||||
(True, raw_default)
|
||||
if isinstance(raw_default, (int, float)) and not isinstance(raw_default, bool) and math.isfinite(raw_default)
|
||||
else (False, None)
|
||||
)
|
||||
if field_type == "integer":
|
||||
if raw_default is _SETTINGS_DEFAULT_MISSING:
|
||||
return True, 0
|
||||
return (True, raw_default) if isinstance(raw_default, int) and not isinstance(raw_default, bool) else (False, None)
|
||||
if field_type == "enum" and options:
|
||||
values = [option["value"] for option in options]
|
||||
if raw_default is _SETTINGS_DEFAULT_MISSING:
|
||||
return True, values[0]
|
||||
return (True, raw_default) if isinstance(raw_default, str) and raw_default in values else (False, None)
|
||||
return False, None
|
||||
|
||||
def _settings_schema_values(raw_schema: object) -> List[object]:
|
||||
if isinstance(raw_schema, list):
|
||||
return raw_schema
|
||||
if isinstance(raw_schema, dict) and isinstance(raw_schema.get("fields"), list):
|
||||
return raw_schema["fields"]
|
||||
return []
|
||||
|
||||
def _sanitize_settings_schema(entry: Dict[str, object]) -> List[Dict[str, object]]:
|
||||
if not _manifest_entry_storage_owned(entry):
|
||||
return []
|
||||
fields: List[Dict[str, object]] = []
|
||||
seen_keys: Set[str] = set()
|
||||
for raw_field in _settings_schema_values(entry.get("settings_schema")):
|
||||
if not isinstance(raw_field, dict):
|
||||
continue
|
||||
if raw_field.get("sensitive") is True:
|
||||
continue
|
||||
key = raw_field.get("key")
|
||||
if not isinstance(key, str):
|
||||
continue
|
||||
key = key.strip()
|
||||
if not _EXTENSION_SETTINGS_KEY_RE.fullmatch(key):
|
||||
continue
|
||||
field_type = raw_field.get("type")
|
||||
if not isinstance(field_type, str):
|
||||
continue
|
||||
field_type = field_type.strip().lower()
|
||||
if field_type not in _EXTENSION_SETTING_TYPES:
|
||||
continue
|
||||
options = _normalize_enum_options(raw_field.get("options")) if field_type == "enum" else None
|
||||
if field_type == "enum" and options is None:
|
||||
continue
|
||||
ok, default = _normalize_settings_default(field_type, raw_field.get("default", _SETTINGS_DEFAULT_MISSING), options)
|
||||
if not ok:
|
||||
continue
|
||||
if key in seen_keys:
|
||||
continue
|
||||
seen_keys.add(key)
|
||||
field: Dict[str, object] = {
|
||||
"key": key,
|
||||
"type": field_type,
|
||||
"label": _settings_text(raw_field.get("label")) or key,
|
||||
"description": _settings_text(raw_field.get("description"), max_len=300),
|
||||
"default": default,
|
||||
}
|
||||
if options is not None:
|
||||
field["options"] = options
|
||||
fields.append(field)
|
||||
return fields
|
||||
|
||||
|
||||
def _normalize_loopback_sidecar_origin(value: object) -> Optional[str]:
|
||||
"""Return a canonical loopback origin or None when unsafe.
|
||||
@@ -850,22 +593,6 @@ def _normalize_sidecar_health_path(value: object) -> Optional[str]:
|
||||
return decoded_path
|
||||
|
||||
|
||||
def _is_valid_sidecar_proxy_path(decoded_path: str) -> bool:
|
||||
if any(ch in decoded_path for ch in ("?", "#")):
|
||||
return False
|
||||
if any(ch.isspace() for ch in decoded_path):
|
||||
return False
|
||||
if not decoded_path.startswith("/") or decoded_path.startswith("//"):
|
||||
return False
|
||||
segments = decoded_path.split("/")[1:]
|
||||
if not segments:
|
||||
return False
|
||||
for segment in segments:
|
||||
if not segment or segment in (".", ".."):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _sidecar_from_manifest_entry(
|
||||
entry: Dict[str, object], diagnostics: Optional[Dict[str, Any]] = None
|
||||
) -> Optional[Dict[str, str]]:
|
||||
@@ -898,21 +625,6 @@ def _sidecar_from_manifest_entry(
|
||||
return None
|
||||
else:
|
||||
health_path = _DEFAULT_SIDECAR_HEALTH_PATH
|
||||
# proxy_auth negotiation (token-v1). Absent = explicit legacy (unauthenticated
|
||||
# proxy→sidecar, unchanged behavior for any pre-existing declaration). token-v1
|
||||
# = core mints+injects a per-extension token and the sidecar validates it.
|
||||
# Any unknown value fails closed (reject the sidecar) so a typo can never
|
||||
# silently downgrade to unauthenticated.
|
||||
raw_proxy_auth = raw.get("proxy_auth")
|
||||
if raw_proxy_auth is None:
|
||||
proxy_auth = "legacy"
|
||||
elif raw_proxy_auth in ("legacy", "token-v1"):
|
||||
proxy_auth = str(raw_proxy_auth)
|
||||
else:
|
||||
_add_diagnostic_warning(
|
||||
diagnostics, "sidecar_proxy_auth_unsupported", _SIDECAR_WARNING_SOURCE
|
||||
)
|
||||
return None
|
||||
sidecar_id = _manifest_entry_text(entry, "id")
|
||||
name = _manifest_entry_text(entry, "name")
|
||||
return {
|
||||
@@ -922,7 +634,6 @@ def _sidecar_from_manifest_entry(
|
||||
"origin": origin,
|
||||
"health_path": health_path,
|
||||
"health_url": f"{origin}{health_path}",
|
||||
"proxy_auth": proxy_auth,
|
||||
}
|
||||
|
||||
|
||||
@@ -1024,13 +735,10 @@ def _gallery_installed_runtime_manifest(
|
||||
asset_base = ext_id
|
||||
if isinstance(manifest, dict):
|
||||
top_entry: Dict[str, object] = {"id": ext_id}
|
||||
for key in ("name", "enabled", "scripts", "stylesheets", "sidecar", "permissions", "settings_schema"):
|
||||
for key in ("name", "enabled", "scripts", "stylesheets", "sidecar"):
|
||||
if key in manifest:
|
||||
top_entry[key] = manifest[key]
|
||||
if any(
|
||||
key in top_entry
|
||||
for key in ("scripts", "stylesheets", "sidecar", "permissions", "settings_schema")
|
||||
):
|
||||
if any(key in top_entry for key in ("scripts", "stylesheets", "sidecar")):
|
||||
entries.append(_copy_manifest_entry_with_asset_base(top_entry, asset_base))
|
||||
for _source, _index, entry in _manifest_extension_entries(manifest):
|
||||
copied = _copy_manifest_entry_with_asset_base(entry, asset_base)
|
||||
@@ -1101,10 +809,7 @@ def _load_manifest_with_status(
|
||||
|
||||
|
||||
def _manifest_extension_state(
|
||||
manifest: object,
|
||||
disabled_ids: Set[str],
|
||||
diagnostics: Optional[Dict[str, Any]] = None,
|
||||
consent_ids: Optional[Set[str]] = None,
|
||||
manifest: object, disabled_ids: Set[str], diagnostics: Optional[Dict[str, Any]] = None
|
||||
) -> Dict[str, Any]:
|
||||
"""Return sanitized per-extension state for manifest extension entries."""
|
||||
extension_entries: List[Dict[str, Any]] = []
|
||||
@@ -1131,7 +836,6 @@ def _manifest_extension_state(
|
||||
effective_enabled = manifest_enabled and not user_disabled
|
||||
if not manifest_enabled:
|
||||
manifest_disabled_ids.add(ext_id)
|
||||
settings_schema = _sanitize_settings_schema(entry)
|
||||
extension_entries.append(
|
||||
{
|
||||
"id": ext_id,
|
||||
@@ -1142,8 +846,6 @@ def _manifest_extension_state(
|
||||
"effective_enabled": effective_enabled,
|
||||
"can_toggle": can_toggle,
|
||||
"reload_required": True,
|
||||
"storage_owned": _manifest_entry_storage_owned(entry),
|
||||
"settings_schema": settings_schema,
|
||||
"status": (
|
||||
"manifest_disabled"
|
||||
if not manifest_enabled
|
||||
@@ -1155,7 +857,7 @@ def _manifest_extension_state(
|
||||
_add_diagnostic_warning(diagnostics, "manifest_extension_id_invalid", "manifest:extensions")
|
||||
if duplicate_seen:
|
||||
_add_diagnostic_warning(diagnostics, "manifest_extension_id_duplicate", "manifest:extensions")
|
||||
stale_ids = sorted((disabled_ids | (consent_ids or set())) - known_ids)
|
||||
stale_ids = sorted(disabled_ids - known_ids)
|
||||
if stale_ids:
|
||||
_add_diagnostic_warning(diagnostics, "extension_state_unknown_ids", _EXTENSION_STATE_WARNING_SOURCE)
|
||||
return {
|
||||
@@ -1165,164 +867,13 @@ def _manifest_extension_state(
|
||||
}
|
||||
|
||||
|
||||
def _extension_sidecar_proxy_path(extension_id: str) -> str:
|
||||
return f"/api/extensions/{extension_id}/sidecar/"
|
||||
|
||||
|
||||
def _sidecar_proxy_public_status(
|
||||
extension_id: str,
|
||||
origin: str,
|
||||
approved_origin: Optional[str],
|
||||
*,
|
||||
available: bool,
|
||||
proxy_auth: str = "legacy",
|
||||
) -> Dict[str, Any]:
|
||||
consented = bool(available and approved_origin == origin)
|
||||
origin_changed = bool(available and approved_origin and approved_origin != origin)
|
||||
payload: Dict[str, Any] = {
|
||||
"available": available,
|
||||
"consented": consented,
|
||||
"consent_required": bool(available and not consented),
|
||||
"path": _extension_sidecar_proxy_path(extension_id),
|
||||
"origin_changed": origin_changed,
|
||||
}
|
||||
# proxy_auth / posture are token-v1-only fields. Legacy sidecars keep the
|
||||
# original payload shape byte-for-byte (the public status contract tests pin
|
||||
# it), so a token-v1 negotiation field never leaks onto a legacy record.
|
||||
# (Frank #6331 blocker 2 — fields were previously emitted on every record and
|
||||
# broke the legacy exact-shape contract on shard 4.)
|
||||
if proxy_auth == "token-v1":
|
||||
# token-v1 posture (§9.1). "protected" = WebUI auth ON (the only posture
|
||||
# in which token-v1 proxying is permitted; auth-off is now fail-closed at
|
||||
# both consent and resolution, so "local_unprotected" signals the panel
|
||||
# to tell the operator to enable authentication before the proxy will work).
|
||||
posture = None
|
||||
if available:
|
||||
try:
|
||||
from api.auth import is_auth_enabled
|
||||
posture = "protected" if is_auth_enabled() else "local_unprotected"
|
||||
except Exception:
|
||||
posture = None
|
||||
payload["proxy_auth"] = proxy_auth
|
||||
payload["posture"] = posture
|
||||
return payload
|
||||
|
||||
|
||||
def _extension_sidecar_records(
|
||||
manifest: object,
|
||||
disabled_ids: Optional[Set[str]] = None,
|
||||
state: Optional[Dict[str, Any]] = None,
|
||||
diagnostics: Optional[Dict[str, Any]] = None,
|
||||
) -> Tuple[List[Dict[str, Any]], Dict[str, Dict[str, Any]]]:
|
||||
disabled_ids = disabled_ids or set()
|
||||
consent_map = {}
|
||||
if isinstance(state, dict) and isinstance(state.get("sidecar_proxy_consents"), dict):
|
||||
consent_map = state["sidecar_proxy_consents"]
|
||||
id_counts: Dict[str, int] = {}
|
||||
by_id: Dict[str, Dict[str, Any]] = {}
|
||||
for _source, _index, entry in _manifest_extension_entries(manifest):
|
||||
raw_id = _manifest_entry_text(entry, "id")
|
||||
if not _valid_extension_id(raw_id):
|
||||
continue
|
||||
ext_id = raw_id.strip()
|
||||
id_counts[ext_id] = id_counts.get(ext_id, 0) + 1
|
||||
if ext_id in by_id:
|
||||
continue
|
||||
manifest_enabled = entry.get("enabled", True) is not False
|
||||
user_disabled = ext_id in disabled_ids
|
||||
effective_enabled = manifest_enabled and not user_disabled
|
||||
sidecar = _sidecar_from_manifest_entry(entry, diagnostics) if effective_enabled else None
|
||||
approved_origin = consent_map.get(ext_id) if isinstance(consent_map.get(ext_id), str) else None
|
||||
by_id[ext_id] = {
|
||||
"id": ext_id,
|
||||
"name": _manifest_entry_text(entry, "name"),
|
||||
"manifest_enabled": manifest_enabled,
|
||||
"user_disabled": user_disabled,
|
||||
"effective_enabled": effective_enabled,
|
||||
"sidecar": sidecar,
|
||||
"approved_origin": approved_origin,
|
||||
}
|
||||
records: List[Dict[str, Any]] = []
|
||||
for ext_id, item in by_id.items():
|
||||
sidecar = item.get("sidecar")
|
||||
if sidecar is None:
|
||||
continue
|
||||
available = bool(item["effective_enabled"] and id_counts.get(ext_id, 0) == 1)
|
||||
proxy = _sidecar_proxy_public_status(
|
||||
ext_id,
|
||||
sidecar["origin"],
|
||||
item.get("approved_origin"),
|
||||
available=available,
|
||||
proxy_auth=sidecar.get("proxy_auth", "legacy"),
|
||||
)
|
||||
item["duplicate_id"] = id_counts.get(ext_id, 0) > 1
|
||||
item["proxy"] = proxy
|
||||
if len(records) < _MAX_URL_LIST:
|
||||
# Build the PUBLIC sidecar record explicitly. The internal `sidecar`
|
||||
# dict carries a top-level `proxy_auth` for the consent/resolution
|
||||
# logic, but the public status contract only exposes proxy_auth (and
|
||||
# posture) INSIDE `proxy`, and only for token-v1 records — legacy
|
||||
# sidecars keep their original public shape byte-for-byte. Spreading
|
||||
# `**sidecar` leaked a top-level `proxy_auth: legacy` and broke the
|
||||
# exact-shape contract tests (Frank #6331 blocker 2).
|
||||
public_record = {k: v for k, v in sidecar.items() if k != "proxy_auth"}
|
||||
public_record["proxy"] = proxy
|
||||
records.append(public_record)
|
||||
else:
|
||||
_add_diagnostic_warning(diagnostics, "sidecar_list_truncated", _SIDECAR_WARNING_SOURCE)
|
||||
break
|
||||
return records, by_id
|
||||
|
||||
|
||||
def _normalize_sidecar_proxy_path(value: object) -> Optional[str]:
|
||||
if value is None:
|
||||
return "/"
|
||||
raw = str(value)
|
||||
if raw == "":
|
||||
return "/"
|
||||
if raw.startswith("/"):
|
||||
return None
|
||||
candidate = f"/{raw}"
|
||||
if not _is_valid_sidecar_proxy_path(_fully_unquote_path(candidate)):
|
||||
return None
|
||||
return candidate
|
||||
|
||||
def _extension_runtime_entries(
|
||||
manifest: object, disabled_ids: Optional[Set[str]] = None
|
||||
) -> List[Dict[str, object]]:
|
||||
"""Return enabled extension metadata injected before extension scripts run."""
|
||||
disabled_ids = disabled_ids or set()
|
||||
extensions: List[Dict[str, object]] = []
|
||||
seen_ids: Set[str] = set()
|
||||
for _source, _index, entry in _manifest_extension_entries(manifest):
|
||||
raw_id = _manifest_entry_text(entry, "id")
|
||||
if not _valid_extension_id(raw_id):
|
||||
continue
|
||||
ext_id = raw_id.strip()
|
||||
if ext_id in seen_ids:
|
||||
continue
|
||||
seen_ids.add(ext_id)
|
||||
if entry.get("enabled", True) is False or ext_id in disabled_ids:
|
||||
continue
|
||||
extensions.append(
|
||||
{
|
||||
"id": ext_id,
|
||||
"name": _manifest_entry_text(entry, "name") or ext_id,
|
||||
"storage_owned": _manifest_entry_storage_owned(entry),
|
||||
"settings_schema": _sanitize_settings_schema(entry),
|
||||
}
|
||||
)
|
||||
return extensions
|
||||
|
||||
|
||||
def _read_manifest_urls_with_diagnostics(
|
||||
root: Path,
|
||||
diagnostics: Optional[Dict[str, Any]] = None,
|
||||
disabled_ids: Optional[Set[str]] = None,
|
||||
manifest: Optional[object] = None,
|
||||
manifest_status: Optional[Dict[str, Any]] = None,
|
||||
state: Optional[Dict[str, Any]] = None,
|
||||
) -> Tuple[List[str], List[str], List[Dict[str, Any]], Dict[str, Any]]:
|
||||
) -> Tuple[List[str], List[str], List[Dict[str, str]], Dict[str, Any]]:
|
||||
disabled_ids = disabled_ids or set()
|
||||
if manifest is None or manifest_status is None:
|
||||
manifest, manifest_status = _load_manifest_with_status(root, diagnostics)
|
||||
@@ -1331,7 +882,7 @@ def _read_manifest_urls_with_diagnostics(
|
||||
|
||||
scripts: List[str] = []
|
||||
stylesheets: List[str] = []
|
||||
sidecars: List[Dict[str, Any]] = []
|
||||
sidecars: List[Dict[str, str]] = []
|
||||
asset_base = str(manifest_status.get("_asset_base", "") or "")
|
||||
entries = _iter_manifest_entries(manifest, disabled_ids=disabled_ids)
|
||||
manifest_status["entry_count"] = len(entries)
|
||||
@@ -1340,6 +891,15 @@ def _read_manifest_urls_with_diagnostics(
|
||||
for _source, entry in entries:
|
||||
if not isinstance(entry, dict):
|
||||
continue
|
||||
if _source.startswith("manifest.extensions["):
|
||||
sidecar = _sidecar_from_manifest_entry(entry, diagnostics)
|
||||
if sidecar is not None:
|
||||
if len(sidecars) < _MAX_URL_LIST:
|
||||
sidecars.append(sidecar)
|
||||
else:
|
||||
_add_diagnostic_warning(
|
||||
diagnostics, "sidecar_list_truncated", _SIDECAR_WARNING_SOURCE
|
||||
)
|
||||
script_source = "manifest:scripts"
|
||||
stylesheet_source = "manifest:stylesheets"
|
||||
if not scripts_full:
|
||||
@@ -1362,12 +922,6 @@ def _read_manifest_urls_with_diagnostics(
|
||||
):
|
||||
stylesheets_full = True
|
||||
break
|
||||
sidecars, _ = _extension_sidecar_records(
|
||||
manifest,
|
||||
disabled_ids=disabled_ids,
|
||||
state=state,
|
||||
diagnostics=diagnostics,
|
||||
)
|
||||
manifest_status.update(
|
||||
{
|
||||
"loaded": True,
|
||||
@@ -1396,14 +950,8 @@ def get_extension_config() -> Dict[str, Any]:
|
||||
return {"enabled": False, "script_urls": [], "stylesheet_urls": []}
|
||||
state = _load_extension_state()
|
||||
disabled_ids = set(state.get("disabled_extensions") or [])
|
||||
manifest, manifest_status = _load_manifest_with_status(root)
|
||||
manifest_scripts, manifest_stylesheets, _, _ = _read_manifest_urls_with_diagnostics(
|
||||
root,
|
||||
disabled_ids=disabled_ids,
|
||||
manifest=manifest,
|
||||
manifest_status=manifest_status,
|
||||
)
|
||||
config = {
|
||||
manifest_scripts, manifest_stylesheets = _read_manifest_urls(root, disabled_ids=disabled_ids)
|
||||
return {
|
||||
"enabled": True,
|
||||
"script_urls": _read_url_list(
|
||||
_EXTENSION_SCRIPT_URLS_ENV, manifest_scripts or None
|
||||
@@ -1412,10 +960,6 @@ def get_extension_config() -> Dict[str, Any]:
|
||||
_EXTENSION_STYLESHEET_URLS_ENV, manifest_stylesheets or None
|
||||
),
|
||||
}
|
||||
runtime_entries = _extension_runtime_entries(manifest, disabled_ids) if manifest is not None else []
|
||||
if runtime_entries:
|
||||
config["extensions"] = runtime_entries
|
||||
return config
|
||||
|
||||
|
||||
|
||||
@@ -1464,13 +1008,7 @@ def get_extension_status() -> Dict[str, Any]:
|
||||
}
|
||||
|
||||
manifest, manifest_status = _load_manifest_with_status(root, diagnostics)
|
||||
consent_ids = set((state.get("sidecar_proxy_consents") or {}).keys())
|
||||
extension_state = _manifest_extension_state(
|
||||
manifest,
|
||||
disabled_ids,
|
||||
diagnostics,
|
||||
consent_ids=consent_ids,
|
||||
) if manifest is not None else {
|
||||
extension_state = _manifest_extension_state(manifest, disabled_ids, diagnostics) if manifest is not None else {
|
||||
"extensions": [],
|
||||
"known_ids": set(),
|
||||
"manifest_disabled_ids": set(),
|
||||
@@ -1481,7 +1019,6 @@ def get_extension_status() -> Dict[str, Any]:
|
||||
disabled_ids=disabled_ids,
|
||||
manifest=manifest,
|
||||
manifest_status=manifest_status,
|
||||
state=state,
|
||||
)
|
||||
extensions = extension_state["extensions"]
|
||||
known_ids = extension_state["known_ids"]
|
||||
@@ -1537,13 +1074,7 @@ def set_extension_user_enabled(extension_id: object, enabled: object) -> Dict[st
|
||||
manifest, manifest_status = _load_manifest_with_status(root, diagnostics)
|
||||
if manifest is None or not manifest_status.get("loaded", False):
|
||||
raise ExtensionToggleError("Extension manifest is not loaded", status=409)
|
||||
consent_map = state.get("sidecar_proxy_consents") or {}
|
||||
extension_state = _manifest_extension_state(
|
||||
manifest,
|
||||
disabled_ids,
|
||||
diagnostics,
|
||||
consent_ids=set(consent_map.keys()),
|
||||
)
|
||||
extension_state = _manifest_extension_state(manifest, disabled_ids, diagnostics)
|
||||
known_ids: Set[str] = extension_state["known_ids"]
|
||||
manifest_disabled_ids: Set[str] = extension_state["manifest_disabled_ids"]
|
||||
if ext_id not in known_ids:
|
||||
@@ -1554,16 +1085,7 @@ def set_extension_user_enabled(extension_id: object, enabled: object) -> Dict[st
|
||||
disabled_ids.discard(ext_id)
|
||||
else:
|
||||
disabled_ids.add(ext_id)
|
||||
_write_extension_state(
|
||||
{
|
||||
"disabled_extensions": sorted(disabled_ids),
|
||||
"sidecar_proxy_consents": {
|
||||
consent_ext_id: origin
|
||||
for consent_ext_id, origin in consent_map.items()
|
||||
if consent_ext_id in known_ids
|
||||
},
|
||||
}
|
||||
)
|
||||
_write_extension_state({"disabled_extensions": sorted(disabled_ids)})
|
||||
# Return a fresh status snapshot after the atomic write is visible. Keeping
|
||||
# the readback outside the lock avoids doing the full manifest/status parse
|
||||
# while blocking other toggles; a concurrent toggle may be reflected too,
|
||||
@@ -1571,176 +1093,6 @@ def set_extension_user_enabled(extension_id: object, enabled: object) -> Dict[st
|
||||
return get_extension_status()
|
||||
|
||||
|
||||
def set_extension_sidecar_proxy_consent(extension_id: object, approved: object) -> Dict[str, Any]:
|
||||
"""Persist or revoke proxy consent for the current sidecar origin."""
|
||||
if not _valid_extension_id(extension_id):
|
||||
raise ExtensionSidecarProxyError("Invalid extension id", status=400)
|
||||
ext_id = str(extension_id).strip()
|
||||
if not isinstance(approved, bool):
|
||||
raise ExtensionSidecarProxyError("approved must be a boolean", status=400)
|
||||
root = _extension_root()
|
||||
if root is None:
|
||||
raise ExtensionSidecarProxyError("Extensions are not configured", status=404)
|
||||
with _EXTENSION_STATE_LOCK:
|
||||
diagnostics = _new_diagnostics()
|
||||
state = _load_extension_state(diagnostics)
|
||||
disabled_ids = set(state.get("disabled_extensions") or [])
|
||||
consent_map = dict(state.get("sidecar_proxy_consents") or {})
|
||||
manifest, manifest_status = _load_manifest_with_status(root, diagnostics)
|
||||
if manifest is None or not manifest_status.get("loaded", False):
|
||||
raise ExtensionSidecarProxyError("Extension manifest is not loaded", status=409)
|
||||
extension_state = _manifest_extension_state(
|
||||
manifest,
|
||||
disabled_ids,
|
||||
diagnostics,
|
||||
consent_ids=set(consent_map.keys()),
|
||||
)
|
||||
known_ids: Set[str] = extension_state["known_ids"]
|
||||
if ext_id not in known_ids:
|
||||
raise ExtensionSidecarProxyError("Extension not found", status=404)
|
||||
_sidecars, by_id = _extension_sidecar_records(
|
||||
manifest,
|
||||
disabled_ids=disabled_ids,
|
||||
state=state,
|
||||
diagnostics=diagnostics,
|
||||
)
|
||||
item = by_id.get(ext_id) or {}
|
||||
sidecar = item.get("sidecar")
|
||||
proxy = item.get("proxy") or {}
|
||||
if approved:
|
||||
if sidecar is None or proxy.get("available") is not True:
|
||||
raise ExtensionSidecarProxyError("Extension sidecar proxy is unavailable", status=409)
|
||||
consent_map[ext_id] = sidecar["origin"]
|
||||
# For token-v1, mint the per-extension token NOW and fail the consent
|
||||
# if it can't be persisted+read — otherwise we'd persist consent and
|
||||
# then 503 on every request (a mint failure must surface at grant
|
||||
# time, not as a mysterious later error).
|
||||
if sidecar.get("proxy_auth") == "token-v1":
|
||||
# token-v1 REQUIRES configured WebUI authentication. Without it,
|
||||
# this consent endpoint is itself unauthenticated, so any caller
|
||||
# that can reach the WebUI listener could self-grant consent and
|
||||
# then drive the token-bearing proxy (a forwarding oracle) — the
|
||||
# sidecar token file blocks direct port access but does NOT stop a
|
||||
# caller who simply asks core to inject it. Loopback origin is not
|
||||
# sufficient: another local UID can still reach the listener
|
||||
# without reading the 0600 token file. Fail closed regardless of
|
||||
# origin. (Frank #6331 blocker 1.)
|
||||
from api.auth import is_auth_enabled
|
||||
if not is_auth_enabled():
|
||||
raise ExtensionSidecarProxyError(
|
||||
"Sidecar token-v1 proxy requires WebUI authentication; "
|
||||
"set a password in Settings before granting consent.",
|
||||
status=403,
|
||||
)
|
||||
from api import extension_sidecar_auth as _sc_auth
|
||||
if not _sc_auth.ensure_token(ext_id):
|
||||
raise ExtensionSidecarProxyError(
|
||||
"Could not provision the sidecar auth token", status=503
|
||||
)
|
||||
else:
|
||||
consent_map.pop(ext_id, None)
|
||||
_write_extension_state(
|
||||
{
|
||||
"disabled_extensions": sorted(disabled_ids),
|
||||
"sidecar_proxy_consents": {
|
||||
consent_ext_id: origin
|
||||
for consent_ext_id, origin in consent_map.items()
|
||||
if consent_ext_id in known_ids
|
||||
},
|
||||
}
|
||||
)
|
||||
return get_extension_status()
|
||||
|
||||
|
||||
def resolve_extension_sidecar_proxy_target(
|
||||
extension_id: object,
|
||||
proxy_path: object,
|
||||
query: str = "",
|
||||
) -> Dict[str, Any]:
|
||||
"""Resolve the current approved sidecar proxy target for an extension."""
|
||||
if not _valid_extension_id(extension_id):
|
||||
raise ExtensionSidecarProxyError("Invalid extension id", status=400)
|
||||
normalized_path = _normalize_sidecar_proxy_path(proxy_path)
|
||||
if normalized_path is None:
|
||||
raise ExtensionSidecarProxyError("Invalid sidecar proxy path", status=400)
|
||||
ext_id = str(extension_id).strip()
|
||||
root = _extension_root()
|
||||
if root is None:
|
||||
raise ExtensionSidecarProxyError("Extensions are not configured", status=404)
|
||||
diagnostics = _new_diagnostics()
|
||||
state = _load_extension_state(diagnostics)
|
||||
disabled_ids = set(state.get("disabled_extensions") or [])
|
||||
manifest, manifest_status = _load_manifest_with_status(root, diagnostics)
|
||||
if manifest is None or not manifest_status.get("loaded", False):
|
||||
raise ExtensionSidecarProxyError("Extension manifest is not loaded", status=409)
|
||||
consent_ids = set((state.get("sidecar_proxy_consents") or {}).keys())
|
||||
extension_state = _manifest_extension_state(
|
||||
manifest,
|
||||
disabled_ids,
|
||||
diagnostics,
|
||||
consent_ids=consent_ids,
|
||||
)
|
||||
if ext_id not in extension_state["known_ids"]:
|
||||
raise ExtensionSidecarProxyError("Extension not found", status=404)
|
||||
_sidecars, by_id = _extension_sidecar_records(
|
||||
manifest,
|
||||
disabled_ids=disabled_ids,
|
||||
state=state,
|
||||
diagnostics=diagnostics,
|
||||
)
|
||||
item = by_id.get(ext_id) or {}
|
||||
sidecar = item.get("sidecar")
|
||||
proxy = item.get("proxy") or {}
|
||||
if sidecar is None or proxy.get("available") is not True:
|
||||
raise ExtensionSidecarProxyError("Extension sidecar proxy is unavailable", status=409)
|
||||
if proxy.get("consented") is not True:
|
||||
raise ExtensionSidecarProxyError("Extension sidecar proxy consent required", status=403)
|
||||
upstream_url = f"{sidecar['origin']}{normalized_path}"
|
||||
if query:
|
||||
upstream_url = f"{upstream_url}?{query}"
|
||||
|
||||
# token-v1 auth (§9.1/§9.2). Legacy sidecars proxy unchanged; token-v1
|
||||
# sidecars get a per-extension secret injected, and only proxy when the
|
||||
# trust posture is sound.
|
||||
proxy_auth = sidecar.get("proxy_auth", "legacy")
|
||||
inject_token: Optional[str] = None
|
||||
if proxy_auth == "token-v1":
|
||||
from api.auth import is_auth_enabled
|
||||
from api import extension_sidecar_auth as _sc_auth
|
||||
|
||||
# token-v1 REQUIRES configured WebUI authentication — fail closed
|
||||
# regardless of the upstream origin (loopback included). The consent
|
||||
# endpoint and this resolution path are both unauthenticated when auth is
|
||||
# off, so a caller that can reach the WebUI listener could otherwise drive
|
||||
# a token-bearing proxy to the sidecar (a forwarding oracle). Loopback is
|
||||
# NOT a sufficient guard: another local UID can reach the listener without
|
||||
# ever reading the 0600 token file. (Frank #6331 blocker 1 — this
|
||||
# mirrors the consent-time check so a pre-existing consent granted while
|
||||
# auth was enabled cannot be exercised after auth is turned off.)
|
||||
if not is_auth_enabled():
|
||||
raise ExtensionSidecarProxyError(
|
||||
"Sidecar token-v1 proxy requires WebUI authentication; "
|
||||
"set a password in Settings.",
|
||||
status=403,
|
||||
)
|
||||
token = _sc_auth.current_token(ext_id) or _sc_auth.ensure_token(ext_id)
|
||||
if not token:
|
||||
# Token could not be persisted+read → fail closed rather than proxy
|
||||
# unauthenticated (never inject an ephemeral secret).
|
||||
raise ExtensionSidecarProxyError(
|
||||
"Sidecar proxy auth token is unavailable", status=503
|
||||
)
|
||||
inject_token = token
|
||||
return {
|
||||
"extension_id": ext_id,
|
||||
"origin": sidecar["origin"],
|
||||
"proxy_path": proxy["path"],
|
||||
"upstream_url": upstream_url,
|
||||
"proxy_auth": proxy_auth,
|
||||
"auth_token": inject_token,
|
||||
}
|
||||
|
||||
|
||||
def _install_manifest_file() -> Path:
|
||||
return _extension_state_dir() / _GALLERY_INSTALL_STATE_FILENAME
|
||||
|
||||
@@ -1854,7 +1206,7 @@ def install_extension(id: object, download_url: object, sha256: object) -> Dict[
|
||||
ext_dir_resolved = ext_dir.resolve()
|
||||
for member_name in file_members:
|
||||
decoded = _fully_unquote_path(_stripped(member_name))
|
||||
if not decoded or not _is_safe_archive_member(decoded):
|
||||
if not decoded or not _is_safe_relative_path(decoded):
|
||||
raise ExtensionInstallError("Unsafe archive member")
|
||||
resolved = (ext_dir / decoded).resolve()
|
||||
try:
|
||||
@@ -1956,7 +1308,7 @@ def uninstall_extension(id: object) -> Dict[str, Any]:
|
||||
raise ExtensionInstallError("Extension not installed", 404)
|
||||
ext_dir = root / ext_id
|
||||
for rel_path in entry.get("files", []):
|
||||
if not _is_safe_archive_member(rel_path):
|
||||
if not _is_safe_relative_path(rel_path):
|
||||
continue
|
||||
target = (ext_dir / rel_path).resolve()
|
||||
try:
|
||||
@@ -1998,8 +1350,7 @@ def get_extension_registry() -> Dict[str, Any]:
|
||||
if cached is not None and (now - cached_at) < _REGISTRY_TTL_SECONDS:
|
||||
return {"entries": cached}
|
||||
try:
|
||||
opener = _build_gallery_opener()
|
||||
raw = opener.open(_REGISTRY_URL, timeout=10).read(2 * 1024 * 1024)
|
||||
raw = urlopen(_REGISTRY_URL, timeout=10).read(2 * 1024 * 1024)
|
||||
data = json.loads(raw.decode("utf-8"))
|
||||
if isinstance(data, list):
|
||||
entries = data
|
||||
@@ -2036,15 +1387,6 @@ def inject_extension_tags(index_html: str) -> str:
|
||||
'<script src="{}" defer></script>'.format(html.escape(url, quote=True))
|
||||
for url in config["script_urls"]
|
||||
]
|
||||
runtime_config = {
|
||||
"extensions": config.get("extensions", []),
|
||||
}
|
||||
runtime_json = json.dumps(runtime_config, ensure_ascii=False, separators=(",", ":")).replace("<", "\\u003c")
|
||||
runtime_tag = (
|
||||
"<script>window.__HERMES_EXTENSION_CONFIG__={};"
|
||||
"if(window.HermesExtensionSettings)window.HermesExtensionSettings.primeFromStatus(window.__HERMES_EXTENSION_CONFIG__);"
|
||||
"</script>"
|
||||
).format(runtime_json)
|
||||
|
||||
if stylesheet_tags:
|
||||
head_marker = "</head>"
|
||||
@@ -2054,11 +1396,9 @@ def inject_extension_tags(index_html: str) -> str:
|
||||
else:
|
||||
result = block + result
|
||||
|
||||
if runtime_config["extensions"] or script_tags:
|
||||
if script_tags:
|
||||
body_marker = "</body>"
|
||||
block = runtime_tag + "\n"
|
||||
if script_tags:
|
||||
block += "\n".join(script_tags) + "\n"
|
||||
block = "\n".join(script_tags) + "\n"
|
||||
if body_marker in result:
|
||||
result = result.replace(body_marker, block + body_marker, 1)
|
||||
else:
|
||||
@@ -2068,9 +1408,6 @@ def inject_extension_tags(index_html: str) -> str:
|
||||
|
||||
|
||||
def _is_safe_relative_path(rel: str) -> bool:
|
||||
# Strict: reject empty, traversal, AND any dot-prefixed segment. This is shared
|
||||
# by static serving, asset URLs and manifest paths, where a hidden file must
|
||||
# never become reachable. Archive members use _is_safe_archive_member() below.
|
||||
if not rel or "\x00" in rel or "\\" in rel:
|
||||
return False
|
||||
for segment in rel.split("/"):
|
||||
@@ -2079,31 +1416,6 @@ def _is_safe_relative_path(rel: str) -> bool:
|
||||
return True
|
||||
|
||||
|
||||
# Benign hidden LEAF files that extension archives legitimately ship. A dot-prefixed
|
||||
# name is permitted only as the final path segment and only if it is one of these;
|
||||
# dot-directories (e.g. ".git/") and every other dotfile (".env", ".secret") stay rejected.
|
||||
_ALLOWED_ARCHIVE_DOTFILES = frozenset({".gitkeep", ".gitignore", ".gitattributes", ".env.example"})
|
||||
|
||||
|
||||
def _is_safe_archive_member(rel: str) -> bool:
|
||||
"""Path validator for archive install/uninstall ONLY. Same containment as
|
||||
_is_safe_relative_path(), but permits a narrow allowlist of benign hidden leaf
|
||||
files so extensions can ship ".gitkeep"/".env.example" without loosening the
|
||||
shared validator that also gates static serving."""
|
||||
if not rel or "\x00" in rel or "\\" in rel:
|
||||
return False
|
||||
segments = rel.split("/")
|
||||
for segment in segments[:-1]:
|
||||
if not segment or segment in (".", "..") or segment.startswith("."):
|
||||
return False
|
||||
leaf = segments[-1]
|
||||
if not leaf or leaf in (".", ".."):
|
||||
return False
|
||||
if leaf.startswith(".") and leaf not in _ALLOWED_ARCHIVE_DOTFILES:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _not_found(handler) -> bool:
|
||||
j(handler, {"error": "not found"}, status=404)
|
||||
return True
|
||||
|
||||
+117
-714
File diff suppressed because it is too large
Load Diff
@@ -1,195 +0,0 @@
|
||||
"""Helpers for restarting the active-profile Hermes gateway."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import os
|
||||
import shutil
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
from api.profiles import (
|
||||
_PROFILE_ID_RE,
|
||||
_is_root_profile,
|
||||
get_active_hermes_home,
|
||||
get_active_profile_name,
|
||||
get_hermes_home_for_profile,
|
||||
)
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_GATEWAY_RESTART_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def _resolve_hermes_command() -> str:
|
||||
"""Resolve the CLI path used for active-profile gateway restarts."""
|
||||
hermes_cmd = shutil.which("hermes")
|
||||
if hermes_cmd:
|
||||
return hermes_cmd
|
||||
|
||||
sibling = Path(sys.executable).parent / "hermes"
|
||||
if sibling.exists():
|
||||
return str(sibling)
|
||||
return "hermes"
|
||||
|
||||
|
||||
def _consume_stream(stream) -> None:
|
||||
"""Drain a subprocess stream to prevent stdout/stderr pipe deadlocks."""
|
||||
try:
|
||||
while stream and stream.read(4096):
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def _release_lock() -> None:
|
||||
try:
|
||||
_GATEWAY_RESTART_LOCK.release()
|
||||
except RuntimeError:
|
||||
# The lock may already have been released by another path.
|
||||
pass
|
||||
|
||||
|
||||
def _gateway_restart_profile_context(profile: str | None = None) -> tuple[Path, str | None]:
|
||||
"""Return the HERMES_HOME and CLI profile arg for a gateway restart."""
|
||||
if profile is None:
|
||||
raw_profile = str(get_active_profile_name() or "default").strip()
|
||||
active_home = Path(get_active_hermes_home())
|
||||
else:
|
||||
raw_profile = str(profile or "")
|
||||
if not raw_profile or not _PROFILE_ID_RE.fullmatch(raw_profile):
|
||||
raise ValueError(f"Invalid profile for gateway restart: {profile!r}")
|
||||
active_home = Path(get_hermes_home_for_profile(raw_profile))
|
||||
|
||||
if (
|
||||
raw_profile == "default"
|
||||
and active_home.name == "default"
|
||||
and active_home.parent.name == "profiles"
|
||||
):
|
||||
return active_home, None
|
||||
if not raw_profile or not _PROFILE_ID_RE.fullmatch(raw_profile) or _is_root_profile(raw_profile):
|
||||
return active_home, "default"
|
||||
return active_home, raw_profile
|
||||
|
||||
|
||||
def restart_active_profile_gateway(
|
||||
*,
|
||||
profile: str | None = None,
|
||||
quick_timeout_seconds: float = 2.0,
|
||||
background_wait_seconds: float = 240.0,
|
||||
) -> dict:
|
||||
"""Run a non-blocking ``hermes gateway restart`` for the active profile.
|
||||
|
||||
Returns a short status dict with these values:
|
||||
- completed: command finished quickly and succeeded.
|
||||
- in_progress: command did not finish within ``quick_timeout_seconds``.
|
||||
- failed: command finished quickly with non-zero exit status.
|
||||
- busy: restart already in progress from another caller.
|
||||
"""
|
||||
if not _GATEWAY_RESTART_LOCK.acquire(blocking=False):
|
||||
return {
|
||||
"status": "busy",
|
||||
"message": "Restart already in progress. Please wait a moment and try again.",
|
||||
}
|
||||
|
||||
try:
|
||||
active_home, cli_profile = _gateway_restart_profile_context(profile)
|
||||
env = os.environ.copy()
|
||||
env["HERMES_HOME"] = str(active_home)
|
||||
hermes_cmd = _resolve_hermes_command()
|
||||
cmd = [hermes_cmd]
|
||||
if cli_profile is not None:
|
||||
cmd.extend(["--profile", cli_profile])
|
||||
cmd.extend(["gateway", "restart"])
|
||||
|
||||
if cli_profile is None:
|
||||
logger.info(
|
||||
"Restarting gateway service via CLI command: %s gateway restart (HERMES_HOME=%s)",
|
||||
hermes_cmd,
|
||||
active_home,
|
||||
)
|
||||
else:
|
||||
logger.info(
|
||||
"Restarting gateway service via CLI command: %s --profile %s gateway restart (HERMES_HOME=%s)",
|
||||
hermes_cmd,
|
||||
cli_profile,
|
||||
active_home,
|
||||
)
|
||||
proc = subprocess.Popen(
|
||||
cmd,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
text=True,
|
||||
env=env,
|
||||
)
|
||||
|
||||
try:
|
||||
stdout, stderr = proc.communicate(timeout=quick_timeout_seconds)
|
||||
_release_lock()
|
||||
stdout = (stdout or "").strip()
|
||||
stderr = (stderr or "").strip()
|
||||
if proc.returncode == 0:
|
||||
logger.info("Gateway service restarted successfully: %s", stdout)
|
||||
return {
|
||||
"status": "completed",
|
||||
"message": "Gateway service restarted successfully",
|
||||
"detail": stdout or stderr,
|
||||
}
|
||||
|
||||
logger.error("Gateway service restart failed with code %s: %s", proc.returncode, stderr)
|
||||
return {
|
||||
"status": "failed",
|
||||
"message": f"Restart failed: {stderr or stdout}",
|
||||
"detail": stdout or stderr,
|
||||
"returncode": proc.returncode,
|
||||
}
|
||||
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.info(
|
||||
"Gateway restart is taking longer than %.1fs (likely draining in-flight runs);"
|
||||
" continuing in background",
|
||||
quick_timeout_seconds,
|
||||
)
|
||||
|
||||
threading.Thread(target=_consume_stream, args=(proc.stdout,), daemon=True).start()
|
||||
threading.Thread(target=_consume_stream, args=(proc.stderr,), daemon=True).start()
|
||||
|
||||
def _wait_and_release() -> None:
|
||||
try:
|
||||
proc.wait(timeout=background_wait_seconds)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error(
|
||||
"Gateway restart process timed out after %.1fs. Terminating process.",
|
||||
background_wait_seconds,
|
||||
)
|
||||
try:
|
||||
proc.terminate()
|
||||
try:
|
||||
proc.wait(timeout=5.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
try:
|
||||
proc.wait(timeout=5.0)
|
||||
except subprocess.TimeoutExpired:
|
||||
logger.error(
|
||||
"Gateway restart process refused to die even after SIGKILL.",
|
||||
)
|
||||
except Exception:
|
||||
logger.exception("Failed to terminate timed out gateway restart process.")
|
||||
finally:
|
||||
_release_lock()
|
||||
|
||||
threading.Thread(target=_wait_and_release, daemon=True).start()
|
||||
return {
|
||||
"status": "in_progress",
|
||||
"message": "Gateway service restart initiated (in progress)",
|
||||
}
|
||||
except Exception as exc:
|
||||
_release_lock()
|
||||
logger.exception("Failed to run gateway restart command")
|
||||
return {
|
||||
"status": "failed",
|
||||
"message": f"Internal error running restart: {type(exc).__name__}: {exc}",
|
||||
}
|
||||
+59
-73
@@ -9,6 +9,7 @@ This enables real-time session list updates in the sidebar without
|
||||
requiring any changes to hermes-agent.
|
||||
"""
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import queue
|
||||
@@ -19,7 +20,7 @@ from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
from api.config import HOME
|
||||
from api.agent_sessions import open_state_db_readonly, read_importable_agent_session_rows
|
||||
from api.agent_sessions import read_importable_agent_session_rows
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@@ -43,7 +44,7 @@ _WATCHER_EXCLUDED_SOURCES = ("cron", "webui")
|
||||
|
||||
|
||||
def _cheap_change_fingerprint(db_path: Path) -> str | None:
|
||||
"""Compute a cheap fingerprint with an index-covered message aggregate.
|
||||
"""Compute a cheap change-detection fingerprint without the messages JOIN.
|
||||
|
||||
The expensive projection (``read_importable_agent_session_rows``) runs a CTE
|
||||
plus a per-session ``MAX(messages.timestamp)`` aggregation over an oversampled
|
||||
@@ -52,15 +53,25 @@ def _cheap_change_fingerprint(db_path: Path) -> str | None:
|
||||
scan, and the watcher runs it forever on a 5s timer even when nothing changed
|
||||
(issue #3506).
|
||||
|
||||
This hashes every sessions-table column the projection uses, plus a
|
||||
per-session ``COUNT`` / ``MAX(messages.timestamp)`` aggregate scoped to the
|
||||
same non-cron/webui rows. The message aggregate stays on the agent's existing
|
||||
``(session_id, timestamp)`` covering index, avoiding a table-page lookup for
|
||||
every historical message.
|
||||
This computes a fingerprint from a ``sessions``-table-only scan (no messages
|
||||
JOIN), scoped to the same non-cron/webui rows as the projection. To guarantee
|
||||
it never skips a change the projection would reflect, it hashes **every
|
||||
sessions-table column the projection reads or uses for visibility/collapse**
|
||||
-- not just the columns surfaced to the sidebar. That matters because the
|
||||
projection collapses compression lineage and hides/shows rows based on
|
||||
``parent_session_id`` / ``ended_at`` / ``end_reason`` / ``source``, so a change
|
||||
to one of those alters *which rows* appear even when no displayed field on a
|
||||
given row moved.
|
||||
|
||||
``role`` is intentionally absent because it is not in that index. A bounded
|
||||
periodic full projection in ``GatewayWatcher._poll_once`` covers rare
|
||||
role-only visibility mutations without restoring the five-second table scan.
|
||||
The one projection input that does not live in the ``sessions`` table is the
|
||||
per-session message aggregate (``COUNT`` / ``MAX(messages.timestamp)`` ->
|
||||
``last_activity``). That is fully proxied by ``sessions.message_count``: the
|
||||
agent's state layer bumps ``message_count`` on every appended message and
|
||||
rewrites it to the absolute count on truncate/rewind/compaction, so a message
|
||||
insert or delete (the only events that can move ``MAX(timestamp)``) always
|
||||
changes ``message_count``. The fingerprint is therefore a strict superset of
|
||||
the projection's change surface (it also fires on out-of-order inserts that
|
||||
would not raise ``MAX(timestamp)``).
|
||||
|
||||
Returns the fingerprint string, or ``None`` on any error / a pre-source
|
||||
schema so the caller falls back to running the expensive projection rather
|
||||
@@ -76,7 +87,7 @@ def _cheap_change_fingerprint(db_path: Path) -> str | None:
|
||||
'origin_chat_id', 'origin_user_id', 'platform',
|
||||
)
|
||||
try:
|
||||
with closing(open_state_db_readonly(db_path)) as conn:
|
||||
with closing(sqlite3.connect(str(db_path))) as conn:
|
||||
cur = conn.cursor()
|
||||
cur.execute("PRAGMA table_info(sessions)")
|
||||
cols = {row[1] for row in cur.fetchall()}
|
||||
@@ -99,17 +110,21 @@ def _cheap_change_fingerprint(db_path: Path) -> str | None:
|
||||
# timestamps but can leave sessions.message_count unchanged — so the
|
||||
# sessions-only scan above would miss it and the watcher would skip a
|
||||
# projection whose last_activity (MAX(messages.timestamp)) actually
|
||||
# moved. Fold in a PER-SESSION COUNT/MAX aggregate, scoped to the same
|
||||
# non-excluded sessions as the projection. COUNT preserves drift
|
||||
# detection; MAX catches same-count rewrites because replacement rows
|
||||
# receive fresh timestamps. Do not read ``role`` here: the normal
|
||||
# (session_id, timestamp) index can then cover this five-second scan
|
||||
# instead of forcing a table-page lookup for every historical row.
|
||||
# moved. Fold in a PER-SESSION message aggregate, scoped to the same
|
||||
# non-excluded sessions as the projection. It must be per-session
|
||||
# (grouped), NOT a single global MAX: rewriting an OLDER, non-newest
|
||||
# session moves that session's last_activity but not the global max,
|
||||
# so a global aggregate would still miss it (#3536 review round 2).
|
||||
# cron/webui churn is excluded by the JOIN filter so it still does
|
||||
# NOT trigger a re-projection. This is one GROUP BY over the already-
|
||||
# filtered set — far cheaper than the projection's oversampled
|
||||
# correlated CTE — so it preserves the cheap-fingerprint property.
|
||||
if 'messages' in {r[0] for r in conn.execute(
|
||||
"SELECT name FROM sqlite_master WHERE type='table'").fetchall()}:
|
||||
try:
|
||||
msg_rows = conn.execute(
|
||||
"SELECT s.id, COUNT(m.id), "
|
||||
"COUNT(CASE WHEN LOWER(m.role) = 'user' THEN 1 END), "
|
||||
"COALESCE(MAX(m.timestamp), 0) "
|
||||
"FROM sessions s LEFT JOIN messages m ON m.session_id = s.id "
|
||||
f"WHERE s.source IS NOT NULL AND s.source NOT IN ({placeholders}) "
|
||||
@@ -142,11 +157,9 @@ def _get_state_db_path(hermes_home: Path | None = None) -> Path:
|
||||
return hermes_home / 'state.db'
|
||||
|
||||
|
||||
def _get_agent_sessions_from_db(db_path: Path | None = None) -> list | None:
|
||||
def _get_agent_sessions_from_db(db_path: Path | None = None) -> list:
|
||||
"""Read all non-webui sessions from state.db.
|
||||
|
||||
Returns a list of session dicts (including an empty list for a successful
|
||||
empty projection), or ``None`` when the projection fails.
|
||||
Returns list of session dicts, or empty list on any error.
|
||||
"""
|
||||
db_path = Path(db_path) if db_path is not None else _get_state_db_path()
|
||||
if not db_path.exists():
|
||||
@@ -169,7 +182,7 @@ def _get_agent_sessions_from_db(db_path: Path | None = None) -> list | None:
|
||||
})
|
||||
return sessions
|
||||
except Exception:
|
||||
return None
|
||||
return []
|
||||
|
||||
|
||||
# ── GatewayWatcher ──────────────────────────────────────────────────────────
|
||||
@@ -187,11 +200,6 @@ class GatewayWatcher:
|
||||
"""
|
||||
|
||||
POLL_INTERVAL = 5 # seconds between polls
|
||||
# ``messages.role`` is not present in the agent's covering
|
||||
# ``(session_id, timestamp)`` index, but the full projection uses it for CLI
|
||||
# visibility. Keep the hot poll index-only and bound detection of rare
|
||||
# role-only mutations with a periodic parity projection.
|
||||
PROJECTION_PARITY_INTERVAL = 60.0
|
||||
SUBSCRIBER_TIMEOUT = 30 # seconds before sending keepalive comment
|
||||
|
||||
def __init__(
|
||||
@@ -218,7 +226,6 @@ class GatewayWatcher:
|
||||
# unchanged we skip the expensive messages-JOIN projection entirely
|
||||
# (issue #3506). Empty string forces the first poll to run the full read.
|
||||
self._last_cheap_fp: str = ''
|
||||
self._last_full_projection_at: float | None = None
|
||||
|
||||
def start(self):
|
||||
"""Start the watcher daemon thread."""
|
||||
@@ -310,55 +317,34 @@ class GatewayWatcher:
|
||||
except Exception:
|
||||
logger.debug("Failed to send sentinel to dead subscriber")
|
||||
|
||||
def _poll_once(self, *, now: float | None = None) -> bool:
|
||||
"""Run one change-detection pass and report whether projection ran.
|
||||
|
||||
Most passes stay on the covering fingerprint. A bounded parity pass
|
||||
protects projection fields (notably role-derived CLI visibility) that
|
||||
the agent's existing index cannot see.
|
||||
"""
|
||||
db_path = self._state_db_path
|
||||
# A watcher may start before the agent has created state.db. Publishing an
|
||||
# empty first snapshot would make an already-rendered sidebar disappear;
|
||||
# wait for the first real database instead. If a previously observed DB
|
||||
# disappears, the normal projection path still publishes that change.
|
||||
if (
|
||||
not db_path.exists()
|
||||
and self._last_full_projection_at is None
|
||||
and not self._last_hash
|
||||
):
|
||||
return False
|
||||
|
||||
cheap_fp = _cheap_change_fingerprint(db_path) if db_path.exists() else ''
|
||||
current_time = time.monotonic() if now is None else now
|
||||
fingerprint_changed = cheap_fp is None or cheap_fp != self._last_cheap_fp
|
||||
parity_due = (
|
||||
self._last_full_projection_at is None
|
||||
or current_time - self._last_full_projection_at
|
||||
>= self.PROJECTION_PARITY_INTERVAL
|
||||
)
|
||||
if not fingerprint_changed and not parity_due:
|
||||
return False
|
||||
|
||||
sessions = _get_agent_sessions_from_db(db_path)
|
||||
if sessions is None:
|
||||
return False
|
||||
current_hash = _snapshot_hash(sessions)
|
||||
if cheap_fp is not None:
|
||||
self._last_cheap_fp = cheap_fp
|
||||
self._last_full_projection_at = current_time
|
||||
|
||||
if current_hash != self._last_hash:
|
||||
self._last_hash = current_hash
|
||||
self._last_sessions = sessions
|
||||
self._notify_subscribers(sessions)
|
||||
return True
|
||||
|
||||
def _poll_loop(self):
|
||||
"""Main polling loop. Runs in a daemon thread."""
|
||||
while not self._stop_event.is_set():
|
||||
try:
|
||||
self._poll_once()
|
||||
# Phase 1: cheap sessions-only fingerprint. The expensive
|
||||
# messages-JOIN projection (_get_agent_sessions_from_db) only
|
||||
# runs when this fingerprint actually changes, so an idle server
|
||||
# with a large state.db stops re-aggregating tens of thousands
|
||||
# of message rows every 5 seconds (issue #3506). A None
|
||||
# fingerprint (error / unreadable db) forces the full read so we
|
||||
# never silently skip a real change.
|
||||
db_path = self._state_db_path
|
||||
cheap_fp = _cheap_change_fingerprint(db_path) if db_path.exists() else ''
|
||||
if cheap_fp is not None and cheap_fp == self._last_cheap_fp:
|
||||
# Nothing changed in the sidebar-visible session set; skip
|
||||
# the expensive projection and the notify entirely.
|
||||
pass
|
||||
else:
|
||||
# Phase 2: only now pay for the full projection.
|
||||
sessions = _get_agent_sessions_from_db(db_path)
|
||||
current_hash = _snapshot_hash(sessions)
|
||||
if cheap_fp is not None:
|
||||
self._last_cheap_fp = cheap_fp
|
||||
|
||||
if current_hash != self._last_hash:
|
||||
self._last_hash = current_hash
|
||||
self._last_sessions = sessions
|
||||
self._notify_subscribers(sessions)
|
||||
except Exception:
|
||||
logger.debug("Error in gateway watcher poll loop", exc_info=True)
|
||||
|
||||
|
||||
+3
-105
@@ -73,99 +73,8 @@ def _profile_db(profile_home: str | Path):
|
||||
return db
|
||||
|
||||
|
||||
def _profile_home_context_api():
|
||||
"""Return the native context-local Hermes home setters when available."""
|
||||
try:
|
||||
from hermes_constants import reset_hermes_home_override, set_hermes_home_override
|
||||
except Exception: # pragma: no cover - depends on installed hermes-agent
|
||||
return None
|
||||
return set_hermes_home_override, reset_hermes_home_override
|
||||
|
||||
|
||||
def _native_profile_context_api(profile_home: str | Path):
|
||||
"""Return context setters only when SessionDB resolves that context at call time."""
|
||||
context_api = _profile_home_context_api()
|
||||
if context_api is None:
|
||||
return None
|
||||
try:
|
||||
from hermes_state import _default_db_path # type: ignore
|
||||
except Exception: # pragma: no cover - depends on installed hermes-agent
|
||||
return None
|
||||
if not callable(_default_db_path):
|
||||
return None
|
||||
|
||||
home = Path(profile_home).expanduser().resolve()
|
||||
set_home, reset_home = context_api
|
||||
try:
|
||||
token = set_home(home)
|
||||
except Exception: # pragma: no cover - depends on installed hermes-agent
|
||||
return None
|
||||
try:
|
||||
resolved_db_path = Path(_default_db_path()).expanduser().resolve()
|
||||
except Exception: # pragma: no cover - depends on installed hermes-agent
|
||||
return None
|
||||
finally:
|
||||
reset_home(token)
|
||||
if resolved_db_path != home / "state.db":
|
||||
return None
|
||||
return context_api
|
||||
|
||||
|
||||
class _ProfileGoalManager:
|
||||
"""Run the native GoalManager under a context-local profile home."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
session_id: str,
|
||||
*,
|
||||
profile_home: str | Path,
|
||||
default_max_turns: int = 20,
|
||||
context_api=None,
|
||||
):
|
||||
if _NativeGoalManager is None:
|
||||
raise RuntimeError("Hermes goal manager unavailable")
|
||||
context_api = context_api or _profile_home_context_api()
|
||||
if context_api is None:
|
||||
raise RuntimeError("Hermes profile context unavailable")
|
||||
self.session_id = session_id
|
||||
self.profile_home = Path(profile_home).expanduser().resolve()
|
||||
self._set_home, self._reset_home = context_api
|
||||
self._manager = self._scoped(
|
||||
_NativeGoalManager,
|
||||
session_id=session_id,
|
||||
default_max_turns=default_max_turns,
|
||||
)
|
||||
|
||||
def _scoped(self, func, *args, **kwargs):
|
||||
token = self._set_home(self.profile_home)
|
||||
try:
|
||||
return func(*args, **kwargs)
|
||||
finally:
|
||||
self._reset_home(token)
|
||||
|
||||
@property
|
||||
def state(self):
|
||||
return self._manager.state
|
||||
|
||||
def __getattr__(self, name):
|
||||
value = getattr(self._manager, name)
|
||||
if not callable(value):
|
||||
return value
|
||||
|
||||
def scoped_call(*args, **kwargs):
|
||||
return self._scoped(value, *args, **kwargs)
|
||||
|
||||
return scoped_call
|
||||
|
||||
def _restore_state(self, snapshot) -> None:
|
||||
from hermes_cli.goals import save_goal # type: ignore
|
||||
|
||||
self._manager._state = snapshot
|
||||
self._scoped(save_goal, self.session_id, snapshot)
|
||||
|
||||
|
||||
class _LegacyProfileGoalManager:
|
||||
"""Explicit-DB fallback for Hermes versions without profile context."""
|
||||
"""Small WebUI-local GoalManager adapter with explicit profile persistence."""
|
||||
|
||||
def __init__(self, session_id: str, *, profile_home: str | Path, default_max_turns: int = 20):
|
||||
if GoalState is None:
|
||||
@@ -284,7 +193,7 @@ class _LegacyProfileGoalManager:
|
||||
if judge_goal is None:
|
||||
verdict, reason = "continue", "goal judge unavailable"
|
||||
else:
|
||||
verdict, reason, *_ = judge_goal(state.goal, str(last_response or ""))
|
||||
verdict, reason = judge_goal(state.goal, str(last_response or ""))
|
||||
state.last_verdict = verdict
|
||||
state.last_reason = reason
|
||||
|
||||
@@ -337,15 +246,7 @@ def _manager(session_id: str, *, profile_home: str | Path | None = None):
|
||||
return None
|
||||
if profile_home and GoalManager is _NativeGoalManager and GoalState is not None:
|
||||
try:
|
||||
context_api = _native_profile_context_api(profile_home)
|
||||
if context_api is not None:
|
||||
return _ProfileGoalManager(
|
||||
session_id=session_id,
|
||||
profile_home=profile_home,
|
||||
default_max_turns=_default_max_turns(),
|
||||
context_api=context_api,
|
||||
)
|
||||
return _LegacyProfileGoalManager(
|
||||
return _ProfileGoalManager(
|
||||
session_id=session_id,
|
||||
profile_home=profile_home,
|
||||
default_max_turns=_default_max_turns(),
|
||||
@@ -506,9 +407,6 @@ def restore_goal_state(session_id: str, snapshot: Any, *, profile_home: str | Pa
|
||||
pass
|
||||
return
|
||||
if isinstance(mgr, _ProfileGoalManager):
|
||||
mgr._restore_state(snapshot)
|
||||
return
|
||||
if isinstance(mgr, _LegacyProfileGoalManager):
|
||||
mgr._state = snapshot
|
||||
mgr._save(snapshot)
|
||||
return
|
||||
|
||||
+34
-723
@@ -1,9 +1,6 @@
|
||||
"""
|
||||
Hermes Web UI -- HTTP helper functions.
|
||||
"""
|
||||
import base64 as _base64
|
||||
import binascii as _binascii
|
||||
import functools
|
||||
import json as _json
|
||||
import logging
|
||||
import os
|
||||
@@ -14,17 +11,6 @@ from api.config import IMAGE_EXTS, MD_EXTS
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_PUBLIC_MESSAGE_INTERNAL_FIELDS = frozenset({
|
||||
"api_content",
|
||||
"_row_id",
|
||||
"_state_db_row_id",
|
||||
"_db_row_id",
|
||||
"state_db_row_id",
|
||||
"_active_turn_token",
|
||||
"_active_turn_user",
|
||||
"_fork_child_turn",
|
||||
})
|
||||
|
||||
|
||||
# Treat stalled/closed HTTP clients as normal disconnects. Long-lived SSE
|
||||
# connections often end this way when a browser tab sleeps, a phone switches
|
||||
@@ -201,15 +187,6 @@ def _security_headers(handler):
|
||||
)
|
||||
|
||||
|
||||
def flush_pending_auth_cookies(handler) -> None:
|
||||
pending = getattr(handler, '_pending_set_cookies', None)
|
||||
if not pending:
|
||||
return
|
||||
handler._pending_set_cookies = []
|
||||
for cookie in pending:
|
||||
handler.send_header('Set-Cookie', cookie)
|
||||
|
||||
|
||||
def _accepts_gzip(handler) -> bool:
|
||||
"""Check if the client accepts gzip encoding."""
|
||||
headers = getattr(handler, 'headers', None)
|
||||
@@ -272,20 +249,13 @@ def j(handler, payload, status: int=200, extra_headers: dict=None, *, pretty: bo
|
||||
handler.send_header('Content-Length', str(len(body)))
|
||||
handler.send_header('Cache-Control', 'no-store')
|
||||
_security_headers(handler)
|
||||
flush_pending_auth_cookies(handler)
|
||||
if extra_headers:
|
||||
for k, v in extra_headers.items():
|
||||
handler.send_header(k, v)
|
||||
_safe_write(handler, body)
|
||||
|
||||
|
||||
def t(
|
||||
handler,
|
||||
payload,
|
||||
status: int=200,
|
||||
content_type: str='text/plain; charset=utf-8',
|
||||
extra_headers: dict=None,
|
||||
) -> None:
|
||||
def t(handler, payload, status: int=200, content_type: str='text/plain; charset=utf-8') -> None:
|
||||
"""Send a plain text or HTML response."""
|
||||
body = payload if isinstance(payload, bytes) else str(payload).encode('utf-8')
|
||||
handler.send_response(status)
|
||||
@@ -293,10 +263,6 @@ def t(
|
||||
handler.send_header('Content-Length', str(len(body)))
|
||||
handler.send_header('Cache-Control', 'no-store')
|
||||
_security_headers(handler)
|
||||
if extra_headers:
|
||||
for k, v in extra_headers.items():
|
||||
handler.send_header(k, v)
|
||||
flush_pending_auth_cookies(handler)
|
||||
_safe_write(handler, body)
|
||||
|
||||
|
||||
@@ -354,13 +320,6 @@ def _build_redact_fn():
|
||||
r"""(Authorization:\s*(?:Bearer|Bot)\s+)([^\s'",\]\)]+)""",
|
||||
_re.IGNORECASE,
|
||||
)
|
||||
# A rejected image data URI can place a syntactically valid AWS key at an
|
||||
# arbitrary base64 alignment, immediately after another base64 character.
|
||||
# The general credential regex uses token boundaries to avoid rewriting
|
||||
# prose identifiers; this defense-in-depth pass ensures the fail-closed
|
||||
# image path still removes the maintainer's embedded-suffix attack even
|
||||
# when hermes-agent is unavailable and the local fallback owns redaction.
|
||||
_EMBEDDED_AWS_ACCESS_KEY_RE = _re.compile(r"AKIA[A-Z0-9]{16}")
|
||||
_ENV_RE = _re.compile(
|
||||
r"([A-Z0-9_]{0,50}(?:API_?KEY|TOKEN|SECRET|PASSWORD|PASSWD|CREDENTIAL|AUTH)[A-Z0-9_]{0,50})"
|
||||
r"\s*=\s*(['\"]?)(\S+)\2"
|
||||
@@ -428,7 +387,6 @@ def _build_redact_fn():
|
||||
if not isinstance(text, str) or not text:
|
||||
return text
|
||||
text = _CRED_RE.sub(lambda m: _mask(m.group(1)), text)
|
||||
text = _EMBEDDED_AWS_ACCESS_KEY_RE.sub(lambda m: _mask(m.group(0)), text)
|
||||
text = _AUTH_HDR_RE.sub(lambda m: m.group(1) + _mask(m.group(2)), text)
|
||||
text = _ENV_RE.sub(_env_replacement, text)
|
||||
text = _PRIVKEY_RE.sub("[REDACTED PRIVATE KEY]", text)
|
||||
@@ -458,25 +416,7 @@ def _build_redact_fn():
|
||||
return _combined_redact
|
||||
|
||||
|
||||
_redact_fn_uncached = _build_redact_fn()
|
||||
|
||||
# Repeated dashboard polls re-request the same unchanged session payloads, so
|
||||
# the combined redactor (~15 regex passes per string) was the dominant CPU cost
|
||||
# under concurrent polling — enough to wedge the single-process server behind
|
||||
# the GIL and surface as "Mất kết nối" in the browser. The redactor is pure and
|
||||
# deterministic (force=True, fixed masking), so identical strings always map to
|
||||
# identical output and are safe to memoize without invalidation.
|
||||
_redact_fn_lru = functools.lru_cache(maxsize=4096)(_redact_fn_uncached)
|
||||
|
||||
# Cap per-entry size so a handful of giant tool-output dumps can't evict the
|
||||
# thousands of small recurring strings that actually benefit, or balloon RSS.
|
||||
_REDACT_CACHE_MAX_TEXT_LEN = 16384
|
||||
|
||||
|
||||
def _redact_fn_cached(text):
|
||||
if len(text) > _REDACT_CACHE_MAX_TEXT_LEN:
|
||||
return _redact_fn_uncached(text)
|
||||
return _redact_fn_lru(text)
|
||||
_redact_fn_cached = _build_redact_fn()
|
||||
|
||||
|
||||
_SENSITIVE_CASE_MARKERS = (
|
||||
@@ -597,360 +537,6 @@ def _redact_text(text: str, *, _enabled: bool | None = None) -> str:
|
||||
return _redact_fn_cached(text)
|
||||
|
||||
|
||||
_RASTER_IMAGE_DATA_URI_PREFIXES = (
|
||||
("data:image/png;base64,", "png"),
|
||||
("data:image/jpeg;base64,", "jpeg"),
|
||||
("data:image/jpg;base64,", "jpeg"),
|
||||
("data:image/gif;base64,", "gif"),
|
||||
("data:image/webp;base64,", "webp"),
|
||||
("data:image/bmp;base64,", "bmp"),
|
||||
)
|
||||
|
||||
|
||||
def _is_native_raster_data_uri(text: str) -> bool:
|
||||
"""Return whether *text* is one complete, canonical raster data URI.
|
||||
|
||||
Native image content is opaque binary, not text that the credential regexes
|
||||
can safely rewrite. The exemption is a credential-boundary decision, so a
|
||||
matching header or magic prefix is not enough: decode the entire canonical
|
||||
base64 payload and require the image format to terminate exactly at the end
|
||||
of the decoded bytes. Any malformed, ambiguous, or trailing content falls
|
||||
through to normal text redaction.
|
||||
"""
|
||||
if not isinstance(text, str):
|
||||
return False
|
||||
image_kind = None
|
||||
payload_start = 0
|
||||
for prefix, candidate_kind in _RASTER_IMAGE_DATA_URI_PREFIXES:
|
||||
# URI schemes and MIME type tokens are case-insensitive. Only normalize
|
||||
# this short header slice — never the multi-megabyte base64 payload.
|
||||
if text[:len(prefix)].lower() == prefix:
|
||||
image_kind = candidate_kind
|
||||
payload_start = len(prefix)
|
||||
break
|
||||
if image_kind is None:
|
||||
return False
|
||||
|
||||
payload = text[payload_start:]
|
||||
if not payload:
|
||||
return False
|
||||
try:
|
||||
raw = _base64.b64decode(payload, validate=True)
|
||||
except (_binascii.Error, ValueError):
|
||||
return False
|
||||
# validate=True rejects foreign characters and misplaced padding; this
|
||||
# round-trip also rejects non-canonical pad bits and missing/extra padding.
|
||||
if _base64.b64encode(raw).decode("ascii") != payload:
|
||||
return False
|
||||
|
||||
if image_kind == "png":
|
||||
return _is_complete_png(raw)
|
||||
if image_kind == "jpeg":
|
||||
return _is_complete_jpeg(raw)
|
||||
if image_kind == "gif":
|
||||
return _is_complete_gif(raw)
|
||||
if image_kind == "webp":
|
||||
return _is_complete_webp(raw)
|
||||
if image_kind == "bmp":
|
||||
return _is_complete_bmp(raw)
|
||||
return False
|
||||
|
||||
|
||||
def _is_complete_png(raw: bytes) -> bool:
|
||||
if not raw.startswith(b"\x89PNG\r\n\x1a\n"):
|
||||
return False
|
||||
pos = 8
|
||||
chunk_index = 0
|
||||
saw_idat = False
|
||||
while pos < len(raw):
|
||||
if pos + 12 > len(raw):
|
||||
return False
|
||||
length = int.from_bytes(raw[pos:pos + 4], "big")
|
||||
chunk_type = raw[pos + 4:pos + 8]
|
||||
data_start = pos + 8
|
||||
data_end = data_start + length
|
||||
chunk_end = data_end + 4
|
||||
if chunk_end > len(raw):
|
||||
return False
|
||||
if not all((65 <= value <= 90) or (97 <= value <= 122) for value in chunk_type):
|
||||
return False
|
||||
if not 65 <= chunk_type[2] <= 90: # PNG reserved bit must be zero.
|
||||
return False
|
||||
expected_crc = int.from_bytes(raw[data_end:chunk_end], "big")
|
||||
actual_crc = _binascii.crc32(chunk_type + raw[data_start:data_end]) & 0xFFFFFFFF
|
||||
if actual_crc != expected_crc:
|
||||
return False
|
||||
|
||||
if chunk_index == 0:
|
||||
if chunk_type != b"IHDR" or length != 13:
|
||||
return False
|
||||
width = int.from_bytes(raw[data_start:data_start + 4], "big")
|
||||
height = int.from_bytes(raw[data_start + 4:data_start + 8], "big")
|
||||
bit_depth = raw[data_start + 8]
|
||||
color_type = raw[data_start + 9]
|
||||
valid_depths = {
|
||||
0: {1, 2, 4, 8, 16},
|
||||
2: {8, 16},
|
||||
3: {1, 2, 4, 8},
|
||||
4: {8, 16},
|
||||
6: {8, 16},
|
||||
}
|
||||
if (
|
||||
not width
|
||||
or not height
|
||||
or bit_depth not in valid_depths.get(color_type, set())
|
||||
or raw[data_start + 10] != 0
|
||||
or raw[data_start + 11] != 0
|
||||
or raw[data_start + 12] not in {0, 1}
|
||||
):
|
||||
return False
|
||||
elif chunk_type == b"IHDR":
|
||||
return False
|
||||
|
||||
if chunk_type == b"IDAT":
|
||||
saw_idat = True
|
||||
if chunk_type == b"IEND":
|
||||
return length == 0 and saw_idat and chunk_end == len(raw)
|
||||
pos = chunk_end
|
||||
chunk_index += 1
|
||||
return False
|
||||
|
||||
|
||||
_JPEG_SOF_MARKERS = {
|
||||
0xC0, 0xC1, 0xC2, 0xC3,
|
||||
0xC5, 0xC6, 0xC7,
|
||||
0xC9, 0xCA, 0xCB,
|
||||
0xCD, 0xCE, 0xCF,
|
||||
}
|
||||
|
||||
|
||||
def _is_complete_jpeg(raw: bytes) -> bool:
|
||||
if len(raw) < 4 or raw[:2] != b"\xff\xd8":
|
||||
return False
|
||||
pos = 2
|
||||
saw_sof = False
|
||||
saw_scan = False
|
||||
while pos < len(raw):
|
||||
marker_start = pos
|
||||
if raw[pos] != 0xFF:
|
||||
return False
|
||||
while pos < len(raw) and raw[pos] == 0xFF:
|
||||
pos += 1
|
||||
if pos >= len(raw):
|
||||
return False
|
||||
marker = raw[pos]
|
||||
pos += 1
|
||||
if marker == 0xD9:
|
||||
return saw_sof and saw_scan and pos == len(raw)
|
||||
if marker in {0x00, 0x01, 0xD8} or 0xD0 <= marker <= 0xD7:
|
||||
return False
|
||||
if pos + 2 > len(raw):
|
||||
return False
|
||||
segment_length = int.from_bytes(raw[pos:pos + 2], "big")
|
||||
if segment_length < 2:
|
||||
return False
|
||||
segment_end = pos + segment_length
|
||||
if segment_end > len(raw):
|
||||
return False
|
||||
if marker in _JPEG_SOF_MARKERS:
|
||||
if segment_length < 8:
|
||||
return False
|
||||
saw_sof = True
|
||||
if marker != 0xDA:
|
||||
pos = segment_end
|
||||
continue
|
||||
|
||||
saw_scan = True
|
||||
pos = segment_end
|
||||
while pos < len(raw):
|
||||
if raw[pos] != 0xFF:
|
||||
pos += 1
|
||||
continue
|
||||
marker_start = pos
|
||||
while pos < len(raw) and raw[pos] == 0xFF:
|
||||
pos += 1
|
||||
if pos >= len(raw):
|
||||
return False
|
||||
scan_marker = raw[pos]
|
||||
if scan_marker == 0x00 or 0xD0 <= scan_marker <= 0xD7:
|
||||
pos += 1
|
||||
continue
|
||||
pos = marker_start
|
||||
break
|
||||
else:
|
||||
return False
|
||||
return False
|
||||
|
||||
|
||||
def _gif_subblocks_end(raw: bytes, pos: int) -> int | None:
|
||||
while pos < len(raw):
|
||||
size = raw[pos]
|
||||
pos += 1
|
||||
if size == 0:
|
||||
return pos
|
||||
if pos + size > len(raw):
|
||||
return None
|
||||
pos += size
|
||||
return None
|
||||
|
||||
|
||||
def _is_complete_gif(raw: bytes) -> bool:
|
||||
if len(raw) < 14 or not raw.startswith((b"GIF87a", b"GIF89a")):
|
||||
return False
|
||||
width = int.from_bytes(raw[6:8], "little")
|
||||
height = int.from_bytes(raw[8:10], "little")
|
||||
if not width or not height:
|
||||
return False
|
||||
packed = raw[10]
|
||||
pos = 13
|
||||
if packed & 0x80:
|
||||
pos += 3 * (1 << ((packed & 0x07) + 1))
|
||||
if pos > len(raw):
|
||||
return False
|
||||
saw_image = False
|
||||
while pos < len(raw):
|
||||
introducer = raw[pos]
|
||||
pos += 1
|
||||
if introducer == 0x3B:
|
||||
return saw_image and pos == len(raw)
|
||||
if introducer == 0x21:
|
||||
if pos >= len(raw):
|
||||
return False
|
||||
pos += 1 # extension label
|
||||
end = _gif_subblocks_end(raw, pos)
|
||||
if end is None:
|
||||
return False
|
||||
pos = end
|
||||
continue
|
||||
if introducer != 0x2C or pos + 9 > len(raw):
|
||||
return False
|
||||
image_width = int.from_bytes(raw[pos + 4:pos + 6], "little")
|
||||
image_height = int.from_bytes(raw[pos + 6:pos + 8], "little")
|
||||
image_packed = raw[pos + 8]
|
||||
if not image_width or not image_height:
|
||||
return False
|
||||
pos += 9
|
||||
if image_packed & 0x80:
|
||||
pos += 3 * (1 << ((image_packed & 0x07) + 1))
|
||||
if pos >= len(raw):
|
||||
return False
|
||||
lzw_minimum_code_size = raw[pos]
|
||||
if not 2 <= lzw_minimum_code_size <= 11:
|
||||
return False
|
||||
pos += 1
|
||||
end = _gif_subblocks_end(raw, pos)
|
||||
if end is None:
|
||||
return False
|
||||
pos = end
|
||||
saw_image = True
|
||||
return False
|
||||
|
||||
|
||||
def _is_webp_image_chunk(chunk_type: bytes, data: bytes) -> bool:
|
||||
if chunk_type == b"VP8 ":
|
||||
if len(data) < 10 or data[3:6] != b"\x9d\x01\x2a":
|
||||
return False
|
||||
width = int.from_bytes(data[6:8], "little") & 0x3FFF
|
||||
height = int.from_bytes(data[8:10], "little") & 0x3FFF
|
||||
return bool(width and height)
|
||||
if chunk_type == b"VP8L":
|
||||
# The three high bits of the fifth byte are the version number. The
|
||||
# current lossless bitstream defines only version zero.
|
||||
return len(data) >= 5 and data[0] == 0x2F and not data[4] & 0xE0
|
||||
return False
|
||||
|
||||
|
||||
def _is_complete_webp_frame(data: bytes) -> bool:
|
||||
"""Validate the nested chunks in one extended-WebP animation frame."""
|
||||
if len(data) < 16 or data[15] & 0xFC:
|
||||
return False
|
||||
pos = 16
|
||||
saw_image = False
|
||||
while pos < len(data):
|
||||
if pos + 8 > len(data):
|
||||
return False
|
||||
chunk_type = data[pos:pos + 4]
|
||||
chunk_size = int.from_bytes(data[pos + 4:pos + 8], "little")
|
||||
data_start = pos + 8
|
||||
data_end = data_start + chunk_size
|
||||
chunk_end = data_end + (chunk_size & 1)
|
||||
if chunk_end > len(data):
|
||||
return False
|
||||
chunk_data = data[data_start:data_end]
|
||||
if chunk_type in {b"VP8 ", b"VP8L"}:
|
||||
if saw_image or not _is_webp_image_chunk(chunk_type, chunk_data):
|
||||
return False
|
||||
saw_image = True
|
||||
elif chunk_type != b"ALPH":
|
||||
return False
|
||||
pos = chunk_end
|
||||
return saw_image and pos == len(data)
|
||||
|
||||
|
||||
def _is_complete_webp(raw: bytes) -> bool:
|
||||
if (
|
||||
len(raw) < 20
|
||||
or raw[:4] != b"RIFF"
|
||||
or raw[8:12] != b"WEBP"
|
||||
or int.from_bytes(raw[4:8], "little") + 8 != len(raw)
|
||||
):
|
||||
return False
|
||||
pos = 12
|
||||
saw_image = False
|
||||
while pos < len(raw):
|
||||
if pos + 8 > len(raw):
|
||||
return False
|
||||
chunk_type = raw[pos:pos + 4]
|
||||
chunk_size = int.from_bytes(raw[pos + 4:pos + 8], "little")
|
||||
data_start = pos + 8
|
||||
data_end = data_start + chunk_size
|
||||
chunk_end = data_end + (chunk_size & 1)
|
||||
if chunk_end > len(raw):
|
||||
return False
|
||||
data = raw[data_start:data_end]
|
||||
if chunk_type in {b"VP8 ", b"VP8L"}:
|
||||
if saw_image or not _is_webp_image_chunk(chunk_type, data):
|
||||
return False
|
||||
saw_image = True
|
||||
elif chunk_type == b"VP8X":
|
||||
if len(data) != 10 or data[0] & 0x81 or any(data[1:4]):
|
||||
return False
|
||||
elif chunk_type == b"ANMF":
|
||||
if not _is_complete_webp_frame(data):
|
||||
return False
|
||||
saw_image = True
|
||||
pos = chunk_end
|
||||
return saw_image and pos == len(raw)
|
||||
|
||||
|
||||
def _is_complete_bmp(raw: bytes) -> bool:
|
||||
if len(raw) < 26 or raw[:2] != b"BM":
|
||||
return False
|
||||
if int.from_bytes(raw[2:6], "little") != len(raw):
|
||||
return False
|
||||
pixel_offset = int.from_bytes(raw[10:14], "little")
|
||||
dib_size = int.from_bytes(raw[14:18], "little")
|
||||
if dib_size == 12:
|
||||
width = int.from_bytes(raw[18:20], "little")
|
||||
height = int.from_bytes(raw[20:22], "little")
|
||||
planes = int.from_bytes(raw[22:24], "little")
|
||||
bits_per_pixel = int.from_bytes(raw[24:26], "little")
|
||||
elif dib_size >= 40 and 14 + dib_size <= len(raw):
|
||||
width = int.from_bytes(raw[18:22], "little", signed=True)
|
||||
height = int.from_bytes(raw[22:26], "little", signed=True)
|
||||
planes = int.from_bytes(raw[26:28], "little")
|
||||
bits_per_pixel = int.from_bytes(raw[28:30], "little")
|
||||
else:
|
||||
return False
|
||||
return (
|
||||
bool(width)
|
||||
and bool(height)
|
||||
and planes == 1
|
||||
and bits_per_pixel in {1, 2, 4, 8, 16, 24, 32}
|
||||
and 14 + dib_size <= pixel_offset < len(raw)
|
||||
)
|
||||
|
||||
|
||||
def _redact_value(v, *, _enabled: bool | None = None):
|
||||
"""Recursively redact credentials from strings, dicts, and lists.
|
||||
|
||||
@@ -960,287 +546,41 @@ def _redact_value(v, *, _enabled: bool | None = None):
|
||||
if isinstance(v, str):
|
||||
return _redact_text(v, _enabled=_enabled)
|
||||
if isinstance(v, dict):
|
||||
return {key: _redact_value(value, _enabled=_enabled) for key, value in v.items()}
|
||||
return {k: _redact_value(val, _enabled=_enabled) for k, val in v.items()}
|
||||
if isinstance(v, list):
|
||||
return [_redact_value(item, _enabled=_enabled) for item in v]
|
||||
return v
|
||||
|
||||
|
||||
def _redact_message_content_part(part, *, _enabled: bool):
|
||||
"""Redact one canonical ``messages[*].content[*]`` part.
|
||||
|
||||
The raster exemption exists only at this authoritative schema position.
|
||||
Image-shaped dictionaries in metadata, tools, todos, journals, or arbitrary
|
||||
nested values remain on the normal fail-closed redaction path.
|
||||
"""
|
||||
if not _enabled or not (
|
||||
isinstance(part, dict)
|
||||
and part.get("type") == "image_url"
|
||||
and isinstance(part.get("image_url"), dict)
|
||||
):
|
||||
return _redact_value(part, _enabled=_enabled)
|
||||
result = {}
|
||||
for key, value in part.items():
|
||||
if key != "image_url":
|
||||
result[key] = _redact_value(value, _enabled=_enabled)
|
||||
continue
|
||||
result[key] = {
|
||||
image_key: image_value
|
||||
if image_key == "url" and _is_native_raster_data_uri(image_value)
|
||||
else _redact_value(image_value, _enabled=_enabled)
|
||||
for image_key, image_value in value.items()
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
def _scrub_alias_record(record):
|
||||
"""Copy one schema record while removing private replay aliases.
|
||||
|
||||
This deliberately copies only one record level. Values such as a tool's
|
||||
``args`` or a function's opaque ``arguments`` are business payloads, not
|
||||
nested WebUI records, so parsing or recursively walking them would corrupt
|
||||
valid user data.
|
||||
"""
|
||||
if not isinstance(record, dict):
|
||||
return _copy_json_value(record)
|
||||
return {
|
||||
key: _copy_json_value(value)
|
||||
for key, value in record.items()
|
||||
if key not in _PUBLIC_MESSAGE_INTERNAL_FIELDS
|
||||
}
|
||||
|
||||
|
||||
def _scrub_content_part(part):
|
||||
"""Scrub one canonical ``messages[*].content[*]`` part."""
|
||||
return _scrub_alias_record(part)
|
||||
|
||||
|
||||
def _scrub_function_record(function):
|
||||
"""Scrub a tool-call function envelope without parsing arguments."""
|
||||
return _scrub_alias_record(function)
|
||||
|
||||
|
||||
def _scrub_tool_call_record(tool_call):
|
||||
"""Scrub one canonical tool-call record and its function envelope."""
|
||||
result = _scrub_alias_record(tool_call)
|
||||
if not isinstance(tool_call, dict):
|
||||
return result
|
||||
function = tool_call.get("function")
|
||||
if isinstance(function, dict):
|
||||
result["function"] = _scrub_function_record(function)
|
||||
return result
|
||||
|
||||
|
||||
def _scrub_message_record(message, *, preserve_api_content: bool = False):
|
||||
"""Scrub one message record along its authoritative nested schema paths."""
|
||||
if not isinstance(message, dict):
|
||||
return _copy_json_value(message)
|
||||
result = _scrub_alias_record(message)
|
||||
if preserve_api_content and isinstance(message.get("api_content"), str) and message.get("api_content"):
|
||||
result["api_content"] = message["api_content"]
|
||||
content = message.get("content")
|
||||
if isinstance(content, list):
|
||||
result["content"] = [_scrub_content_part(part) for part in content]
|
||||
tool_calls = message.get("tool_calls")
|
||||
if isinstance(tool_calls, list):
|
||||
result["tool_calls"] = [_scrub_tool_call_record(call) for call in tool_calls]
|
||||
return result
|
||||
|
||||
|
||||
def _scrub_message_records(messages, *, preserve_api_content: bool = False):
|
||||
if not isinstance(messages, list):
|
||||
return _copy_json_value(messages)
|
||||
return [
|
||||
_scrub_message_record(message, preserve_api_content=preserve_api_content)
|
||||
for message in messages
|
||||
]
|
||||
|
||||
|
||||
def _scrub_tool_call_records(tool_calls):
|
||||
if not isinstance(tool_calls, list):
|
||||
return _copy_json_value(tool_calls)
|
||||
return [_scrub_tool_call_record(call) for call in tool_calls]
|
||||
|
||||
|
||||
def _scrub_runtime_journal_snapshot(snapshot):
|
||||
"""Scrub only the snapshot's real message/tool-call arrays.
|
||||
|
||||
A live tool's ``args`` dictionary is intentionally opaque. In particular,
|
||||
``tool_calls[].args.messages[]`` is ordinary business input even though the
|
||||
field name happens to be ``messages``; it is not a transcript container.
|
||||
"""
|
||||
if not isinstance(snapshot, dict):
|
||||
return _copy_json_value(snapshot)
|
||||
result = _copy_json_value(snapshot)
|
||||
if isinstance(snapshot.get("messages"), list):
|
||||
result["messages"] = _scrub_message_records(snapshot["messages"])
|
||||
if isinstance(snapshot.get("tool_calls"), list):
|
||||
result["tool_calls"] = _scrub_tool_call_records(snapshot["tool_calls"])
|
||||
return result
|
||||
|
||||
|
||||
def scrub_internal_replay_fields(
|
||||
value,
|
||||
*,
|
||||
preserve_message_api_content: bool = False,
|
||||
message_records: bool | None = None,
|
||||
):
|
||||
"""Copy runtime data and scrub aliases at authoritative schema positions.
|
||||
|
||||
``message_records`` selects the shape of a bare list (messages versus
|
||||
session-level tool calls). A session-shaped dictionary follows only its
|
||||
named ``messages``, ``context_messages``, ``tool_calls``, and
|
||||
``runtime_journal_snapshot`` fields. No generic key-name recursion is
|
||||
performed, so arbitrary nested tool arguments remain byte-for-byte intact.
|
||||
The Agent boundary may retain a message record's trusted ``api_content``;
|
||||
public/import boundaries use the default, stripping it everywhere.
|
||||
"""
|
||||
if isinstance(value, list):
|
||||
if message_records is False:
|
||||
return _scrub_tool_call_records(value)
|
||||
return _scrub_message_records(
|
||||
value,
|
||||
preserve_api_content=preserve_message_api_content,
|
||||
)
|
||||
if not isinstance(value, dict):
|
||||
return _copy_json_value(value)
|
||||
result = _copy_json_value(value)
|
||||
for key, child in value.items():
|
||||
if key in {"messages", "context_messages"} and isinstance(child, list):
|
||||
result[key] = _scrub_message_records(
|
||||
child,
|
||||
preserve_api_content=preserve_message_api_content,
|
||||
)
|
||||
elif key == "tool_calls" and isinstance(child, list):
|
||||
result[key] = _scrub_tool_call_records(child)
|
||||
elif key == "runtime_journal_snapshot" and isinstance(child, dict):
|
||||
result[key] = _scrub_runtime_journal_snapshot(child)
|
||||
return result
|
||||
|
||||
|
||||
def _public_message_projection(message, *, _enabled: bool, _active_turn_token=None):
|
||||
"""Return one public transcript message without internal replay fields."""
|
||||
is_active = (
|
||||
isinstance(message, dict)
|
||||
and message.get("role") == "user"
|
||||
and _active_turn_token is not None
|
||||
and message.get("_active_turn_token") == _active_turn_token
|
||||
)
|
||||
message = scrub_internal_replay_fields([message], message_records=True)[0]
|
||||
if not isinstance(message, dict):
|
||||
return _redact_value(message, _enabled=_enabled)
|
||||
item = {}
|
||||
allow_native_image = message.get("role") == "user"
|
||||
for key, value in message.items():
|
||||
if key in _PUBLIC_MESSAGE_INTERNAL_FIELDS:
|
||||
continue
|
||||
if allow_native_image and key == "content" and isinstance(value, list):
|
||||
item[key] = [
|
||||
_redact_message_content_part(part, _enabled=_enabled)
|
||||
for part in value
|
||||
]
|
||||
else:
|
||||
item[key] = _redact_value(value, _enabled=_enabled)
|
||||
if is_active:
|
||||
item["_active_turn_user"] = True
|
||||
return item
|
||||
|
||||
|
||||
def _redact_messages(messages, *, _enabled: bool, _active_turn_token=None):
|
||||
if not isinstance(messages, list):
|
||||
return _redact_value(messages, _enabled=_enabled)
|
||||
return [
|
||||
_public_message_projection(
|
||||
message,
|
||||
_enabled=_enabled,
|
||||
_active_turn_token=_active_turn_token,
|
||||
)
|
||||
for message in messages
|
||||
]
|
||||
|
||||
|
||||
def _redact_tool_calls(tool_calls, *, _enabled: bool):
|
||||
scrubbed = scrub_internal_replay_fields(tool_calls, message_records=False)
|
||||
return _redact_value(scrubbed, _enabled=_enabled)
|
||||
|
||||
|
||||
def _redact_nested_message_containers(value, *, _enabled: bool):
|
||||
"""Redact only the runtime snapshot's authoritative message arrays."""
|
||||
scrubbed = scrub_internal_replay_fields(value)
|
||||
if not isinstance(scrubbed, dict):
|
||||
return _redact_value(scrubbed, _enabled=_enabled)
|
||||
result = {}
|
||||
for key, child in scrubbed.items():
|
||||
if key in {"messages", "context_messages"} and isinstance(child, list):
|
||||
result[key] = _redact_messages(child, _enabled=_enabled)
|
||||
elif key == "tool_calls" and isinstance(child, list):
|
||||
result[key] = _redact_tool_calls(child, _enabled=_enabled)
|
||||
elif key == "runtime_journal_snapshot" and isinstance(child, dict):
|
||||
result[key] = _redact_nested_message_containers(child, _enabled=_enabled)
|
||||
else:
|
||||
result[key] = _redact_value(child, _enabled=_enabled)
|
||||
return result
|
||||
|
||||
|
||||
def public_session_projection(session_dict: dict) -> dict:
|
||||
"""Return a public session payload with redaction and alias stripping.
|
||||
|
||||
Callers use this for every response/export/SSE session payload. It never
|
||||
mutates the in-memory session or the caller's dictionary.
|
||||
"""
|
||||
return redact_session_data(session_dict)
|
||||
|
||||
|
||||
def strip_public_internal_fields(value, *, message_records: bool = False):
|
||||
"""Deep-copy imported records through the shared schema scrubber.
|
||||
|
||||
JSON import uses this before constructing or saving a ``Session``. The
|
||||
Five replay aliases belong to a message/content-part/tool-call/function
|
||||
record itself; matching names inside user content or tool arguments are
|
||||
ordinary JSON and must be preserved. This is intentionally independent of
|
||||
the credential-redaction setting: caller-supplied provider sidecars must
|
||||
never become durable WebUI session state.
|
||||
"""
|
||||
return scrub_internal_replay_fields(value, message_records=message_records)
|
||||
|
||||
|
||||
def _copy_json_value(value):
|
||||
"""Deep-copy JSON-shaped data without applying message-field filtering."""
|
||||
if isinstance(value, dict):
|
||||
return {key: _copy_json_value(child) for key, child in value.items()}
|
||||
if isinstance(value, list):
|
||||
return [_copy_json_value(child) for child in value]
|
||||
return value
|
||||
|
||||
|
||||
def redact_session_data(session_dict: dict) -> dict:
|
||||
"""Redact credentials in the public session response without mutation."""
|
||||
"""Redact credentials from message content, tool data, and session sidecars.
|
||||
|
||||
Applies to: messages[], tool_calls[], todo_state, runtime_journal_snapshot,
|
||||
and title.
|
||||
The underlying session file is not modified; redaction is response-layer only.
|
||||
|
||||
Reads the ``api_redact_enabled`` setting ONCE for the entire response and
|
||||
threads it through to avoid hundreds of settings.json reads per session
|
||||
payload (a 50-message session has hundreds of nested strings). When the
|
||||
setting is disabled this is also a fast path: the recursion still walks
|
||||
but every string returns early.
|
||||
"""
|
||||
from api.config import load_settings
|
||||
_enabled = bool(load_settings().get("api_redact_enabled", True))
|
||||
if not isinstance(session_dict, dict):
|
||||
return {}
|
||||
result = {}
|
||||
from api.process_event_utils import build_active_turn_token
|
||||
_active_turn_token = build_active_turn_token(session_dict.get("active_stream_id"), session_dict.get("pending_started_at"))
|
||||
for key, value in session_dict.items():
|
||||
if key in _PUBLIC_MESSAGE_INTERNAL_FIELDS:
|
||||
continue
|
||||
if key == 'title' and isinstance(value, str):
|
||||
result[key] = _redact_text(value, _enabled=_enabled)
|
||||
elif key in {'messages', 'context_messages'}:
|
||||
result[key] = _redact_messages(value, _enabled=_enabled, _active_turn_token=_active_turn_token)
|
||||
elif key == 'tool_calls' and isinstance(value, list):
|
||||
result[key] = _redact_tool_calls(value, _enabled=_enabled)
|
||||
elif key in {'todo_state', 'runtime_journal_snapshot'}:
|
||||
result[key] = _redact_nested_message_containers(value, _enabled=_enabled)
|
||||
else:
|
||||
# Operational fields (workspace path, ids, config, timestamps, etc.)
|
||||
# are NOT credential-masked: a valid workspace path may legitimately
|
||||
# contain a credential-shaped component, and masking it would corrupt
|
||||
# the authoritative value the client echoes back on the next send.
|
||||
# Deep-copy them through unchanged (only transcript-bearing fields
|
||||
# above carry free-form user/model text worth redacting).
|
||||
result[key] = _copy_json_value(value)
|
||||
result = dict(session_dict)
|
||||
if isinstance(result.get('title'), str):
|
||||
result['title'] = _redact_text(result['title'], _enabled=_enabled)
|
||||
if 'messages' in result:
|
||||
result['messages'] = _redact_value(result['messages'], _enabled=_enabled)
|
||||
if 'tool_calls' in result:
|
||||
result['tool_calls'] = _redact_value(result['tool_calls'], _enabled=_enabled)
|
||||
if 'todo_state' in result:
|
||||
result['todo_state'] = _redact_value(result['todo_state'], _enabled=_enabled)
|
||||
if 'runtime_journal_snapshot' in result:
|
||||
result['runtime_journal_snapshot'] = _redact_value(
|
||||
result['runtime_journal_snapshot'],
|
||||
_enabled=_enabled,
|
||||
)
|
||||
return result
|
||||
|
||||
|
||||
@@ -1268,18 +608,10 @@ def read_body(handler) -> dict:
|
||||
pass
|
||||
raise ValueError(f'Request body too large ({length} bytes, max {MAX_BODY_BYTES})')
|
||||
raw = handler.rfile.read(length) if length else b'{}'
|
||||
# A body-optional endpoint (e.g. DELETE /api/mcp/servers/{name}) may send a
|
||||
# whitespace-only body; treat it like an empty body ({}) rather than a 400,
|
||||
# matching the pre-#7336 lenient behavior for that shape (see gate finding).
|
||||
if not raw.strip():
|
||||
return {}
|
||||
try:
|
||||
parsed = _json.loads(raw)
|
||||
return _json.loads(raw)
|
||||
except Exception:
|
||||
raise ValueError('Invalid JSON body') from None
|
||||
if not isinstance(parsed, dict):
|
||||
raise ValueError('JSON body must be an object')
|
||||
return parsed
|
||||
return {}
|
||||
|
||||
|
||||
# ── Profile cookie helpers (issue #798) ─────────────────────────────────────
|
||||
@@ -1360,7 +692,7 @@ def get_profile_cookie(handler) -> str | None:
|
||||
return raw_val if _valid_profile_name(raw_val) else None
|
||||
|
||||
|
||||
def build_profile_cookie(name: str, handler=None, *, session_cookie_value: str | None = None) -> str:
|
||||
def build_profile_cookie(name: str, handler=None) -> str:
|
||||
"""Build a Set-Cookie header value for the active-profile cookie.
|
||||
|
||||
Always persist the selected profile in the cookie, including 'default'.
|
||||
@@ -1386,16 +718,8 @@ def build_profile_cookie(name: str, handler=None, *, session_cookie_value: str |
|
||||
except Exception:
|
||||
_auth_on = False
|
||||
if _auth_on and handler is None:
|
||||
if session_cookie_value is None:
|
||||
raise RuntimeError("build_profile_cookie requires a request handler when auth is enabled (to bind the profile cookie to the session)")
|
||||
if session_cookie_value is not None:
|
||||
try:
|
||||
from api.auth import sign_profile_cookie_value
|
||||
value = sign_profile_cookie_value(name, session_cookie_value)
|
||||
except Exception as exc:
|
||||
logger.warning("Failed to sign active profile cookie", exc_info=True)
|
||||
raise RuntimeError("could not sign active profile cookie") from exc
|
||||
elif handler is not None:
|
||||
raise RuntimeError("build_profile_cookie requires a request handler when auth is enabled (to bind the profile cookie to the session)")
|
||||
if handler is not None:
|
||||
try:
|
||||
from api.auth import is_auth_enabled, parse_cookie, sign_profile_cookie_value
|
||||
if is_auth_enabled():
|
||||
@@ -1408,16 +732,3 @@ def build_profile_cookie(name: str, handler=None, *, session_cookie_value: str |
|
||||
cookie[cookie_name]['httponly'] = True
|
||||
cookie[cookie_name]['samesite'] = 'Lax'
|
||||
return cookie[cookie_name].OutputString()
|
||||
|
||||
|
||||
def clear_profile_cookie(handler) -> None:
|
||||
import http.cookies as _hc
|
||||
|
||||
cookie = _hc.SimpleCookie()
|
||||
cookie_name = get_profile_cookie_name()
|
||||
cookie[cookie_name] = ''
|
||||
cookie[cookie_name]['path'] = '/'
|
||||
cookie[cookie_name]['httponly'] = True
|
||||
cookie[cookie_name]['samesite'] = 'Lax'
|
||||
cookie[cookie_name]['max-age'] = '0'
|
||||
handler.send_header('Set-Cookie', cookie[cookie_name].OutputString())
|
||||
|
||||
@@ -20,7 +20,6 @@ from dataclasses import asdict, is_dataclass
|
||||
from urllib.parse import parse_qs, unquote
|
||||
|
||||
from api.helpers import bad, j
|
||||
from api.workspace import resolve_trusted_workspace
|
||||
|
||||
BOARD_COLUMNS = ["triage", "todo", "ready", "running", "blocked", "done"]
|
||||
_TASK_PREFIX = "/api/kanban/tasks/"
|
||||
@@ -856,13 +855,6 @@ def _create_board_payload(body):
|
||||
slug = str(body.get("slug") or "").strip()
|
||||
if not slug:
|
||||
raise ValueError("slug is required")
|
||||
board_kwargs = {}
|
||||
if "default_workdir" in body:
|
||||
raw_workdir = str(body.get("default_workdir") or "").strip()
|
||||
if raw_workdir:
|
||||
board_kwargs["default_workdir"] = str(resolve_trusted_workspace(raw_workdir))
|
||||
else:
|
||||
board_kwargs["default_workdir"] = ""
|
||||
try:
|
||||
meta = kb.create_board(
|
||||
slug,
|
||||
@@ -870,7 +862,6 @@ def _create_board_payload(body):
|
||||
description=body.get("description") or None,
|
||||
icon=body.get("icon") or None,
|
||||
color=body.get("color") or None,
|
||||
**board_kwargs,
|
||||
)
|
||||
except (ValueError, AttributeError) as exc:
|
||||
raise ValueError(str(exc)) from exc
|
||||
@@ -903,10 +894,6 @@ def _update_board_payload(slug, body):
|
||||
raise ValueError(f"invalid board slug: {slug!r}") from exc
|
||||
if not normed or not kb.board_exists(normed):
|
||||
raise LookupError(f"board {slug!r} does not exist")
|
||||
board_kwargs = {}
|
||||
if "default_workdir" in body:
|
||||
raw_workdir = str(body.get("default_workdir") or "").strip()
|
||||
board_kwargs["default_workdir"] = str(resolve_trusted_workspace(raw_workdir)) if raw_workdir else ""
|
||||
archived = body.get("archived")
|
||||
if isinstance(archived, str):
|
||||
archived = archived.strip().lower() in {"1", "true", "yes", "on"}
|
||||
@@ -917,7 +904,6 @@ def _update_board_payload(slug, body):
|
||||
icon=body.get("icon"),
|
||||
color=body.get("color"),
|
||||
archived=archived if isinstance(archived, bool) else None,
|
||||
**board_kwargs,
|
||||
)
|
||||
return {"board": _board_meta_dict(meta), "read_only": False}
|
||||
|
||||
|
||||
@@ -1,500 +0,0 @@
|
||||
"""Per-message media snapshots: freeze file bytes at message-settle time.
|
||||
|
||||
Problem
|
||||
-------
|
||||
``/api/media`` serves a file's CURRENT bytes. Since the ETag-revalidation
|
||||
work (#6922) made browsers revalidate on every use, an in-place overwrite of
|
||||
``report.html`` also rewrites every historical chat preview that referenced
|
||||
it — the user loses the old/new comparison they had when the agent emitted
|
||||
the file twice under the same name.
|
||||
|
||||
This module is the storage half of the fix. When a turn settles
|
||||
(``api/streaming.py``), every local-file ``MEDIA:`` reference in the new
|
||||
assistant messages is snapshotted into a content-addressed store:
|
||||
|
||||
<STATE_DIR>/media_snapshots/<sha256>.snap
|
||||
|
||||
Content addressing gives free dedup (an unchanged file re-settles to the
|
||||
same digest and the copy is skipped) and makes every stored object
|
||||
IMMUTABLE — the serving side can therefore cache snapshots aggressively.
|
||||
Messages carry a ``_media_snapshots`` annotation mapping absolute path to
|
||||
digest (sidecar JSON preserves extra message fields); the frontend appends
|
||||
``&snap=<digest>`` to ``/api/media`` URLs and ``api/routes.py`` serves the
|
||||
stored bytes instead of the live file.
|
||||
|
||||
Security model
|
||||
--------------
|
||||
* Digests are validated by whole-string ``fullmatch`` against ``[0-9a-f]{64}``
|
||||
before any path is built — a crafted ``snap=`` value cannot traverse out of
|
||||
the store.
|
||||
* Capture uses the SAME deny predicate as the ``/api/media`` serve path
|
||||
(``routes._media_deny_reason``): anything the endpoint refuses to serve is
|
||||
never captured in the first place.
|
||||
* Every captured digest carries a server-owned source-path binding; a digest
|
||||
is only ever served back for the exact canonical path it was captured from,
|
||||
so it can never act as a bearer capability through a different allowed path.
|
||||
* The store lives under STATE_DIR; capture is restricted to files that are
|
||||
regular files within caller-approved roots (the caller reuses the same
|
||||
allow-list reasoning as ``/api/media`` — this module never decides what a
|
||||
user may see, only stores bytes it is handed).
|
||||
* Writes are tmp-file + fsync + rename: a crash never leaves a torn
|
||||
snapshot that would serve corrupt bytes forever (content addressing means
|
||||
a torn file could never be overwritten with good bytes later).
|
||||
|
||||
Caps
|
||||
----
|
||||
* Per-file cap (default 50 MB): larger files are not snapshotted; previews
|
||||
for them gracefully degrade to the live file.
|
||||
* Total store cap (default 2 GB): when exceeded after a capture, oldest
|
||||
snapshots (by mtime) are evicted until the store fits. Eviction only
|
||||
degrades OLD previews to live-file behaviour; it never corrupts anything.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger("hermes.webui")
|
||||
|
||||
# Strict whole-string shape. ``\\Z`` (not ``$``) so a terminal newline cannot
|
||||
# sneak past the gate; ``fullmatch`` is used at call sites.
|
||||
_DIGEST_RE = re.compile(r"[0-9a-f]{64}\Z")
|
||||
|
||||
# Default caps. Overridable via env var for operators with unusual disks.
|
||||
DEFAULT_MAX_FILE_BYTES = 50 * 1024 * 1024 # 50 MB per snapshot
|
||||
DEFAULT_TOTAL_CAP_BYTES = 2 * 1024 * 1024 * 1024 # 2 GB total store
|
||||
|
||||
_SNAPSHOT_DIR_ENV = "HERMES_WEBUI_MEDIA_SNAPSHOT_DIR"
|
||||
|
||||
# Capture and eviction run on the streaming worker thread; serialize them so
|
||||
# two concurrent settles cannot race the same tmp file or the quota scan.
|
||||
_LOCK = threading.Lock()
|
||||
|
||||
def media_capture_allowed(path: Path) -> bool:
|
||||
"""Allow-list predicate for snapshot capture (same gate as serve).
|
||||
|
||||
True only when ``path`` is a regular file inside an allowed root and NOT
|
||||
inside a denied Hermes-internal state location. The deny half is the
|
||||
SAME predicate the ``/api/media`` serve path uses (``routes._media_deny_reason``,
|
||||
the #3234 state/profile deny set): anything the endpoint would refuse to
|
||||
serve is never captured in the first place — capture and serve can never
|
||||
diverge on what is denied. Any failure mode returns False —
|
||||
snapshotting is best-effort durability, never a reason to widen file
|
||||
access.
|
||||
"""
|
||||
import stat as stat_mod
|
||||
|
||||
try:
|
||||
resolved = path.resolve()
|
||||
st = resolved.stat()
|
||||
except OSError:
|
||||
return False
|
||||
if not stat_mod.S_ISREG(st.st_mode):
|
||||
return False
|
||||
|
||||
within_any_root = any(
|
||||
_path_within(resolved, root) for root in _allowed_roots_for_capture()
|
||||
)
|
||||
if not within_any_root:
|
||||
return False
|
||||
# Authoritative deny parity with the serve path (#6979 Round 2 MUST-FIX 1):
|
||||
# STATE_DIR subdirs, webui_state subdirs, named-profile roots, secret
|
||||
# basenames and the snapshot store itself are all denied exactly as the
|
||||
# route denies them.
|
||||
try:
|
||||
from api.routes import _media_deny_reason
|
||||
|
||||
if _media_deny_reason(resolved):
|
||||
return False
|
||||
except Exception:
|
||||
return False # fail closed: never snapshot when the gate is unclear
|
||||
return True
|
||||
|
||||
|
||||
def _allowed_roots_for_capture() -> list[Path]:
|
||||
"""Roots capture is permitted in — same shape as ``/api/media``'s list."""
|
||||
roots: list[Path] = []
|
||||
home = Path(os.path.expanduser("~"))
|
||||
hermes_home = Path(os.getenv("HERMES_HOME", str(home / ".hermes"))).expanduser()
|
||||
for candidate in (hermes_home, Path("/tmp"), home / ".hermes"):
|
||||
try:
|
||||
resolved = candidate.resolve()
|
||||
except OSError:
|
||||
continue
|
||||
if resolved not in roots:
|
||||
roots.append(resolved)
|
||||
try:
|
||||
from api.workspace import get_last_workspace
|
||||
|
||||
ws = Path(get_last_workspace()).resolve()
|
||||
if ws.is_dir() and ws not in roots:
|
||||
roots.append(ws)
|
||||
except Exception:
|
||||
pass
|
||||
extra = os.environ.get("MEDIA_ALLOWED_ROOTS", "").strip()
|
||||
if extra:
|
||||
for root in extra.split(os.pathsep):
|
||||
root = root.strip()
|
||||
if not root:
|
||||
continue
|
||||
try:
|
||||
rp = Path(root).resolve()
|
||||
except OSError:
|
||||
continue
|
||||
if rp.is_dir() and rp not in roots:
|
||||
roots.append(rp)
|
||||
return roots
|
||||
|
||||
|
||||
def _path_within(child: Path, root: Path) -> bool:
|
||||
try:
|
||||
child.resolve().relative_to(root)
|
||||
return True
|
||||
except (ValueError, OSError):
|
||||
return False
|
||||
|
||||
|
||||
def resolve_media_ref(raw_ref: str) -> Path | None:
|
||||
"""Map a raw ``MEDIA:`` token to an absolute local path, or None.
|
||||
|
||||
Only bare local paths and ``file://`` URLs resolve; http(s)/data/other
|
||||
schemes return None (nothing to snapshot — they are not server files).
|
||||
``~`` is expanded. No existence check: callers decide whether absence
|
||||
matters (capture skips missing files).
|
||||
"""
|
||||
ref = str(raw_ref or "").strip()
|
||||
if not ref:
|
||||
return None
|
||||
if ref.startswith("data:") or re.match(r"^[a-zA-Z][a-zA-Z0-9+.-]*://", ref):
|
||||
if ref.lower().startswith("file://"):
|
||||
from urllib.parse import unquote, urlparse
|
||||
|
||||
try:
|
||||
parsed = urlparse(ref)
|
||||
ref = unquote(parsed.path or "")
|
||||
except Exception:
|
||||
ref = ref[len("file://"):]
|
||||
else:
|
||||
return None
|
||||
if not ref or ref.startswith(("http:", "https:")):
|
||||
return None
|
||||
try:
|
||||
return Path(ref).expanduser().resolve()
|
||||
except (OSError, RuntimeError):
|
||||
return None
|
||||
|
||||
|
||||
def _default_snapshot_dir() -> Path:
|
||||
from api.config import STATE_DIR
|
||||
|
||||
return Path(STATE_DIR) / "media_snapshots"
|
||||
|
||||
|
||||
def get_snapshot_dir() -> Path:
|
||||
"""Snapshot store root (created lazily by capture)."""
|
||||
override = os.getenv(_SNAPSHOT_DIR_ENV, "").strip()
|
||||
if override:
|
||||
return Path(override).expanduser()
|
||||
return _default_snapshot_dir()
|
||||
|
||||
|
||||
def is_valid_digest(digest: str) -> bool:
|
||||
"""Strict digest shape check — the ONLY gate before path construction.
|
||||
|
||||
``fullmatch`` with a ``\\Z``-anchored pattern: ``$`` would also match
|
||||
before a terminal newline, letting ``<64 hex>\\n`` slip through.
|
||||
"""
|
||||
try:
|
||||
return _DIGEST_RE.fullmatch(str(digest or "")) is not None
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
|
||||
|
||||
def snapshot_path_for_digest(digest: str) -> Path | None:
|
||||
"""Return the on-disk path for a digest, or None if absent/invalid.
|
||||
|
||||
Never creates anything; serving uses this to decide snapshot vs live-file
|
||||
fallback.
|
||||
"""
|
||||
if not is_valid_digest(digest):
|
||||
return None
|
||||
candidate = get_snapshot_dir() / f"{digest}.snap"
|
||||
return candidate if candidate.is_file() else None
|
||||
|
||||
|
||||
def _binding_path_for_digest(digest: str) -> Path:
|
||||
"""Sidecar holding the source-path set for a digest (``<digest>.src.json``)."""
|
||||
return get_snapshot_dir() / f"{digest}.src.json"
|
||||
|
||||
|
||||
def _record_source_binding(digest: str, source: Path) -> None:
|
||||
"""Persist the server-owned canonical source-path ↔ digest association.
|
||||
|
||||
A digest is only ever served back for the EXACT path it was captured from
|
||||
(see :func:`snapshot_servable_for_path`): a digest must never become a
|
||||
bearer capability readable through any other allowed path. The sidecar is
|
||||
a tiny path list, rewritten tmp+rename atomically. Caller holds
|
||||
``_LOCK`` (capture is serialized), so concurrent settles cannot race it.
|
||||
"""
|
||||
try:
|
||||
binding_file = _binding_path_for_digest(digest)
|
||||
sources: set[str] = set()
|
||||
try:
|
||||
data = json.loads(binding_file.read_text(encoding="utf-8"))
|
||||
sources = set(data.get("sources", []))
|
||||
except (OSError, ValueError, TypeError):
|
||||
sources = set()
|
||||
sources.add(str(Path(source).resolve()))
|
||||
tmp = binding_file.with_name(binding_file.name + ".tmp")
|
||||
tmp.write_text(
|
||||
json.dumps({"digest": digest, "sources": sorted(sources)}, sort_keys=True),
|
||||
encoding="utf-8",
|
||||
)
|
||||
os.replace(tmp, binding_file)
|
||||
except OSError as exc:
|
||||
logger.debug("media snapshot source binding failed for %s: %s", source, exc)
|
||||
|
||||
|
||||
def snapshot_servable_for_path(digest: str, target: Path) -> bool:
|
||||
"""True only when ``digest`` was captured from canonical path ``target``.
|
||||
|
||||
Serve-side half of the source-path binding (#6979 Round 2 MUST-FIX 1):
|
||||
the snapshot branch serves a digest only for the exact authorized path it
|
||||
was captured from, so replaying a digest through a different (allowed)
|
||||
path can never read stored bytes the normal ``path=`` gate would refuse.
|
||||
A missing/invalid sidecar returns False — the caller falls back to the
|
||||
live file.
|
||||
"""
|
||||
if not is_valid_digest(digest):
|
||||
return False
|
||||
try:
|
||||
want = str(target.resolve())
|
||||
except OSError:
|
||||
return False
|
||||
try:
|
||||
data = json.loads(_binding_path_for_digest(digest).read_text(encoding="utf-8"))
|
||||
except (OSError, ValueError, TypeError):
|
||||
return False
|
||||
return want in set(data.get("sources", []))
|
||||
|
||||
|
||||
def _total_cap_bytes() -> int:
|
||||
try:
|
||||
return max(0, int(os.getenv("HERMES_WEBUI_MEDIA_SNAPSHOT_CAP_BYTES", "")))
|
||||
except ValueError:
|
||||
return DEFAULT_TOTAL_CAP_BYTES
|
||||
|
||||
|
||||
def _max_file_bytes() -> int:
|
||||
try:
|
||||
return max(0, int(os.getenv("HERMES_WEBUI_MEDIA_SNAPSHOT_MAX_FILE_BYTES", "")))
|
||||
except ValueError:
|
||||
return DEFAULT_MAX_FILE_BYTES
|
||||
|
||||
|
||||
def _store_size_and_entries(directory: Path) -> tuple[int, list[tuple[float, int, Path]]]:
|
||||
"""Scan the store: (total bytes, [(mtime, size, path), ...])."""
|
||||
total = 0
|
||||
entries: list[tuple[float, int, Path]] = []
|
||||
try:
|
||||
for child in directory.iterdir():
|
||||
if child.suffix != ".snap":
|
||||
continue
|
||||
try:
|
||||
st = child.stat()
|
||||
except OSError:
|
||||
continue
|
||||
total += st.st_size
|
||||
entries.append((st.st_mtime, st.st_size, child))
|
||||
except OSError:
|
||||
pass
|
||||
return total, entries
|
||||
|
||||
|
||||
def _enforce_quota_locked(directory: Path) -> None:
|
||||
"""Evict oldest snapshots until the store is under the total cap.
|
||||
|
||||
Caller holds ``_LOCK``. Eviction is best-effort: an unlink failure is
|
||||
logged and skipped, never raised into the settle path.
|
||||
"""
|
||||
cap = _total_cap_bytes()
|
||||
total, entries = _store_size_and_entries(directory)
|
||||
if total <= cap:
|
||||
return
|
||||
entries.sort() # oldest mtime first
|
||||
for _mtime, size, path in entries:
|
||||
if total <= cap:
|
||||
break
|
||||
try:
|
||||
path.unlink()
|
||||
total -= size
|
||||
logger.info("media snapshot quota: evicted %s (%d bytes)", path.name, size)
|
||||
# Drop the digest's source-binding sidecar with its blob.
|
||||
try:
|
||||
_binding_path_for_digest(path.stem).unlink()
|
||||
except OSError:
|
||||
pass
|
||||
except OSError as exc:
|
||||
logger.debug("media snapshot eviction failed for %s: %s", path, exc)
|
||||
|
||||
|
||||
def capture_snapshot(source: Path, *, max_file_bytes: int | None = None) -> str | None:
|
||||
"""Copy ``source`` into the content-addressed store; return its digest.
|
||||
|
||||
Returns None (caller falls back to live-file previews) when:
|
||||
* the file is missing / not a regular file / unreadable,
|
||||
* the file exceeds the per-file cap,
|
||||
* any I/O error occurs mid-copy (the torn tmp is removed).
|
||||
|
||||
Never raises — snapshotting is a durability enhancement and must not be
|
||||
able to break the settle path.
|
||||
"""
|
||||
cap = _max_file_bytes() if max_file_bytes is None else max_file_bytes
|
||||
try:
|
||||
st = source.stat()
|
||||
except OSError:
|
||||
return None
|
||||
import stat as stat_mod
|
||||
|
||||
if not stat_mod.S_ISREG(st.st_mode):
|
||||
return None
|
||||
if cap and st.st_size > cap:
|
||||
return None
|
||||
|
||||
with _LOCK:
|
||||
directory = get_snapshot_dir()
|
||||
try:
|
||||
directory.mkdir(parents=True, exist_ok=True)
|
||||
except OSError:
|
||||
return None
|
||||
|
||||
digest = hashlib.sha256()
|
||||
tmp_path: Path | None = None
|
||||
try:
|
||||
tmp_path = directory / f".tmp.{os.getpid()}.{threading.get_ident()}"
|
||||
with open(source, "rb") as src, open(tmp_path, "wb") as dst:
|
||||
while True:
|
||||
chunk = src.read(1024 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
digest.update(chunk)
|
||||
dst.write(chunk)
|
||||
dst.flush()
|
||||
os.fsync(dst.fileno())
|
||||
hex_digest = digest.hexdigest()
|
||||
final_path = directory / f"{hex_digest}.snap"
|
||||
if final_path.exists():
|
||||
# Content already stored (dedup) — drop the duplicate copy.
|
||||
try:
|
||||
tmp_path.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
else:
|
||||
os.replace(tmp_path, final_path)
|
||||
tmp_path = None
|
||||
# Server-owned source-path binding: this digest may only be served
|
||||
# back for THIS canonical path (see snapshot_servable_for_path).
|
||||
_record_source_binding(hex_digest, source)
|
||||
_enforce_quota_locked(directory)
|
||||
return hex_digest
|
||||
except OSError as exc:
|
||||
logger.debug("media snapshot capture failed for %s: %s", source, exc)
|
||||
if tmp_path is not None:
|
||||
try:
|
||||
tmp_path.unlink()
|
||||
except OSError:
|
||||
pass
|
||||
return None
|
||||
|
||||
|
||||
def annotate_media_snapshots(
|
||||
messages: list,
|
||||
*,
|
||||
resolve_ref=None,
|
||||
allowed_predicate=None,
|
||||
) -> int:
|
||||
"""Scan settled messages and snapshot every local-file MEDIA: reference.
|
||||
|
||||
Writes a ``_media_snapshots`` dict ({absolute path: digest}) onto each
|
||||
assistant message that carries at least one local-file ``MEDIA:`` ref.
|
||||
Messages whose refs are already fully annotated are skipped (idempotent
|
||||
across repeated settles). A recorded digest is FINAL: even if its blob is
|
||||
later evicted by quota, a re-settle must not re-capture the CURRENT live
|
||||
bytes and silently rebind the historical message — evicted blobs degrade
|
||||
to live-file serving instead (#6979 Round 2 SHOULD-FIX).
|
||||
|
||||
``resolve_ref(raw_ref) -> Path | None`` maps a raw MEDIA token to an
|
||||
absolute file path (defaults to :func:`resolve_media_ref`); refs it cannot
|
||||
resolve (remote URLs, data: URIs) are ignored. ``allowed_predicate(path)
|
||||
-> bool`` applies the same allow/deny reasoning as ``/api/media``
|
||||
(defaults to :func:`media_capture_allowed`) so the store never receives
|
||||
bytes the endpoint would not serve.
|
||||
|
||||
Returns the number of new snapshots captured (0 on a repeat settle).
|
||||
"""
|
||||
import re as _re
|
||||
|
||||
if resolve_ref is None:
|
||||
resolve_ref = resolve_media_ref
|
||||
if allowed_predicate is None:
|
||||
allowed_predicate = media_capture_allowed
|
||||
media_re = _re.compile(r"MEDIA:([^\s\)\]]+)")
|
||||
captured = 0
|
||||
for msg in messages or []:
|
||||
if not isinstance(msg, dict) or msg.get("role") != "assistant":
|
||||
continue
|
||||
content = msg.get("content")
|
||||
if not isinstance(content, str) or "MEDIA:" not in content:
|
||||
continue
|
||||
refs = media_re.findall(content)
|
||||
if not refs:
|
||||
continue
|
||||
existing = msg.get("_media_snapshots")
|
||||
snaps = dict(existing) if isinstance(existing, dict) else {}
|
||||
changed = False
|
||||
for raw_ref in refs:
|
||||
if resolve_ref is not None:
|
||||
try:
|
||||
path = resolve_ref(raw_ref)
|
||||
except Exception:
|
||||
path = None
|
||||
else:
|
||||
path = None
|
||||
if path is None:
|
||||
continue
|
||||
# Index by BOTH the resolved absolute path and the raw token as the
|
||||
# frontend embeds it (file:// unwrapped, ~/ kept verbatim). One of
|
||||
# the two always matches the path= query param in the rendered URL.
|
||||
keys = [str(path)]
|
||||
if raw_ref not in keys:
|
||||
keys.append(raw_ref)
|
||||
# A recorded digest is FINAL once stamped (blob presence is NOT
|
||||
# re-checked): quota eviction must not cause a re-settle to
|
||||
# re-capture the current live bytes and rebind the historical
|
||||
# message — that would defeat per-message immutability and thrash
|
||||
# the store with re-capture I/O. Evicted blobs simply fall back to
|
||||
# live-file serving on the serve side.
|
||||
pending = [k for k in keys if not (snaps.get(k) and is_valid_digest(snaps[k]))]
|
||||
if not pending:
|
||||
continue # already stored under every key — zero-I/O fast path
|
||||
if allowed_predicate is not None:
|
||||
try:
|
||||
if not allowed_predicate(path):
|
||||
continue
|
||||
except Exception:
|
||||
continue
|
||||
digest = capture_snapshot(path)
|
||||
if digest:
|
||||
for k in keys:
|
||||
snaps[k] = digest
|
||||
changed = True
|
||||
captured += 1
|
||||
if changed:
|
||||
msg["_media_snapshots"] = snaps
|
||||
return captured
|
||||
+2
-29
@@ -44,7 +44,6 @@ The SSE `metering` event payload:
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import math
|
||||
import threading
|
||||
import time
|
||||
from dataclasses import dataclass
|
||||
@@ -59,8 +58,6 @@ class _SessionMeter:
|
||||
reasoning_tokens: int = 0
|
||||
first_token_ts: float = 0.0 # time.monotonic() of first token received
|
||||
last_token_ts: float = 0.0 # time.monotonic() of last token received
|
||||
pending_started_at: float | None = None
|
||||
ttft_ms: int | None = None
|
||||
|
||||
def total_tokens(self) -> int:
|
||||
return self.output_tokens + self.reasoning_tokens
|
||||
@@ -97,18 +94,6 @@ class GlobalMeter:
|
||||
with self._lock:
|
||||
self._sessions[stream_id] = _SessionMeter()
|
||||
|
||||
def set_pending_started_at(self, stream_id: str, pending_started_at) -> None:
|
||||
try:
|
||||
value = float(pending_started_at)
|
||||
except (TypeError, ValueError):
|
||||
return
|
||||
if not math.isfinite(value) or value <= 0:
|
||||
return
|
||||
with self._lock:
|
||||
session = self._sessions.get(stream_id)
|
||||
if session is not None:
|
||||
session.pending_started_at = value
|
||||
|
||||
def get_interval(self) -> float:
|
||||
"""Return 1.0 when sessions are actively receiving tokens, 10.0 when idle.
|
||||
|
||||
@@ -134,8 +119,6 @@ class GlobalMeter:
|
||||
s.first_token_ts = now
|
||||
s.last_token_ts = now
|
||||
s.output_tokens = running_output_tokens
|
||||
if s.ttft_ms is None and s.pending_started_at is not None:
|
||||
s.ttft_ms = max(0, round((time.time() - s.pending_started_at) * 1000))
|
||||
|
||||
def record_reasoning(self, stream_id: str, running_reasoning_tokens: int) -> None:
|
||||
now = time.monotonic()
|
||||
@@ -152,12 +135,7 @@ class GlobalMeter:
|
||||
with self._lock:
|
||||
self._sessions.pop(stream_id, None)
|
||||
|
||||
def get_ttft_ms(self, stream_id: str) -> int | None:
|
||||
with self._lock:
|
||||
session = self._sessions.get(stream_id)
|
||||
return session.ttft_ms if session is not None else None
|
||||
|
||||
def get_stats(self, stream_id: str | None = None) -> dict:
|
||||
def get_stats(self) -> dict:
|
||||
now = time.monotonic()
|
||||
with self._lock:
|
||||
# Prune stale sessions
|
||||
@@ -197,7 +175,7 @@ class GlobalMeter:
|
||||
high = max(active_readings) if active_readings else 0.0
|
||||
low = min(active_readings) if active_readings else 0.0
|
||||
|
||||
stats = {
|
||||
return {
|
||||
'tps': round(global_tps, 1) if global_tps is not None else None,
|
||||
'tps_available': global_tps is not None,
|
||||
'estimated': False,
|
||||
@@ -205,11 +183,6 @@ class GlobalMeter:
|
||||
'low': round(low, 1) if low else None,
|
||||
'active': len(self._sessions),
|
||||
}
|
||||
if stream_id is not None:
|
||||
session = self._sessions.get(stream_id)
|
||||
if session is not None and session.ttft_ms is not None:
|
||||
stats['ttft_ms'] = session.ttft_ms
|
||||
return stats
|
||||
|
||||
|
||||
# ── Module-level singleton ─────────────────────────────────────────────────────
|
||||
|
||||
+355
-5631
File diff suppressed because it is too large
Load Diff
+18
-24
@@ -191,10 +191,7 @@ def _write_auth_json(data: dict[str, Any], auth_path: Path | None = None) -> Pat
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_name(f"{path.name}.tmp.{os.getpid()}.{uuid.uuid4().hex}")
|
||||
try:
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
f.write(json.dumps(data, indent=2, ensure_ascii=False) + "\n")
|
||||
f.flush()
|
||||
os.fsync(f.fileno()) # durable before rename: no truncated auth.json on power loss
|
||||
tmp.write_text(json.dumps(data, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
|
||||
try:
|
||||
tmp.chmod(0o600)
|
||||
except OSError as exc:
|
||||
@@ -218,23 +215,6 @@ def _now_iso() -> str:
|
||||
return datetime.now(timezone.utc).isoformat().replace("+00:00", "Z")
|
||||
|
||||
|
||||
def _invalidate_provider_state_caches(provider: str) -> None:
|
||||
"""Invalidate caches derived from provider credential state."""
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
|
||||
invalidate_credential_pool_cache(provider)
|
||||
except Exception:
|
||||
logger.debug("Failed to invalidate %s credential cache", provider, exc_info=True)
|
||||
|
||||
try:
|
||||
from api.providers import invalidate_providers_cache
|
||||
|
||||
invalidate_providers_cache()
|
||||
except Exception:
|
||||
logger.debug("Failed to invalidate providers cache after %s credential change", provider, exc_info=True)
|
||||
|
||||
|
||||
def _persist_codex_credentials(hermes_home: Path, token_data: dict[str, Any]) -> Path:
|
||||
"""Persist Codex OAuth credentials to active-profile auth.json."""
|
||||
access_token = str(token_data.get("access_token") or "").strip()
|
||||
@@ -294,7 +274,12 @@ def _persist_codex_credentials(hermes_home: Path, token_data: dict[str, Any]) ->
|
||||
auth["updated_at"] = now
|
||||
path = _write_auth_json(auth, auth_path)
|
||||
|
||||
_invalidate_provider_state_caches("openai-codex")
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
|
||||
invalidate_credential_pool_cache("openai-codex")
|
||||
except Exception:
|
||||
logger.debug("Failed to invalidate openai-codex credential cache", exc_info=True)
|
||||
|
||||
return path
|
||||
|
||||
@@ -401,7 +386,12 @@ def _link_anthropic_credentials(hermes_home: Path) -> None:
|
||||
})
|
||||
auth["updated_at"] = now
|
||||
_write_auth_json(auth, auth_path)
|
||||
_invalidate_provider_state_caches("anthropic")
|
||||
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
invalidate_credential_pool_cache("anthropic")
|
||||
except Exception:
|
||||
logger.debug("Failed to invalidate anthropic credential cache", exc_info=True)
|
||||
|
||||
|
||||
def _anthropic_public_start_payload(flow_id: str, flow: dict[str, Any]) -> dict[str, Any]:
|
||||
@@ -522,7 +512,11 @@ def _remove_anthropic_link_marker(hermes_home: Path) -> None:
|
||||
pool.pop("anthropic", None)
|
||||
auth["updated_at"] = _now_iso()
|
||||
_write_auth_json(auth, auth_path)
|
||||
_invalidate_provider_state_caches("anthropic")
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
invalidate_credential_pool_cache("anthropic")
|
||||
except Exception:
|
||||
logger.debug("Failed to invalidate anthropic credential cache", exc_info=True)
|
||||
|
||||
|
||||
# ── Codex protocol ──────────────────────────────────────────────────────────
|
||||
|
||||
@@ -1,647 +0,0 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import io
|
||||
import posixpath
|
||||
import zipfile
|
||||
import zlib
|
||||
from pathlib import Path
|
||||
|
||||
CLAIMED_OFFICE_EXTENSIONS = frozenset({".docx", ".xlsx", ".pptx"})
|
||||
CLAIMED_OFFICE_FORMATS = frozenset({"docx", "xlsx", "pptx"})
|
||||
OFFICE_PREVIEW_KIND = "office"
|
||||
OFFICE_RENDER_MODE = "code"
|
||||
OFFICE_DEPENDENCY_HINT = (
|
||||
"Office preview is not available on this server. Install python-docx, "
|
||||
"openpyxl, and python-pptx to enable it: pip install python-docx openpyxl "
|
||||
"python-pptx"
|
||||
)
|
||||
OFFICE_PREVIEW_TRUNCATED_NOTICE = "[Preview truncated: Office content exceeds safe limits]"
|
||||
MAX_OFFICE_PREVIEW_CHARS = 120_000
|
||||
MAX_DOCX_PREVIEW_BLOCKS = 2_000
|
||||
MAX_DOCX_TABLE_CELLS = 5_000
|
||||
MAX_XLSX_PREVIEW_SHEETS = 20
|
||||
MAX_XLSX_PREVIEW_ROWS_PER_SHEET = 500
|
||||
MAX_XLSX_PREVIEW_CELLS_PER_SHEET = 5_000
|
||||
MAX_PPTX_PREVIEW_SLIDES = 100
|
||||
MAX_PPTX_PREVIEW_SHAPES_PER_SLIDE = 200
|
||||
MAX_OFFICE_ARCHIVE_MEMBERS = 256
|
||||
MAX_OFFICE_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES = 8_000_000
|
||||
MAX_OFFICE_ARCHIVE_MEMBER_BYTES = 4_000_000
|
||||
MAX_OFFICE_ARCHIVE_MAX_COMPRESSION_RATIO = 200
|
||||
MAX_DOCX_ARCHIVE_DOCUMENT_BYTES = 4_000_000
|
||||
MAX_XLSX_ARCHIVE_SHARED_STRINGS_BYTES = 4_000_000
|
||||
MAX_XLSX_ARCHIVE_WORKSHEET_BYTES = 2_000_000
|
||||
MAX_XLSX_ARCHIVE_METADATA_BYTES = 512_000
|
||||
MAX_PPTX_ARCHIVE_SLIDE_BYTES = 1_000_000
|
||||
MAX_PPTX_ARCHIVE_MEDIA_BYTES = 2_000_000
|
||||
|
||||
_WORD_NAMESPACE = "{http://schemas.openxmlformats.org/wordprocessingml/2006/main}"
|
||||
_DEFAULT_DOCX_SECTION_SIGNATURE = None
|
||||
|
||||
_DOCX_BODY_CHILDREN = {f"{_WORD_NAMESPACE}p", f"{_WORD_NAMESPACE}sectPr"}
|
||||
_DOCX_PARAGRAPH_CHILDREN = {f"{_WORD_NAMESPACE}pPr", f"{_WORD_NAMESPACE}r"}
|
||||
_DOCX_SAFE_PARAGRAPH_PROPERTY_CHILDREN = {f"{_WORD_NAMESPACE}pStyle"}
|
||||
_DOCX_RUN_CHILDREN = {f"{_WORD_NAMESPACE}t"}
|
||||
|
||||
|
||||
def _office_dependency_import_error() -> ImportError:
|
||||
return ImportError(OFFICE_DEPENDENCY_HINT)
|
||||
|
||||
def _office_archive_limit_error() -> ValueError:
|
||||
return ValueError("Office preview exceeds safe archive limits")
|
||||
|
||||
def _office_preview_read_error(office_format: str) -> ValueError:
|
||||
return ValueError(f"Unable to read {office_format.upper()} preview")
|
||||
|
||||
|
||||
def _load_docx_document():
|
||||
try:
|
||||
from docx import Document as document_factory
|
||||
except ImportError as exc: # pragma: no cover - depends on local install shape
|
||||
raise _office_dependency_import_error() from exc
|
||||
return document_factory
|
||||
|
||||
|
||||
def _load_workbook_reader():
|
||||
try:
|
||||
from openpyxl import load_workbook as workbook_reader
|
||||
except ImportError as exc: # pragma: no cover - depends on local install shape
|
||||
raise _office_dependency_import_error() from exc
|
||||
return workbook_reader
|
||||
|
||||
|
||||
def _load_presentation_ctor():
|
||||
try:
|
||||
from pptx import Presentation as presentation_ctor
|
||||
except ImportError as exc: # pragma: no cover - depends on local install shape
|
||||
raise _office_dependency_import_error() from exc
|
||||
return presentation_ctor
|
||||
|
||||
def _normalise_archive_member_name(name: str) -> str:
|
||||
normalized = posixpath.normpath(name.replace("\\", "/"))
|
||||
if name.startswith(("/", "\\")) or normalized in {"", ".", ".."} or normalized.startswith("../"):
|
||||
raise _office_archive_limit_error()
|
||||
return normalized
|
||||
|
||||
def _archive_member_byte_limit(office_format: str, member_name: str) -> int:
|
||||
if office_format == "docx":
|
||||
if member_name == "word/document.xml":
|
||||
return MAX_DOCX_ARCHIVE_DOCUMENT_BYTES
|
||||
elif office_format == "xlsx":
|
||||
if member_name == "xl/sharedStrings.xml":
|
||||
return MAX_XLSX_ARCHIVE_SHARED_STRINGS_BYTES
|
||||
if member_name.startswith("xl/worksheets/"):
|
||||
return MAX_XLSX_ARCHIVE_WORKSHEET_BYTES
|
||||
if member_name.startswith("xl/theme/") or member_name.startswith("docProps/") or member_name == "xl/workbook.xml":
|
||||
return MAX_XLSX_ARCHIVE_METADATA_BYTES
|
||||
elif office_format == "pptx":
|
||||
if member_name.startswith("ppt/slides/"):
|
||||
return MAX_PPTX_ARCHIVE_SLIDE_BYTES
|
||||
if member_name.startswith("ppt/media/"):
|
||||
return MAX_PPTX_ARCHIVE_MEDIA_BYTES
|
||||
return MAX_OFFICE_ARCHIVE_MEMBER_BYTES
|
||||
|
||||
def _preflight_office_archive(office_format: str, raw: bytes) -> None:
|
||||
try:
|
||||
archive = zipfile.ZipFile(io.BytesIO(raw))
|
||||
except zipfile.BadZipFile as exc:
|
||||
raise _office_preview_read_error(office_format) from exc
|
||||
|
||||
with archive:
|
||||
file_infos = [info for info in archive.infolist() if not info.is_dir()]
|
||||
if len(file_infos) > MAX_OFFICE_ARCHIVE_MEMBERS:
|
||||
raise _office_archive_limit_error()
|
||||
|
||||
total_uncompressed = 0
|
||||
for info in file_infos:
|
||||
member_name = _normalise_archive_member_name(info.filename)
|
||||
member_limit = _archive_member_byte_limit(office_format, member_name)
|
||||
member_size = 0
|
||||
try:
|
||||
member = archive.open(info)
|
||||
except Exception as exc: # pragma: no cover - malformed archive path
|
||||
raise _office_preview_read_error(office_format) from exc
|
||||
with member:
|
||||
try:
|
||||
while True:
|
||||
chunk = member.read(64 * 1024)
|
||||
if not chunk:
|
||||
break
|
||||
member_size += len(chunk)
|
||||
total_uncompressed += len(chunk)
|
||||
if member_size > member_limit or total_uncompressed > MAX_OFFICE_ARCHIVE_TOTAL_UNCOMPRESSED_BYTES:
|
||||
raise _office_archive_limit_error()
|
||||
except (zipfile.BadZipFile, zlib.error, EOFError, OSError) as exc:
|
||||
# Corrupt/truncated member data (bad CRC, short read, etc.) —
|
||||
# mundane for partially-uploaded files, not adversarial. Raise
|
||||
# the module's intended read-error ValueError (handled as a
|
||||
# clean 4xx) instead of letting BadZipFile/zlib.error escape to
|
||||
# the route's catch-all as an unhandled 500 + traceback. The
|
||||
# limit-exceeded ValueError above is deliberately NOT caught
|
||||
# here (it is not one of these decompression exception types).
|
||||
raise _office_preview_read_error(office_format) from exc
|
||||
if member_size > max(info.compress_size, 1) * MAX_OFFICE_ARCHIVE_MAX_COMPRESSION_RATIO:
|
||||
raise _office_archive_limit_error()
|
||||
|
||||
|
||||
def is_claimed_office_path(path: str | Path) -> bool:
|
||||
return Path(str(path)).suffix.lower() in CLAIMED_OFFICE_EXTENSIONS
|
||||
|
||||
|
||||
def _office_format_for_path(path: str | Path) -> str:
|
||||
return Path(str(path)).suffix.lower().lstrip(".")
|
||||
|
||||
|
||||
def _normalise_preview_text(value, max_chars: int | None = None) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
text = str(value)
|
||||
if max_chars is not None and max_chars >= 0 and len(text) > max_chars:
|
||||
text = text[:max_chars]
|
||||
return text.replace("\r", "\n").replace("\n", " ").strip()
|
||||
|
||||
|
||||
def _preview_line_count(content: str) -> int:
|
||||
if not content:
|
||||
return 1
|
||||
return content.count("\n") + 1
|
||||
|
||||
|
||||
def _finalize_preview_text(content: str, truncated: bool = False, strip_edges: bool = True) -> tuple[str, bool]:
|
||||
# docx passes strip_edges=False: leading/trailing blank paragraphs are
|
||||
# meaningful body content and the editor textarea is prefilled from this
|
||||
# text, so stripping edge whitespace would silently drop those paragraphs on
|
||||
# an unedited save (interior blanks already round-trip). xlsx/pptx keep the
|
||||
# strip — their previews are read-only and edge whitespace is just noise.
|
||||
text = (content or "").strip() if strip_edges else (content or "")
|
||||
if len(text) > MAX_OFFICE_PREVIEW_CHARS:
|
||||
text = text[:MAX_OFFICE_PREVIEW_CHARS].rstrip()
|
||||
truncated = True
|
||||
if truncated:
|
||||
text = f"{text}\n\n{OFFICE_PREVIEW_TRUNCATED_NOTICE}" if text else OFFICE_PREVIEW_TRUNCATED_NOTICE
|
||||
return text, truncated
|
||||
|
||||
|
||||
class _PreviewBuilder:
|
||||
def __init__(self, char_limit: int | None = None) -> None:
|
||||
self._char_limit = MAX_OFFICE_PREVIEW_CHARS if char_limit is None else max(char_limit, 0)
|
||||
self._parts: list[str] = []
|
||||
self._length = 0
|
||||
self._started = False
|
||||
self.truncated = False
|
||||
|
||||
@property
|
||||
def remaining_chars(self) -> int:
|
||||
return max(self._char_limit - self._length, 0)
|
||||
|
||||
@property
|
||||
def started(self) -> bool:
|
||||
return self._started
|
||||
|
||||
@property
|
||||
def has_content(self) -> bool:
|
||||
return bool(self._parts)
|
||||
|
||||
@property
|
||||
def text(self) -> str:
|
||||
return "".join(self._parts)
|
||||
|
||||
def _append_piece(self, piece: str) -> bool:
|
||||
if self.truncated:
|
||||
return False
|
||||
if not piece:
|
||||
return True
|
||||
remaining = self.remaining_chars
|
||||
if remaining <= 0:
|
||||
self.truncated = True
|
||||
return False
|
||||
if len(piece) > remaining:
|
||||
piece = piece[:remaining].rstrip()
|
||||
self.truncated = True
|
||||
if piece:
|
||||
self._parts.append(piece)
|
||||
self._length += len(piece)
|
||||
return not self.truncated
|
||||
|
||||
def start_line(self) -> bool:
|
||||
if self._started:
|
||||
return self._append_piece("\n")
|
||||
self._started = True
|
||||
return True
|
||||
|
||||
def start_section(self) -> bool:
|
||||
if self._started:
|
||||
return self._append_piece("\n\n")
|
||||
self._started = True
|
||||
return True
|
||||
|
||||
def append_text(self, text: str) -> bool:
|
||||
if not self._started:
|
||||
self._started = True
|
||||
return self._append_piece(text)
|
||||
|
||||
def finish(self, strip_edges: bool = True) -> tuple[str, bool]:
|
||||
return _finalize_preview_text(self.text, self.truncated, strip_edges=strip_edges)
|
||||
|
||||
|
||||
def _append_normalized_preview_text(builder: _PreviewBuilder, value) -> bool:
|
||||
if builder.truncated:
|
||||
return False
|
||||
remaining = builder.remaining_chars
|
||||
if remaining <= 0:
|
||||
builder.truncated = True
|
||||
return False
|
||||
raw_text = "" if value is None else str(value)
|
||||
clipped = len(raw_text) > remaining
|
||||
if not builder.append_text(_normalise_preview_text(raw_text, remaining)):
|
||||
return False
|
||||
if clipped:
|
||||
builder.truncated = True
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _append_verbatim_preview_text(builder: _PreviewBuilder, value) -> bool:
|
||||
"""Append run text WITHOUT per-node strip/space-normalization (docx runs).
|
||||
|
||||
A docx paragraph is split across multiple ``<w:t>`` runs and the whitespace
|
||||
at run boundaries is significant: ``p.add_run("Hello ") + p.add_run("world")``
|
||||
stores two runs whose text is ``"Hello "`` and ``"world"``. The general
|
||||
``_normalise_preview_text`` path ``.strip()``s each node and concatenates
|
||||
with no separator, which corrupts that to ``"Helloworld"`` — and because the
|
||||
editor textarea is prefilled from the preview text, opening + saving such a
|
||||
file (even with no edits) persisted the corruption to disk. That
|
||||
normalization is still correct for xlsx cells / pptx shapes (whole-cell
|
||||
values, not run fragments), so only the docx run path switches to verbatim
|
||||
append. Text is budget-clipped but never stripped/space-collapsed.
|
||||
"""
|
||||
if builder.truncated:
|
||||
return False
|
||||
remaining = builder.remaining_chars
|
||||
if remaining <= 0:
|
||||
builder.truncated = True
|
||||
return False
|
||||
raw_text = "" if value is None else str(value)
|
||||
clipped = len(raw_text) > remaining
|
||||
if not builder.append_text(raw_text):
|
||||
return False
|
||||
if clipped:
|
||||
builder.truncated = True
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _iter_docx_text_nodes(element):
|
||||
for node in element.iter():
|
||||
if node.tag == f"{_WORD_NAMESPACE}t" and node.text:
|
||||
yield node.text
|
||||
|
||||
|
||||
def _append_docx_element_text(builder: _PreviewBuilder, element) -> bool:
|
||||
for text in _iter_docx_text_nodes(element):
|
||||
# Verbatim (not _normalise_preview_text): preserve run-boundary
|
||||
# whitespace so multi-run paragraphs round-trip losslessly. See
|
||||
# _append_verbatim_preview_text for why docx differs from xlsx/pptx.
|
||||
if not _append_verbatim_preview_text(builder, text):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _append_docx_cell_text(builder: _PreviewBuilder, cell_element) -> bool:
|
||||
first_paragraph = True
|
||||
for child in cell_element:
|
||||
if child.tag != f"{_WORD_NAMESPACE}p":
|
||||
continue
|
||||
if not first_paragraph and not builder.append_text("\n"):
|
||||
return False
|
||||
if not _append_docx_element_text(builder, child):
|
||||
return False
|
||||
first_paragraph = False
|
||||
return True
|
||||
|
||||
|
||||
def _docx_preview_text(document) -> tuple[str, bool]:
|
||||
builder = _PreviewBuilder()
|
||||
body_blocks_seen = 0
|
||||
table_cells_seen = 0
|
||||
table_index = 0
|
||||
for child in document._element.body:
|
||||
if child.tag == f"{_WORD_NAMESPACE}sectPr":
|
||||
continue
|
||||
body_blocks_seen += 1
|
||||
if body_blocks_seen > MAX_DOCX_PREVIEW_BLOCKS:
|
||||
builder.truncated = True
|
||||
break
|
||||
if child.tag == f"{_WORD_NAMESPACE}p":
|
||||
if not builder.start_line() or not _append_docx_element_text(builder, child):
|
||||
break
|
||||
continue
|
||||
if child.tag != f"{_WORD_NAMESPACE}tbl":
|
||||
continue
|
||||
table_index += 1
|
||||
if not builder.start_line() or not builder.append_text(f"Table {table_index}"):
|
||||
break
|
||||
for row in child:
|
||||
if row.tag != f"{_WORD_NAMESPACE}tr":
|
||||
continue
|
||||
if not builder.start_line():
|
||||
break
|
||||
first_cell = True
|
||||
for cell in row:
|
||||
if cell.tag != f"{_WORD_NAMESPACE}tc":
|
||||
continue
|
||||
table_cells_seen += 1
|
||||
if table_cells_seen > MAX_DOCX_TABLE_CELLS:
|
||||
builder.truncated = True
|
||||
break
|
||||
if not first_cell and not builder.append_text("\t"):
|
||||
break
|
||||
if not _append_docx_cell_text(builder, cell):
|
||||
break
|
||||
first_cell = False
|
||||
if builder.truncated:
|
||||
break
|
||||
if builder.truncated:
|
||||
break
|
||||
# strip_edges=False: preserve leading/trailing blank paragraphs so an
|
||||
# unedited open->save round-trips them (the editor prefills from this text).
|
||||
return builder.finish(strip_edges=False)
|
||||
|
||||
|
||||
def _docx_paragraph_properties_are_safe(properties) -> bool:
|
||||
for child in properties:
|
||||
if child.tag not in _DOCX_SAFE_PARAGRAPH_PROPERTY_CHILDREN:
|
||||
return False
|
||||
if child.tag == f"{_WORD_NAMESPACE}pStyle" and child.get(f"{_WORD_NAMESPACE}val") != "Normal":
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _docx_xml_signature(element) -> tuple:
|
||||
attributes = tuple(
|
||||
sorted(
|
||||
(key, value)
|
||||
for key, value in element.attrib.items()
|
||||
if not key.rsplit("}", 1)[-1].startswith("rsid")
|
||||
)
|
||||
)
|
||||
children = tuple(_docx_xml_signature(child) for child in element)
|
||||
text = (element.text or "").strip()
|
||||
return element.tag, attributes, text, children
|
||||
|
||||
|
||||
def _default_docx_section_signature() -> tuple:
|
||||
global _DEFAULT_DOCX_SECTION_SIGNATURE
|
||||
if _DEFAULT_DOCX_SECTION_SIGNATURE is None:
|
||||
document = _load_docx_document()()
|
||||
_DEFAULT_DOCX_SECTION_SIGNATURE = tuple(
|
||||
_docx_xml_signature(child) for child in document._element.body.sectPr
|
||||
)
|
||||
return _DEFAULT_DOCX_SECTION_SIGNATURE
|
||||
|
||||
|
||||
def _docx_section_properties_are_safe(section_properties) -> bool:
|
||||
return tuple(_docx_xml_signature(child) for child in section_properties) == _default_docx_section_signature()
|
||||
|
||||
|
||||
def _docx_editability(document) -> tuple[bool, str | None]:
|
||||
body = document._element.body
|
||||
for child in body:
|
||||
if child.tag not in _DOCX_BODY_CHILDREN:
|
||||
return False, "docx contains unsupported structures"
|
||||
if child.tag == f"{_WORD_NAMESPACE}sectPr" and not _docx_section_properties_are_safe(child):
|
||||
return False, "docx contains unsupported section content"
|
||||
for paragraph in document.paragraphs:
|
||||
for child in paragraph._p:
|
||||
if child.tag not in _DOCX_PARAGRAPH_CHILDREN:
|
||||
return False, "docx contains unsupported paragraph structures"
|
||||
if child.tag == f"{_WORD_NAMESPACE}pPr" and not _docx_paragraph_properties_are_safe(child):
|
||||
return False, "docx contains unsupported paragraph structures"
|
||||
for run in paragraph.runs:
|
||||
for child in run._r:
|
||||
if child.tag not in _DOCX_RUN_CHILDREN:
|
||||
return False, "docx contains unsupported inline content"
|
||||
return True, None
|
||||
|
||||
|
||||
def _preview_docx(raw: bytes) -> tuple[str, bool, str | None, bool]:
|
||||
_preflight_office_archive("docx", raw)
|
||||
try:
|
||||
document = _load_docx_document()(io.BytesIO(raw))
|
||||
except ImportError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - library-specific failure mode
|
||||
raise ValueError("Unable to read DOCX preview") from exc
|
||||
content, truncated = _docx_preview_text(document)
|
||||
if truncated:
|
||||
return content, False, "docx preview exceeds safe limits", True
|
||||
editable, reason = _docx_editability(document)
|
||||
return content, editable, reason, truncated
|
||||
|
||||
|
||||
def _preview_xlsx(raw: bytes) -> tuple[str, bool]:
|
||||
_preflight_office_archive("xlsx", raw)
|
||||
try:
|
||||
workbook = _load_workbook_reader()(io.BytesIO(raw), data_only=True, read_only=True)
|
||||
except ImportError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - library-specific failure mode
|
||||
raise ValueError("Unable to read XLSX preview") from exc
|
||||
builder = _PreviewBuilder()
|
||||
try:
|
||||
for sheet_index, sheet in enumerate(workbook.worksheets, start=1):
|
||||
if sheet_index > MAX_XLSX_PREVIEW_SHEETS:
|
||||
builder.truncated = True
|
||||
break
|
||||
if not builder.start_section() or not builder.append_text(f"Sheet: {sheet.title}"):
|
||||
break
|
||||
rows_seen = 0
|
||||
cells_seen = 0
|
||||
# openpyxl read-only mode reports max_row/max_column as None for any
|
||||
# workbook lacking a <dimension> record — which includes everything
|
||||
# openpyxl.Workbook(write_only=True) produces. `getattr(..., DEFAULT)`
|
||||
# does NOT help (the attribute exists; its value is None).
|
||||
#
|
||||
# When the dimension IS known, bound iter_rows to the capped extent.
|
||||
# When it's UNKNOWN, pass None so openpyxl yields each row's NATURAL
|
||||
# width — passing the cap instead would make openpyxl pad every row
|
||||
# out to `max_col` with None cells, exhausting the per-sheet cell
|
||||
# budget on row 1 and dropping the rest of the sheet. The rows_seen /
|
||||
# cells_seen counters below bound the work in the unknown case.
|
||||
sheet_max_row = getattr(sheet, "max_row", None)
|
||||
sheet_max_col = getattr(sheet, "max_column", None)
|
||||
max_row = min(sheet_max_row, MAX_XLSX_PREVIEW_ROWS_PER_SHEET) if sheet_max_row else None
|
||||
max_col = min(sheet_max_col, MAX_XLSX_PREVIEW_CELLS_PER_SHEET) if sheet_max_col else None
|
||||
for row in sheet.iter_rows(values_only=True, max_row=max_row, max_col=max_col):
|
||||
rows_seen += 1
|
||||
if rows_seen > MAX_XLSX_PREVIEW_ROWS_PER_SHEET:
|
||||
builder.truncated = True
|
||||
break
|
||||
row_budget = builder.remaining_chars - (1 if builder.started else 0)
|
||||
row_builder = _PreviewBuilder(row_budget)
|
||||
for value in row:
|
||||
cells_seen += 1
|
||||
if cells_seen > MAX_XLSX_PREVIEW_CELLS_PER_SHEET:
|
||||
builder.truncated = True
|
||||
break
|
||||
if row_builder.has_content and not row_builder.append_text("\t"):
|
||||
break
|
||||
if not _append_normalized_preview_text(row_builder, value):
|
||||
if not row_builder.truncated:
|
||||
builder.truncated = True
|
||||
break
|
||||
if builder.truncated:
|
||||
break
|
||||
if row_builder.has_content:
|
||||
if not builder.start_line() or not builder.append_text(row_builder.text):
|
||||
break
|
||||
if row_builder.truncated:
|
||||
builder.truncated = True
|
||||
break
|
||||
if builder.truncated:
|
||||
break
|
||||
if (getattr(sheet, "max_row", None) or rows_seen) > MAX_XLSX_PREVIEW_ROWS_PER_SHEET:
|
||||
builder.truncated = True
|
||||
break
|
||||
if (getattr(sheet, "max_column", None) or 0) > MAX_XLSX_PREVIEW_CELLS_PER_SHEET:
|
||||
builder.truncated = True
|
||||
break
|
||||
finally:
|
||||
close = getattr(workbook, "close", None)
|
||||
if callable(close):
|
||||
close()
|
||||
if not builder.has_content:
|
||||
return _finalize_preview_text("Empty workbook", builder.truncated)
|
||||
return builder.finish()
|
||||
|
||||
|
||||
def _preview_pptx(raw: bytes) -> tuple[str, bool]:
|
||||
_preflight_office_archive("pptx", raw)
|
||||
try:
|
||||
presentation = _load_presentation_ctor()(io.BytesIO(raw))
|
||||
except ImportError:
|
||||
raise
|
||||
except Exception as exc: # pragma: no cover - library-specific failure mode
|
||||
raise ValueError("Unable to read PPTX preview") from exc
|
||||
builder = _PreviewBuilder()
|
||||
for slide_index, slide in enumerate(presentation.slides, start=1):
|
||||
if slide_index > MAX_PPTX_PREVIEW_SLIDES:
|
||||
builder.truncated = True
|
||||
break
|
||||
if not builder.start_section() or not builder.append_text(f"Slide {slide_index}"):
|
||||
break
|
||||
shapes_seen = 0
|
||||
has_text = False
|
||||
for shape in slide.shapes:
|
||||
shapes_seen += 1
|
||||
if shapes_seen > MAX_PPTX_PREVIEW_SHAPES_PER_SLIDE:
|
||||
builder.truncated = True
|
||||
break
|
||||
text = getattr(shape, "text", "")
|
||||
if not _normalise_preview_text(text):
|
||||
continue
|
||||
if not builder.start_line() or not _append_normalized_preview_text(builder, text):
|
||||
break
|
||||
has_text = True
|
||||
if builder.truncated:
|
||||
break
|
||||
if not has_text and (not builder.start_line() or not builder.append_text("(empty slide)")):
|
||||
break
|
||||
if not builder.has_content:
|
||||
return _finalize_preview_text("Empty presentation", builder.truncated)
|
||||
return builder.finish()
|
||||
|
||||
|
||||
def preview_office_document(path: str | Path, raw: bytes) -> dict:
|
||||
office_format = _office_format_for_path(path)
|
||||
if office_format not in CLAIMED_OFFICE_FORMATS:
|
||||
raise ValueError(f"Unsupported Office format: {path}")
|
||||
|
||||
truncated = False
|
||||
if office_format == "docx":
|
||||
content, editable, reason, truncated = _preview_docx(raw)
|
||||
elif office_format == "xlsx":
|
||||
content, truncated = _preview_xlsx(raw)
|
||||
editable, reason = False, "xlsx preview is read-only in this slice"
|
||||
elif office_format == "pptx":
|
||||
content, truncated = _preview_pptx(raw)
|
||||
editable, reason = False, "pptx preview is read-only in this slice"
|
||||
else: # pragma: no cover - exhaustive guard
|
||||
raise ValueError(f"Unsupported Office format: {path}")
|
||||
|
||||
payload = {
|
||||
"path": str(path),
|
||||
"content": content,
|
||||
"size": len(raw),
|
||||
"lines": _preview_line_count(content),
|
||||
"preview_kind": OFFICE_PREVIEW_KIND,
|
||||
"office_format": office_format,
|
||||
"render_mode": OFFICE_RENDER_MODE,
|
||||
"editable": editable,
|
||||
}
|
||||
if reason:
|
||||
payload["edit_blocked_reason"] = reason
|
||||
if truncated:
|
||||
payload["truncated"] = True
|
||||
return payload
|
||||
|
||||
|
||||
def _docx_bytes_from_text(content: str, current_bytes: bytes | None = None) -> bytes:
|
||||
"""Rebuild a docx from edited plain text, preserving the original package.
|
||||
|
||||
We reload the CURRENT document (styles.xml, docProps/core.xml, theme,
|
||||
settings, sectPr) and replace only the body's paragraph content, rather than
|
||||
starting from python-docx's blank template. Building from the blank template
|
||||
silently wiped author/title/custom styles on an unedited open→save — the
|
||||
same fail-closed round-trip class the sectPr and run-whitespace fixes closed.
|
||||
When current_bytes is unavailable we fall back to a fresh document.
|
||||
|
||||
Content is bounded BEFORE the (quadratic — python-docx scans for sectPr on
|
||||
every add_paragraph) build loop: an editable preview is capped at
|
||||
MAX_OFFICE_PREVIEW_CHARS and MAX_DOCX_PREVIEW_BLOCKS lines, so any legitimate
|
||||
editor save fits well within these bounds. Rejecting oversized input up front
|
||||
prevents a write-surface CPU/RSS DoS (measured ~20s CPU for 50k lines).
|
||||
"""
|
||||
text = str(content or "").replace("\r\n", "\n").replace("\r", "\n")
|
||||
if len(text) > MAX_OFFICE_PREVIEW_CHARS:
|
||||
raise ValueError("DOCX content exceeds the editable size limit")
|
||||
lines = text.split("\n")
|
||||
if len(lines) > MAX_DOCX_PREVIEW_BLOCKS:
|
||||
raise ValueError("DOCX content exceeds the editable paragraph limit")
|
||||
|
||||
if current_bytes is not None:
|
||||
try:
|
||||
document = _load_docx_document()(io.BytesIO(current_bytes))
|
||||
except ImportError:
|
||||
raise
|
||||
except Exception:
|
||||
document = _load_docx_document()()
|
||||
else:
|
||||
document = _load_docx_document()()
|
||||
body = document._element.body
|
||||
for child in list(body):
|
||||
if child.tag != f"{_WORD_NAMESPACE}sectPr":
|
||||
body.remove(child)
|
||||
for line in lines:
|
||||
document.add_paragraph(line)
|
||||
buffer = io.BytesIO()
|
||||
document.save(buffer)
|
||||
return buffer.getvalue()
|
||||
|
||||
|
||||
def save_office_document(path: str | Path, current_bytes: bytes, content: str) -> tuple[dict, bytes]:
|
||||
office_format = _office_format_for_path(path)
|
||||
if office_format != "docx":
|
||||
raise ValueError(f"{office_format or 'office file'} is preview-only in this slice")
|
||||
|
||||
current_preview = preview_office_document(path, current_bytes)
|
||||
if not current_preview.get("editable"):
|
||||
raise ValueError(current_preview.get("edit_blocked_reason") or "DOCX document is not editable")
|
||||
|
||||
# Rebuild from the CURRENT package so styles/docProps/theme survive an
|
||||
# unedited round-trip; the body is still fully replaced and re-verified.
|
||||
saved_bytes = _docx_bytes_from_text(content, current_bytes)
|
||||
saved_preview = preview_office_document(path, saved_bytes)
|
||||
if not saved_preview.get("editable"):
|
||||
raise ValueError(saved_preview.get("edit_blocked_reason") or "Saved DOCX is not editable")
|
||||
return saved_preview, saved_bytes
|
||||
+4
-121
@@ -17,7 +17,6 @@ from api.config import (
|
||||
DEFAULT_WORKSPACE,
|
||||
_FALLBACK_MODELS,
|
||||
_HERMES_FOUND,
|
||||
invalidate_models_cache,
|
||||
_PROVIDER_DISPLAY,
|
||||
_PROVIDER_MODELS,
|
||||
_get_config_path,
|
||||
@@ -28,41 +27,12 @@ from api.config import (
|
||||
save_settings,
|
||||
verify_hermes_imports,
|
||||
)
|
||||
from api.paths import _atomic_write_text
|
||||
from api.providers import _write_env_file # shared impl with _ENV_LOCK (#1164)
|
||||
from api.workspace import get_last_workspace, load_workspaces
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
# ── OpenRouter namespace translation (#7514) ────────────────────────────────
|
||||
#
|
||||
# ``_FALLBACK_MODELS`` (api/config.py) is authored for the *direct* provider
|
||||
# endpoints, so its Z.AI entries carry the provider's own ``zai/`` prefix.
|
||||
# OpenRouter serves the same models under its canonical ``z-ai/`` slug, so
|
||||
# projecting that list into the OpenRouter setup verbatim handed the wizard
|
||||
# ids that 404 on openrouter.ai (``zai/glm-5.3`` instead of
|
||||
# ``z-ai/glm-5.3``). Translate at this projection boundary only — the
|
||||
# fallback catalog and the direct ``zai`` setup keep their own namespace.
|
||||
#
|
||||
# Every other namespace in the fallback list is already OpenRouter-canonical:
|
||||
# openai/, anthropic/, google/, deepseek/, qwen/, x-ai/, mistralai/, minimax/,
|
||||
# openrouter/, tencent/, nvidia/, arcee-ai/ — hence a single mapping entry.
|
||||
_OPENROUTER_NAMESPACE_MAP = {"zai/": "z-ai/"}
|
||||
|
||||
|
||||
def _to_openrouter_namespace(model_id: str) -> str:
|
||||
"""Return *model_id* with direct-provider prefixes mapped to OpenRouter's.
|
||||
|
||||
Ids whose namespace already matches OpenRouter (or that carry no
|
||||
namespace) are returned unchanged.
|
||||
"""
|
||||
for direct_prefix, openrouter_prefix in _OPENROUTER_NAMESPACE_MAP.items():
|
||||
if model_id.startswith(direct_prefix):
|
||||
return openrouter_prefix + model_id[len(direct_prefix):]
|
||||
return model_id
|
||||
|
||||
|
||||
_SUPPORTED_PROVIDER_SETUPS = {
|
||||
# ── Easy start ──────────────────────────────────────────────────────
|
||||
"openrouter": {
|
||||
@@ -71,8 +41,7 @@ _SUPPORTED_PROVIDER_SETUPS = {
|
||||
"default_model": "anthropic/claude-sonnet-4.6",
|
||||
"requires_base_url": False,
|
||||
"models": [
|
||||
{"id": _to_openrouter_namespace(model["id"]), "label": model["label"]}
|
||||
for model in _FALLBACK_MODELS
|
||||
{"id": model["id"], "label": model["label"]} for model in _FALLBACK_MODELS
|
||||
],
|
||||
"category": "easy_start",
|
||||
"quick": True,
|
||||
@@ -281,8 +250,7 @@ def _save_yaml_config(config_path: Path, config: dict) -> None:
|
||||
raise RuntimeError("PyYAML is required to write Hermes config.yaml") from exc
|
||||
|
||||
config_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
_atomic_write_text(
|
||||
config_path,
|
||||
config_path.write_text(
|
||||
_yaml.safe_dump(config, sort_keys=False, allow_unicode=True),
|
||||
encoding="utf-8",
|
||||
)
|
||||
@@ -763,18 +731,15 @@ def _status_from_runtime(cfg: dict, imports_ok: bool) -> dict:
|
||||
)
|
||||
|
||||
chat_ready = bool(_HERMES_FOUND and imports_ok and provider_ready)
|
||||
note_args: list[str] = []
|
||||
|
||||
if not _HERMES_FOUND or not imports_ok:
|
||||
state = "agent_unavailable"
|
||||
note_key = "onboarding_notice_system_unavailable"
|
||||
note = (
|
||||
"Hermes is not fully importable from the Web UI yet. Finish bootstrap or fix the "
|
||||
"agent install before provider setup will work."
|
||||
)
|
||||
elif chat_ready:
|
||||
state = "ready"
|
||||
note_key = "onboarding_notice_system_ready"
|
||||
provider_name = _PROVIDER_DISPLAY.get(
|
||||
provider, provider.title() if provider else "Hermes"
|
||||
)
|
||||
@@ -782,15 +747,11 @@ def _status_from_runtime(cfg: dict, imports_ok: bool) -> dict:
|
||||
elif provider_configured:
|
||||
state = "provider_incomplete"
|
||||
if provider == "custom" and not base_url:
|
||||
note_key = "onboarding_notice_custom_base_url_required"
|
||||
note = (
|
||||
"Hermes has a saved provider/model selection, but the custom "
|
||||
"provider still needs a base URL. Add the API key too if that "
|
||||
"server requires one."
|
||||
"Hermes has a saved provider/model selection but still needs the "
|
||||
"base URL and API key required to chat."
|
||||
)
|
||||
elif provider not in _SUPPORTED_PROVIDER_SETUPS:
|
||||
note_key = "onboarding_notice_provider_auth_required"
|
||||
note_args = [provider]
|
||||
# OAuth / unsupported provider: avoid misleading "API key" wording.
|
||||
note = (
|
||||
f"Provider '{provider}' is configured but not yet authenticated. "
|
||||
@@ -798,14 +759,12 @@ def _status_from_runtime(cfg: dict, imports_ok: bool) -> dict:
|
||||
"setup, then reload the Web UI."
|
||||
)
|
||||
else:
|
||||
note_key = "onboarding_notice_provider_api_key_required"
|
||||
note = (
|
||||
"Hermes has a saved provider/model selection but still needs the "
|
||||
"API key required to chat."
|
||||
)
|
||||
else:
|
||||
state = "needs_provider"
|
||||
note_key = "onboarding_notice_provider_choice_required"
|
||||
note = "Hermes is installed, but you still need to choose a provider and save working credentials."
|
||||
|
||||
return {
|
||||
@@ -814,8 +773,6 @@ def _status_from_runtime(cfg: dict, imports_ok: bool) -> dict:
|
||||
"chat_ready": chat_ready,
|
||||
"setup_state": state,
|
||||
"provider_note": note,
|
||||
"provider_note_key": note_key,
|
||||
"provider_note_args": note_args,
|
||||
"current_provider": provider or None,
|
||||
"current_model": model or None,
|
||||
"current_base_url": base_url or None,
|
||||
@@ -1087,80 +1044,6 @@ def apply_onboarding_setup(body: dict) -> dict:
|
||||
return get_onboarding_status()
|
||||
|
||||
|
||||
def apply_self_hosted_provider_setup(body: dict) -> dict:
|
||||
provider = str(body.get("provider") or "").strip().lower()
|
||||
model = str(body.get("model") or "").strip()
|
||||
api_key = str(body.get("api_key") or "").strip()
|
||||
base_url = _normalize_base_url(str(body.get("base_url") or ""))
|
||||
activate = body.get("activate")
|
||||
do_activate = activate is None or bool(activate)
|
||||
|
||||
if provider not in {"ollama", "lmstudio"}:
|
||||
raise ValueError(f"unsupported self-hosted provider: {provider}")
|
||||
if not model:
|
||||
raise ValueError("model is required")
|
||||
|
||||
provider_meta = _SUPPORTED_PROVIDER_SETUPS.get(provider, {})
|
||||
if provider_meta.get("requires_base_url"):
|
||||
if not base_url:
|
||||
raise ValueError("base_url is required for this provider")
|
||||
parsed = urlparse(base_url)
|
||||
if parsed.scheme not in {"http", "https"}:
|
||||
raise ValueError("base_url must start with http:// or https://")
|
||||
|
||||
config_path = _get_config_path()
|
||||
cfg = _load_yaml_config(config_path)
|
||||
providers_cfg = cfg.setdefault("providers", {})
|
||||
if not isinstance(providers_cfg, dict):
|
||||
providers_cfg = {}
|
||||
cfg["providers"] = providers_cfg
|
||||
|
||||
provider_cfg = providers_cfg.setdefault(provider, {})
|
||||
if not isinstance(provider_cfg, dict):
|
||||
provider_cfg = {}
|
||||
providers_cfg[provider] = provider_cfg
|
||||
|
||||
provider_cfg["base_url"] = base_url
|
||||
|
||||
model_cfg = cfg.get("model", {})
|
||||
if not isinstance(model_cfg, dict):
|
||||
model_cfg = {}
|
||||
original_model_cfg = dict(model_cfg)
|
||||
env_var = provider_meta.get("env_var")
|
||||
|
||||
if do_activate:
|
||||
model_cfg["provider"] = provider
|
||||
model_cfg["default"] = _normalize_model_for_provider(provider, model)
|
||||
model_cfg["base_url"] = base_url
|
||||
cfg["model"] = model_cfg
|
||||
elif "model" in cfg:
|
||||
cfg["model"] = original_model_cfg
|
||||
_save_yaml_config(config_path, cfg)
|
||||
|
||||
if api_key and env_var:
|
||||
_write_env_file(_get_active_hermes_home() / ".env", {env_var: api_key})
|
||||
os.environ[env_var] = api_key
|
||||
|
||||
try:
|
||||
from api.profiles import _reload_dotenv
|
||||
_reload_dotenv(_get_active_hermes_home())
|
||||
except Exception:
|
||||
logger.debug("Failed to reload dotenv")
|
||||
|
||||
try:
|
||||
# hermes_cli may cache config at import time; ask it to reload if possible.
|
||||
from hermes_cli.config import reload as _cli_reload
|
||||
_cli_reload()
|
||||
except Exception:
|
||||
logger.debug("Failed to reload hermes_cli config")
|
||||
|
||||
invalidate_models_cache()
|
||||
result = {"ok": True, "provider": provider, "base_url": base_url}
|
||||
if do_activate:
|
||||
result["model"] = model_cfg.get("default")
|
||||
return result
|
||||
|
||||
|
||||
def complete_onboarding() -> dict:
|
||||
save_settings({"onboarding_completed": True})
|
||||
return get_onboarding_status()
|
||||
|
||||
@@ -72,8 +72,6 @@ def _atomic_write_json(path: Path, payload: Any) -> None:
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as f:
|
||||
json.dump(payload, f, indent=2, sort_keys=True)
|
||||
f.flush()
|
||||
os.fsync(f.fileno()) # durable before rename: no zero-length passkey file on power loss
|
||||
os.chmod(tmp, 0o600)
|
||||
os.replace(tmp, path)
|
||||
except Exception:
|
||||
@@ -182,41 +180,6 @@ def _host_without_port(host: str) -> str:
|
||||
|
||||
|
||||
def rp_context(handler) -> tuple[str, str]:
|
||||
# A reverse proxy that rewrites Host to its own upstream address (common
|
||||
# when a separate SPA front-end proxies /api to this backend) leaves Host
|
||||
# useless for deriving the RPID, while Origin still carries the origin the
|
||||
# browser is actually on. Prefer Origin's hostname: WebAuthn's RPID-origin
|
||||
# check compares against the page's origin, not the backend's.
|
||||
#
|
||||
# Origin is client-supplied, so this is deliberately NOT a trust decision:
|
||||
# it only selects which name the ceremony is scoped to. The actual security
|
||||
# gates are unchanged and live elsewhere — the authenticator will only
|
||||
# release a credential whose RPID is a registrable suffix of the real page
|
||||
# origin, `_client_data()` requires clientDataJSON.origin to equal the
|
||||
# origin stored with the challenge, `_parse_auth_data()` compares the
|
||||
# authenticator's rpIdHash against that same stored RPID, and the assertion
|
||||
# signature is verified against the stored credential public key. A forged
|
||||
# Origin therefore yields a self-consistent ceremony that still cannot
|
||||
# produce a valid signature.
|
||||
#
|
||||
# Accept it only as a syntactically valid http(s) origin with a hostname,
|
||||
# and rebuild the origin string from the parsed parts rather than echoing
|
||||
# the raw header, so a malformed or non-http Origin falls through to Host.
|
||||
browser_origin = handler.headers.get("Origin", "")
|
||||
if browser_origin:
|
||||
try:
|
||||
from urllib.parse import urlparse
|
||||
parsed = urlparse(browser_origin.strip())
|
||||
if parsed.scheme in ("http", "https") and parsed.hostname:
|
||||
# Re-bracket IPv6 literals: parsed.hostname strips the [] that the
|
||||
# browser's clientDataJSON.origin carries, so an unbracketed
|
||||
# "http://::1:8787" would never match the stored origin.
|
||||
host_part = f"[{parsed.hostname}]" if ":" in parsed.hostname else parsed.hostname
|
||||
netloc = host_part if parsed.port is None else f"{host_part}:{parsed.port}"
|
||||
return parsed.hostname, f"{parsed.scheme}://{netloc}"
|
||||
except Exception:
|
||||
pass
|
||||
# Fallback: derive from Host header (direct/internal access)
|
||||
host = _host_without_port(handler.headers.get("Host", "localhost"))
|
||||
proto = handler.headers.get("X-Forwarded-Proto", "").split(",", 1)[0].strip().lower()
|
||||
if proto not in {"http", "https"}:
|
||||
|
||||
-246
@@ -5,258 +5,12 @@ that need the default Hermes home can import them without triggering config's
|
||||
larger startup side effects.
|
||||
"""
|
||||
|
||||
import errno
|
||||
import os
|
||||
import secrets
|
||||
import stat
|
||||
import tempfile
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
HOME = Path.home()
|
||||
|
||||
|
||||
def _create_atomic_temp_file(write_path: Path, *, existing: bool) -> tuple[int, str]:
|
||||
"""Create a same-directory temp file without reading the process umask.
|
||||
|
||||
``tempfile.mkstemp`` deliberately creates files as ``0600``. For a brand-new
|
||||
config, use ``os.open(..., 0666)`` instead so the kernel applies the active
|
||||
process umask as it would for ``Path.write_text``. Reading umask by calling
|
||||
``os.umask`` is unsafe because that syscall mutates process-wide state, and
|
||||
imports may happen after the threaded server has started.
|
||||
"""
|
||||
if existing:
|
||||
return tempfile.mkstemp(
|
||||
dir=str(write_path.parent), prefix=f".{write_path.name}_", suffix=".tmp"
|
||||
)
|
||||
|
||||
flags = os.O_RDWR | os.O_CREAT | os.O_EXCL
|
||||
for _ in range(100):
|
||||
tmp = write_path.parent / f".{write_path.name}_{secrets.token_hex(16)}.tmp"
|
||||
try:
|
||||
return os.open(tmp, flags, 0o666), str(tmp)
|
||||
except FileExistsError:
|
||||
continue
|
||||
raise FileExistsError("unable to allocate unique config temporary file")
|
||||
|
||||
# In-place compatibility writes cannot use rename isolation. Serialize every
|
||||
# such write in this process so concurrent WebUI saves cannot interleave their
|
||||
# truncate/write/fsync sequences. A single lock avoids a permanent per-path
|
||||
# lock cache for arbitrary profile/config locations.
|
||||
_IN_PLACE_WRITE_LOCK = threading.RLock()
|
||||
|
||||
|
||||
def _fsync_directory(directory: Path) -> None:
|
||||
"""Persist a completed rename on filesystems that support directory fsync."""
|
||||
if os.name == "nt":
|
||||
return
|
||||
flags = os.O_RDONLY | getattr(os, "O_DIRECTORY", 0)
|
||||
try:
|
||||
fd = os.open(directory, flags)
|
||||
except PermissionError:
|
||||
return
|
||||
try:
|
||||
try:
|
||||
os.fsync(fd)
|
||||
except OSError as exc:
|
||||
if exc.errno not in {errno.EINVAL, errno.ENOTSUP}:
|
||||
raise
|
||||
finally:
|
||||
os.close(fd)
|
||||
|
||||
|
||||
def _has_extended_attributes(path: Path) -> bool:
|
||||
"""Return whether *path* has xattrs that an inode replacement would discard."""
|
||||
listxattr = getattr(os, "listxattr", None)
|
||||
if listxattr is None:
|
||||
return False
|
||||
try:
|
||||
return bool(listxattr(path))
|
||||
except OSError as exc:
|
||||
unsupported_errnos = {
|
||||
errno.ENOSYS,
|
||||
errno.ENOTSUP,
|
||||
getattr(errno, "EOPNOTSUPP", errno.ENOTSUP),
|
||||
}
|
||||
if exc.errno in unsupported_errnos:
|
||||
return False
|
||||
raise
|
||||
|
||||
|
||||
def _require_writable_target(write_path: Path) -> os.stat_result | None:
|
||||
"""Reject writes to a read-only target before any replacement work.
|
||||
|
||||
``os.replace`` happily swaps a fresh inode over a ``0444`` file when the
|
||||
directory is writable, which would silently defeat a deliberately locked
|
||||
config. The old in-place ``Path.write_text`` raised ``PermissionError``
|
||||
there; preserve that contract with a non-truncating ``O_WRONLY`` probe,
|
||||
letting the failure propagate. Returns the ``fstat`` of the inode the
|
||||
probe actually opened (so the caller's metadata describes the verified
|
||||
file even if a concurrent writer replaced it since its own ``stat``), or
|
||||
``None`` if the target vanished concurrently.
|
||||
"""
|
||||
try:
|
||||
probe_fd = os.open(write_path, os.O_WRONLY)
|
||||
except FileNotFoundError:
|
||||
return None
|
||||
try:
|
||||
return os.fstat(probe_fd)
|
||||
finally:
|
||||
os.close(probe_fd)
|
||||
|
||||
|
||||
def _atomic_write_text(path: Path, text: str, *, encoding: str = "utf-8") -> None:
|
||||
"""Atomically replace *path* with *text*.
|
||||
|
||||
Writes to a temp file in the same directory, flushes + ``os.fsync``, then
|
||||
``os.replace``s it into place, so a crash (or exception) mid-write can never
|
||||
truncate an existing file — the old contents stay intact until the rename
|
||||
commits the new ones in one step. Mirrors the tempfile+fsync+os.replace
|
||||
pattern already used for ``.env`` and cost snapshots in ``api.providers``;
|
||||
extracted here so ``config.yaml`` / profile ``config.yaml`` writers can share
|
||||
it without pulling in env-file-specific logic.
|
||||
|
||||
Permissions and ownership are preserved: ``os.replace`` carries the temp
|
||||
file's metadata onto the target, and ``tempfile.mkstemp`` starts with the
|
||||
writer's owner/group and mode ``0600``. Without correcting those values,
|
||||
every rewrite could silently reset a shared config's group and tighten a
|
||||
group/other-readable ``config.yaml`` (commonly ``0644``/``0664``) down to
|
||||
owner-only. Existing files copy uid/gid and mode onto the temp descriptor;
|
||||
new files use the active process umask, exactly as a normal
|
||||
``open(..., "w")`` would.
|
||||
(Unlike ``.env``, ``config.yaml`` holds no secrets and is not meant to be
|
||||
forced to ``0600``.)
|
||||
|
||||
Symlinks and hard links keep the same follow-through semantics as
|
||||
``Path.write_text``: writing ``config.yaml`` through a symlink updates the
|
||||
referent, while writing an inode with multiple hard links updates every
|
||||
alias rather than severing the selected path. A symlink target change
|
||||
observed before commit is rejected rather than updating a stale referent.
|
||||
|
||||
The temp+rename dance needs WRITE PERMISSION ON THE DIRECTORY, which the
|
||||
plain ``Path.write_text`` it replaced did not: hardened deployments
|
||||
(``HERMES_CONFIG_PATH`` pointing into a root-owned, read-only dir with a
|
||||
writable ``config.yaml`` inside) would otherwise lose the ability to save
|
||||
settings at all. When temp creation is denied, or the original uid/gid
|
||||
cannot be transferred to the temp inode, we fall back to a guarded
|
||||
descriptor write only if the target is still the regular-file inode
|
||||
inspected above. Hard-linked files and files with extended attributes use
|
||||
that same path because replacing their directory entry would respectively
|
||||
break ``Path.write_text`` compatibility or discard metadata such as POSIX
|
||||
ACLs. These in-place writes are serialized across process-local WebUI
|
||||
writers. They give up crash-atomicity (and raw concurrent readers may
|
||||
observe the write in progress) only where atomic replace with preserved
|
||||
semantics is impossible. Other ``PermissionError``s still propagate.
|
||||
Successful renames fsync the parent directory where supported.
|
||||
|
||||
The caller is responsible for ensuring ``path.parent`` exists.
|
||||
"""
|
||||
path = Path(path)
|
||||
symlink_target = path.resolve(strict=False) if path.is_symlink() else None
|
||||
write_path = symlink_target or path
|
||||
try:
|
||||
existing_stat = os.stat(write_path)
|
||||
except FileNotFoundError:
|
||||
existing_stat = None
|
||||
mode = stat.S_IMODE(existing_stat.st_mode) if existing_stat else None
|
||||
|
||||
def _verify_symlink_target() -> None:
|
||||
if symlink_target is not None and path.resolve(strict=False) != symlink_target:
|
||||
raise RuntimeError("config symlink target changed during atomic write")
|
||||
|
||||
def _write_in_place() -> None:
|
||||
with _IN_PLACE_WRITE_LOCK:
|
||||
if existing_stat is None or not stat.S_ISREG(existing_stat.st_mode):
|
||||
raise PermissionError(f"Cannot replace config path: {write_path}")
|
||||
_verify_symlink_target()
|
||||
fallback_fd = os.open(write_path, os.O_WRONLY)
|
||||
owns_fallback_fd = True
|
||||
try:
|
||||
opened_stat = os.fstat(fallback_fd)
|
||||
if (opened_stat.st_dev, opened_stat.st_ino) != (
|
||||
existing_stat.st_dev,
|
||||
existing_stat.st_ino,
|
||||
):
|
||||
raise PermissionError("config target changed before fallback write")
|
||||
_verify_symlink_target()
|
||||
os.ftruncate(fallback_fd, 0)
|
||||
fallback_file = os.fdopen(fallback_fd, "w", encoding=encoding)
|
||||
owns_fallback_fd = False
|
||||
with fallback_file:
|
||||
fallback_file.write(text)
|
||||
fallback_file.flush()
|
||||
os.fsync(fallback_file.fileno())
|
||||
finally:
|
||||
if owns_fallback_fd:
|
||||
os.close(fallback_fd)
|
||||
|
||||
if existing_stat is not None and stat.S_ISREG(existing_stat.st_mode):
|
||||
probed_stat = _require_writable_target(write_path)
|
||||
if probed_stat is not None and stat.S_ISREG(probed_stat.st_mode):
|
||||
existing_stat = probed_stat
|
||||
mode = stat.S_IMODE(existing_stat.st_mode)
|
||||
else:
|
||||
existing_stat = probed_stat
|
||||
mode = None
|
||||
if existing_stat is not None and (
|
||||
existing_stat.st_nlink > 1 or _has_extended_attributes(write_path)
|
||||
):
|
||||
_write_in_place()
|
||||
return
|
||||
|
||||
try:
|
||||
fd, tmp = _create_atomic_temp_file(write_path, existing=existing_stat is not None)
|
||||
except PermissionError:
|
||||
_write_in_place()
|
||||
return
|
||||
owns_fd = True
|
||||
try:
|
||||
if existing_stat is not None and hasattr(os, "fchown"):
|
||||
try:
|
||||
os.fchown(fd, existing_stat.st_uid, existing_stat.st_gid)
|
||||
except (OSError, NotImplementedError) as exc:
|
||||
unsupported_ownership_errnos = {
|
||||
errno.EINVAL,
|
||||
errno.ENOSYS,
|
||||
errno.ENOTSUP,
|
||||
}
|
||||
if (
|
||||
isinstance(exc, OSError)
|
||||
and not isinstance(exc, PermissionError)
|
||||
and exc.errno not in unsupported_ownership_errnos
|
||||
):
|
||||
raise
|
||||
os.close(fd)
|
||||
owns_fd = False
|
||||
os.unlink(tmp)
|
||||
_write_in_place()
|
||||
return
|
||||
if mode is not None and hasattr(os, "fchmod"):
|
||||
os.fchmod(fd, mode)
|
||||
elif mode is not None:
|
||||
os.chmod(tmp, mode)
|
||||
f = os.fdopen(fd, "w", encoding=encoding)
|
||||
owns_fd = False
|
||||
with f:
|
||||
f.write(text)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
_verify_symlink_target()
|
||||
os.replace(tmp, write_path)
|
||||
_fsync_directory(write_path.parent)
|
||||
except BaseException:
|
||||
if owns_fd:
|
||||
try:
|
||||
os.close(fd)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def _hermes_home_has_webui_state(base: Path) -> bool:
|
||||
"""Return True when *base* holds real WebUI state under its ``webui/`` dir.
|
||||
|
||||
|
||||
@@ -0,0 +1,182 @@
|
||||
"""
|
||||
Plugin subprocess lifecycle manager.
|
||||
|
||||
Spawns, monitors, restarts, and kills plugin subprocesses.
|
||||
Each plugin gets its own subprocess with resource limits and
|
||||
a minimal environment.
|
||||
"""
|
||||
|
||||
import logging
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
import threading
|
||||
from pathlib import Path
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
PLUGIN_PROCESSES: dict[str, subprocess.Popen] = {}
|
||||
_PROCESS_LOCK = threading.Lock()
|
||||
_PIPE_LOCKS: dict[str, threading.Lock] = {}
|
||||
|
||||
# Environment variable prefixes allowed in plugin subprocess
|
||||
_ENV_ALLOWED_PREFIXES = {
|
||||
"HERMES_HOME",
|
||||
"HERMES_PLUGIN_",
|
||||
"HERMES_WEBUI_PLUGIN_",
|
||||
"HOME",
|
||||
"LANG",
|
||||
"LC_",
|
||||
"PATH",
|
||||
"PYTHONPATH",
|
||||
"PYTHONUNBUFFERED",
|
||||
"USER",
|
||||
}
|
||||
|
||||
|
||||
def _build_plugin_env(plugin_name: str, plugin_dir: Path) -> dict:
|
||||
env = {}
|
||||
for k, v in os.environ.items():
|
||||
for prefix in _ENV_ALLOWED_PREFIXES:
|
||||
if k == prefix or k.startswith(prefix):
|
||||
env[k] = v
|
||||
break
|
||||
env["HERMES_PLUGIN_NAME"] = plugin_name
|
||||
env["HERMES_PLUGIN_DIR"] = str(plugin_dir)
|
||||
env["PYTHONUNBUFFERED"] = "1"
|
||||
return env
|
||||
|
||||
|
||||
def get_plugin_pipe_lock(plugin_name: str) -> threading.Lock:
|
||||
with _PROCESS_LOCK:
|
||||
if plugin_name not in _PIPE_LOCKS:
|
||||
_PIPE_LOCKS[plugin_name] = threading.Lock()
|
||||
return _PIPE_LOCKS[plugin_name]
|
||||
|
||||
|
||||
def _read_stderr(proc: subprocess.Popen, plugin_name: str):
|
||||
if proc.stderr is None:
|
||||
return
|
||||
try:
|
||||
for line in proc.stderr:
|
||||
line_str = line.decode("utf-8", errors="replace")[:2000] # cap per-line
|
||||
line_str = line_str.rstrip()
|
||||
if line_str:
|
||||
logger.warning("[plugin:%s] %s", plugin_name, line_str)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
|
||||
def spawn_plugin(plugin_name: str, plugin_dir: Path) -> subprocess.Popen | None:
|
||||
"""Spawn a plugin subprocess if not already running.
|
||||
|
||||
Args:
|
||||
plugin_name: e.g. 'my-plugin'
|
||||
plugin_dir: Path to plugin root (contains __init__.py)
|
||||
|
||||
Returns the subprocess.Popen handle, or None on failure.
|
||||
"""
|
||||
plugin_dir = plugin_dir.resolve()
|
||||
runner = Path(__file__).parent / "plugin_runner.py"
|
||||
|
||||
with _PROCESS_LOCK:
|
||||
existing = PLUGIN_PROCESSES.get(plugin_name)
|
||||
if existing and existing.poll() is None:
|
||||
return existing # already running
|
||||
|
||||
# Kill stale process
|
||||
if existing:
|
||||
try:
|
||||
existing.kill()
|
||||
existing.wait(timeout=2)
|
||||
except Exception:
|
||||
pass
|
||||
PLUGIN_PROCESSES.pop(plugin_name, None)
|
||||
|
||||
env = _build_plugin_env(plugin_name, plugin_dir)
|
||||
|
||||
try:
|
||||
proc = subprocess.Popen(
|
||||
[sys.executable, str(runner)],
|
||||
stdin=subprocess.PIPE,
|
||||
stdout=subprocess.PIPE,
|
||||
stderr=subprocess.PIPE,
|
||||
env=env,
|
||||
text=False,
|
||||
)
|
||||
|
||||
# Wait for ready signal from the subprocess
|
||||
if proc.stdout is not None:
|
||||
try:
|
||||
import json as _json
|
||||
import select
|
||||
ready, _, _ = select.select([proc.stdout], [], [], 10)
|
||||
if not ready:
|
||||
logger.warning("Plugin %s: no ready signal within 10s, killing", plugin_name)
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
return None
|
||||
ready_line = proc.stdout.readline()
|
||||
if ready_line:
|
||||
ready_msg = _json.loads(ready_line.decode("utf-8").strip())
|
||||
if ready_msg.get("ready"):
|
||||
logger.info(
|
||||
"Plugin %s ready (pid=%d, routes=%s)",
|
||||
plugin_name, proc.pid, ready_msg.get("routes", []),
|
||||
)
|
||||
else:
|
||||
logger.warning("Plugin %s: unexpected startup message", plugin_name)
|
||||
proc.kill(); proc.wait(); return None
|
||||
else:
|
||||
logger.warning("Plugin %s: no startup message, process may have exited", plugin_name)
|
||||
proc.kill(); proc.wait(); return None
|
||||
except Exception:
|
||||
logger.exception("Plugin %s: startup failed, killing", plugin_name)
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
return None
|
||||
|
||||
PLUGIN_PROCESSES[plugin_name] = proc
|
||||
# Start stderr reader thread
|
||||
threading.Thread(
|
||||
target=_read_stderr, args=(proc, plugin_name), daemon=True,
|
||||
).start()
|
||||
return proc
|
||||
except Exception:
|
||||
logger.exception("Failed to spawn plugin subprocess: %s", plugin_name)
|
||||
return None
|
||||
|
||||
|
||||
def get_plugin_process(plugin_name: str) -> subprocess.Popen | None:
|
||||
with _PROCESS_LOCK:
|
||||
proc = PLUGIN_PROCESSES.get(plugin_name)
|
||||
if proc is None:
|
||||
return None
|
||||
if proc.poll() is not None:
|
||||
logger.warning(
|
||||
"Plugin subprocess %s exited (code=%d)", plugin_name, proc.returncode,
|
||||
)
|
||||
with _PROCESS_LOCK:
|
||||
if PLUGIN_PROCESSES.get(plugin_name) is proc:
|
||||
PLUGIN_PROCESSES.pop(plugin_name, None)
|
||||
return None
|
||||
return proc
|
||||
|
||||
|
||||
def kill_plugin(plugin_name: str, timeout: float = 5.0):
|
||||
with _PROCESS_LOCK:
|
||||
proc = PLUGIN_PROCESSES.pop(plugin_name, None)
|
||||
if proc is None or proc.poll() is not None:
|
||||
return
|
||||
try:
|
||||
proc.terminate()
|
||||
proc.wait(timeout=timeout)
|
||||
except subprocess.TimeoutExpired:
|
||||
proc.kill()
|
||||
proc.wait()
|
||||
logger.info("Terminated plugin subprocess: %s", plugin_name)
|
||||
|
||||
|
||||
def kill_all_plugins():
|
||||
for name in list(PLUGIN_PROCESSES):
|
||||
kill_plugin(name)
|
||||
@@ -0,0 +1,234 @@
|
||||
"""
|
||||
Plugin subprocess runner — stdin/stdout JSON-line protocol.
|
||||
|
||||
Runs inside the plugin subprocess. Reads JSON requests from stdin,
|
||||
dispatches to registered handlers via CompatHandler, writes JSON
|
||||
responses to stdout.
|
||||
|
||||
Set by plugin_manager.py at spawn time:
|
||||
HERMES_PLUGIN_NAME=<plugin_name>
|
||||
HERMES_PLUGIN_DIR=/path/to/plugin/root
|
||||
|
||||
Protocol:
|
||||
Request (stdin, one JSON line):
|
||||
{"id": int, "method": "GET|POST", "path": "/sub_route",
|
||||
"query": "...", "headers": {...}, "body_b64": "..."}
|
||||
|
||||
Response (stdout, one JSON line):
|
||||
{"id": int, "status": 200, "headers": {...}, "body_b64": "..."}
|
||||
|
||||
Lifecycle:
|
||||
- On stdin EOF: exit 0
|
||||
- On handler exception: write error response, continue
|
||||
- On JSON decode error: write error, continue
|
||||
"""
|
||||
|
||||
import base64
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
import traceback
|
||||
from types import SimpleNamespace
|
||||
|
||||
|
||||
class CompatHandler:
|
||||
"""Accumulates handler.write() calls and serializes to a JSON response."""
|
||||
|
||||
def __init__(self, request_headers: dict, body_bytes: bytes):
|
||||
self._status = 200
|
||||
self._resp_headers: dict[str, str] = {}
|
||||
self._body_parts: list[bytes] = []
|
||||
self._body_bytes = body_bytes
|
||||
self._hdrs = {k.lower(): v for k, v in (request_headers or {}).items()}
|
||||
self._read_pos = 0
|
||||
|
||||
@property
|
||||
def headers(self):
|
||||
return self
|
||||
|
||||
def get(self, key, default=None):
|
||||
return self._hdrs.get(key.lower() if isinstance(key, str) else key, default)
|
||||
|
||||
def send_response(self, status: int):
|
||||
self._status = status
|
||||
|
||||
def send_header(self, key: str, value: str):
|
||||
self._resp_headers[key] = value
|
||||
|
||||
def end_headers(self):
|
||||
pass
|
||||
|
||||
@property
|
||||
def wfile(self):
|
||||
return self
|
||||
|
||||
def write(self, data: bytes | str):
|
||||
if isinstance(data, str):
|
||||
data = data.encode("utf-8")
|
||||
self._body_parts.append(data)
|
||||
|
||||
@property
|
||||
def rfile(self):
|
||||
return self
|
||||
|
||||
def read(self, length: int) -> bytes:
|
||||
remaining = len(self._body_bytes) - self._read_pos
|
||||
if remaining <= 0:
|
||||
return b""
|
||||
chunk = self._body_bytes[self._read_pos:self._read_pos + length]
|
||||
self._read_pos += len(chunk)
|
||||
return chunk
|
||||
|
||||
def to_response(self, request_id: int) -> dict:
|
||||
body = b"".join(self._body_parts)
|
||||
resp = {
|
||||
"id": request_id,
|
||||
"status": self._status,
|
||||
"headers": self._resp_headers,
|
||||
}
|
||||
if body:
|
||||
resp["body_b64"] = base64.b64encode(body).decode("ascii")
|
||||
return resp
|
||||
|
||||
def j(self, data: dict, status: int = 200):
|
||||
body = json.dumps(data, ensure_ascii=False).encode("utf-8")
|
||||
self.send_response(status)
|
||||
self.send_header("Content-Type", "application/json; charset=utf-8")
|
||||
self.send_header("Content-Length", str(len(body)))
|
||||
self.end_headers()
|
||||
self.wfile.write(body)
|
||||
|
||||
def bad(self, msg: str, status: int = 400):
|
||||
self.j({"error": msg}, status=status)
|
||||
|
||||
|
||||
def _load_plugin(plugin_name: str, plugin_dir: str):
|
||||
import importlib.util
|
||||
|
||||
init_path = os.path.join(plugin_dir, "__init__.py")
|
||||
if not os.path.isfile(init_path):
|
||||
sys.stderr.write(f"[plugin:{plugin_name}] __init__.py not found\n")
|
||||
sys.stderr.flush()
|
||||
return None
|
||||
|
||||
spec = importlib.util.spec_from_file_location(plugin_name, init_path)
|
||||
if spec is None:
|
||||
sys.stderr.write(f"[plugin:{plugin_name}] failed to create module spec\n")
|
||||
sys.stderr.flush()
|
||||
return None
|
||||
mod = importlib.util.module_from_spec(spec)
|
||||
|
||||
# Make adjacent modules (e.g. tools.py) importable
|
||||
parent = os.path.dirname(plugin_dir)
|
||||
if parent not in sys.path:
|
||||
sys.path.insert(0, parent)
|
||||
|
||||
spec.loader.exec_module(mod)
|
||||
if not hasattr(mod, "register"):
|
||||
sys.stderr.write(f"[plugin:{plugin_name}] no register() found\n")
|
||||
sys.stderr.flush()
|
||||
return None
|
||||
return mod
|
||||
|
||||
|
||||
def main():
|
||||
plugin_name = os.environ.get("HERMES_PLUGIN_NAME", "unknown")
|
||||
plugin_dir = os.environ.get("HERMES_PLUGIN_DIR", "")
|
||||
|
||||
# Apply resource limits to this process (POSIX; silently skipped elsewhere).
|
||||
# NOTE: RLIMIT_AS caps *virtual address space*, not resident RAM. CPython
|
||||
# plus shared libs already map 200-400MB before any plugin code runs, and a
|
||||
# plugin importing numpy / Pillow / a cffi-backed lib can map well past that
|
||||
# at startup. A tight value (e.g. 512MB) would make the interpreter abort
|
||||
# before emitting the ready signal, so the plugin appears permanently
|
||||
# unavailable. We set a generous ceiling that only trips on runaway
|
||||
# allocation; true physical-RAM limiting belongs at the cgroup layer.
|
||||
for name, value in {
|
||||
"RLIMIT_AS": 3 * 1024 * 1024 * 1024, "RLIMIT_NPROC": 64,
|
||||
"RLIMIT_NOFILE": 256, "RLIMIT_CPU": 300,
|
||||
}.items():
|
||||
try:
|
||||
import resource
|
||||
soft, hard = resource.getrlimit(getattr(resource, name))
|
||||
resource.setrlimit(getattr(resource, name), (min(value, hard), hard))
|
||||
except (OSError, ValueError, AttributeError, ImportError):
|
||||
pass
|
||||
try:
|
||||
os.setpgrp()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
out = os.fdopen(os.dup(sys.stdout.fileno()), "w")
|
||||
os.dup2(sys.stderr.fileno(), sys.stdout.fileno())
|
||||
|
||||
def send(data):
|
||||
out.write(json.dumps(data) + "\n")
|
||||
out.flush()
|
||||
|
||||
def err(status, message, request_id=0):
|
||||
send({
|
||||
"id": request_id,
|
||||
"status": status,
|
||||
"headers": {"content-type": "application/json; charset=utf-8"},
|
||||
"body_b64": base64.b64encode(json.dumps({"error": message}).encode()).decode("ascii"),
|
||||
})
|
||||
|
||||
if not plugin_dir:
|
||||
err(500, "HERMES_PLUGIN_DIR not set")
|
||||
return
|
||||
|
||||
mod = _load_plugin(plugin_name, plugin_dir)
|
||||
if mod is None:
|
||||
sys.stderr.write(f"[plugin:{plugin_name}] failed to load\n")
|
||||
sys.stderr.flush()
|
||||
return
|
||||
|
||||
routes = mod.register()
|
||||
if not isinstance(routes, dict):
|
||||
err(500, "register() did not return a route map")
|
||||
return
|
||||
|
||||
send({"ready": True, "routes": list(routes.keys())})
|
||||
|
||||
for line in sys.stdin:
|
||||
line = line.strip()
|
||||
if not line:
|
||||
continue
|
||||
try:
|
||||
req = json.loads(line)
|
||||
except json.JSONDecodeError:
|
||||
err(400, "invalid JSON request")
|
||||
continue
|
||||
|
||||
req_id = req.get("id", 0)
|
||||
method = req.get("method", "GET")
|
||||
path = req.get("path", "/")
|
||||
query = req.get("query", "")
|
||||
req_headers = req.get("headers", {})
|
||||
body_b64 = req.get("body_b64") or ""
|
||||
body_bytes = base64.b64decode(body_b64) if body_b64 else b""
|
||||
|
||||
parsed = SimpleNamespace(path=path, query=query)
|
||||
|
||||
method_handlers = routes.get(path, {})
|
||||
handler_fn = method_handlers.get(method)
|
||||
if handler_fn is None:
|
||||
err(404, "not found", request_id=req_id)
|
||||
continue
|
||||
|
||||
try:
|
||||
ch = CompatHandler(req_headers, body_bytes)
|
||||
handler_fn(ch, parsed)
|
||||
resp = ch.to_response(req_id)
|
||||
except Exception:
|
||||
err(500, "plugin internal error", request_id=req_id)
|
||||
sys.stderr.write(f"[plugin:{plugin_name}] handler error:\n")
|
||||
traceback.print_exc(file=sys.stderr)
|
||||
sys.stderr.flush()
|
||||
continue
|
||||
|
||||
send(resp)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
+35
-1
@@ -1,5 +1,5 @@
|
||||
"""
|
||||
Plugin discovery and static serving for Hermes Web UI.
|
||||
Plugin discovery, static serving, and subprocess management for Hermes Web UI.
|
||||
|
||||
Scans ~/.hermes/plugins/<name>/dashboard/ for manifest.json files,
|
||||
matching the official Hermes dashboard plugin format.
|
||||
@@ -11,7 +11,13 @@ Each plugin may have:
|
||||
index.js -- plugin JS bundle (IIFE)
|
||||
style.css -- optional plugin stylesheet
|
||||
plugin_api.py -- optional backend API (not used in WebUI MVP)
|
||||
|
||||
Each plugin runs in its own subprocess (stdin/stdout JSON protocol).
|
||||
The plugin's __init__.py must define a register() function that returns
|
||||
a route map. WebUI serves plugin API traffic under the unified path
|
||||
prefix /api/plugins/<plugin_name>/<sub_route>.
|
||||
"""
|
||||
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
@@ -37,6 +43,14 @@ PLUGIN_MANIFESTS: dict[str, dict] = {}
|
||||
_PLUGIN_STATIC_ROOTS: dict[str, Path] = {}
|
||||
|
||||
|
||||
def get_plugin_root(plugin_name: str) -> Path | None:
|
||||
"""Return the root directory (parent of dashboard/) for a plugin."""
|
||||
base = _PLUGIN_STATIC_ROOTS.get(plugin_name)
|
||||
if base:
|
||||
return base.parent
|
||||
return None
|
||||
|
||||
|
||||
def _get_plugin_base() -> Path:
|
||||
return Path(os.environ.get("HERMES_WEBUI_PLUGINS_DIR", str(Path.home() / ".hermes" / "plugins")))
|
||||
|
||||
@@ -97,6 +111,26 @@ def load_plugins() -> None:
|
||||
_PLUGIN_STATIC_ROOTS[name] = dashboard_dir.resolve()
|
||||
|
||||
|
||||
def load_plugin_api_routes() -> None:
|
||||
"""Spawn subprocesses for each plugin that has an __init__.py."""
|
||||
for plugin_name in list(PLUGIN_MANIFESTS):
|
||||
base = _PLUGIN_STATIC_ROOTS.get(plugin_name)
|
||||
if not base:
|
||||
continue
|
||||
init_file = base.parent / "__init__.py"
|
||||
if not init_file.exists():
|
||||
logger.debug("Plugin %s has no __init__.py, skipping API routes", plugin_name)
|
||||
continue
|
||||
|
||||
from api.plugin_manager import spawn_plugin
|
||||
plugin_root = base.parent
|
||||
proc = spawn_plugin(plugin_name, plugin_root)
|
||||
if proc is None:
|
||||
logger.warning("Plugin %s: subprocess failed to spawn (dir=%s)", plugin_name, plugin_root)
|
||||
else:
|
||||
logger.info("Plugin %s: spawned subprocess (dir=%s)", plugin_name, plugin_root)
|
||||
|
||||
|
||||
def serve_plugin_static(plugin_name: str, rel_path: str) -> tuple[bytes, str] | None:
|
||||
"""
|
||||
Serve a built static asset from a plugin's dashboard/dist/ (or static/) dir.
|
||||
|
||||
@@ -1,495 +0,0 @@
|
||||
"""Shared helpers for WebUI completion/delegation delivery."""
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import OrderedDict
|
||||
from dataclasses import dataclass
|
||||
import logging
|
||||
import math
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Older Hermes Agent builds do not expose durable claim/complete/release APIs.
|
||||
# Keep their in-process compatibility dedupe bounded so long-lived WebUI
|
||||
# processes cannot retain every delegation id forever.
|
||||
LEGACY_ASYNC_DELIVERY_DEDUPE_MAX = 1024
|
||||
ASYNC_DELIVERY_CLAIM_RETRY_SECONDS = 301.0
|
||||
ASYNC_DELIVERY_ROUTING_RETRY_SECONDS = 5.0
|
||||
_LEGACY_ASYNC_DELIVERY_LOCK = threading.Lock()
|
||||
_LEGACY_ASYNC_DELIVERY_IDS: OrderedDict[str, None] = OrderedDict()
|
||||
_ASYNC_DELIVERY_RETRY_LOCK = threading.Lock()
|
||||
_ASYNC_DELIVERY_RETRY_TIMER: threading.Timer | None = None
|
||||
_ASYNC_DELIVERY_RETRY_DEADLINE = 0.0
|
||||
_ASYNC_DELIVERY_RETRY_QUEUE: Any = None
|
||||
_ASYNC_DELIVERY_RETRY_GENERATION = 0
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class AsyncDelegationDeliveryClaim:
|
||||
"""Opaque ownership token for one WebUI async-delegation consumer."""
|
||||
|
||||
delegation_id: str
|
||||
claim_id: str
|
||||
durable: bool
|
||||
|
||||
|
||||
def completion_delivery_id(evt: Any) -> str:
|
||||
"""Return the stable WebUI delivery/dedupe id for a completion event.
|
||||
|
||||
Terminal background-process events use ``session_id`` for the process id.
|
||||
Async ``delegate_task`` completions carry ``delegation_id`` instead, so both
|
||||
WebUI delivery paths must key those events by ``delegation_id``.
|
||||
"""
|
||||
if not isinstance(evt, dict):
|
||||
return ""
|
||||
if evt.get("type") == "async_delegation":
|
||||
return str(
|
||||
evt.get("delegation_id")
|
||||
or evt.get("session_id")
|
||||
or evt.get("task_id")
|
||||
or ""
|
||||
).strip()
|
||||
return str(evt.get("session_id") or "").strip()
|
||||
|
||||
|
||||
# ── process-wakeup display metadata (#6345) ────────────────────────────────
|
||||
# Inverse of the two structured ``format_wakeup_prompt`` shapes (completion,
|
||||
# watch_match). Those shapes are pinned by
|
||||
# tests/test_background_process_wakeup_format.py; the other event kinds
|
||||
# (watch_overflow/watch_disabled free-text, async_delegation agent-side
|
||||
# formatter) intentionally return None so the UI keeps its raw fallback.
|
||||
_WAKEUP_COMPLETION_RE = re.compile(
|
||||
r"\A\[IMPORTANT: Background process (?P<sid>[^\n]*?) completed "
|
||||
r"\(exit_code=(?P<exit_code>[^)\n]*)\)\.\n"
|
||||
r"Command: (?P<cmd>[^\n]*)\n"
|
||||
r"Output:\n"
|
||||
)
|
||||
_WAKEUP_WATCH_MATCH_RE = re.compile(
|
||||
r"\A\[IMPORTANT: Background process (?P<sid>[^\n]*?) matched watch pattern "
|
||||
r"\"(?P<pattern>.*)\"\.\n"
|
||||
r"Command: (?P<cmd>[^\n]*)\n"
|
||||
r"Matched output:\n"
|
||||
)
|
||||
|
||||
|
||||
def wakeup_display_meta(text: Any) -> dict | None:
|
||||
"""Parse a ``format_wakeup_prompt`` body into display-only metadata.
|
||||
|
||||
Returns ``{type, task_id, command, exit_code}`` for completion events and
|
||||
``{type, task_id, command, pattern}`` for watch matches, or None when the
|
||||
text is not one of those pinned shapes. Header fields only — the output
|
||||
section stays in the message body (the UI extracts it there), so the
|
||||
metadata never duplicates multi-KB process output in the store.
|
||||
|
||||
Header fields are anchored to the pinned single-line grammar (``sid``,
|
||||
``exit_code``, ``command``, ``pattern`` never contain newlines). The
|
||||
optional watch suppression note is deliberately NOT parsed out: it lives in
|
||||
the free-form output tail, where process output can contain the exact same
|
||||
"(N earlier matches were suppressed…)" text, so inferring it from the body
|
||||
would misclassify legitimate output and drop it. The note stays part of the
|
||||
rendered output verbatim (#6350 review finding 2).
|
||||
"""
|
||||
body = str(text or "")
|
||||
m = _WAKEUP_COMPLETION_RE.match(body)
|
||||
if m:
|
||||
exit_code: Any = m.group("exit_code")
|
||||
try:
|
||||
exit_code = int(exit_code)
|
||||
except ValueError:
|
||||
pass
|
||||
return {
|
||||
"type": "completion",
|
||||
"task_id": m.group("sid"),
|
||||
"command": m.group("cmd"),
|
||||
"exit_code": exit_code,
|
||||
}
|
||||
m = _WAKEUP_WATCH_MATCH_RE.match(body)
|
||||
if m:
|
||||
return {
|
||||
"type": "watch_match",
|
||||
"task_id": m.group("sid"),
|
||||
"command": m.group("cmd"),
|
||||
"pattern": m.group("pattern"),
|
||||
}
|
||||
return None
|
||||
|
||||
|
||||
def attach_wakeup_display_meta(msg: Any, source: Any) -> None:
|
||||
"""Stamp ``_wakeup_meta`` on a process-wakeup user message, best-effort.
|
||||
|
||||
Companion to the ``_source`` stamp: display-only (``_wakeup_meta`` is not
|
||||
in ``_API_SAFE_MSG_KEYS``, so it never reaches a provider) and never
|
||||
raises — an unparseable body simply leaves the message unstamped and the
|
||||
UI falls back to parsing/raw rendering.
|
||||
"""
|
||||
if source != "process_wakeup" or not isinstance(msg, dict):
|
||||
return
|
||||
if msg.get("_wakeup_meta"):
|
||||
return
|
||||
try:
|
||||
meta = wakeup_display_meta(msg.get("content"))
|
||||
except Exception:
|
||||
logger.debug("wakeup display-meta derivation failed", exc_info=True)
|
||||
return
|
||||
if meta:
|
||||
msg["_wakeup_meta"] = meta
|
||||
|
||||
|
||||
def build_active_turn_token(stream_id: Any, started_at: Any) -> str | None:
|
||||
"""Return the exact eager-row token for one active WebUI turn."""
|
||||
if not stream_id:
|
||||
return None
|
||||
try:
|
||||
started = float(started_at)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
if not math.isfinite(started) or started <= 0:
|
||||
return None
|
||||
return f"{str(stream_id).strip()}:{started:.17g}"
|
||||
|
||||
|
||||
def stamp_message_source(
|
||||
msg: Any,
|
||||
source: Any,
|
||||
*,
|
||||
active_turn_token: Any = None,
|
||||
) -> None:
|
||||
"""Stamp ``_source`` and any display metadata on a materialized user turn.
|
||||
|
||||
Single choke point for every path that persists a non-``webui`` user turn
|
||||
(result merges, eager checkpoint, and the pending-turn recovery paths) so a
|
||||
future source-bearing recovery site cannot silently skip the ``_wakeup_meta``
|
||||
stamp — the gap #6350 review flagged in ``_append_recovered_pending_turn``
|
||||
and the cancel outer-finally recovery. ``webui`` turns are left untouched to
|
||||
preserve the existing "``_source`` omitted for the default source" contract.
|
||||
"""
|
||||
if isinstance(msg, dict) and active_turn_token:
|
||||
msg["_active_turn_token"] = str(active_turn_token)
|
||||
if not isinstance(msg, dict) or not source or source == "webui":
|
||||
return
|
||||
msg["_source"] = source
|
||||
attach_wakeup_display_meta(msg, source)
|
||||
|
||||
|
||||
def _claim_bounded_local(delegation_id: str) -> bool:
|
||||
with _LEGACY_ASYNC_DELIVERY_LOCK:
|
||||
if delegation_id in _LEGACY_ASYNC_DELIVERY_IDS:
|
||||
return False
|
||||
_LEGACY_ASYNC_DELIVERY_IDS[delegation_id] = None
|
||||
while len(_LEGACY_ASYNC_DELIVERY_IDS) > LEGACY_ASYNC_DELIVERY_DEDUPE_MAX:
|
||||
_LEGACY_ASYNC_DELIVERY_IDS.popitem(last=False)
|
||||
return True
|
||||
|
||||
|
||||
def _release_bounded_local(delegation_id: str) -> None:
|
||||
with _LEGACY_ASYNC_DELIVERY_LOCK:
|
||||
_LEGACY_ASYNC_DELIVERY_IDS.pop(delegation_id, None)
|
||||
|
||||
|
||||
def _arm_async_delegation_restore_sweep(completion_queue: Any, delay: float) -> bool:
|
||||
"""Arm one process-wide durable restore sweep at the earliest deadline.
|
||||
|
||||
The durable database is the backlog. Keeping one shared timer avoids both
|
||||
one-thread-per-event growth and lossy eviction of individual retry entries.
|
||||
The sweep restores every still-pending record; atomic claims suppress races
|
||||
and delivered rows are excluded by the core query.
|
||||
"""
|
||||
global _ASYNC_DELIVERY_RETRY_TIMER
|
||||
global _ASYNC_DELIVERY_RETRY_DEADLINE
|
||||
global _ASYNC_DELIVERY_RETRY_QUEUE
|
||||
global _ASYNC_DELIVERY_RETRY_GENERATION
|
||||
|
||||
if completion_queue is None:
|
||||
return False
|
||||
retry_delay = max(0.0, float(delay))
|
||||
deadline = time.monotonic() + retry_delay
|
||||
|
||||
with _ASYNC_DELIVERY_RETRY_LOCK:
|
||||
if (
|
||||
_ASYNC_DELIVERY_RETRY_TIMER is not None
|
||||
and deadline >= _ASYNC_DELIVERY_RETRY_DEADLINE
|
||||
):
|
||||
return True
|
||||
previous = _ASYNC_DELIVERY_RETRY_TIMER
|
||||
if previous is not None:
|
||||
previous.cancel()
|
||||
_ASYNC_DELIVERY_RETRY_GENERATION += 1
|
||||
generation = _ASYNC_DELIVERY_RETRY_GENERATION
|
||||
_ASYNC_DELIVERY_RETRY_DEADLINE = deadline
|
||||
_ASYNC_DELIVERY_RETRY_QUEUE = completion_queue
|
||||
|
||||
def _restore() -> None:
|
||||
global _ASYNC_DELIVERY_RETRY_TIMER
|
||||
global _ASYNC_DELIVERY_RETRY_DEADLINE
|
||||
global _ASYNC_DELIVERY_RETRY_QUEUE
|
||||
|
||||
with _ASYNC_DELIVERY_RETRY_LOCK:
|
||||
if generation != _ASYNC_DELIVERY_RETRY_GENERATION:
|
||||
return
|
||||
target_queue = _ASYNC_DELIVERY_RETRY_QUEUE
|
||||
_ASYNC_DELIVERY_RETRY_TIMER = None
|
||||
_ASYNC_DELIVERY_RETRY_DEADLINE = 0.0
|
||||
_ASYNC_DELIVERY_RETRY_QUEUE = None
|
||||
try:
|
||||
from tools.async_delegation import restore_undelivered_completions
|
||||
|
||||
restore_undelivered_completions(target_queue)
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"Failed to restore pending async delegations; retrying sweep",
|
||||
exc_info=True,
|
||||
)
|
||||
_arm_async_delegation_restore_sweep(
|
||||
target_queue,
|
||||
ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
)
|
||||
|
||||
timer = threading.Timer(retry_delay, _restore)
|
||||
timer.daemon = True
|
||||
_ASYNC_DELIVERY_RETRY_TIMER = timer
|
||||
timer.start()
|
||||
return True
|
||||
|
||||
|
||||
def schedule_async_delegation_claim_retry(
|
||||
evt: Any,
|
||||
completion_queue: Any,
|
||||
*,
|
||||
delay: float | None = None,
|
||||
) -> bool:
|
||||
"""Schedule a durable restore sweep after a claim or routing lease delay."""
|
||||
if not isinstance(evt, dict) or evt.get("type") != "async_delegation":
|
||||
return False
|
||||
delegation_id = str(evt.get("delegation_id") or "").strip()
|
||||
if not delegation_id or completion_queue is None:
|
||||
return False
|
||||
try:
|
||||
from tools.async_delegation import get_durable_delegation
|
||||
|
||||
durable = get_durable_delegation(delegation_id)
|
||||
except (ImportError, AttributeError):
|
||||
return False
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Failed to inspect durable async delegation %s for retry",
|
||||
delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return False
|
||||
if not isinstance(durable, dict) or durable.get("delivery_state") != "pending":
|
||||
return False
|
||||
|
||||
retry_delay = (
|
||||
ASYNC_DELIVERY_CLAIM_RETRY_SECONDS if delay is None else max(0.0, float(delay))
|
||||
)
|
||||
return _arm_async_delegation_restore_sweep(completion_queue, retry_delay)
|
||||
|
||||
|
||||
def requeue_async_delegation_event(
|
||||
evt: Any,
|
||||
completion_queue: Any,
|
||||
*,
|
||||
delay: float = 0.0,
|
||||
stop_event: threading.Event | None = None,
|
||||
durable: bool | None = None,
|
||||
) -> bool:
|
||||
"""Requeue an async event, falling back to the durable restore sweep.
|
||||
|
||||
Callers pass the queue reference they already resolved so an import failure
|
||||
cannot strand a released durable claim. Legacy events without a durable row
|
||||
still get one best-effort direct requeue; durable events additionally arm a
|
||||
restore sweep when the direct queue write fails.
|
||||
"""
|
||||
if not isinstance(evt, dict) or evt.get("type") != "async_delegation":
|
||||
return False
|
||||
if completion_queue is None:
|
||||
return False
|
||||
retry_delay = max(0.0, float(delay))
|
||||
if retry_delay:
|
||||
if stop_event is not None:
|
||||
if stop_event.wait(retry_delay):
|
||||
return False
|
||||
else:
|
||||
time.sleep(retry_delay)
|
||||
try:
|
||||
completion_queue.put(dict(evt))
|
||||
return True
|
||||
except Exception:
|
||||
logger.warning("Failed to requeue async delegation event", exc_info=True)
|
||||
if durable is True:
|
||||
return _arm_async_delegation_restore_sweep(
|
||||
completion_queue,
|
||||
ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
)
|
||||
return schedule_async_delegation_claim_retry(
|
||||
evt,
|
||||
completion_queue,
|
||||
delay=ASYNC_DELIVERY_ROUTING_RETRY_SECONDS,
|
||||
)
|
||||
|
||||
|
||||
def _cancel_async_delegation_claim_retry(_delegation_id: str) -> None:
|
||||
# Retry is a shared durable-store sweep, not a per-delegation timer. It must
|
||||
# remain armed because other pending records may rely on the same sweep.
|
||||
return None
|
||||
|
||||
|
||||
def async_delivery_retry_timer_count() -> int:
|
||||
with _ASYNC_DELIVERY_RETRY_LOCK:
|
||||
return 1 if _ASYNC_DELIVERY_RETRY_TIMER is not None else 0
|
||||
|
||||
|
||||
def claim_async_delegation_delivery(
|
||||
evt: Any,
|
||||
consumer: str,
|
||||
) -> AsyncDelegationDeliveryClaim | None:
|
||||
"""Atomically claim an async completion for one WebUI delivery path.
|
||||
|
||||
Current Hermes Agent builds provide a durable SQLite-backed claim contract.
|
||||
Older builds fall back to the bounded process-local claim above. The local
|
||||
claim also serializes duplicate legacy events on current cores where no
|
||||
durable row exists yet.
|
||||
"""
|
||||
if not isinstance(evt, dict) or evt.get("type") != "async_delegation":
|
||||
return None
|
||||
delegation_id = completion_delivery_id(evt)
|
||||
if not delegation_id or not _claim_bounded_local(delegation_id):
|
||||
return None
|
||||
|
||||
try:
|
||||
from tools.async_delegation import (
|
||||
claim_event_delivery,
|
||||
complete_event_delivery, # noqa: F401 - capability contract probe
|
||||
release_event_delivery, # noqa: F401 - capability contract probe
|
||||
)
|
||||
except (ImportError, AttributeError):
|
||||
return AsyncDelegationDeliveryClaim(
|
||||
delegation_id=delegation_id,
|
||||
claim_id="",
|
||||
durable=False,
|
||||
)
|
||||
|
||||
try:
|
||||
claim_id = claim_event_delivery(evt, str(consumer or "webui"))
|
||||
except Exception:
|
||||
_release_bounded_local(delegation_id)
|
||||
logger.warning(
|
||||
"Failed to claim durable async delegation delivery for %s",
|
||||
delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
raise
|
||||
if claim_id is None:
|
||||
_release_bounded_local(delegation_id)
|
||||
return None
|
||||
return AsyncDelegationDeliveryClaim(
|
||||
delegation_id=delegation_id,
|
||||
claim_id=str(claim_id or ""),
|
||||
durable=True,
|
||||
)
|
||||
|
||||
|
||||
def _mark_legacy_async_delivery_complete(delegation_id: str) -> bool:
|
||||
"""Acknowledge completion through progressively older core APIs."""
|
||||
try:
|
||||
from tools import async_delegation as async_delivery
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
marker = getattr(async_delivery, "mark_completion_delivered", None)
|
||||
if callable(marker):
|
||||
try:
|
||||
if marker(delegation_id) is not False:
|
||||
return True
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"mark_completion_delivered failed for %s; trying legacy marker",
|
||||
delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
|
||||
legacy_marker = getattr(async_delivery, "mark_async_delegation_consumed", None)
|
||||
if callable(legacy_marker):
|
||||
try:
|
||||
legacy_marker(delegation_id)
|
||||
return True
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Legacy async delegation marker failed for %s",
|
||||
delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
return False
|
||||
|
||||
|
||||
def complete_async_delegation_delivery(
|
||||
evt: Any,
|
||||
claim: AsyncDelegationDeliveryClaim,
|
||||
) -> None:
|
||||
"""Complete a claim after WebUI has accepted the event for delivery."""
|
||||
if claim.durable:
|
||||
from tools.async_delegation import complete_event_delivery
|
||||
|
||||
try:
|
||||
complete_event_delivery(evt, claim.claim_id)
|
||||
_cancel_async_delegation_claim_retry(claim.delegation_id)
|
||||
return
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"Durable async delegation completion ACK failed for %s; "
|
||||
"trying compatibility marker",
|
||||
claim.delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
if _mark_legacy_async_delivery_complete(claim.delegation_id):
|
||||
_cancel_async_delegation_claim_retry(claim.delegation_id)
|
||||
return
|
||||
raise
|
||||
_mark_legacy_async_delivery_complete(claim.delegation_id)
|
||||
|
||||
|
||||
def release_async_delegation_delivery(
|
||||
evt: Any,
|
||||
claim: AsyncDelegationDeliveryClaim,
|
||||
) -> None:
|
||||
"""Release a failed claim so a later WebUI consumer can retry it."""
|
||||
try:
|
||||
if claim.durable:
|
||||
from tools.async_delegation import release_event_delivery
|
||||
|
||||
release_event_delivery(evt, claim.claim_id)
|
||||
except Exception:
|
||||
logger.warning(
|
||||
"Failed to release durable async delegation delivery for %s",
|
||||
claim.delegation_id,
|
||||
exc_info=True,
|
||||
)
|
||||
finally:
|
||||
_release_bounded_local(claim.delegation_id)
|
||||
|
||||
|
||||
def legacy_async_delivery_dedupe_size() -> int:
|
||||
"""Return bounded compatibility-dedupe size for regression coverage."""
|
||||
with _LEGACY_ASYNC_DELIVERY_LOCK:
|
||||
return len(_LEGACY_ASYNC_DELIVERY_IDS)
|
||||
|
||||
|
||||
def _reset_legacy_async_delivery_dedupe_for_tests() -> None:
|
||||
global _ASYNC_DELIVERY_RETRY_TIMER
|
||||
global _ASYNC_DELIVERY_RETRY_DEADLINE
|
||||
global _ASYNC_DELIVERY_RETRY_QUEUE
|
||||
global _ASYNC_DELIVERY_RETRY_GENERATION
|
||||
|
||||
with _LEGACY_ASYNC_DELIVERY_LOCK:
|
||||
_LEGACY_ASYNC_DELIVERY_IDS.clear()
|
||||
with _ASYNC_DELIVERY_RETRY_LOCK:
|
||||
timer = _ASYNC_DELIVERY_RETRY_TIMER
|
||||
_ASYNC_DELIVERY_RETRY_TIMER = None
|
||||
_ASYNC_DELIVERY_RETRY_DEADLINE = 0.0
|
||||
_ASYNC_DELIVERY_RETRY_QUEUE = None
|
||||
_ASYNC_DELIVERY_RETRY_GENERATION += 1
|
||||
if timer is not None:
|
||||
timer.cancel()
|
||||
+65
-306
@@ -21,7 +21,6 @@ from typing import Optional
|
||||
|
||||
import yaml
|
||||
|
||||
from api.paths import _atomic_write_text
|
||||
from api.session_events import publish_session_list_changed
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -57,7 +56,6 @@ _loaded_profile_env_keys: set[str] = set()
|
||||
_tls = threading.local()
|
||||
|
||||
_SKILL_HOME_MODULES = ("tools.skills_tool", "tools.skill_manager_tool")
|
||||
_SKILL_HOME_MODULE_PATCH_LOCK = threading.RLock()
|
||||
|
||||
|
||||
def snapshot_skill_home_modules() -> dict[str, dict[str, object]]:
|
||||
@@ -78,71 +76,6 @@ def snapshot_skill_home_modules() -> dict[str, dict[str, object]]:
|
||||
return snapshot
|
||||
|
||||
|
||||
def _skill_modules_support_profile_home(profile_home: Path) -> bool:
|
||||
"""Return ``True`` when both skill modules resolve to this profile home."""
|
||||
expected_dir = (Path(profile_home) / 'skills').expanduser()
|
||||
|
||||
for module_name in _SKILL_HOME_MODULES:
|
||||
module = sys.modules.get(module_name)
|
||||
if module is None:
|
||||
logger.debug("Skill capability check: %s not pre-imported in sys.modules", module_name)
|
||||
return False
|
||||
|
||||
if not hasattr(module, 'SKILLS_DIR'):
|
||||
logger.debug("Skill capability check: %s missing SKILLS_DIR", module_name)
|
||||
return False
|
||||
|
||||
if not hasattr(module, '_SKILLS_DIR_AT_IMPORT'):
|
||||
logger.debug("Skill capability check: %s missing _SKILLS_DIR_AT_IMPORT", module_name)
|
||||
return False
|
||||
|
||||
try:
|
||||
current_skills_dir = Path(module.SKILLS_DIR).expanduser()
|
||||
import_skills_dir = Path(module._SKILLS_DIR_AT_IMPORT).expanduser()
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Skill capability check: %s has invalid SKILLS_DIR or _SKILLS_DIR_AT_IMPORT",
|
||||
module_name,
|
||||
exc_info=True,
|
||||
)
|
||||
return False
|
||||
|
||||
if current_skills_dir != import_skills_dir:
|
||||
logger.debug(
|
||||
"Skill capability check: %s.SKILLS_DIR %r does not match imported baseline %r",
|
||||
module_name,
|
||||
current_skills_dir,
|
||||
import_skills_dir,
|
||||
)
|
||||
return False
|
||||
|
||||
skills_dir = getattr(module, '_skills_dir', None)
|
||||
if not callable(skills_dir):
|
||||
logger.debug("Skill capability check: %s._skills_dir is not callable", module_name)
|
||||
return False
|
||||
|
||||
try:
|
||||
resolved = Path(skills_dir()).expanduser()
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Skill capability check: %s._skills_dir() failed",
|
||||
module_name,
|
||||
exc_info=True,
|
||||
)
|
||||
return False
|
||||
|
||||
if resolved != expected_dir:
|
||||
logger.debug(
|
||||
"Skill capability check: %s resolves %r instead of %r",
|
||||
module_name,
|
||||
str(resolved),
|
||||
str(expected_dir),
|
||||
)
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def patch_skill_home_modules(home: Path) -> None:
|
||||
"""Patch imported skill modules that cache HERMES_HOME at import time."""
|
||||
for module_name in _SKILL_HOME_MODULES:
|
||||
@@ -1127,48 +1060,11 @@ def _resolve_secret_scope_module():
|
||||
return None
|
||||
|
||||
|
||||
# #5567: hermes-agent v0.18.0+ exposes a CONTEXT-LOCAL Hermes-home override
|
||||
# (`hermes_constants.set_hermes_home_override`) that `get_hermes_home()` — and
|
||||
# therefore `hermes_cli.config.get_config_path()` / `load_config()` — consults
|
||||
# BEFORE the process-global `os.environ["HERMES_HOME"]`. Installing it inside the
|
||||
# profile worker scope eliminates the cross-profile HERMES_HOME race at the
|
||||
# reader (a config read resolves the task-local profile home even if another
|
||||
# thread clobbers os.environ mid-body) WITHOUT serializing workers or mutating
|
||||
# shared state. Resolved lazily + optionally so OLDER agents (no override symbol)
|
||||
# degrade gracefully to the pre-existing os.environ-mirror behavior — unchanged.
|
||||
_hermes_home_override_available = None
|
||||
|
||||
|
||||
def _resolve_hermes_home_override():
|
||||
"""Return the hermes_constants module iff it exposes the v0.18.0+ context-local
|
||||
home override (set/reset), else None. Cached; import-safe on older agents."""
|
||||
global _hermes_home_override_available
|
||||
import sys as _sys
|
||||
if _hermes_home_override_available is False:
|
||||
return None
|
||||
mod = _sys.modules.get('hermes_constants')
|
||||
if mod is None and _hermes_home_override_available is None:
|
||||
try:
|
||||
import hermes_constants as mod # noqa: F811
|
||||
except Exception:
|
||||
_hermes_home_override_available = False
|
||||
return None
|
||||
if mod is not None and hasattr(mod, 'set_hermes_home_override') and hasattr(
|
||||
mod, 'reset_hermes_home_override'
|
||||
):
|
||||
_hermes_home_override_available = True
|
||||
return mod
|
||||
_hermes_home_override_available = False
|
||||
return None
|
||||
|
||||
|
||||
@contextmanager
|
||||
def profile_env_for_background_worker(
|
||||
session,
|
||||
purpose: str = "background worker",
|
||||
logger_override: Optional[logging.Logger] = None,
|
||||
*,
|
||||
scope_skill_modules: bool = True,
|
||||
):
|
||||
"""Temporarily route detached worker config reads through a profile.
|
||||
|
||||
@@ -1221,15 +1117,6 @@ def profile_env_for_background_worker(
|
||||
)
|
||||
_scope_token = None
|
||||
_has_scope = False
|
||||
_secret_scope_mod = None
|
||||
# #5567: context-local Hermes-home override (hermes-agent v0.18.0+). None on
|
||||
# older agents → graceful no-op (falls back to the os.environ mirror below).
|
||||
_home_override_mod = None
|
||||
_home_override_token = None
|
||||
_home_override_installed = False
|
||||
has_profile_skill_home = False
|
||||
should_restore_skill_modules = False
|
||||
_acquired_skill_home_patch_lock = False
|
||||
try:
|
||||
_set_thread_env(**thread_env)
|
||||
_thread_ctx.block_process_env_fallback = True
|
||||
@@ -1242,55 +1129,7 @@ def profile_env_for_background_worker(
|
||||
_has_scope = True
|
||||
except Exception:
|
||||
pass
|
||||
# #5567: install the context-local Hermes-home override so the agent
|
||||
# config reader (get_hermes_home -> get_config_path/load_config) resolves
|
||||
# THIS profile's home from task-local state, immune to a concurrent
|
||||
# cross-profile os.environ["HERMES_HOME"] clobber during the worker body.
|
||||
# No-op on agents < v0.18.0 (resolver returns None) → os.environ mirror
|
||||
# below remains the behavior, exactly as today.
|
||||
_home_override_mod = _resolve_hermes_home_override()
|
||||
if _home_override_mod is not None:
|
||||
try:
|
||||
_home_override_token = _home_override_mod.set_hermes_home_override(
|
||||
str(profile_home_path)
|
||||
)
|
||||
_home_override_installed = True
|
||||
except Exception:
|
||||
_home_override_token = None
|
||||
_home_override_installed = False
|
||||
|
||||
if scope_skill_modules:
|
||||
if _home_override_mod is not None and _home_override_installed:
|
||||
try:
|
||||
has_profile_skill_home = _skill_modules_support_profile_home(
|
||||
profile_home_path
|
||||
)
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Failed to evaluate profile-home skill module capability for %s in %s",
|
||||
profile,
|
||||
purpose,
|
||||
exc_info=True,
|
||||
)
|
||||
has_profile_skill_home = False
|
||||
|
||||
# #5567-fallback: if override is unavailable, or module-side
|
||||
# profile resolution is missing/failed, serialize the full worker
|
||||
# lifespan under the shared legacy patch lock.
|
||||
should_restore_skill_modules = not (
|
||||
_home_override_installed and has_profile_skill_home
|
||||
)
|
||||
if should_restore_skill_modules:
|
||||
_SKILL_HOME_MODULE_PATCH_LOCK.acquire()
|
||||
_acquired_skill_home_patch_lock = True
|
||||
|
||||
with _ENV_LOCK:
|
||||
if scope_skill_modules and should_restore_skill_modules:
|
||||
# Snapshot and patch before mutating process env so setup
|
||||
# failures can unwind without leaking either state.
|
||||
skill_home_snapshot = snapshot_skill_home_modules()
|
||||
patch_skill_home_modules(profile_home_path)
|
||||
|
||||
old_runtime_env = _apply_profile_env_to_process(
|
||||
os.environ,
|
||||
safe_runtime_env,
|
||||
@@ -1298,43 +1137,42 @@ def profile_env_for_background_worker(
|
||||
)
|
||||
had_hermes_home = "HERMES_HOME" in os.environ
|
||||
old_hermes_home = os.environ.get("HERMES_HOME")
|
||||
skill_home_snapshot = snapshot_skill_home_modules()
|
||||
os.environ.update(safe_runtime_env)
|
||||
os.environ["HERMES_HOME"] = str(profile_home_path)
|
||||
try:
|
||||
patch_skill_home_modules(profile_home_path)
|
||||
except Exception:
|
||||
log.debug(
|
||||
"Failed to patch skill modules for %s profile %s",
|
||||
purpose,
|
||||
profile,
|
||||
exc_info=True,
|
||||
)
|
||||
yield
|
||||
finally:
|
||||
try:
|
||||
with _ENV_LOCK:
|
||||
for key, old_value in old_runtime_env.items():
|
||||
if old_value is None:
|
||||
os.environ.pop(key, None)
|
||||
else:
|
||||
os.environ[key] = old_value
|
||||
if had_hermes_home:
|
||||
os.environ["HERMES_HOME"] = old_hermes_home or ""
|
||||
if _has_scope and _secret_scope_mod is not None:
|
||||
try:
|
||||
_secret_scope_mod.reset_secret_scope(_scope_token)
|
||||
except Exception:
|
||||
pass
|
||||
_thread_ctx.block_process_env_fallback = previous_block_process_env
|
||||
if previous_thread_env:
|
||||
_set_thread_env(**previous_thread_env)
|
||||
else:
|
||||
_clear_thread_env()
|
||||
with _ENV_LOCK:
|
||||
for key, old_value in old_runtime_env.items():
|
||||
if old_value is None:
|
||||
os.environ.pop(key, None)
|
||||
else:
|
||||
os.environ.pop("HERMES_HOME", None)
|
||||
if should_restore_skill_modules and skill_home_snapshot is not None:
|
||||
restore_skill_home_modules(skill_home_snapshot)
|
||||
finally:
|
||||
if _acquired_skill_home_patch_lock:
|
||||
_SKILL_HOME_MODULE_PATCH_LOCK.release()
|
||||
_acquired_skill_home_patch_lock = False
|
||||
# Reset context-local state after the fallback globals are restored.
|
||||
if _home_override_mod is not None and _home_override_installed:
|
||||
try:
|
||||
_home_override_mod.reset_hermes_home_override(_home_override_token)
|
||||
except Exception:
|
||||
pass
|
||||
if _has_scope and _secret_scope_mod is not None:
|
||||
try:
|
||||
_secret_scope_mod.reset_secret_scope(_scope_token)
|
||||
except Exception:
|
||||
pass
|
||||
_thread_ctx.block_process_env_fallback = previous_block_process_env
|
||||
if previous_thread_env:
|
||||
_set_thread_env(**previous_thread_env)
|
||||
os.environ[key] = old_value
|
||||
if had_hermes_home:
|
||||
os.environ["HERMES_HOME"] = old_hermes_home or ""
|
||||
else:
|
||||
_clear_thread_env()
|
||||
os.environ.pop("HERMES_HOME", None)
|
||||
if skill_home_snapshot is not None:
|
||||
restore_skill_home_modules(skill_home_snapshot)
|
||||
|
||||
|
||||
@contextmanager
|
||||
@@ -1395,7 +1233,6 @@ def profile_env_for_active_request_readonly(
|
||||
getattr(_thread_ctx, "block_process_env_fallback", False)
|
||||
)
|
||||
home_override_token = None
|
||||
home_override_installed = False
|
||||
_scope_token = None
|
||||
_has_scope = False
|
||||
try:
|
||||
@@ -1411,16 +1248,7 @@ def profile_env_for_active_request_readonly(
|
||||
except Exception:
|
||||
pass
|
||||
if set_hermes_home_override is not None:
|
||||
try:
|
||||
home_override_token = set_hermes_home_override(profile_home_path)
|
||||
home_override_installed = True
|
||||
except Exception:
|
||||
logger.debug(
|
||||
"Failed to install Hermes-home override for active request profile %s in %s",
|
||||
profile,
|
||||
purpose,
|
||||
exc_info=True,
|
||||
)
|
||||
home_override_token = set_hermes_home_override(profile_home_path)
|
||||
yield
|
||||
finally:
|
||||
if _has_scope and _secret_scope_mod is not None:
|
||||
@@ -1428,7 +1256,7 @@ def profile_env_for_active_request_readonly(
|
||||
_secret_scope_mod.reset_secret_scope(_scope_token)
|
||||
except Exception:
|
||||
pass
|
||||
if reset_hermes_home_override is not None and home_override_installed:
|
||||
if home_override_token is not None and reset_hermes_home_override is not None:
|
||||
try:
|
||||
reset_hermes_home_override(home_override_token)
|
||||
except Exception:
|
||||
@@ -1711,22 +1539,19 @@ def switch_profile(name: str, *, process_wide: bool = True) -> dict:
|
||||
default_workspace = None
|
||||
try:
|
||||
from api.config import DEFAULT_WORKSPACE as _DW
|
||||
from api.workspace import _resolve_path, _remote_terminal_workspace_candidate
|
||||
lw_file = home / 'webui_state' / 'last_workspace.txt'
|
||||
if lw_file.exists():
|
||||
_p = lw_file.read_text(encoding='utf-8').strip()
|
||||
if _p:
|
||||
_pp = _resolve_path(_p, profile=name)
|
||||
remote_cand = _remote_terminal_workspace_candidate(_p, profile=name)
|
||||
if remote_cand is not None or _pp.is_dir():
|
||||
default_workspace = str(_pp)
|
||||
_pp = Path(_p).expanduser()
|
||||
if _pp.is_dir():
|
||||
default_workspace = str(_pp.resolve())
|
||||
if default_workspace is None:
|
||||
for _key in ('workspace', 'default_workspace'):
|
||||
_v = cfg.get(_key)
|
||||
if _v:
|
||||
_pp = _resolve_path(str(_v), profile=name)
|
||||
remote_cand = _remote_terminal_workspace_candidate(str(_v), profile=name)
|
||||
if remote_cand is not None or _pp.is_dir():
|
||||
_pp = Path(str(_v)).expanduser().resolve()
|
||||
if _pp.is_dir():
|
||||
default_workspace = str(_pp)
|
||||
break
|
||||
if default_workspace is None:
|
||||
@@ -1734,9 +1559,8 @@ def switch_profile(name: str, *, process_wide: bool = True) -> dict:
|
||||
if isinstance(_tc, dict):
|
||||
_cwd = _tc.get('cwd', '')
|
||||
if _cwd and str(_cwd) not in ('.', ''):
|
||||
_pp = _resolve_path(str(_cwd), profile=name)
|
||||
remote_cand = _remote_terminal_workspace_candidate(str(_cwd), profile=name)
|
||||
if remote_cand is not None or _pp.is_dir():
|
||||
_pp = Path(str(_cwd)).expanduser().resolve()
|
||||
if _pp.is_dir():
|
||||
default_workspace = str(_pp)
|
||||
if default_workspace is None:
|
||||
default_workspace = str(_DW)
|
||||
@@ -1760,31 +1584,6 @@ def switch_profile(name: str, *, process_wide: bool = True) -> dict:
|
||||
_SKILLS_STATS_CACHE: dict[Path, tuple[int, int, int, float]] = {}
|
||||
_SKILLS_STATS_CACHE_TTL = 300.0 # seconds — long because .clear() handles programmatic changes
|
||||
|
||||
# Per-profile compute locks (#5364). Without these, concurrent cold-startup
|
||||
# requests (ThreadingHTTPServer runs one OS thread per request) all miss the
|
||||
# unlocked _SKILLS_STATS_CACHE at once and each walks + parses the whole skill
|
||||
# tree simultaneously — a thundering herd that stalled workers 57–70s under
|
||||
# Docker overlay2. A per-profile lock lets independent profiles compute in
|
||||
# parallel while collapsing concurrent misses on the SAME profile to a single
|
||||
# shared compute (double-checked locking below). The lock registry is guarded by
|
||||
# its own meta-lock and is bounded by the (small) number of profiles.
|
||||
_SKILLS_STATS_LOCKS: dict[Path, threading.Lock] = {}
|
||||
_SKILLS_STATS_LOCKS_GUARD = threading.Lock()
|
||||
|
||||
|
||||
def _skills_stats_lock_for(profile_dir: Path) -> threading.Lock:
|
||||
"""Return (creating if needed) the per-profile compute lock for profile_dir.
|
||||
|
||||
profile_dir must already be resolved so distinct spellings of the same
|
||||
directory share one lock.
|
||||
"""
|
||||
with _SKILLS_STATS_LOCKS_GUARD:
|
||||
lock = _SKILLS_STATS_LOCKS.get(profile_dir)
|
||||
if lock is None:
|
||||
lock = threading.Lock()
|
||||
_SKILLS_STATS_LOCKS[profile_dir] = lock
|
||||
return lock
|
||||
|
||||
|
||||
def _skill_tree_max_mtime_ns(skills_dir: Path, config_path: Path) -> int:
|
||||
"""Return the max st_mtime_ns across config.yaml, skill dirs, and SKILL.md files."""
|
||||
@@ -1926,33 +1725,19 @@ def _get_profile_skills_stats(profile_dir: Path) -> tuple[int, int]:
|
||||
if current_mtime_ns == cached_mtime_ns and now < expiry:
|
||||
return enabled, compat
|
||||
|
||||
# Cache miss, mtime changed, or TTL expired — serialize per-profile so a
|
||||
# burst of concurrent misses (cold startup) collapses to ONE compute instead
|
||||
# of a thundering herd of simultaneous os.walk + SKILL.md parses (#5364).
|
||||
lock = _skills_stats_lock_for(profile_dir)
|
||||
with lock:
|
||||
# Double-checked locking: another thread may have populated a fresh entry
|
||||
# while we waited for the lock. Reuse it when the mtime we already probed
|
||||
# still matches and the entry is within its TTL — no second compute.
|
||||
cached = _SKILLS_STATS_CACHE.get(profile_dir)
|
||||
if cached is not None:
|
||||
enabled, compat, cached_mtime_ns, expiry = cached
|
||||
if current_mtime_ns == cached_mtime_ns and time.time() < expiry:
|
||||
return enabled, compat
|
||||
|
||||
# Snapshot mtime BEFORE compute so any concurrent SKILL.md write during
|
||||
# the compute window causes a mismatch on the next probe instead of
|
||||
# silently serving stale data (TOCTOU).
|
||||
new_mtime_ns = _skill_tree_max_mtime_ns(skills_dir, config_path)
|
||||
res = _compute_profile_skills_stats(profile_dir)
|
||||
_SKILLS_STATS_CACHE[profile_dir] = (
|
||||
res[0], res[1], new_mtime_ns, time.time() + _SKILLS_STATS_CACHE_TTL
|
||||
)
|
||||
return res
|
||||
# Cache miss, mtime changed, or TTL expired — snapshot mtime BEFORE compute
|
||||
# so any concurrent SKILL.md write during the compute window causes a mismatch
|
||||
# on the next probe instead of silently serving stale data (TOCTOU).
|
||||
new_mtime_ns = _skill_tree_max_mtime_ns(skills_dir, config_path)
|
||||
res = _compute_profile_skills_stats(profile_dir)
|
||||
_SKILLS_STATS_CACHE[profile_dir] = (res[0], res[1], new_mtime_ns, now + _SKILLS_STATS_CACHE_TTL)
|
||||
return res
|
||||
|
||||
|
||||
_LIST_PROFILES_CACHE: tuple[list, float] | None = None
|
||||
_LIST_PROFILES_CACHE_TTL = 4.0 # seconds. The perf(session-load-latency) pass bumped this to 60s, but that was reverted: profile-row mutations (defaults / providers / skills / gateway config) do NOT invalidate this cache, so a 60s TTL served stale profile rows for too long after such a change. 4s keeps the os.walk frequent enough that mutation→poll staleness is negligible while still making rapid dropdown re-opens free. The create/delete invalidation hooks below clear the cache immediately on those specific mutations.
|
||||
_LIST_PROFILES_CACHE_TTL = 4.0 # seconds — short enough that gateway dots / new
|
||||
# profiles stay near-live, long enough that rapid
|
||||
# re-opens of the dropdown are free.
|
||||
_LIST_PROFILES_CACHE_LOCK = threading.Lock()
|
||||
|
||||
|
||||
@@ -2105,21 +1890,14 @@ def list_profiles_api() -> list:
|
||||
'total_skills': total_count,
|
||||
}]
|
||||
|
||||
# Single-flight the build (#5364): hold the cache lock across the row build
|
||||
# so a cold-startup burst of concurrent requests collapses to ONE build while
|
||||
# the others wait and then serve the freshly-cached rows — instead of every
|
||||
# thread rebuilding (each walking all profiles' skill trees) at once. The
|
||||
# per-profile skills locks taken inside _build_profile_rows_fast are always
|
||||
# acquired AFTER this lock (never the reverse), so there is no deadlock.
|
||||
with _LIST_PROFILES_CACHE_LOCK:
|
||||
cached = _LIST_PROFILES_CACHE
|
||||
if cached is not None and now - cached[1] < _LIST_PROFILES_CACHE_TTL:
|
||||
rows = cached[0]
|
||||
else:
|
||||
rows = _build_profile_rows_fast()
|
||||
if rows is not None:
|
||||
_LIST_PROFILES_CACHE = (rows, time.time())
|
||||
if cached is not None and now - cached[1] < _LIST_PROFILES_CACHE_TTL:
|
||||
active = get_active_profile_name()
|
||||
# Return a fresh copy with is_active recomputed (cheap, per-request).
|
||||
return [{**p, 'is_active': p['name'] == active} for p in cached[0]]
|
||||
|
||||
rows = _build_profile_rows_fast()
|
||||
if rows is None:
|
||||
# Fallback: cheap helpers unavailable — use the original (slow) path,
|
||||
# or the default-only dict if hermes_cli isn't importable at all.
|
||||
@@ -2153,6 +1931,9 @@ def list_profiles_api() -> list:
|
||||
})
|
||||
return result
|
||||
|
||||
with _LIST_PROFILES_CACHE_LOCK:
|
||||
_LIST_PROFILES_CACHE = (rows, now)
|
||||
|
||||
active = get_active_profile_name()
|
||||
return [{**p, 'is_active': p['name'] == active} for p in rows]
|
||||
|
||||
@@ -2379,7 +2160,7 @@ def _write_endpoint_to_config(profile_dir: Path, base_url: str = None, api_key:
|
||||
if base_url:
|
||||
model_section['base_url'] = base_url
|
||||
cfg['model'] = model_section
|
||||
_atomic_write_text(config_path, _yaml.dump(cfg, default_flow_style=False, allow_unicode=True), encoding='utf-8')
|
||||
config_path.write_text(_yaml.dump(cfg, default_flow_style=False, allow_unicode=True), encoding='utf-8')
|
||||
|
||||
|
||||
def _clean_profile_config_value(value: Optional[str], field: str) -> Optional[str]:
|
||||
@@ -2397,41 +2178,20 @@ def _clean_profile_config_value(value: Optional[str], field: str) -> Optional[st
|
||||
|
||||
|
||||
def _split_webui_provider_model_value(default_model: Optional[str], model_provider: Optional[str]) -> tuple[Optional[str], Optional[str]]:
|
||||
"""Normalize WebUI-internal @provider:model picker values for config.yaml.
|
||||
|
||||
Parsing is delegated to ``config._parse_provider_qualified_model_id()`` so
|
||||
this agrees with the grammar every other call site uses (#6722, #6723). A
|
||||
positional ``rsplit(":", 1)`` cannot tell a colon-tagged model
|
||||
(``@ollama:qwen3.8:27b-mtp-q8_0``) from a multi-segment custom provider ID
|
||||
(``@custom:backup:model-a``); it truncated the model name and persisted the
|
||||
fragment into profile config, which the provider API then 404'd on (#7182).
|
||||
"""
|
||||
"""Normalize WebUI-internal @provider:model picker values for config.yaml."""
|
||||
model = _clean_profile_config_value(default_model, "default_model")
|
||||
provider = _clean_profile_config_value(model_provider, "model_provider")
|
||||
if model and model.startswith("@") and ":" in model:
|
||||
from api.config import _parse_provider_qualified_model_id
|
||||
|
||||
parsed = _parse_provider_qualified_model_id(model)
|
||||
if parsed:
|
||||
model_part, provider_part = parsed
|
||||
provider = provider or _clean_profile_config_value(provider_part, "model_provider")
|
||||
model = _clean_profile_config_value(model_part, "default_model")
|
||||
provider_part, model_part = model[1:].rsplit(":", 1)
|
||||
provider = provider or _clean_profile_config_value(provider_part, "model_provider")
|
||||
model = _clean_profile_config_value(model_part, "default_model")
|
||||
return model, provider
|
||||
|
||||
|
||||
def _strip_webui_provider_prefix(model_id: object) -> str:
|
||||
"""Return the bare model name from a WebUI ``@provider:model`` value.
|
||||
|
||||
Uses the same shared grammar as ``_split_webui_provider_model_value()`` so a
|
||||
tagged model name survives the round trip (#7182).
|
||||
"""
|
||||
value = str(model_id or "").strip()
|
||||
if value.startswith("@") and ":" in value:
|
||||
from api.config import _parse_provider_qualified_model_id
|
||||
|
||||
parsed = _parse_provider_qualified_model_id(value)
|
||||
if parsed:
|
||||
return str(parsed[0] or "").strip()
|
||||
return value.rsplit(":", 1)[1]
|
||||
return value
|
||||
|
||||
|
||||
@@ -2456,8 +2216,7 @@ def _profile_model_selection_exists(
|
||||
continue
|
||||
if model_provider and provider_id == model_provider:
|
||||
provider_seen = True
|
||||
all_group_models = (group.get("models") or []) + (group.get("extra_models") or [])
|
||||
for model in all_group_models:
|
||||
for model in group.get("models", []) or []:
|
||||
if not isinstance(model, dict):
|
||||
continue
|
||||
model_id = str(model.get("id") or "").strip()
|
||||
@@ -2539,7 +2298,7 @@ def _write_model_defaults_to_config(
|
||||
if model_provider:
|
||||
model_section['provider'] = model_provider
|
||||
cfg['model'] = model_section
|
||||
_atomic_write_text(config_path, _yaml.dump(cfg, default_flow_style=False, allow_unicode=True), encoding='utf-8')
|
||||
config_path.write_text(_yaml.dump(cfg, default_flow_style=False, allow_unicode=True), encoding='utf-8')
|
||||
|
||||
|
||||
def create_profile_api(name: str, clone_from: str = None,
|
||||
|
||||
+9
-253
@@ -9,7 +9,6 @@ from __future__ import annotations
|
||||
|
||||
import atexit
|
||||
import base64
|
||||
import copy
|
||||
import hashlib
|
||||
import json
|
||||
import logging
|
||||
@@ -35,8 +34,6 @@ except ImportError: # pragma: no cover - exercised only where fcntl is unavaila
|
||||
from api.config import (
|
||||
_PROVIDER_DISPLAY,
|
||||
_PROVIDER_MODELS,
|
||||
_coerce_provider_cost_budget,
|
||||
_configured_model_ids,
|
||||
_custom_provider_slug_from_name,
|
||||
_get_label_for_model,
|
||||
_models_from_live_provider_ids,
|
||||
@@ -80,7 +77,6 @@ _OPENROUTER_KEY_URL = "https://openrouter.ai/api/v1/key"
|
||||
_PROVIDER_QUOTA_TIMEOUT_SECONDS = 3.0
|
||||
_ACCOUNT_USAGE_SUBPROCESS_TIMEOUT_SECONDS = 35.0
|
||||
_ACCOUNT_USAGE_CACHE_TTL_SECONDS = 45.0
|
||||
_PROVIDERS_CACHE_TTL_SECONDS = 30.0
|
||||
_ACCOUNT_USAGE_CACHE_MAX_ENTRIES = 64
|
||||
_ACCOUNT_USAGE_WORKER_IDLE_SECONDS = 5 * 60
|
||||
_ACCOUNT_USAGE_PROVIDERS = frozenset({"openai-codex", "anthropic"})
|
||||
@@ -137,8 +133,6 @@ _account_usage_probe_semaphore: threading.BoundedSemaphore | None = None
|
||||
# represented as non-None snapshots and remain cacheable.
|
||||
_account_usage_status_cache: dict[tuple[str, str, str], tuple[float, Any]] = {}
|
||||
_account_usage_status_cache_lock = threading.Lock()
|
||||
_providers_cache: dict[tuple[Any, ...], tuple[float, dict[str, Any]]] = {}
|
||||
_providers_cache_lock = threading.Lock()
|
||||
_account_usage_worker_pool: dict[str, list["_AccountUsageProbeWorker"]] = {}
|
||||
_account_usage_worker_pool_lock = threading.Lock()
|
||||
|
||||
@@ -425,19 +419,6 @@ def _entry_exhausted_ttl_seconds(error_code):
|
||||
code = str(error_code or "").strip()
|
||||
if code == "401":
|
||||
return 5 * 60
|
||||
if code == "402":
|
||||
# #6626: keep WebUI's eligibility decision tied to the installed
|
||||
# runtime contract. The runtime routes 402 via
|
||||
# credential_pool._exhausted_ttl() (120s when the new
|
||||
# EXHAUSTED_TTL_402_SECONDS is present, 1h fallback otherwise).
|
||||
# Hard-coding 120s here would let display/probe code mark an entry
|
||||
# usable before CredentialPool.select() is willing to lease it on
|
||||
# mixed-version installations.
|
||||
try:
|
||||
from agent.credential_pool import _exhausted_ttl as _runtime_exhausted_ttl
|
||||
return _runtime_exhausted_ttl(int(code))
|
||||
except Exception:
|
||||
return 60 * 60
|
||||
return 60 * 60
|
||||
|
||||
|
||||
@@ -720,7 +701,6 @@ _PROVIDER_ENV_VAR: dict[str, str] = {
|
||||
"mistralai": "MISTRAL_API_KEY",
|
||||
"x-ai": "XAI_API_KEY",
|
||||
"xiaomi": "XIAOMI_API_KEY",
|
||||
"neuralwatt": "NEURALWATT_API_KEY",
|
||||
"opencode-zen": "OPENCODE_ZEN_API_KEY",
|
||||
"opencode-go": "OPENCODE_GO_API_KEY",
|
||||
# NOTE: bare "ollama" (local) deliberately omitted — local Ollama is keyless
|
||||
@@ -763,8 +743,6 @@ _PROVIDER_ENV_VAR_ALIASES: dict[str, tuple[str, ...]] = {
|
||||
"opencode-go": ("OPENCODE_API_KEY",),
|
||||
}
|
||||
|
||||
_SELF_HOSTED_PROVIDER_IDS = frozenset({"ollama", "lmstudio"})
|
||||
|
||||
|
||||
def _provider_credential_env_vars() -> tuple[str, ...]:
|
||||
names = {name for name in _PROVIDER_ENV_VAR.values() if name}
|
||||
@@ -837,19 +815,6 @@ def _entry_exhausted_ttl_seconds(error_code):
|
||||
code = str(error_code or "").strip()
|
||||
if code == "401":
|
||||
return 5 * 60
|
||||
if code == "402":
|
||||
# #6626: keep WebUI's eligibility decision tied to the installed
|
||||
# runtime contract. The runtime routes 402 via
|
||||
# credential_pool._exhausted_ttl() (120s when the new
|
||||
# EXHAUSTED_TTL_402_SECONDS is present, 1h fallback otherwise).
|
||||
# Hard-coding 120s here would let display/probe code mark an entry
|
||||
# usable before CredentialPool.select() is willing to lease it on
|
||||
# mixed-version installations.
|
||||
try:
|
||||
from agent.credential_pool import _exhausted_ttl as _runtime_exhausted_ttl
|
||||
return _runtime_exhausted_ttl(int(code))
|
||||
except Exception:
|
||||
return 60 * 60
|
||||
return 60 * 60
|
||||
|
||||
|
||||
@@ -998,74 +963,6 @@ def _get_hermes_home() -> Path:
|
||||
return Path.home() / ".hermes"
|
||||
|
||||
|
||||
def _providers_file_mtime_ns(path: Path) -> int:
|
||||
"""Best-effort file mtime for providers-cache invalidation."""
|
||||
try:
|
||||
return path.stat().st_mtime_ns
|
||||
except OSError:
|
||||
return 0
|
||||
|
||||
|
||||
def _providers_config_fingerprint(cfg: Any) -> str:
|
||||
"""Stable fingerprint for config fields that shape the Providers response."""
|
||||
try:
|
||||
return hashlib.sha256(
|
||||
json.dumps(cfg, sort_keys=True, default=str, separators=(",", ":")).encode("utf-8")
|
||||
).hexdigest()
|
||||
except Exception:
|
||||
return repr(cfg)
|
||||
|
||||
|
||||
def _providers_cache_key(cfg: Any) -> tuple[Any, ...]:
|
||||
"""Return a profile-scoped cache key for ``get_providers()`` (#6010).
|
||||
|
||||
The endpoint reads provider state from the active Hermes home plus the
|
||||
current config. Include the home path and the two files users commonly
|
||||
mutate from Settings so a short TTL never crosses profile boundaries or
|
||||
masks immediate credential/config changes.
|
||||
"""
|
||||
home = _get_hermes_home()
|
||||
try:
|
||||
home_key = str(home.resolve())
|
||||
except OSError:
|
||||
home_key = str(home)
|
||||
return (
|
||||
home_key,
|
||||
_providers_file_mtime_ns(home / ".env"),
|
||||
_providers_file_mtime_ns(home / "config.yaml"),
|
||||
_providers_config_fingerprint(cfg),
|
||||
)
|
||||
|
||||
|
||||
def _get_cached_providers(cache_key: tuple[Any, ...]) -> dict[str, Any] | None:
|
||||
now = time.monotonic()
|
||||
with _providers_cache_lock:
|
||||
cached = _providers_cache.get(cache_key)
|
||||
if cached is None:
|
||||
return None
|
||||
ts, payload = cached
|
||||
if now - ts >= _PROVIDERS_CACHE_TTL_SECONDS:
|
||||
_providers_cache.pop(cache_key, None)
|
||||
return None
|
||||
return copy.deepcopy(payload)
|
||||
|
||||
|
||||
def _store_cached_providers(cache_key: tuple[Any, ...], payload: dict[str, Any]) -> dict[str, Any]:
|
||||
with _providers_cache_lock:
|
||||
# Single-entry by design: /api/providers is cacheable only for the
|
||||
# active profile/config snapshot, so clear older snapshots to avoid
|
||||
# retaining unbounded provider metadata across profile switches.
|
||||
_providers_cache.clear()
|
||||
_providers_cache[cache_key] = (time.monotonic(), copy.deepcopy(payload))
|
||||
return payload
|
||||
|
||||
|
||||
def invalidate_providers_cache() -> None:
|
||||
"""Clear cached ``GET /api/providers`` responses."""
|
||||
with _providers_cache_lock:
|
||||
_providers_cache.clear()
|
||||
|
||||
|
||||
def _load_env_file(env_path: Path) -> dict[str, str]:
|
||||
"""Read key=value pairs from a .env file."""
|
||||
values: dict[str, str] = {}
|
||||
@@ -1404,110 +1301,6 @@ def _get_provider_api_key(provider_id: str) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def provider_has_usable_credential(provider_id: str, *, refresh: bool = False) -> bool:
|
||||
"""Return True when a provider has a currently usable configured credential."""
|
||||
provider = str(provider_id or "").strip().lower()
|
||||
if not provider:
|
||||
return False
|
||||
if refresh:
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
|
||||
invalidate_credential_pool_cache(provider)
|
||||
except Exception:
|
||||
logger.debug("Failed to refresh credential pool before provider availability check", exc_info=True)
|
||||
return _get_provider_api_key(provider) is not None
|
||||
|
||||
|
||||
def provider_has_usable_pool_credential(provider_id: str, *, refresh: bool = False) -> bool:
|
||||
"""Return True only when the provider's credential-pool lane has a usable entry."""
|
||||
provider = str(provider_id or "").strip().lower()
|
||||
if not provider:
|
||||
return False
|
||||
if refresh:
|
||||
try:
|
||||
from api.config import invalidate_credential_pool_cache
|
||||
|
||||
invalidate_credential_pool_cache(provider)
|
||||
except Exception:
|
||||
logger.debug("Failed to refresh credential pool before pool availability check", exc_info=True)
|
||||
for entry in _pool_entry_payloads(provider):
|
||||
status = str(entry.get("last_status") or "").strip().lower()
|
||||
if status == "dead":
|
||||
continue
|
||||
if status == "exhausted":
|
||||
ns = SimpleNamespace(**entry)
|
||||
if _entry_is_pool_exhausted(ns):
|
||||
continue
|
||||
key = str(
|
||||
entry.get("runtime_api_key")
|
||||
or entry.get("agent_key")
|
||||
or entry.get("access_token")
|
||||
or ""
|
||||
).strip()
|
||||
if key:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _credential_secret_fingerprint(secret: str) -> str:
|
||||
value = str(secret or "").strip()
|
||||
if not value:
|
||||
return ""
|
||||
return hashlib.sha256(value.encode("utf-8", "ignore")).hexdigest()[:16]
|
||||
|
||||
|
||||
def _entry_secret_fingerprint(entry: dict) -> str:
|
||||
value = str(entry.get("secret_fingerprint") or "").strip().lower()
|
||||
if value.startswith("sha256:"):
|
||||
value = value[len("sha256:"):]
|
||||
if not value:
|
||||
return ""
|
||||
if all(ch in "0123456789abcdef" for ch in value):
|
||||
return value[:16]
|
||||
return ""
|
||||
|
||||
|
||||
def _pool_entry_currently_unusable(entry: dict) -> bool:
|
||||
status = str(entry.get("last_status") or "").strip().lower()
|
||||
if status == "dead":
|
||||
return True
|
||||
if status == "exhausted":
|
||||
ns = SimpleNamespace(**entry)
|
||||
return _entry_is_pool_exhausted(ns)
|
||||
return False
|
||||
|
||||
|
||||
def provider_has_process_wakeup_recovery_credential(provider_id: str, *, refresh: bool = False) -> bool:
|
||||
"""Return True when a paused credential-pool wakeup lane can safely retry."""
|
||||
provider = str(provider_id or "").strip().lower()
|
||||
if not provider:
|
||||
return False
|
||||
if provider_has_usable_pool_credential(provider, refresh=refresh):
|
||||
return True
|
||||
configured_key = _get_provider_api_key(provider)
|
||||
if not configured_key:
|
||||
return False
|
||||
configured_fingerprint = _credential_secret_fingerprint(configured_key)
|
||||
if not configured_fingerprint:
|
||||
return False
|
||||
has_unusable_pool_entry = False
|
||||
has_unknown_unusable_pool_entry = False
|
||||
for entry in _pool_entry_payloads(provider):
|
||||
entry_fingerprint = _entry_secret_fingerprint(entry)
|
||||
if not _pool_entry_currently_unusable(entry):
|
||||
if entry_fingerprint and entry_fingerprint == configured_fingerprint:
|
||||
return True
|
||||
continue
|
||||
has_unusable_pool_entry = True
|
||||
if not entry_fingerprint:
|
||||
has_unknown_unusable_pool_entry = True
|
||||
continue
|
||||
if entry_fingerprint == configured_fingerprint:
|
||||
return False
|
||||
return has_unusable_pool_entry and not has_unknown_unusable_pool_entry
|
||||
|
||||
|
||||
def _active_provider_id() -> str | None:
|
||||
cfg = get_config()
|
||||
model_cfg = cfg.get("model", {})
|
||||
@@ -2257,16 +2050,6 @@ _COST_SNAPSHOT_MAX_DAYS = 365 # hard cap to prevent unbounded growth
|
||||
_COST_SNAPSHOT_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def _get_provider_cost_budget() -> float | None:
|
||||
"""Return the user-configured monthly spend budget, or None if unset."""
|
||||
try:
|
||||
from api.config import load_settings
|
||||
raw = load_settings().get("provider_cost_budget")
|
||||
return _coerce_provider_cost_budget(raw)
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
|
||||
def _cost_snapshots_dir() -> Path:
|
||||
"""Return the directory for cost-snapshot JSON files.
|
||||
|
||||
@@ -2486,7 +2269,6 @@ def get_provider_cost_history(provider_id: str | None = None, days: int = 7) ->
|
||||
}
|
||||
|
||||
display_name = _PROVIDER_DISPLAY.get("openrouter", "OpenRouter")
|
||||
monthly_budget = _get_provider_cost_budget()
|
||||
api_key = _get_provider_api_key("openrouter")
|
||||
if not api_key:
|
||||
return {
|
||||
@@ -2495,7 +2277,6 @@ def get_provider_cost_history(provider_id: str | None = None, days: int = 7) ->
|
||||
"display_name": display_name,
|
||||
"supported": True,
|
||||
"status": "no_key",
|
||||
"monthly_budget": monthly_budget,
|
||||
"message": "OpenRouter cost history needs an OPENROUTER_API_KEY configured on the server.",
|
||||
}
|
||||
|
||||
@@ -2516,7 +2297,6 @@ def get_provider_cost_history(provider_id: str | None = None, days: int = 7) ->
|
||||
"snapshots": deltas,
|
||||
"limit": None,
|
||||
"label": None,
|
||||
"monthly_budget": monthly_budget,
|
||||
"message": "OpenRouter cost history is temporarily unavailable. Showing last known data.",
|
||||
}
|
||||
|
||||
@@ -2538,7 +2318,6 @@ def get_provider_cost_history(provider_id: str | None = None, days: int = 7) ->
|
||||
"snapshots": deltas,
|
||||
"limit": key_info.get("limit"),
|
||||
"label": key_info.get("label") or "OpenRouter credits",
|
||||
"monthly_budget": monthly_budget,
|
||||
"message": "OpenRouter cost history loaded.",
|
||||
}
|
||||
|
||||
@@ -2558,18 +2337,14 @@ def get_providers() -> dict[str, Any]:
|
||||
``config_yaml``, ``oauth``, ``none``)
|
||||
- ``models``: list of known model IDs for this provider
|
||||
"""
|
||||
providers = []
|
||||
|
||||
# Collect all known provider IDs from multiple sources
|
||||
known_ids = set(_PROVIDER_DISPLAY.keys()) | set(_PROVIDER_MODELS.keys())
|
||||
known_ids.update(plugin_model_provider_ids())
|
||||
|
||||
# Also detect providers from config.yaml providers section
|
||||
cfg = get_config()
|
||||
cache_key = _providers_cache_key(cfg)
|
||||
cached = _get_cached_providers(cache_key)
|
||||
if cached is not None:
|
||||
return cached
|
||||
|
||||
providers = []
|
||||
providers_cfg = cfg.get("providers") or {}
|
||||
if isinstance(providers_cfg, dict):
|
||||
known_ids.update(providers_cfg.keys())
|
||||
@@ -2793,20 +2568,12 @@ def get_providers() -> dict[str, Any]:
|
||||
if pid != "nous":
|
||||
models_total = len(models)
|
||||
|
||||
is_self_hosted = pid in _SELF_HOSTED_PROVIDER_IDS
|
||||
try:
|
||||
from api.config import _get_provider_base_url
|
||||
provider_base_url = _get_provider_base_url(pid) if is_self_hosted else None
|
||||
except Exception:
|
||||
provider_base_url = None
|
||||
_is_plugin = is_plugin_model_provider(pid)
|
||||
providers.append({
|
||||
"id": pid,
|
||||
"display_name": display_name,
|
||||
"has_key": has_key,
|
||||
"configurable": not is_oauth and bool(_provider_env_var_for(pid)),
|
||||
"is_self_hosted": is_self_hosted,
|
||||
"base_url": provider_base_url,
|
||||
"is_plugin_provider": _is_plugin,
|
||||
"is_oauth": is_oauth,
|
||||
"key_source": key_source,
|
||||
@@ -2836,20 +2603,12 @@ def get_providers() -> dict[str, Any]:
|
||||
cp_name,
|
||||
)
|
||||
continue
|
||||
# Build the model list using the same sticky-before-plural
|
||||
# ordering as the model picker (api/config.py:7308-7314):
|
||||
# the singular ``model`` field goes first, then unique IDs from
|
||||
# the ``models`` catalog are appended via _configured_model_ids
|
||||
# (which strips whitespace, drops empty IDs, and de-duplicates).
|
||||
# This keeps the Providers card consistent with the picker.
|
||||
cp_model_ids: list[str] = []
|
||||
_singular_model = str(cp.get("model") or "").strip()
|
||||
if _singular_model:
|
||||
cp_model_ids.append(_singular_model)
|
||||
for _mid in _configured_model_ids(cp.get("models")):
|
||||
if _mid not in cp_model_ids:
|
||||
cp_model_ids.append(_mid)
|
||||
cp_models = [{"id": mid, "label": mid} for mid in cp_model_ids]
|
||||
# Collect models from `models` list or `model` single
|
||||
cp_models = []
|
||||
if isinstance(cp.get("models"), list):
|
||||
cp_models = [{"id": str(m), "label": str(m)} for m in cp["models"]]
|
||||
elif cp.get("model"):
|
||||
cp_models = [{"id": cp["model"], "label": cp["model"]}]
|
||||
# Check for env var reference (${VAR_NAME} pattern)
|
||||
cp_api_key = str(cp.get("api_key") or "")
|
||||
cp_has_key = bool(cp_api_key.strip())
|
||||
@@ -2894,11 +2653,10 @@ def get_providers() -> dict[str, Any]:
|
||||
return (3, pid)
|
||||
providers.sort(key=_provider_sort_key)
|
||||
|
||||
result = {
|
||||
return {
|
||||
"providers": providers,
|
||||
"active_provider": active_provider,
|
||||
}
|
||||
return _store_cached_providers(cache_key, result)
|
||||
|
||||
|
||||
def set_provider_key(provider_id: str, api_key: str | None) -> dict[str, Any]:
|
||||
@@ -2951,7 +2709,6 @@ def set_provider_key(provider_id: str, api_key: str | None) -> dict[str, Any]:
|
||||
# disrupting active streaming sessions that may be reading config.cfg.
|
||||
invalidate_models_cache()
|
||||
invalidate_account_usage_status_cache(provider_id)
|
||||
invalidate_providers_cache()
|
||||
|
||||
return {
|
||||
"ok": True,
|
||||
@@ -3054,6 +2811,5 @@ def _clean_provider_key_from_config(provider_id: str) -> None:
|
||||
# reload_config() also acquires _cfg_lock internally.
|
||||
if changed:
|
||||
reload_config()
|
||||
invalidate_providers_cache()
|
||||
except Exception:
|
||||
logger.exception("Failed to clean provider key from config.yaml for %s", provider_id)
|
||||
|
||||
@@ -106,22 +106,11 @@ class RequestDiagnostics:
|
||||
logger: logging.Logger | None = None,
|
||||
timeout_seconds: float | None = None,
|
||||
auto_start: bool = True,
|
||||
print_fn: "callable | None" = None,
|
||||
) -> None:
|
||||
self.request_id = uuid.uuid4().hex[:10]
|
||||
self.method = str(method or "-")
|
||||
self.path = str(path or "-").split("?", 1)[0]
|
||||
self.logger = logger or logging.getLogger(__name__)
|
||||
# perf(webui/session-load-latency) tier2c: when the WebUI process
|
||||
# boots without configuring a root-logger handler, logger.warning
|
||||
# calls are silently dropped (the [SLOW] / "Slow WebUI request"
|
||||
# lines have been invisible in production for some time — only the
|
||||
# per-request ms line, which goes through _safe_webui_print, kept
|
||||
# working). Callers that need the structured log to actually land
|
||||
# in the systemd journal can pass print_fn=handler._safe_webui_print.
|
||||
# When provided, finish() and _on_timeout() route through it
|
||||
# instead of self.logger.warning.
|
||||
self._print_fn = print_fn
|
||||
self.timeout_seconds = _slow_request_seconds() if timeout_seconds is None else max(0.0, float(timeout_seconds))
|
||||
self.started_monotonic = time.monotonic()
|
||||
self.started_wall = time.time()
|
||||
@@ -145,28 +134,14 @@ class RequestDiagnostics:
|
||||
path: str,
|
||||
*,
|
||||
logger: logging.Logger | None = None,
|
||||
print_fn: "callable | None" = None,
|
||||
) -> "RequestDiagnostics | None":
|
||||
clean_path = str(path or "").split("?", 1)[0]
|
||||
if (method.upper(), clean_path) not in {
|
||||
("GET", "/api/sessions"),
|
||||
("POST", "/api/chat/start"),
|
||||
# perf(session-load-latency) Phase 0 instrumentation gaps.
|
||||
# These endpoints fire on every session click and were the source
|
||||
# of the multi-second waterfalls in the slow-request logs.
|
||||
("GET", "/api/profiles"),
|
||||
("GET", "/api/models"),
|
||||
# perf(webui/session-load-latency) tier2c: /api/session is the
|
||||
# chat-open hot path. The handler also calls maybe_start() and
|
||||
# finish() at its _t0/_t6 boundaries, so adding this entry to
|
||||
# the allowlist is NOT a dead-code addition (the Greptile P1
|
||||
# trap for previous similar entries): without the handler call
|
||||
# the watchdog entry would never register and the structured
|
||||
# log would never fire.
|
||||
("GET", "/api/session"),
|
||||
}:
|
||||
return None
|
||||
return cls(method, clean_path, logger=logger, print_fn=print_fn)
|
||||
return cls(method, clean_path, logger=logger)
|
||||
|
||||
def stage(self, name: str) -> None:
|
||||
now = time.monotonic()
|
||||
@@ -183,18 +158,6 @@ class RequestDiagnostics:
|
||||
self._current_stage = clean
|
||||
self._current_stage_started = now
|
||||
|
||||
def _emit_slow(self, prefix: str, record: dict) -> None:
|
||||
payload = json.dumps(record, sort_keys=True)
|
||||
log_msg = f"{prefix} %s"
|
||||
if self._print_fn is not None:
|
||||
try:
|
||||
self._print_fn(log_msg % payload)
|
||||
except Exception:
|
||||
# print_fn is best-effort; never break a request thread.
|
||||
pass
|
||||
else:
|
||||
self.logger.warning(log_msg, payload)
|
||||
|
||||
def finish(self) -> None:
|
||||
record = None
|
||||
with self._lock:
|
||||
@@ -207,7 +170,10 @@ class RequestDiagnostics:
|
||||
# bounded by the number of in-flight requests).
|
||||
_watchdog_unregister(self.request_id)
|
||||
if record and self.timeout_seconds > 0 and record["elapsed_ms"] >= self.timeout_seconds * 1000:
|
||||
self._emit_slow("Slow WebUI request completed:", record)
|
||||
self.logger.warning(
|
||||
"Slow WebUI request completed: %s",
|
||||
json.dumps(record, sort_keys=True),
|
||||
)
|
||||
|
||||
def _on_timeout(self) -> None:
|
||||
with self._lock:
|
||||
@@ -215,7 +181,10 @@ class RequestDiagnostics:
|
||||
return
|
||||
self._watchdog_logged = True
|
||||
record = self._build_record_locked(include_stacks=True)
|
||||
self._emit_slow("Slow WebUI request still running:", record)
|
||||
self.logger.warning(
|
||||
"Slow WebUI request still running: %s",
|
||||
json.dumps(record, sort_keys=True),
|
||||
)
|
||||
|
||||
def _build_record_locked(self, *, include_stacks: bool) -> dict[str, Any]:
|
||||
now = time.monotonic()
|
||||
|
||||
@@ -17,8 +17,6 @@ from datetime import datetime
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from api.subprocess_utils import windows_hide_flags
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Checkpoint identifiers are SHA-style hex hashes from the agent's
|
||||
@@ -104,7 +102,6 @@ def _checkpoint_entry_modes(git: str, ckpt_dir: Path) -> dict[str, int]:
|
||||
result = subprocess.run(
|
||||
[git, "-C", str(ckpt_dir), "ls-files", "-s"],
|
||||
capture_output=True, text=True, timeout=10,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
raise ValueError("Failed to list checkpoint files")
|
||||
@@ -139,7 +136,6 @@ def _read_checkpoint_blob(git: str, ckpt_dir: Path, rel_path: str) -> bytes | No
|
||||
result = subprocess.run(
|
||||
[git, "-C", str(ckpt_dir), "show", f"HEAD:{rel_path}"],
|
||||
capture_output=True, timeout=10,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if result.returncode != 0:
|
||||
return None
|
||||
@@ -254,7 +250,6 @@ def _inspect_checkpoint(ckpt_path: Path, git: str) -> dict[str, Any] | None:
|
||||
result = subprocess.run(
|
||||
[git, "-C", str(ckpt_path), "log", "--format=%H%n%s%n%aI", "-1"],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if result.returncode != 0 or not result.stdout.strip():
|
||||
return None
|
||||
@@ -277,7 +272,6 @@ def _inspect_checkpoint(ckpt_path: Path, git: str) -> dict[str, Any] | None:
|
||||
files_result = subprocess.run(
|
||||
[git, "-C", str(ckpt_path), "ls-files"],
|
||||
capture_output=True, text=True, timeout=5,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
file_count = len(files_result.stdout.strip().split("\n")) if files_result.stdout.strip() else 0
|
||||
|
||||
|
||||
+25
-747
@@ -6,7 +6,6 @@ Extracts approval state, not handlers, by design.
|
||||
import queue
|
||||
import threading
|
||||
import uuid
|
||||
from contextlib import contextmanager
|
||||
|
||||
from api.session_events import publish_session_list_changed
|
||||
|
||||
@@ -47,99 +46,7 @@ except ImportError:
|
||||
_approval_sse_subscribers: dict[str, list[queue.Queue]] = {}
|
||||
_GATEWAY_MIRROR_FLAG = "_gateway_mirror"
|
||||
_GATEWAY_MIRROR_TOKEN = "_gateway_mirror_token"
|
||||
_GATEWAY_MIRROR_RETAINED = "_gateway_mirror_retained"
|
||||
_GATEWAY_ENTRY_DATA_TOKEN_KEY = "_webui_mirror_token"
|
||||
_GATEWAY_AGENT_IDENTITY_V1 = "_gateway_agent_identity_v1"
|
||||
_gateway_relay_owners: dict[tuple[str, str], str] = {}
|
||||
_yolo_transition_lock = threading.Lock()
|
||||
_yolo_transitions: dict[str, dict] = {}
|
||||
_gateway_yolo_handoff_guard = threading.Lock()
|
||||
_gateway_yolo_handoffs: dict[str, dict] = {}
|
||||
|
||||
|
||||
@contextmanager
|
||||
def gateway_yolo_handoff(session_key: str):
|
||||
"""Serialize one session's YOLO toggles with gateway approval dispatch."""
|
||||
session_key = str(session_key or "").strip()
|
||||
with _gateway_yolo_handoff_guard:
|
||||
entry = _gateway_yolo_handoffs.get(session_key)
|
||||
if entry is None:
|
||||
entry = {"lock": threading.Lock(), "users": 0}
|
||||
_gateway_yolo_handoffs[session_key] = entry
|
||||
entry["users"] += 1
|
||||
lock = entry["lock"]
|
||||
lock.acquire()
|
||||
try:
|
||||
yield
|
||||
finally:
|
||||
lock.release()
|
||||
with _gateway_yolo_handoff_guard:
|
||||
entry["users"] -= 1
|
||||
if entry["users"] == 0:
|
||||
_gateway_yolo_handoffs.pop(session_key, None)
|
||||
|
||||
|
||||
def begin_session_yolo_transition(session_key: str) -> object | None:
|
||||
"""Register a pending YOLO enable until one approval relay settles.
|
||||
|
||||
Multiple tabs may relay approvals for different runs in the same session.
|
||||
Track every in-flight enable intent so one failed relay cannot undo another
|
||||
successful or explicit enable. Do not publish an unconfirmed enable to the
|
||||
shared session flag: the gateway stream may only auto-approve later prompts
|
||||
after a relay succeeds or an explicit enable wins.
|
||||
"""
|
||||
session_key = str(session_key or "").strip()
|
||||
if not session_key:
|
||||
return None
|
||||
token = object()
|
||||
with _yolo_transition_lock:
|
||||
transition = _yolo_transitions.get(session_key)
|
||||
if transition is None:
|
||||
transition = {
|
||||
"was_enabled": bool(is_session_yolo_enabled(session_key)),
|
||||
"tokens": set(),
|
||||
"committed": False,
|
||||
}
|
||||
_yolo_transitions[session_key] = transition
|
||||
transition["tokens"].add(token)
|
||||
return token
|
||||
|
||||
|
||||
def finish_session_yolo_transition(session_key: str, token: object | None, *, succeeded: bool) -> None:
|
||||
"""Settle one pending YOLO enable without exposing or applying stale state."""
|
||||
session_key = str(session_key or "").strip()
|
||||
if not session_key or token is None:
|
||||
return
|
||||
with _yolo_transition_lock:
|
||||
transition = _yolo_transitions.get(session_key)
|
||||
if transition is None or token not in transition["tokens"]:
|
||||
return
|
||||
transition["tokens"].remove(token)
|
||||
if succeeded:
|
||||
transition["committed"] = True
|
||||
# The first confirmed relay commits YOLO immediately. Any remaining
|
||||
# tokens may fail later but cannot revoke this successful enable.
|
||||
enable_session_yolo(session_key)
|
||||
if transition["tokens"]:
|
||||
return
|
||||
_yolo_transitions.pop(session_key, None)
|
||||
if transition["committed"] or transition["was_enabled"]:
|
||||
enable_session_yolo(session_key)
|
||||
else:
|
||||
disable_session_yolo(session_key)
|
||||
|
||||
|
||||
def set_session_yolo_enabled(session_key: str, enabled: bool) -> None:
|
||||
"""Apply an explicit YOLO choice and supersede in-flight rollbacks."""
|
||||
session_key = str(session_key or "").strip()
|
||||
if not session_key:
|
||||
return
|
||||
with _yolo_transition_lock:
|
||||
_yolo_transitions.pop(session_key, None)
|
||||
if enabled:
|
||||
enable_session_yolo(session_key)
|
||||
else:
|
||||
disable_session_yolo(session_key)
|
||||
|
||||
|
||||
def _approval_sse_subscribe(session_id: str) -> queue.Queue:
|
||||
@@ -199,7 +106,7 @@ def _approval_sse_notify(session_id: str, head: dict | None, total: int) -> None
|
||||
_approval_sse_notify_locked(session_id, head, total)
|
||||
|
||||
|
||||
def _gateway_mirror_entry_token(entry) -> str | None:
|
||||
def _gateway_mirror_entry_token(entry) -> str:
|
||||
"""Return a stable token for the current process lifetime of a gateway head.
|
||||
|
||||
Stamps a token key into the entry's `.data` dict so
|
||||
@@ -213,7 +120,7 @@ def _gateway_mirror_entry_token(entry) -> str | None:
|
||||
token = uuid.uuid4().hex
|
||||
data[_GATEWAY_ENTRY_DATA_TOKEN_KEY] = token
|
||||
return token
|
||||
return None
|
||||
return uuid.uuid4().hex
|
||||
|
||||
|
||||
def _is_gateway_mirror_entry(entry: dict | None) -> bool:
|
||||
@@ -240,150 +147,29 @@ def reconcile_gateway_pending_mirror_locked(session_key: str) -> tuple[dict | No
|
||||
|
||||
live_head_entry = live_gateway_queue[0] if live_gateway_queue else None
|
||||
live_head_data = getattr(live_head_entry, "data", None) or {}
|
||||
live_run_id = str(live_head_data.get("run_id") or "").strip()
|
||||
# Tokenize EVERY live no-run producer, and derive `live_token` from the
|
||||
# authoritative head, whenever `_gateway_queues[session_key]` has live
|
||||
# producers. This deliberately does NOT defer to a pre-existing no-run
|
||||
# mirror: an unmatched/tokenless mirror A (which the fail-closed binding in
|
||||
# submit_gateway_pending_mirror leaves deliberately unbound) must never
|
||||
# suppress the live producer's token, or A masks the real pending approval
|
||||
# B — B never surfaces as the head and can't be actioned, while responding
|
||||
# to A resolves nothing. While any producer is live, a mirror survives only
|
||||
# if it is bound to some live producer's own token — the head's via
|
||||
# `live_token`, a non-head producer's via `live_local_tokens` (which is what
|
||||
# keeps a non-head mirror resolvable) — so unmatched and tokenless copies
|
||||
# are discarded instead of masking a real one. Tokenless-orphan retention is
|
||||
# reserved for the genuine no-producer case (#7093), which lands here with an
|
||||
# empty `live_gateway_queue` and therefore a `None` `live_token` anyway.
|
||||
live_local_tokens: set[str] = set()
|
||||
for live_entry in live_gateway_queue:
|
||||
live_data = getattr(live_entry, "data", None) or {}
|
||||
if str(live_data.get("run_id") or "").strip():
|
||||
continue
|
||||
live_entry_token = _gateway_mirror_entry_token(live_entry) or ""
|
||||
if live_entry_token:
|
||||
live_local_tokens.add(live_entry_token)
|
||||
if not str(live_data.get("approval_id") or "").strip():
|
||||
live_data["approval_id"] = f"gwlocal:{live_entry_token}"
|
||||
|
||||
# A raw id-less LOCAL entry in `_pending` (agent-side _pending_result drops
|
||||
# `{command, pattern_key, pattern_keys, description}` verbatim when no
|
||||
# gateway notifier is registered) is unactionable by contract: the frontend
|
||||
# owner-capture requires an approval_id, so its card can neither be
|
||||
# approved nor dismissed — the poll re-serves it forever. Mint a stable id
|
||||
# on the entry itself (the same contract the producer-tokenization loop
|
||||
# above applies to _gateway_queues producers). Minting on the stored dict
|
||||
# keeps the id stable across polls, so frontend dismiss markers persist.
|
||||
for entry in queue_list:
|
||||
if not isinstance(entry, dict) or _is_gateway_mirror_entry(entry):
|
||||
continue
|
||||
if str(entry.get("approval_id") or "").strip():
|
||||
continue
|
||||
if str(entry.get("run_id") or "").strip():
|
||||
continue
|
||||
entry["approval_id"] = f"gwlocal-mirrorless:{uuid.uuid4().hex}"
|
||||
live_token = (
|
||||
_gateway_mirror_entry_token(live_head_entry)
|
||||
if live_head_entry and live_head_data
|
||||
else None
|
||||
)
|
||||
if live_token and live_run_id and not str(live_head_data.get("approval_id") or "").strip():
|
||||
live_head_data["approval_id"] = f"gwrun:{live_run_id}:{live_token}"
|
||||
live_approval_id = str(live_head_data.get("approval_id") or "").strip()
|
||||
live_token = _gateway_mirror_entry_token(live_head_entry) if live_head_entry and live_head_data else None
|
||||
|
||||
rebuilt: list[dict] = []
|
||||
deferred_run_entries: list[dict] = []
|
||||
live_mirror_present = False
|
||||
for entry in queue_list:
|
||||
if not _is_gateway_mirror_entry(entry):
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
entry_run_id = str(entry.get("run_id") or "").strip()
|
||||
entry_approval_id = str(entry.get("approval_id") or "").strip()
|
||||
entry_token = str(entry.get(_GATEWAY_MIRROR_TOKEN) or "").strip()
|
||||
matches_live_head = False
|
||||
if live_token:
|
||||
if entry_token and entry_token == live_token:
|
||||
matches_live_head = True
|
||||
elif (
|
||||
live_approval_id
|
||||
and live_run_id
|
||||
and entry_approval_id == live_approval_id
|
||||
and entry_run_id == live_run_id
|
||||
):
|
||||
matches_live_head = True
|
||||
|
||||
if entry_run_id:
|
||||
if matches_live_head and not live_mirror_present:
|
||||
if entry_token != live_token:
|
||||
entry[_GATEWAY_MIRROR_TOKEN] = live_token
|
||||
changed = True
|
||||
rebuilt.append(entry)
|
||||
live_mirror_present = True
|
||||
continue
|
||||
if live_token:
|
||||
if entry.get(_GATEWAY_MIRROR_RETAINED):
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
if entry_token:
|
||||
changed = True
|
||||
continue
|
||||
deferred_run_entries.append(entry)
|
||||
continue
|
||||
if entry.get(_GATEWAY_MIRROR_RETAINED) or not entry_token:
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
changed = True
|
||||
continue
|
||||
|
||||
# A retained mirror is one whose own producer had already vanished when
|
||||
# the user responded (the missing-producer 409 kept visible until an
|
||||
# explicit teardown). It survives ONLY while no producer is live: once
|
||||
# `_gateway_queues[session_key]` holds a real producer again, an
|
||||
# unresolvable retained mirror must not mask it, so fall through to the
|
||||
# normal matching below (which keeps it if it still matches a live
|
||||
# token and discards it otherwise).
|
||||
if entry.get(_GATEWAY_MIRROR_RETAINED) and not live_gateway_queue:
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
|
||||
if matches_live_head and not live_mirror_present:
|
||||
if entry_token != live_token:
|
||||
entry[_GATEWAY_MIRROR_TOKEN] = live_token
|
||||
changed = True
|
||||
if live_token and entry.get(_GATEWAY_MIRROR_TOKEN) == live_token and not live_mirror_present:
|
||||
rebuilt.append(entry)
|
||||
live_mirror_present = True
|
||||
continue
|
||||
|
||||
if entry_token and entry_token in live_local_tokens:
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
|
||||
if not live_token:
|
||||
if entry_token:
|
||||
changed = True
|
||||
continue
|
||||
rebuilt.append(entry)
|
||||
continue
|
||||
|
||||
changed = True
|
||||
|
||||
if live_token and not live_mirror_present:
|
||||
mirror_entry = dict(live_head_data)
|
||||
mirror_run_id = str(mirror_entry.get("run_id") or "").strip()
|
||||
mirror_entry.setdefault(
|
||||
"approval_id",
|
||||
f"gwrun:{mirror_run_id}:{live_token}" if mirror_run_id else uuid.uuid4().hex,
|
||||
)
|
||||
mirror_entry.setdefault("approval_id", uuid.uuid4().hex)
|
||||
mirror_entry[_GATEWAY_MIRROR_FLAG] = True
|
||||
mirror_entry[_GATEWAY_MIRROR_TOKEN] = live_token
|
||||
rebuilt.append(mirror_entry)
|
||||
live_mirror_present = True
|
||||
changed = True
|
||||
|
||||
if deferred_run_entries:
|
||||
rebuilt.extend(deferred_run_entries)
|
||||
|
||||
if rebuilt:
|
||||
if rebuilt != queue_list:
|
||||
_pending[session_key] = rebuilt
|
||||
@@ -398,546 +184,38 @@ def reconcile_gateway_pending_mirror_locked(session_key: str) -> tuple[dict | No
|
||||
return head, total, changed
|
||||
|
||||
|
||||
def _gateway_pending_mirror_locked(
|
||||
session_key: str,
|
||||
approval_id: str = "",
|
||||
run_id: str = "",
|
||||
mirror_token: str = "",
|
||||
) -> dict | None:
|
||||
"""Return the exact live run-backed mirror under `_lock`."""
|
||||
approval_id = str(approval_id or "").strip()
|
||||
run_id = str(run_id or "").strip()
|
||||
mirror_token = str(mirror_token or "").strip()
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
if approval_id:
|
||||
matched_entry: dict | None = None
|
||||
for entry in entries:
|
||||
if not _is_gateway_mirror_entry(entry):
|
||||
continue
|
||||
if entry.get("approval_id") != approval_id:
|
||||
continue
|
||||
entry_run_id = str(entry.get("run_id") or "").strip()
|
||||
if not entry_run_id:
|
||||
if not run_id:
|
||||
return None
|
||||
continue
|
||||
if run_id and entry_run_id != run_id:
|
||||
continue
|
||||
if mirror_token and str(entry.get(_GATEWAY_MIRROR_TOKEN) or "").strip() != mirror_token:
|
||||
continue
|
||||
if run_id:
|
||||
return entry
|
||||
if matched_entry is not None:
|
||||
return None
|
||||
matched_entry = entry
|
||||
return matched_entry
|
||||
for entry in entries:
|
||||
if not _is_gateway_mirror_entry(entry) or not str(entry.get("run_id") or "").strip():
|
||||
continue
|
||||
if mirror_token and str(entry.get(_GATEWAY_MIRROR_TOKEN) or "").strip() != mirror_token:
|
||||
continue
|
||||
if run_id:
|
||||
if entry.get("run_id") == run_id:
|
||||
return entry
|
||||
continue
|
||||
# With no caller-supplied identity, the queue order is authoritative:
|
||||
# return the current run-backed projection and let its embedded
|
||||
# `(approval_id, run_id)` identify the exact relay owner.
|
||||
return entry
|
||||
return None
|
||||
|
||||
|
||||
def gateway_pending_mirror(
|
||||
session_key: str,
|
||||
approval_id: str = "",
|
||||
run_id: str = "",
|
||||
mirror_token: str = "",
|
||||
) -> dict | None:
|
||||
"""Return an exact live run-backed mirror for this session."""
|
||||
with _lock:
|
||||
reconcile_gateway_pending_mirror_locked(session_key)
|
||||
entry = _gateway_pending_mirror_locked(session_key, approval_id, run_id, mirror_token)
|
||||
return dict(entry) if entry else None
|
||||
|
||||
|
||||
def gateway_pending_mirrors(session_key: str) -> list[dict]:
|
||||
"""Return every currently parked run-backed mirror in queue order."""
|
||||
with _lock:
|
||||
reconcile_gateway_pending_mirror_locked(session_key)
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
return [
|
||||
dict(entry)
|
||||
for entry in entries
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and str(entry.get("run_id") or "").strip()
|
||||
]
|
||||
|
||||
|
||||
def claim_gateway_approval_relay_owner(session_key: str, run_id: str, approval_id: str) -> bool:
|
||||
"""Claim the single-flight relay owner for one `(session, run)` pair."""
|
||||
session_key = str(session_key or "").strip()
|
||||
run_id = str(run_id or "").strip()
|
||||
approval_id = str(approval_id or "").strip()
|
||||
if not session_key or not run_id:
|
||||
return False
|
||||
with _lock:
|
||||
key = (session_key, run_id)
|
||||
if key in _gateway_relay_owners:
|
||||
return False
|
||||
_gateway_relay_owners[key] = approval_id
|
||||
return True
|
||||
|
||||
|
||||
def release_gateway_approval_relay_owner(session_key: str, run_id: str, approval_id: str = "") -> None:
|
||||
"""Release the single-flight relay owner for one `(session, run)` pair."""
|
||||
session_key = str(session_key or "").strip()
|
||||
run_id = str(run_id or "").strip()
|
||||
approval_id = str(approval_id or "").strip()
|
||||
if not session_key or not run_id:
|
||||
return
|
||||
with _lock:
|
||||
key = (session_key, run_id)
|
||||
current = str(_gateway_relay_owners.get(key) or "").strip()
|
||||
if approval_id and current and current != approval_id:
|
||||
return
|
||||
_gateway_relay_owners.pop(key, None)
|
||||
|
||||
|
||||
def retire_gateway_pending_mirror(
|
||||
session_key: str,
|
||||
approval_id: str = "",
|
||||
run_id: str = "",
|
||||
mirror_token: str = "",
|
||||
) -> bool:
|
||||
"""Retire one approval, or every mirror for a terminal run."""
|
||||
with _lock:
|
||||
reconcile_gateway_pending_mirror_locked(session_key)
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
normalized_run_id = str(run_id or "").strip()
|
||||
gateway_queue = _gateway_queues.get(session_key) or []
|
||||
retained_gateway_queue = gateway_queue
|
||||
gateway_queue_changed = False
|
||||
if approval_id:
|
||||
match = _gateway_pending_mirror_locked(
|
||||
session_key,
|
||||
approval_id,
|
||||
run_id,
|
||||
mirror_token,
|
||||
)
|
||||
if match is None and not normalized_run_id:
|
||||
match = next((entry for entry in entries if _is_gateway_mirror_entry(entry)
|
||||
and not str(entry.get("run_id") or "").strip()
|
||||
and str(entry.get("approval_id") or "").strip() == approval_id), None)
|
||||
retired = [match] if match else []
|
||||
else:
|
||||
retired = [
|
||||
entry for entry in entries
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and str(entry.get("run_id") or "").strip() == normalized_run_id
|
||||
] if normalized_run_id else [
|
||||
entry for entry in entries
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and not str(entry.get("run_id") or "").strip()
|
||||
]
|
||||
if normalized_run_id:
|
||||
retained_gateway_queue = []
|
||||
for entry in gateway_queue:
|
||||
data = getattr(entry, "data", None) or {}
|
||||
if str(data.get("run_id") or "").strip() == normalized_run_id:
|
||||
gateway_queue_changed = True
|
||||
continue
|
||||
retained_gateway_queue.append(entry)
|
||||
if not retired and not gateway_queue_changed:
|
||||
head, total, changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
if changed:
|
||||
publish_session_list_changed("attention_resolved")
|
||||
return changed
|
||||
for match in retired:
|
||||
entries.remove(match)
|
||||
if normalized_run_id and not approval_id:
|
||||
if retained_gateway_queue:
|
||||
_gateway_queues[session_key] = retained_gateway_queue
|
||||
else:
|
||||
_gateway_queues.pop(session_key, None)
|
||||
if entries:
|
||||
_pending[session_key] = entries
|
||||
else:
|
||||
_pending.pop(session_key, None)
|
||||
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
publish_session_list_changed("attention_resolved")
|
||||
return True
|
||||
|
||||
|
||||
def _gateway_mirrored_pending_run_id(session_key: str, approval_id: str) -> str | None:
|
||||
"""Compatibility wrapper for exact run-backed lookup."""
|
||||
"""Return the mirrored gateway approval run_id for a matching pending card.
|
||||
|
||||
Reconciles the mirror first so a live gateway head still survives a lost
|
||||
`active_stream_id` pointer.
|
||||
"""
|
||||
approval_id = str(approval_id or "").strip()
|
||||
if not approval_id:
|
||||
return None
|
||||
with _lock:
|
||||
entry = _gateway_pending_mirror_locked(session_key, approval_id=approval_id)
|
||||
if entry:
|
||||
return str(entry.get("run_id") or "").strip() or None
|
||||
reconcile_gateway_pending_mirror_locked(session_key)
|
||||
queue = _pending.get(session_key)
|
||||
if isinstance(queue, list):
|
||||
entries = queue
|
||||
elif queue:
|
||||
entries = [queue]
|
||||
else:
|
||||
return None
|
||||
for entry in entries:
|
||||
if isinstance(entry, dict) and entry.get("approval_id") == approval_id and entry.get(_GATEWAY_MIRROR_FLAG):
|
||||
run_id = str(entry.get("run_id") or "").strip()
|
||||
return run_id or None
|
||||
return None
|
||||
|
||||
|
||||
def submit_gateway_pending_mirror(session_key: str, approval: dict) -> tuple[dict | None, int]:
|
||||
"""Mirror the live gateway head into WebUI polling state under a typed tag.
|
||||
|
||||
Every mirrored entry describes one pending approval to the UI. Run-backed
|
||||
mirrors carry ``run_id`` (remote gateway runs) and are bound to the parked
|
||||
``_ApprovalEntry`` via ``approval_id``. No-run mirrors, which represent an
|
||||
in-process (legacy) approval parked in ``_gateway_queues``, have no
|
||||
``run_id`` and instead bind back to the live entry through
|
||||
``_GATEWAY_MIRROR_TOKEN``: ``_resolve_approval_legacy()`` matches
|
||||
``pending[_GATEWAY_MIRROR_TOKEN]`` against the live entry's
|
||||
``_webui_mirror_token`` before it will call ``resolve_gateway_pending_local()``
|
||||
to unblock the agent thread. A no-run mirror's token is stamped from its
|
||||
OWN live producer — resolved via the mirror's ``request_id``/
|
||||
``approval_id`` — and from nothing else. If no live producer's identity
|
||||
matches, the mirror is left tokenless (fail closed): guessing ownership
|
||||
from "first unclaimed token" or the queue head would let approving THIS
|
||||
(possibly stale/foreign) mirror resolve a DIFFERENT live producer than
|
||||
the one the user actually saw, which is an approval-integrity violation,
|
||||
not a convenience. A tokenless mirror is only wired up automatically when
|
||||
there is truly no live producer at all (#7093); ``reconcile_gateway_
|
||||
pending_mirror_locked`` binds a fresh, correctly-bound mirror to the
|
||||
authoritative live head on its own. Without a token, THIS specific click
|
||||
returns ``ok:true`` and the card clears without unblocking any producer —
|
||||
the correct producer's own card reappears on the next reconcile.
|
||||
"""
|
||||
def submit_gateway_pending_mirror(session_key: str, approval: dict) -> None:
|
||||
"""Mirror the live gateway head into WebUI polling state under a typed tag."""
|
||||
del approval # mirror from the live gateway head under `_lock`, not from callback input
|
||||
with _lock:
|
||||
run_id = str(approval.get("run_id") or "").strip()
|
||||
approval_id = str(approval.get("approval_id") or "").strip()
|
||||
live_gateway_queue = _gateway_queues.get(session_key) or []
|
||||
exact_local_entry = next(
|
||||
(
|
||||
entry for entry in live_gateway_queue
|
||||
if getattr(entry, "data", None) is approval
|
||||
),
|
||||
None,
|
||||
) if not run_id else None
|
||||
if exact_local_entry is None and not run_id and approval_id:
|
||||
exact_local_entry = next(
|
||||
(
|
||||
entry for entry in live_gateway_queue
|
||||
if str(((getattr(entry, "data", None) or {}).get("approval_id") or "")).strip() == approval_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
if exact_local_entry is None and not run_id:
|
||||
# Fall back to matching on the core's per-approval `request_id`.
|
||||
# The gateway core notifies WebUI with a COPY of the entry payload
|
||||
# (`notify_cb(dict(entry.data))`), so the identity match above
|
||||
# (`entry.data is approval`) never holds for a real gateway head,
|
||||
# and a local `_ApprovalEntry` carries a `request_id` but no
|
||||
# `approval_id`, so the approval_id fallback misses too. The
|
||||
# `request_id` is stamped once on the source entry
|
||||
# (`_ApprovalEntry.__init__` -> `data.setdefault("request_id", ...)`)
|
||||
# and preserved through the copy, so it uniquely reunites the
|
||||
# notified copy with its queued entry. Without this, the mirror is
|
||||
# created with no token, reconcile keeps the orphan, and
|
||||
# `_session_has_pending_approval` stays True after the entry is
|
||||
# dropped (the stale-approval-card dead-end, #4948 local variant).
|
||||
request_id = str(approval.get("request_id") or "").strip()
|
||||
if request_id:
|
||||
exact_local_entry = next(
|
||||
(
|
||||
entry for entry in live_gateway_queue
|
||||
if str(((getattr(entry, "data", None) or {}).get("request_id") or "")).strip() == request_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
if exact_local_entry is not None:
|
||||
mirror_entries = _normalize_pending_queue_locked(session_key)
|
||||
entries_to_mirror = [live_gateway_queue[0]] if live_gateway_queue else []
|
||||
if exact_local_entry not in entries_to_mirror:
|
||||
entries_to_mirror.append(exact_local_entry)
|
||||
for entry in entries_to_mirror:
|
||||
local_data = entry.data
|
||||
token = _gateway_mirror_entry_token(entry)
|
||||
entry_approval_id = str(local_data.get("approval_id") or "").strip()
|
||||
if entry is exact_local_entry:
|
||||
entry_approval_id = approval_id or entry_approval_id or f"gwlocal:{token}"
|
||||
approval_id = entry_approval_id
|
||||
approval["approval_id"] = entry_approval_id
|
||||
elif not entry_approval_id:
|
||||
entry_approval_id = f"gwlocal:{token}"
|
||||
local_data["approval_id"] = entry_approval_id
|
||||
if not any(
|
||||
_is_gateway_mirror_entry(mirror)
|
||||
and str(mirror.get(_GATEWAY_MIRROR_TOKEN) or "") == token
|
||||
for mirror in mirror_entries
|
||||
):
|
||||
mirror_entry = dict(local_data)
|
||||
mirror_entry["approval_id"] = entry_approval_id
|
||||
mirror_entry[_GATEWAY_MIRROR_FLAG] = True
|
||||
mirror_entry[_GATEWAY_MIRROR_TOKEN] = token
|
||||
mirror_entries.append(mirror_entry)
|
||||
if run_id:
|
||||
live_head_entry = live_gateway_queue[0] if live_gateway_queue else None
|
||||
live_head_data = getattr(live_head_entry, "data", None) or {}
|
||||
live_head_run_id = str(live_head_data.get("run_id") or "").strip()
|
||||
live_head_approval_id = str(live_head_data.get("approval_id") or "").strip()
|
||||
live_token = (
|
||||
_gateway_mirror_entry_token(live_head_entry)
|
||||
if live_head_entry and live_head_data
|
||||
else None
|
||||
)
|
||||
if (
|
||||
live_token
|
||||
and live_head_run_id == run_id
|
||||
and (
|
||||
not approval_id
|
||||
or not live_head_approval_id
|
||||
or live_head_approval_id == approval_id
|
||||
)
|
||||
):
|
||||
if approval_id:
|
||||
live_head_data["approval_id"] = approval_id
|
||||
else:
|
||||
approval_id = live_head_approval_id
|
||||
if not approval_id:
|
||||
approval_id = f"gwrun:{run_id}:{live_token}"
|
||||
live_head_data["approval_id"] = approval_id
|
||||
approval["approval_id"] = approval_id
|
||||
else:
|
||||
if not approval_id:
|
||||
approval_id = f"gwrun:{run_id}:{uuid.uuid4().hex}"
|
||||
approval["approval_id"] = approval_id
|
||||
mirror_entry = dict(approval)
|
||||
mirror_entry["run_id"] = run_id
|
||||
mirror_entry["approval_id"] = approval_id
|
||||
mirror_entry[_GATEWAY_MIRROR_FLAG] = True
|
||||
mirror_entry[_GATEWAY_MIRROR_TOKEN] = uuid.uuid4().hex
|
||||
mirror_entry[_GATEWAY_MIRROR_RETAINED] = True
|
||||
if not _gateway_pending_mirror_locked(session_key, approval_id=approval_id, run_id=run_id):
|
||||
_normalize_pending_queue_locked(session_key).append(mirror_entry)
|
||||
elif not exact_local_entry:
|
||||
if not approval_id:
|
||||
approval_id = uuid.uuid4().hex
|
||||
approval["approval_id"] = approval_id
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
no_run_mirror = next(
|
||||
(
|
||||
entry for entry in reversed(entries)
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and not str(entry.get("run_id") or "").strip()
|
||||
and str(entry.get("approval_id") or "").strip() == approval_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
if no_run_mirror:
|
||||
approval["approval_id"] = str(no_run_mirror.get("approval_id") or approval_id).strip()
|
||||
elif not _gateway_pending_mirror_locked(session_key, approval_id=approval_id):
|
||||
# Stamp the mirror token from the mirror's OWN live producer so
|
||||
# the first respond can link this mirror to the right
|
||||
# _ApprovalEntry in _gateway_queues. Without it,
|
||||
# _resolve_approval_legacy cannot match the no-run mirror to its
|
||||
# gateway entry (both token fields are empty) and the agent
|
||||
# thread is never unblocked on the first click (#6008 legacy).
|
||||
# We MUST NOT blindly take the live head: for a non-head mirror
|
||||
# (multiple parked producers, #7093 exact-producer isolation)
|
||||
# the head belongs to a sibling, and stamping its token would
|
||||
# bind the mirror to the wrong entry. Only an explicit
|
||||
# request_id/approval_id match may bind a token.
|
||||
#
|
||||
# FAIL CLOSED when no producer's identity matches: never infer
|
||||
# ownership from "first unclaimed token" or the queue head. A
|
||||
# stale/foreign approval (mismatched request_id, or no
|
||||
# identity at all) that borrows another live producer's token
|
||||
# would let approving THIS mirror resolve a DIFFERENT producer
|
||||
# than the one the user actually saw — an approval-integrity
|
||||
# violation, not a convenience (found in review of 818fd2fd).
|
||||
# A tokenless orphan is only legitimate when there is no live
|
||||
# producer at all (#7093); reconcile_gateway_pending_mirror_locked
|
||||
# binds a real mirror to the authoritative live head on its own.
|
||||
live_queue_for_mirror = _gateway_queues.get(session_key) or []
|
||||
request_id_for_mirror = str(approval.get("request_id") or "").strip()
|
||||
mirror_producer = None
|
||||
if request_id_for_mirror or approval_id:
|
||||
for cand in live_queue_for_mirror:
|
||||
cand_data = getattr(cand, "data", None) or {}
|
||||
if (request_id_for_mirror and
|
||||
str(cand_data.get("request_id") or "").strip() == request_id_for_mirror):
|
||||
mirror_producer = cand
|
||||
break
|
||||
if (approval_id and not request_id_for_mirror and
|
||||
str(cand_data.get("approval_id") or "").strip() == approval_id):
|
||||
mirror_producer = cand
|
||||
break
|
||||
mirror_token = (
|
||||
_gateway_mirror_entry_token(mirror_producer)
|
||||
if mirror_producer is not None else None
|
||||
)
|
||||
mirror_entry = dict(approval)
|
||||
mirror_entry["approval_id"] = approval_id
|
||||
mirror_entry[_GATEWAY_MIRROR_FLAG] = True
|
||||
if mirror_token:
|
||||
mirror_entry[_GATEWAY_MIRROR_TOKEN] = mirror_token
|
||||
_normalize_pending_queue_locked(session_key).append(mirror_entry)
|
||||
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
publish_session_list_changed("attention_pending")
|
||||
return (dict(head) if head else None), total
|
||||
|
||||
|
||||
def resolve_gateway_pending_local(
|
||||
session_key: str, approval_id: str, choice: str, reason: str | None = None
|
||||
) -> tuple[int, dict | None, int]:
|
||||
"""Resolve the exact parked local entry bound to an approval mirror."""
|
||||
target = None
|
||||
with _lock:
|
||||
approval_id = str(approval_id or "").strip()
|
||||
gateway_queue = _gateway_queues.get(session_key) or []
|
||||
for index, entry in enumerate(gateway_queue):
|
||||
data = getattr(entry, "data", None) or {}
|
||||
if str(data.get("approval_id") or "").strip() == approval_id:
|
||||
target = gateway_queue.pop(index)
|
||||
break
|
||||
if gateway_queue:
|
||||
_gateway_queues[session_key] = gateway_queue
|
||||
else:
|
||||
_gateway_queues.pop(session_key, None)
|
||||
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
if target is None:
|
||||
return 0, head, total
|
||||
target.result = choice
|
||||
if reason:
|
||||
target.reason = reason
|
||||
target.event.set()
|
||||
publish_session_list_changed("attention_resolved")
|
||||
return 1, head, total
|
||||
|
||||
|
||||
def resolve_gateway_pending_local_no_run_mirror(
|
||||
session_key: str, approval_id: str, choice: str, reason: str | None = None
|
||||
) -> tuple[bool, int, dict | None, int]:
|
||||
"""Resolve an exact no-run mirror only while its parked producer still exists."""
|
||||
target = None
|
||||
with _lock:
|
||||
approval_id = str(approval_id or "").strip()
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
matched_mirror = next(
|
||||
(
|
||||
entry for entry in entries
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and not str(entry.get("run_id") or "").strip()
|
||||
and str(entry.get("approval_id") or "").strip() == approval_id
|
||||
),
|
||||
None,
|
||||
)
|
||||
if matched_mirror is None:
|
||||
return False, 0, entries[0] if entries else None, len(entries)
|
||||
|
||||
gateway_queue = _gateway_queues.get(session_key) or []
|
||||
for index, entry in enumerate(gateway_queue):
|
||||
data = getattr(entry, "data", None) or {}
|
||||
if str(data.get("approval_id") or "").strip() == approval_id:
|
||||
target = gateway_queue.pop(index)
|
||||
break
|
||||
if target is None:
|
||||
matched_mirror[_GATEWAY_MIRROR_RETAINED] = True
|
||||
return True, 0, entries[0] if entries else None, len(entries)
|
||||
|
||||
if gateway_queue:
|
||||
_gateway_queues[session_key] = gateway_queue
|
||||
else:
|
||||
_gateway_queues.pop(session_key, None)
|
||||
entries.remove(matched_mirror)
|
||||
if entries:
|
||||
_pending[session_key] = entries
|
||||
else:
|
||||
_pending.pop(session_key, None)
|
||||
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
target.result = choice
|
||||
if reason:
|
||||
target.reason = reason
|
||||
target.event.set()
|
||||
publish_session_list_changed("attention_resolved")
|
||||
return True, 1, head, total
|
||||
|
||||
|
||||
def resolve_gateway_pending_local_all(
|
||||
session_key: str,
|
||||
choice: str,
|
||||
reason: str | None = None,
|
||||
) -> tuple[int, dict | None, int]:
|
||||
"""Resolve every parked local/no-run approval without touching remote runs."""
|
||||
targets = []
|
||||
removed_pending = False
|
||||
with _lock:
|
||||
reconcile_gateway_pending_mirror_locked(session_key)
|
||||
|
||||
gateway_queue = _gateway_queues.get(session_key) or []
|
||||
retained_gateway_queue = []
|
||||
for entry in gateway_queue:
|
||||
data = getattr(entry, "data", None) or {}
|
||||
if str(data.get("run_id") or "").strip():
|
||||
retained_gateway_queue.append(entry)
|
||||
else:
|
||||
targets.append(entry)
|
||||
if retained_gateway_queue:
|
||||
_gateway_queues[session_key] = retained_gateway_queue
|
||||
else:
|
||||
_gateway_queues.pop(session_key, None)
|
||||
|
||||
queue = _pending.get(session_key)
|
||||
entries = queue if isinstance(queue, list) else [queue] if queue else []
|
||||
retained_pending = [
|
||||
entry
|
||||
for entry in entries
|
||||
if _is_gateway_mirror_entry(entry)
|
||||
and str(entry.get("run_id") or "").strip()
|
||||
]
|
||||
removed_pending = len(retained_pending) != len(entries)
|
||||
if retained_pending:
|
||||
_pending[session_key] = retained_pending
|
||||
else:
|
||||
_pending.pop(session_key, None)
|
||||
|
||||
head, total, _changed = reconcile_gateway_pending_mirror_locked(session_key)
|
||||
_approval_sse_notify_locked(session_key, head, total)
|
||||
|
||||
for entry in targets:
|
||||
entry.result = choice
|
||||
if reason:
|
||||
entry.reason = reason
|
||||
entry.event.set()
|
||||
if targets or removed_pending:
|
||||
publish_session_list_changed("attention_resolved")
|
||||
return len(targets), head, total
|
||||
|
||||
|
||||
def settle_gateway_pending_local_notification(
|
||||
session_key: str,
|
||||
approval: dict,
|
||||
) -> tuple[bool, dict | None, int]:
|
||||
"""Auto-resolve or publish one local approval at the YOLO handoff boundary.
|
||||
|
||||
The Agent adds its blocking entry before invoking WebUI's notify callback.
|
||||
Serialize that callback with session YOLO commit/disable so a waiter arriving
|
||||
after a drain snapshot cannot be parked behind an already-committed enable.
|
||||
Run-backed approvals stay on the Runs API path and are never resolved here.
|
||||
"""
|
||||
with gateway_yolo_handoff(session_key):
|
||||
run_id = str((approval or {}).get("run_id") or "").strip()
|
||||
if not run_id and is_session_yolo_enabled(session_key):
|
||||
_resolved, head, total = resolve_gateway_pending_local_all(
|
||||
session_key,
|
||||
"once",
|
||||
)
|
||||
return True, head, total
|
||||
head, total = submit_gateway_pending_mirror(session_key, approval)
|
||||
return False, head, total
|
||||
|
||||
|
||||
def submit_pending(session_key: str, approval: dict) -> None:
|
||||
|
||||
+17
-318
@@ -1,8 +1,7 @@
|
||||
"""Session-list cache helpers extracted from api.routes."""
|
||||
|
||||
import copy
|
||||
import os
|
||||
import re
|
||||
import copy
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
@@ -13,26 +12,17 @@ from api.models import _active_state_db_path, _active_stream_ids
|
||||
from api.profiles import _profiles_match
|
||||
|
||||
|
||||
# Cron session ids are ``cron_{job_id}_{run_timestamp}`` where the run
|
||||
# timestamp is ``YYYYMMDD_HHMMSS`` (e.g. cron_job6728_20260803_100000). Used to
|
||||
# validate that the text after a matched ``cron_{jid}_`` prefix is EXACTLY a run
|
||||
# timestamp, so a shorter job id (backup) cannot claim a longer job id's session
|
||||
# (backup_full) when the longer job is not itself running (#6728 gate fix).
|
||||
_CRON_RUN_TS_RE = re.compile(r"\d{8}_\d{6}")
|
||||
|
||||
|
||||
_SESSIONS_CACHE_TTL_SECONDS = 2.5
|
||||
# #4808: while a turn is actively streaming the frontend polls /api/sessions on a
|
||||
# fixed cadence (static/sessions.js `_streamingPollMs`). With the idle TTL of
|
||||
# 2.5s, the entry expires between streaming polls, so each poll can find it stale
|
||||
# and force a full all_sessions() rebuild on the hot path under the global store
|
||||
# LOCK — pinning CPU and starving token rendering on large stores (recurrence of
|
||||
# #4672). Hold the sidebar cache steady for longer than one poll interval while
|
||||
# streaming; live runtime state (active stream, sort order, pending flags) is
|
||||
# overlaid on every response regardless of cache, and structural/settings changes
|
||||
# still invalidate immediately. Keep this strictly greater than
|
||||
# `_streamingPollMs`/1000 (see tests/test_streaming_cache_ttl_vs_poll.py).
|
||||
_SESSIONS_CACHE_STREAMING_TTL_SECONDS = 45.0
|
||||
# fixed cadence (static/sessions.js `_streamingPollMs` = 5000ms). With the idle TTL
|
||||
# of 2.5s, every streaming poll lands in a fresh window and forces a full
|
||||
# all_sessions() rebuild on the hot path under the global store LOCK — pinning CPU
|
||||
# and starving token rendering on large stores (recurrence of #4672). Hold the
|
||||
# sidebar cache steady for longer than one poll interval while streaming; live
|
||||
# runtime state (active stream, sort order, pending flags) is overlaid on every
|
||||
# response regardless of cache, and structural/settings changes still evict
|
||||
# immediately via the source stamp.
|
||||
_SESSIONS_CACHE_STREAMING_TTL_SECONDS = 10.0
|
||||
_SESSIONS_CACHE_MAX_ENTRIES = 64
|
||||
_SESSIONS_CACHE_WAIT_SECONDS = 0.25
|
||||
_SESSIONS_CACHE_STALE_WAIT_SECONDS = 0.10
|
||||
@@ -44,178 +34,6 @@ _SESSIONS_CACHE_ALL_PROFILES_INVALIDATION_VERSION = 0
|
||||
_SESSIONS_CACHE_PROFILE_INVALIDATION_VERSION: dict[str, int] = {}
|
||||
|
||||
|
||||
_SIDEBAR_SESSION_RESPONSE_FIELDS = {
|
||||
"session_id",
|
||||
"title",
|
||||
"display_title",
|
||||
"_state_db_title",
|
||||
"workspace",
|
||||
"model",
|
||||
"model_provider",
|
||||
"message_count",
|
||||
"user_message_count",
|
||||
"created_at",
|
||||
"updated_at",
|
||||
"last_message_at",
|
||||
"pinned",
|
||||
"archived",
|
||||
"project_id",
|
||||
"profile",
|
||||
"input_tokens",
|
||||
"output_tokens",
|
||||
"estimated_cost",
|
||||
"cache_read_tokens",
|
||||
"cache_write_tokens",
|
||||
"cache_hit_percent",
|
||||
"personality",
|
||||
"context_length",
|
||||
"config_context_length",
|
||||
"window_usage_percent",
|
||||
"source_tag",
|
||||
"raw_source",
|
||||
"session_source",
|
||||
"source_label",
|
||||
"is_cli_session",
|
||||
"is_messaging_session",
|
||||
"is_streaming",
|
||||
"cron_running",
|
||||
"active_stream_id",
|
||||
"has_pending_user_message",
|
||||
"pending_started_at",
|
||||
"default_hidden",
|
||||
"worktree_path",
|
||||
"worktree_branch",
|
||||
"parent_session_id",
|
||||
"parent_title",
|
||||
"parent_source",
|
||||
"relationship_type",
|
||||
"pre_compression_snapshot",
|
||||
"_lineage_root_id",
|
||||
"_lineage_tip_id",
|
||||
"_compression_segment_count",
|
||||
"_lineage_collapsed_count",
|
||||
"_parent_lineage_root_id",
|
||||
"_parent_lineage_tip_id",
|
||||
"_cross_surface_child_session",
|
||||
"match_type",
|
||||
"match_preview",
|
||||
# Preserved so the sidebar can suppress rename / action-menu / swipe on
|
||||
# read-only sessions and render the detailed gateway model label. Only the
|
||||
# latest bounded routing object is included; routing history stays excluded.
|
||||
"read_only",
|
||||
"is_read_only",
|
||||
"gateway_routing",
|
||||
}
|
||||
|
||||
|
||||
def _session_list_cache_sidebar_fields() -> set[str]:
|
||||
"""Return the canonical bounded field set used by the list response."""
|
||||
return _SIDEBAR_SESSION_RESPONSE_FIELDS
|
||||
|
||||
|
||||
def _session_list_cache_copy_value(value):
|
||||
"""Copy only mutable values after the transcript-bearing projection."""
|
||||
if isinstance(value, (dict, list, set)):
|
||||
return copy.deepcopy(value)
|
||||
return value
|
||||
|
||||
|
||||
def _session_list_cache_copy_row(row: dict) -> dict:
|
||||
return {
|
||||
key: _session_list_cache_copy_value(value)
|
||||
for key, value in row.items()
|
||||
}
|
||||
|
||||
|
||||
def _session_list_cache_bounded_payload(payload: dict) -> dict:
|
||||
"""Project a builder payload to data the sidebar can actually consume.
|
||||
|
||||
Builders operate on full session snapshots because they also perform
|
||||
reconciliation and lineage decisions. The cache must not retain those
|
||||
snapshots: a single long transcript can otherwise make every cache set/get
|
||||
allocate hundreds of megabytes via deepcopy().
|
||||
"""
|
||||
fields = _session_list_cache_sidebar_fields()
|
||||
|
||||
def project_rows(rows):
|
||||
projected = []
|
||||
for row in rows or []:
|
||||
if not isinstance(row, dict):
|
||||
projected.append({})
|
||||
continue
|
||||
item = {key: row[key] for key in fields if key in row}
|
||||
projected.append(_session_list_cache_copy_row(item))
|
||||
return projected
|
||||
|
||||
bounded = {
|
||||
key: _session_list_cache_copy_value(value)
|
||||
for key, value in payload.items()
|
||||
if key not in {"sessions", "sidebar_reference_sessions"}
|
||||
}
|
||||
if "sessions" in payload:
|
||||
bounded["sessions"] = project_rows(payload.get("sessions"))
|
||||
if "sidebar_reference_sessions" in payload:
|
||||
bounded["sidebar_reference_sessions"] = project_rows(
|
||||
payload.get("sidebar_reference_sessions")
|
||||
)
|
||||
return bounded
|
||||
|
||||
|
||||
def _session_list_cache_copy_payload(payload: dict) -> dict:
|
||||
"""Copy only the already-bounded cache shape for request-local mutation."""
|
||||
copied = {
|
||||
key: _session_list_cache_copy_value(value)
|
||||
for key, value in payload.items()
|
||||
if key not in {"sessions", "sidebar_reference_sessions"}
|
||||
}
|
||||
if "sessions" in payload:
|
||||
copied["sessions"] = [
|
||||
_session_list_cache_copy_row(row)
|
||||
for row in payload.get("sessions", [])
|
||||
]
|
||||
if "sidebar_reference_sessions" in payload:
|
||||
copied["sidebar_reference_sessions"] = [
|
||||
_session_list_cache_copy_row(row)
|
||||
for row in payload.get("sidebar_reference_sessions", [])
|
||||
]
|
||||
return copied
|
||||
|
||||
|
||||
def get_session_list_cache_snapshot() -> dict[str, object]:
|
||||
"""Return scalar cache occupancy without waiting or changing LRU state.
|
||||
|
||||
Held-section discipline: between the nonblocking acquire and the release,
|
||||
only ``len()`` and module-constant reads are permitted. Nothing that can
|
||||
resolve config, resolve a profile, touch the filesystem, import a module, or
|
||||
wait on another lock may be added here. ``_SESSIONS_CACHE_LOCK`` is an
|
||||
``RLock``, so a nonblocking acquire from a thread already holding it would
|
||||
report available mid-mutation; the health collector is this helper's only
|
||||
caller and never runs nested inside a cache rebuild.
|
||||
"""
|
||||
result = {
|
||||
"available": False,
|
||||
"entries": 0,
|
||||
"inflight_rebuilds": 0,
|
||||
"cap": 0,
|
||||
}
|
||||
acquired = False
|
||||
try:
|
||||
acquired = _SESSIONS_CACHE_LOCK.acquire(blocking=False)
|
||||
if not acquired:
|
||||
return result
|
||||
return {
|
||||
"available": True,
|
||||
"entries": max(0, int(len(_SESSIONS_CACHE))),
|
||||
"inflight_rebuilds": max(0, int(len(_SESSIONS_CACHE_INFLIGHT))),
|
||||
"cap": max(0, int(_SESSIONS_CACHE_MAX_ENTRIES)),
|
||||
}
|
||||
except Exception:
|
||||
return result
|
||||
finally:
|
||||
if acquired:
|
||||
_SESSIONS_CACHE_LOCK.release()
|
||||
|
||||
|
||||
def _session_list_cache_session_dir() -> Path:
|
||||
try:
|
||||
import api.routes as _routes
|
||||
@@ -279,27 +97,6 @@ def _session_list_cache_active_stream_ids():
|
||||
return _active_stream_ids()
|
||||
|
||||
|
||||
def _session_list_cache_running_cron_jobs() -> dict[str, float]:
|
||||
"""Return {job_id: start_epoch} for cron jobs currently tracked as running.
|
||||
|
||||
Cron liveness lives only in the in-memory ``_RUNNING_CRON_JOBS`` dict in
|
||||
api.routes (#6728): the sidebar polls /api/sessions (not /api/crons/status),
|
||||
so without this overlay a still-running cron job's session row looks
|
||||
completed the moment it appends a message. Fail closed to an empty dict.
|
||||
"""
|
||||
try:
|
||||
import api.routes as _routes
|
||||
|
||||
jobs = getattr(_routes, "_RUNNING_CRON_JOBS", None)
|
||||
lock = getattr(_routes, "_RUNNING_CRON_LOCK", None)
|
||||
if jobs is None or lock is None:
|
||||
return {}
|
||||
with lock:
|
||||
return dict(jobs)
|
||||
except Exception:
|
||||
return {}
|
||||
|
||||
|
||||
def _session_list_cache_resolved_source_stamp(key: tuple):
|
||||
try:
|
||||
import api.routes as _routes
|
||||
@@ -328,23 +125,9 @@ def _session_list_cache_key(
|
||||
include_archived: bool = False,
|
||||
exclude_hidden: bool = False,
|
||||
visible_only: bool = False,
|
||||
show_webhook_sessions: bool = False,
|
||||
show_kanban_sessions: bool = False,
|
||||
source_filter: str | None = None,
|
||||
sidebar_source: str | None = None,
|
||||
archived_limit: int | None = None,
|
||||
archived_offset: int = 0,
|
||||
) -> tuple:
|
||||
normalized_archived_limit = None
|
||||
if archived_limit is not None:
|
||||
try:
|
||||
normalized_archived_limit = max(0, int(archived_limit))
|
||||
except (TypeError, ValueError):
|
||||
normalized_archived_limit = None
|
||||
try:
|
||||
normalized_archived_offset = max(0, int(archived_offset or 0))
|
||||
except (TypeError, ValueError):
|
||||
normalized_archived_offset = 0
|
||||
return (
|
||||
_session_list_cache_profile_scope(active_profile),
|
||||
bool(all_profiles),
|
||||
@@ -354,12 +137,8 @@ def _session_list_cache_key(
|
||||
bool(include_archived),
|
||||
bool(exclude_hidden),
|
||||
bool(visible_only),
|
||||
bool(show_webhook_sessions),
|
||||
bool(show_kanban_sessions),
|
||||
source_filter,
|
||||
sidebar_source,
|
||||
normalized_archived_limit,
|
||||
normalized_archived_offset,
|
||||
)
|
||||
|
||||
|
||||
@@ -377,70 +156,34 @@ def _session_list_cache_get(
|
||||
if stamp != current_stamp:
|
||||
if allow_stale:
|
||||
_SESSIONS_CACHE.move_to_end(key)
|
||||
return _session_list_cache_copy_payload(payload), False
|
||||
return copy.deepcopy(payload), False
|
||||
_SESSIONS_CACHE.pop(key, None)
|
||||
return None, False
|
||||
# #4808: widen the freshness window while a turn is streaming so the fixed
|
||||
# streaming poll cadence doesn't force a full rebuild on every poll.
|
||||
# 5s streaming poll cadence doesn't force a full rebuild on every poll.
|
||||
ttl = _SESSIONS_CACHE_TTL_SECONDS
|
||||
if _session_list_cache_streaming_freeze_marker() is not None:
|
||||
ttl = _SESSIONS_CACHE_STREAMING_TTL_SECONDS
|
||||
fresh = (now - ts) < ttl
|
||||
if fresh:
|
||||
_SESSIONS_CACHE.move_to_end(key)
|
||||
return _session_list_cache_copy_payload(payload), True
|
||||
return copy.deepcopy(payload), True
|
||||
if allow_stale:
|
||||
_SESSIONS_CACHE.move_to_end(key)
|
||||
return _session_list_cache_copy_payload(payload), False
|
||||
return copy.deepcopy(payload), False
|
||||
_SESSIONS_CACHE.pop(key, None)
|
||||
return None, False
|
||||
|
||||
|
||||
def _session_list_cache_stale_reason(key: tuple) -> str | None:
|
||||
"""Return why an existing cache entry is stale, if it is stale."""
|
||||
now = time.monotonic()
|
||||
current_stamp = _session_list_cache_resolved_source_stamp(key)
|
||||
with _SESSIONS_CACHE_LOCK:
|
||||
entry = _SESSIONS_CACHE.get(key)
|
||||
if not entry:
|
||||
return None
|
||||
ts, stamp, _payload = entry
|
||||
if stamp != current_stamp:
|
||||
return "source"
|
||||
ttl = _SESSIONS_CACHE_TTL_SECONDS
|
||||
if _session_list_cache_streaming_freeze_marker() is not None:
|
||||
ttl = _SESSIONS_CACHE_STREAMING_TTL_SECONDS
|
||||
if (now - ts) >= ttl:
|
||||
return "age"
|
||||
return None
|
||||
|
||||
|
||||
def _session_list_cache_set(
|
||||
key: tuple,
|
||||
payload: dict,
|
||||
*,
|
||||
expected_invalidation_stamp: tuple[int, int] | None = None,
|
||||
) -> bool:
|
||||
def _session_list_cache_set(key: tuple, payload: dict) -> None:
|
||||
if not isinstance(payload, dict):
|
||||
return False
|
||||
return
|
||||
stamp = _session_list_cache_resolved_source_stamp(key)
|
||||
bounded = _session_list_cache_bounded_payload(payload)
|
||||
with _SESSIONS_CACHE_LOCK:
|
||||
# Projection intentionally happens outside the lock so large source rows
|
||||
# cannot block cache hits. Re-check the caller's pre-build generation
|
||||
# atomically before insertion, otherwise a rename/archive/delete clear
|
||||
# that lands during projection can be undone by this stale write.
|
||||
if (
|
||||
expected_invalidation_stamp is not None
|
||||
and _session_list_cache_invalidation_stamp(key)
|
||||
!= expected_invalidation_stamp
|
||||
):
|
||||
return False
|
||||
_SESSIONS_CACHE[key] = (time.monotonic(), stamp, bounded)
|
||||
_SESSIONS_CACHE[key] = (time.monotonic(), stamp, copy.deepcopy(payload))
|
||||
_SESSIONS_CACHE.move_to_end(key)
|
||||
while len(_SESSIONS_CACHE) > _SESSIONS_CACHE_MAX_ENTRIES:
|
||||
_SESSIONS_CACHE.popitem(last=False)
|
||||
return True
|
||||
|
||||
|
||||
def _session_list_cache_clear(profile: str | None = None) -> None:
|
||||
@@ -645,11 +388,6 @@ def _session_list_cache_overlay_runtime_rows(rows: list[dict]) -> list[dict]:
|
||||
active_stream_ids = _session_list_cache_active_stream_ids()
|
||||
except Exception:
|
||||
active_stream_ids = set()
|
||||
try:
|
||||
running_cron_jobs = _session_list_cache_running_cron_jobs()
|
||||
except Exception:
|
||||
running_cron_jobs = {}
|
||||
cron_job_prefixes = [(jid, f"cron_{jid}_", started_at) for jid, started_at in running_cron_jobs.items()]
|
||||
session_ids = [
|
||||
str(row.get("session_id") or "").strip()
|
||||
for row in rows
|
||||
@@ -681,50 +419,11 @@ def _session_list_cache_overlay_runtime_rows(rows: list[dict]) -> list[dict]:
|
||||
item[key] = raw_live_value
|
||||
stream_id = item.get("active_stream_id")
|
||||
item["is_streaming"] = bool(stream_id and stream_id in active_stream_ids)
|
||||
# #6728: a still-running cron job's session row must not look completed
|
||||
# in the sidebar. Cron liveness is only exposed via /api/crons/status,
|
||||
# which the sidebar never polls — stamp the flag here so the client can
|
||||
# defer its completion/unread transition until the job actually ends.
|
||||
# Session ids are cron_{job_id}_{run_timestamp}: only the run started at
|
||||
# (or after) the tracked start belongs to the live execution — older runs
|
||||
# of the same job stay completed.
|
||||
item["cron_running"] = _session_list_row_cron_running(
|
||||
sid, item, cron_job_prefixes
|
||||
)
|
||||
overlaid.append(item)
|
||||
overlaid.sort(key=_session_list_runtime_sort_key, reverse=True)
|
||||
return overlaid
|
||||
|
||||
|
||||
def _session_list_row_cron_running(
|
||||
sid: str, row: dict, cron_job_prefixes: list[tuple[str, str, float]]
|
||||
) -> bool:
|
||||
if not cron_job_prefixes or not sid:
|
||||
return False
|
||||
created_at = _session_list_row_numeric_value(row.get("created_at"))
|
||||
# Longest prefix first, no fall-through: job ids may nest (backup vs
|
||||
# backup_full), and the shorter prefix is a valid prefix of the longer one.
|
||||
# A session belongs to the longest matching job id — first-match in
|
||||
# insertion order, or falling through to a shorter prefix after a time-miss,
|
||||
# would let a running shorter-prefix job claim a completed longer-prefix
|
||||
# session. Mirrors the max(matches, key=len) convention in
|
||||
# api.routes._latest_cron_session_info_for_jobs.
|
||||
#
|
||||
# #6728 (gate fix): prefixes are built ONLY from RUNNING jobs, so if the
|
||||
# true longer owner (backup_full) is not running, longest-prefix sorting
|
||||
# never sees it and a running `backup` would otherwise swallow a
|
||||
# `cron_backup_full_YYYYMMDD_HHMMSS` session (the leftover `full_...` still
|
||||
# starts with nothing it should match). Require the text AFTER the prefix to
|
||||
# be exactly a run-timestamp (YYYYMMDD_HHMMSS) so a shorter job id cannot
|
||||
# claim a longer job id's session regardless of which jobs are running.
|
||||
for _jid, prefix, started_at in sorted(
|
||||
cron_job_prefixes, key=lambda item: len(item[1]), reverse=True
|
||||
):
|
||||
if sid.startswith(prefix) and _CRON_RUN_TS_RE.fullmatch(sid[len(prefix):]):
|
||||
return created_at >= started_at
|
||||
return False
|
||||
|
||||
|
||||
def _session_list_row_numeric_value(value) -> float:
|
||||
try:
|
||||
numeric = float(value or 0)
|
||||
|
||||
+1638
-8738
File diff suppressed because it is too large
Load Diff
+17
-486
@@ -10,52 +10,17 @@ import os
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from collections import OrderedDict
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable
|
||||
from typing import Iterable
|
||||
|
||||
RUN_JOURNAL_DIR_NAME = "_run_journal"
|
||||
_SAFE_ID_RE = re.compile(r"^[A-Za-z0-9_.-]+$")
|
||||
_WRITER_LOCKS: dict[tuple[str, str, str], threading.Lock] = {}
|
||||
_WRITER_LOCKS_GUARD = threading.Lock()
|
||||
# Next-seq to assign per run-journal file path, kept in memory so repeat appends
|
||||
# to the same run do not re-parse the whole file on every call. The per-path
|
||||
# ``_lock_for(path)`` serializes same-path reserve→append so seqs stay monotonic
|
||||
# and file order matches; ``_SEQ_CACHE_LOCK`` (below) additionally guards every
|
||||
# *structural* access to the dict (reserve/note/evict) so ``delete_run_journal``
|
||||
# can iterate + drop keys while a concurrent append on ANOTHER path inserts one,
|
||||
# without a ``dictionary changed size during iteration`` crash. See
|
||||
# ``_reserve_next_seq`` and ``delete_run_journal`` (which evicts stale entries).
|
||||
_SEQ_CACHE: dict[str, int] = {}
|
||||
_SEQ_CACHE_LOCK = threading.Lock()
|
||||
# Summary callers only need terminal state and the latest cursor. Re-parsing a
|
||||
# completed journal's full payload (which can include multi-megabyte tool or
|
||||
# session results) on every status/reconnect probe is needless. This process
|
||||
# cache is keyed by a complete stat identity, so it is never used after an
|
||||
# atomic replacement, append, truncate, or same-path file recreation.
|
||||
_SUMMARY_CACHE_MAX_ENTRIES = 128
|
||||
_SUMMARY_CACHE: OrderedDict[str, tuple[tuple[int, int, int, int, int], dict]] = OrderedDict()
|
||||
_SUMMARY_CACHE_LOCK = threading.Lock()
|
||||
# Events that mark a run terminal in the journal / summary sense.
|
||||
TERMINAL_SSE_EVENTS = frozenset({"done", "cancel", "apperror", "error", "stream_end"})
|
||||
# Events that should close an SSE relay drain loop. `done` is intentionally
|
||||
# excluded: background title generation and `stream_end` are emitted after
|
||||
# `done`, and breaking early would drop them. `apperror` is included because
|
||||
# it terminates with no trailing `stream_end`.
|
||||
SSE_RELAY_CLOSE_EVENTS = frozenset({"stream_end", "cancel", "apperror", "error"})
|
||||
# Back-compat alias used by older call sites / tests.
|
||||
_TERMINAL_SSE_EVENTS = TERMINAL_SSE_EVENTS
|
||||
_TERMINAL_SSE_EVENTS = {"done", "cancel", "apperror", "error", "stream_end"}
|
||||
_FSYNC_MODE_ENV = "HERMES_WEBUI_RUN_JOURNAL_FSYNC"
|
||||
_FSYNC_MODE_EAGER = "eager"
|
||||
_FSYNC_MODE_TERMINAL_ONLY = "terminal-only"
|
||||
_SESSION_REPLAY_MAX_BYTES = 4 * 1024 * 1024
|
||||
_SESSION_REPLAY_MAX_ROWS = 4096
|
||||
_SESSION_REPLAY_READ_CHUNK_BYTES = 64 * 1024
|
||||
_SNAPSHOT_ARGS_MAX_ITEMS = 64
|
||||
_SNAPSHOT_ARGS_MAX_DEPTH = 8
|
||||
_SNAPSHOT_ARGS_MAX_STRING_CHARS = 8192
|
||||
_SNAPSHOT_ARGS_MAX_TOTAL_CHARS = 64 * 1024
|
||||
_SNAPSHOT_ARGS_TRUNCATED_SUFFIX = "...[truncated]"
|
||||
|
||||
|
||||
def _default_session_dir() -> Path:
|
||||
@@ -88,68 +53,6 @@ def _lock_for(path: Path) -> threading.Lock:
|
||||
return lock
|
||||
|
||||
|
||||
def _summary_cache_signature(path: Path) -> tuple[int, int, int, int, int] | None:
|
||||
"""Return the complete filesystem identity used for summary-cache validity.
|
||||
|
||||
Includes ``st_ctime_ns`` so a same-inode, same-size rewrite that restores the
|
||||
original ``mtime_ns`` (e.g. an atomic replace) still invalidates the cache —
|
||||
ctime advances on any metadata/content change and cannot be forged back.
|
||||
"""
|
||||
try:
|
||||
stat = path.stat()
|
||||
except OSError:
|
||||
return None
|
||||
return (
|
||||
int(stat.st_dev),
|
||||
int(stat.st_ino),
|
||||
int(stat.st_size),
|
||||
int(stat.st_mtime_ns),
|
||||
int(stat.st_ctime_ns),
|
||||
)
|
||||
|
||||
|
||||
def _get_cached_summary(path: Path) -> dict | None:
|
||||
signature = _summary_cache_signature(path)
|
||||
if signature is None:
|
||||
return None
|
||||
key = str(path)
|
||||
with _SUMMARY_CACHE_LOCK:
|
||||
cached = _SUMMARY_CACHE.get(key)
|
||||
if cached is None:
|
||||
return None
|
||||
cached_signature, summary = cached
|
||||
if cached_signature != signature:
|
||||
_SUMMARY_CACHE.pop(key, None)
|
||||
return None
|
||||
_SUMMARY_CACHE.move_to_end(key)
|
||||
return dict(summary)
|
||||
|
||||
|
||||
def _cache_summary(
|
||||
path: Path,
|
||||
summary: dict,
|
||||
*,
|
||||
expected_signature: tuple[int, int, int, int, int] | None = None,
|
||||
) -> None:
|
||||
signature = _summary_cache_signature(path)
|
||||
# The pre-read signature is an enforced TOCTOU precondition. In particular,
|
||||
# a journal created after a missing-file read has ``None -> signature`` and
|
||||
# must not cache the empty/unknown result under the new file's identity.
|
||||
if signature is None or signature != expected_signature:
|
||||
return
|
||||
key = str(path)
|
||||
with _SUMMARY_CACHE_LOCK:
|
||||
_SUMMARY_CACHE[key] = (signature, dict(summary))
|
||||
_SUMMARY_CACHE.move_to_end(key)
|
||||
while len(_SUMMARY_CACHE) > _SUMMARY_CACHE_MAX_ENTRIES:
|
||||
_SUMMARY_CACHE.popitem(last=False)
|
||||
|
||||
|
||||
def _discard_cached_summary(path: Path) -> None:
|
||||
with _SUMMARY_CACHE_LOCK:
|
||||
_SUMMARY_CACHE.pop(str(path), None)
|
||||
|
||||
|
||||
def _read_jsonl(path: Path) -> tuple[list[dict], list[dict]]:
|
||||
events: list[dict] = []
|
||||
malformed: list[dict] = []
|
||||
@@ -172,129 +75,12 @@ def _read_jsonl(path: Path) -> tuple[list[dict], list[dict]]:
|
||||
return events, malformed
|
||||
|
||||
|
||||
def _parse_run_journal_event_id(raw: str | None) -> tuple[str | None, int | None]:
|
||||
raw = str(raw or "").strip()
|
||||
if not raw:
|
||||
return None, None
|
||||
if ":" in raw:
|
||||
run_id, tail = raw.rsplit(":", 1)
|
||||
else:
|
||||
run_id, tail = None, raw
|
||||
try:
|
||||
seq = max(0, int(tail))
|
||||
except (TypeError, ValueError):
|
||||
return run_id or None, None
|
||||
return run_id or None, seq
|
||||
|
||||
|
||||
def _snapshot_args_take_budget(budget: dict[str, int], amount: int) -> int:
|
||||
remaining = max(0, int(budget.get("remaining") or 0))
|
||||
take = min(remaining, max(0, amount))
|
||||
budget["remaining"] = remaining - take
|
||||
return take
|
||||
|
||||
|
||||
def _bound_snapshot_args_string(value: str, budget: dict[str, int]) -> str:
|
||||
max_chars = min(len(value), _SNAPSHOT_ARGS_MAX_STRING_CHARS)
|
||||
take = _snapshot_args_take_budget(budget, max_chars)
|
||||
out = value[:take]
|
||||
if take < len(value):
|
||||
suffix_take = _snapshot_args_take_budget(budget, len(_SNAPSHOT_ARGS_TRUNCATED_SUFFIX))
|
||||
out += _SNAPSHOT_ARGS_TRUNCATED_SUFFIX[:suffix_take]
|
||||
return out
|
||||
|
||||
|
||||
def _bound_run_journal_snapshot_value(value: Any, budget: dict[str, int], depth: int) -> Any:
|
||||
if budget.get("remaining", 0) <= 0:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
return _bound_snapshot_args_string(value, budget)
|
||||
if isinstance(value, dict):
|
||||
if depth >= _SNAPSHOT_ARGS_MAX_DEPTH:
|
||||
return {}
|
||||
out: dict[str, Any] = {}
|
||||
for index, (key, item) in enumerate(value.items()):
|
||||
if index >= _SNAPSHOT_ARGS_MAX_ITEMS or budget.get("remaining", 0) <= 0:
|
||||
break
|
||||
bounded_key = _bound_snapshot_args_string(str(key), budget)
|
||||
if not bounded_key:
|
||||
continue
|
||||
out[bounded_key] = _bound_run_journal_snapshot_value(item, budget, depth + 1)
|
||||
return out
|
||||
if isinstance(value, (list, tuple)):
|
||||
if depth >= _SNAPSHOT_ARGS_MAX_DEPTH:
|
||||
return []
|
||||
return [
|
||||
_bound_run_journal_snapshot_value(item, budget, depth + 1)
|
||||
for item in value[:_SNAPSHOT_ARGS_MAX_ITEMS]
|
||||
if budget.get("remaining", 0) > 0
|
||||
]
|
||||
if isinstance(value, (bool, int, float)) or value is None:
|
||||
try:
|
||||
_snapshot_args_take_budget(budget, len(json.dumps(value)))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return value
|
||||
return _bound_snapshot_args_string(str(value), budget)
|
||||
|
||||
|
||||
def bound_run_journal_snapshot_args(args: Any) -> Any:
|
||||
"""Return recovery tool args with realistic values intact and pathological payloads bounded."""
|
||||
if args is None:
|
||||
return {}
|
||||
budget = {"remaining": _SNAPSHOT_ARGS_MAX_TOTAL_CHARS}
|
||||
return _bound_run_journal_snapshot_value(args, budget, 0)
|
||||
|
||||
|
||||
def _next_seq(path: Path) -> int:
|
||||
events, _malformed = _read_jsonl(path)
|
||||
seqs = [int(event.get("seq") or 0) for event in events if isinstance(event.get("seq"), int)]
|
||||
return (max(seqs) + 1) if seqs else 1
|
||||
|
||||
|
||||
def _reserve_next_seq(path: Path) -> int:
|
||||
"""Reserve and return the next seq for ``path``, advancing the in-memory cache.
|
||||
|
||||
Callers MUST hold ``_lock_for(path)``. The first append per path in this
|
||||
process seeds the cache from ``_next_seq(path)`` (one file read); every later
|
||||
append is a pure in-memory increment, avoiding the O(n) re-parse that
|
||||
re-reading the whole journal on every append caused (O(n^2) over a run).
|
||||
Because ``RunJournalWriter`` and the free ``append_run_event`` share this one
|
||||
cache under the same per-path lock, their seqs stay monotonic and gapless
|
||||
even when both write the same path. ``_SEQ_CACHE_LOCK`` additionally makes the
|
||||
dict get+set atomic against a concurrent cross-path eviction.
|
||||
"""
|
||||
key = str(path)
|
||||
with _SEQ_CACHE_LOCK:
|
||||
nxt = _SEQ_CACHE.get(key)
|
||||
if nxt is not None:
|
||||
_SEQ_CACHE[key] = nxt + 1
|
||||
return nxt
|
||||
# Cache miss: seed from disk WITHOUT holding the module-global lock, so a
|
||||
# slow first-access file read for one path can't block every other path's
|
||||
# cache ops. The caller holds the per-path lock, so only one thread per path
|
||||
# can reach this branch — no double-seed, and no same-path writer can race
|
||||
# the value in between.
|
||||
seeded = _next_seq(path)
|
||||
with _SEQ_CACHE_LOCK:
|
||||
_SEQ_CACHE[key] = seeded + 1
|
||||
return seeded
|
||||
|
||||
|
||||
def _note_assigned_seq(path: Path, seq: int) -> None:
|
||||
"""Keep the cache at least one past an explicitly-supplied ``seq``.
|
||||
|
||||
Callers MUST hold ``_lock_for(path)``. When an append carries a caller-chosen
|
||||
``seq`` rather than drawing from the cache, advance the cache so a later
|
||||
cache-based append on the same path cannot re-issue an already-used seq.
|
||||
"""
|
||||
key = str(path)
|
||||
nxt = int(seq) + 1
|
||||
with _SEQ_CACHE_LOCK:
|
||||
if _SEQ_CACHE.get(key, 0) < nxt:
|
||||
_SEQ_CACHE[key] = nxt
|
||||
|
||||
|
||||
def _terminal_state_for_event(event_name: str, payload) -> str | None:
|
||||
name = str(event_name or "")
|
||||
if name == "done" or name == "stream_end":
|
||||
@@ -342,49 +128,6 @@ def _fsync_parent_dir(path: Path) -> None:
|
||||
pass
|
||||
|
||||
|
||||
def _event_created_at(event: dict, *, fallback: float = 0.0) -> float:
|
||||
try:
|
||||
return float(event.get("created_at") or fallback)
|
||||
except (TypeError, ValueError):
|
||||
return fallback
|
||||
|
||||
|
||||
def _iter_bounded_raw_jsonl_lines(path: Path, *, max_bytes: int, retained_bytes: int = 0):
|
||||
line_no = 0
|
||||
buffered = bytearray()
|
||||
total_bytes = int(retained_bytes)
|
||||
try:
|
||||
with path.open("rb") as fh:
|
||||
while True:
|
||||
chunk = fh.read(_SESSION_REPLAY_READ_CHUNK_BYTES)
|
||||
if not chunk:
|
||||
if buffered:
|
||||
if total_bytes + len(buffered) > max_bytes:
|
||||
raise ValueError("replay_limit_bytes")
|
||||
line_no += 1
|
||||
total_bytes += len(buffered)
|
||||
yield line_no, bytes(buffered), total_bytes
|
||||
return
|
||||
start = 0
|
||||
while start < len(chunk):
|
||||
newline = chunk.find(b"\n", start)
|
||||
if newline == -1:
|
||||
buffered.extend(chunk[start:])
|
||||
if total_bytes + len(buffered) > max_bytes:
|
||||
raise ValueError("replay_limit_bytes")
|
||||
break
|
||||
buffered.extend(chunk[start : newline + 1])
|
||||
if total_bytes + len(buffered) > max_bytes:
|
||||
raise ValueError("replay_limit_bytes")
|
||||
line_no += 1
|
||||
total_bytes += len(buffered)
|
||||
yield line_no, bytes(buffered), total_bytes
|
||||
buffered.clear()
|
||||
start = newline + 1
|
||||
except FileNotFoundError:
|
||||
return
|
||||
|
||||
|
||||
def append_run_event(
|
||||
session_id: str,
|
||||
run_id: str,
|
||||
@@ -402,11 +145,7 @@ def append_run_event(
|
||||
if not event_name:
|
||||
raise ValueError("event_name is required")
|
||||
with _lock_for(path):
|
||||
if seq is not None:
|
||||
assigned_seq = int(seq)
|
||||
_note_assigned_seq(path, assigned_seq)
|
||||
else:
|
||||
assigned_seq = _reserve_next_seq(path)
|
||||
assigned_seq = int(seq) if seq is not None else _next_seq(path)
|
||||
terminal_state = _terminal_state_for_event(event_name, payload)
|
||||
event = {
|
||||
"version": 1,
|
||||
@@ -430,7 +169,6 @@ def append_run_event(
|
||||
fh.flush()
|
||||
if _should_fsync_event(terminal_state):
|
||||
os.fsync(fh.fileno())
|
||||
_discard_cached_summary(path)
|
||||
if created_file:
|
||||
_fsync_parent_dir(path)
|
||||
return event
|
||||
@@ -445,13 +183,13 @@ class RunJournalWriter:
|
||||
self.session_dir = Path(session_dir) if session_dir is not None else None
|
||||
self._path = _run_path(self.session_id, self.run_id, session_dir=self.session_dir)
|
||||
self._lock = _lock_for(self._path)
|
||||
with self._lock:
|
||||
self._next_seq = _next_seq(self._path)
|
||||
|
||||
def append_sse_event(self, event_name: str, payload=None) -> dict:
|
||||
# Draw from the shared module-level seq cache under the per-path lock so
|
||||
# this writer and any direct append_run_event() call on the same path
|
||||
# agree on one monotonic, gapless sequence.
|
||||
with self._lock:
|
||||
seq = _reserve_next_seq(self._path)
|
||||
seq = self._next_seq
|
||||
self._next_seq += 1
|
||||
return append_run_event(
|
||||
self.session_id,
|
||||
self.run_id,
|
||||
@@ -484,32 +222,14 @@ def read_run_events(
|
||||
}
|
||||
|
||||
|
||||
def select_authoritative_terminal_event(events: Iterable[dict]) -> dict | None:
|
||||
"""Return the terminal event that owns the run's settled outcome.
|
||||
|
||||
``stream_end`` is transport closure, so a preceding semantic terminal event
|
||||
(done, cancel, or error) remains authoritative. Among semantic terminal
|
||||
events, the latest journal row wins.
|
||||
"""
|
||||
terminal_events = [
|
||||
event
|
||||
for event in events
|
||||
if isinstance(event, dict) and event.get("terminal")
|
||||
]
|
||||
return next(
|
||||
(
|
||||
event
|
||||
for event in reversed(terminal_events)
|
||||
if event.get("event") != "stream_end"
|
||||
),
|
||||
terminal_events[-1] if terminal_events else None,
|
||||
)
|
||||
|
||||
|
||||
def _summary_from_events(session_id: str, run_id: str, events: Iterable[dict]) -> dict:
|
||||
ordered = [event for event in events if isinstance(event, dict)]
|
||||
last = ordered[-1] if ordered else None
|
||||
terminal = select_authoritative_terminal_event(ordered)
|
||||
terminal_events = [event for event in ordered if event.get("terminal")]
|
||||
terminal = next(
|
||||
(event for event in reversed(terminal_events) if event.get("event") != "stream_end"),
|
||||
terminal_events[-1] if terminal_events else None,
|
||||
)
|
||||
status = terminal.get("terminal_state") if terminal else ("running" if ordered else "unknown")
|
||||
return {
|
||||
"session_id": str(session_id),
|
||||
@@ -525,49 +245,8 @@ def _summary_from_events(session_id: str, run_id: str, events: Iterable[dict]) -
|
||||
|
||||
|
||||
def latest_run_summary(session_id: str, run_id: str, *, session_dir: Path | None = None) -> dict:
|
||||
path = _run_path(session_id, run_id, session_dir=session_dir)
|
||||
cached = _get_cached_summary(path)
|
||||
if cached is not None:
|
||||
return cached
|
||||
pre_read_signature = _summary_cache_signature(path)
|
||||
events, _malformed = _read_jsonl(path)
|
||||
summary = _summary_from_events(session_id, run_id, events)
|
||||
_cache_summary(path, summary, expected_signature=pre_read_signature)
|
||||
return summary
|
||||
|
||||
|
||||
def session_journal_fingerprint(session_id: str, *, session_dir: Path | None = None) -> tuple[int, float, int]:
|
||||
"""Cheap, bounded fingerprint of a session's run journal: (file_count, max_mtime, total_size).
|
||||
|
||||
Reads only directory + per-file stat metadata (never parses journal bodies), so it stays
|
||||
O(runs) and cannot be tipped over by a large ``done`` row. Used to detect that the journal
|
||||
advanced during an idle live-subscribe wait — a run that starts AND finishes inside a single
|
||||
keepalive tick leaves the journal changed but never materializes a live in-memory stream, so a
|
||||
no-cursor idle subscriber would otherwise miss it until a manual refresh. Returns (0, 0.0, 0)
|
||||
when the session has no journal yet. Invalid ids resolve to the empty fingerprint rather than
|
||||
raising so callers can probe unconditionally.
|
||||
"""
|
||||
try:
|
||||
sid = _validate_id(session_id, "session_id")
|
||||
except ValueError:
|
||||
return (0, 0.0, 0)
|
||||
root = Path(session_dir) if session_dir is not None else _default_session_dir()
|
||||
session_root = root / RUN_JOURNAL_DIR_NAME / sid
|
||||
if not session_root.exists():
|
||||
return (0, 0.0, 0)
|
||||
count = 0
|
||||
max_mtime = 0.0
|
||||
total_size = 0
|
||||
for path in session_root.glob("*.jsonl"):
|
||||
try:
|
||||
st = path.stat()
|
||||
except OSError:
|
||||
continue
|
||||
count += 1
|
||||
total_size += st.st_size
|
||||
if st.st_mtime > max_mtime:
|
||||
max_mtime = st.st_mtime
|
||||
return (count, max_mtime, total_size)
|
||||
journal = read_run_events(session_id, run_id, session_dir=session_dir)
|
||||
return _summary_from_events(session_id, run_id, journal.get("events") or [])
|
||||
|
||||
|
||||
def find_run_summary(run_id: str, *, session_dir: Path | None = None) -> dict | None:
|
||||
@@ -576,133 +255,13 @@ def find_run_summary(run_id: str, *, session_dir: Path | None = None) -> dict |
|
||||
journal_root = root / RUN_JOURNAL_DIR_NAME
|
||||
for path in journal_root.glob(f"*/{rid}.jsonl"):
|
||||
session_id = path.parent.name
|
||||
summary = _get_cached_summary(path)
|
||||
if summary is None:
|
||||
pre_read_signature = _summary_cache_signature(path)
|
||||
events, _malformed = _read_jsonl(path)
|
||||
summary = _summary_from_events(session_id, rid, events)
|
||||
_cache_summary(path, summary, expected_signature=pre_read_signature)
|
||||
events, _malformed = _read_jsonl(path)
|
||||
summary = _summary_from_events(session_id, rid, events)
|
||||
summary["path"] = str(path)
|
||||
return summary
|
||||
return None
|
||||
|
||||
|
||||
def read_session_run_events(
|
||||
session_id: str,
|
||||
*,
|
||||
after_event_id: str | None = None,
|
||||
session_dir: Path | None = None,
|
||||
max_bytes: int = _SESSION_REPLAY_MAX_BYTES,
|
||||
max_rows: int = _SESSION_REPLAY_MAX_ROWS,
|
||||
) -> dict:
|
||||
"""Replay durable run-journal rows for one session after an opaque cursor."""
|
||||
sid = _validate_id(session_id, "session_id")
|
||||
cursor_run_id, cursor_seq = _parse_run_journal_event_id(after_event_id)
|
||||
raw_cursor = str(after_event_id or "").strip()
|
||||
if raw_cursor and cursor_run_id is not None:
|
||||
try:
|
||||
cursor_run_id = _validate_id(cursor_run_id, "run_id")
|
||||
except ValueError:
|
||||
cursor_seq = None
|
||||
if raw_cursor:
|
||||
try:
|
||||
if int(raw_cursor.rsplit(":", 1)[-1]) < 0:
|
||||
cursor_seq = None
|
||||
except (TypeError, ValueError):
|
||||
pass
|
||||
if raw_cursor and (cursor_run_id is None or cursor_seq is None or cursor_seq <= 0):
|
||||
return {
|
||||
"session_id": sid,
|
||||
"cursor_run_id": cursor_run_id,
|
||||
"cursor_seq": cursor_seq,
|
||||
"status": "cursor_invalid",
|
||||
"events": [],
|
||||
}
|
||||
if not raw_cursor:
|
||||
return {
|
||||
"session_id": sid,
|
||||
"cursor_run_id": None,
|
||||
"cursor_seq": None,
|
||||
"status": "ok",
|
||||
"events": [],
|
||||
}
|
||||
root = Path(session_dir) if session_dir is not None else _default_session_dir()
|
||||
session_root = root / RUN_JOURNAL_DIR_NAME / sid
|
||||
runs: list[tuple[float, str, list[dict]]] = []
|
||||
retained_rows = 0
|
||||
retained_bytes = 0
|
||||
for path in sorted(session_root.glob("*.jsonl")) if session_root.exists() else []:
|
||||
run_id = path.stem
|
||||
try:
|
||||
run_id = _validate_id(run_id, "run_id")
|
||||
except ValueError:
|
||||
continue
|
||||
events: list[dict] = []
|
||||
expected_seq = 1
|
||||
try:
|
||||
for _line_no, raw, total_bytes in _iter_bounded_raw_jsonl_lines(
|
||||
path,
|
||||
max_bytes=max_bytes,
|
||||
retained_bytes=retained_bytes,
|
||||
):
|
||||
retained_bytes = total_bytes
|
||||
if not raw.strip():
|
||||
continue
|
||||
try:
|
||||
event = json.loads(raw.decode("utf-8"))
|
||||
seq = int(event.get("seq")) if isinstance(event, dict) else 0
|
||||
except (UnicodeDecodeError, json.JSONDecodeError, TypeError, ValueError):
|
||||
return {"session_id": sid, "cursor_run_id": cursor_run_id, "cursor_seq": cursor_seq, "status": "replay_malformed", "events": []}
|
||||
if (
|
||||
seq != expected_seq
|
||||
or event.get("event_id") != f"{run_id}:{seq}"
|
||||
or event.get("run_id") != run_id
|
||||
or event.get("session_id") != sid
|
||||
):
|
||||
return {"session_id": sid, "cursor_run_id": cursor_run_id, "cursor_seq": cursor_seq, "status": "replay_noncontiguous", "events": []}
|
||||
expected_seq += 1
|
||||
retained_rows += 1
|
||||
if retained_rows > max_rows:
|
||||
return {"session_id": sid, "cursor_run_id": cursor_run_id, "cursor_seq": cursor_seq, "status": "replay_limit_rows", "events": []}
|
||||
events.append(event)
|
||||
except FileNotFoundError:
|
||||
continue
|
||||
except ValueError as exc:
|
||||
if str(exc) == "replay_limit_bytes":
|
||||
return {"session_id": sid, "cursor_run_id": cursor_run_id, "cursor_seq": cursor_seq, "status": "replay_limit_bytes", "events": []}
|
||||
raise
|
||||
created_at = min((_event_created_at(event) for event in events), default=path.stat().st_mtime)
|
||||
runs.append((created_at, run_id, events))
|
||||
runs.sort(key=lambda run: (run[0], run[1]))
|
||||
cursor_index = next((index for index, (_created_at, run_id, _events) in enumerate(runs) if run_id == cursor_run_id), None)
|
||||
if cursor_index is None:
|
||||
foreign_paths = root.joinpath(RUN_JOURNAL_DIR_NAME).glob(f"*/{cursor_run_id}.jsonl") if cursor_run_id else []
|
||||
foreign_session_id = next((path.parent.name for path in foreign_paths if path.parent.name != sid), "")
|
||||
status = "cursor_run_missing"
|
||||
if foreign_session_id:
|
||||
status = "cursor_session_mismatch"
|
||||
return {
|
||||
"session_id": sid,
|
||||
"cursor_run_id": cursor_run_id,
|
||||
"cursor_seq": cursor_seq,
|
||||
"status": status,
|
||||
"events": [],
|
||||
}
|
||||
cursor_events = runs[cursor_index][2]
|
||||
if cursor_seq is None or cursor_seq > len(cursor_events):
|
||||
return {"session_id": sid, "cursor_run_id": cursor_run_id, "cursor_seq": cursor_seq, "status": "cursor_event_missing", "events": []}
|
||||
replay_events = [event for event in cursor_events if event["seq"] > cursor_seq]
|
||||
for _created_at, _run_id, events in runs[cursor_index + 1:]:
|
||||
replay_events.extend(events)
|
||||
return {
|
||||
"session_id": sid,
|
||||
"cursor_run_id": cursor_run_id,
|
||||
"cursor_seq": cursor_seq,
|
||||
"status": "ok",
|
||||
"events": replay_events,
|
||||
}
|
||||
|
||||
|
||||
def delete_run_journal(session_id: str, *, session_dir: Path | None = None) -> bool:
|
||||
"""Remove the entire per-session run-journal directory (``_run_journal/{sid}/``).
|
||||
|
||||
@@ -726,35 +285,7 @@ def delete_run_journal(session_id: str, *, session_dir: Path | None = None) -> b
|
||||
if not session_journal_dir.exists():
|
||||
return False
|
||||
shutil.rmtree(session_journal_dir, ignore_errors=True)
|
||||
removed = not session_journal_dir.exists()
|
||||
# Evict any writer locks the removed runs left behind. `_lock_for` keys are
|
||||
# ``(str(path.parent), path.name, pid)`` and every run file for this session
|
||||
# lives directly under ``session_journal_dir``, so drop all keys whose parent
|
||||
# dir matches — pid-independent — to keep `_WRITER_LOCKS` from growing forever.
|
||||
# Guard on confirmed removal: `rmtree(ignore_errors=True)` can silently leave
|
||||
# the directory (locked files on Windows, permission transients). If the files
|
||||
# still exist their locks are still live — evicting them would hand a later
|
||||
# `_lock_for` caller a brand-new Lock, breaking mutual exclusion with a writer
|
||||
# still holding the old one.
|
||||
if removed:
|
||||
dir_key = str(session_journal_dir)
|
||||
with _WRITER_LOCKS_GUARD:
|
||||
for key in [k for k in _WRITER_LOCKS if k[0] == dir_key]:
|
||||
del _WRITER_LOCKS[key]
|
||||
# Drop cached next-seq entries for the removed runs too. Every run file
|
||||
# for this session lives directly under ``session_journal_dir``, so its
|
||||
# cache key's parent dir matches. Without this, a run re-created at the
|
||||
# same path would resume the stale cached seq instead of restarting at 1.
|
||||
# Hold ``_SEQ_CACHE_LOCK`` — the SAME mutex ``_reserve_next_seq``/
|
||||
# ``_note_assigned_seq`` take — so a concurrent append on another path
|
||||
# cannot mutate the dict mid-iteration (``dictionary changed size``).
|
||||
with _SEQ_CACHE_LOCK:
|
||||
for cache_key in [entry for entry in _SEQ_CACHE if str(Path(entry).parent) == dir_key]:
|
||||
del _SEQ_CACHE[cache_key]
|
||||
with _SUMMARY_CACHE_LOCK:
|
||||
for cache_key in [entry for entry in _SUMMARY_CACHE if str(Path(entry).parent) == dir_key]:
|
||||
del _SUMMARY_CACHE[cache_key]
|
||||
return removed
|
||||
return not session_journal_dir.exists()
|
||||
|
||||
|
||||
def stale_interrupted_event(session_id: str, run_id: str, *, after_seq: int | None = None) -> dict | None:
|
||||
|
||||
@@ -13,11 +13,9 @@ from __future__ import annotations
|
||||
import argparse
|
||||
import json
|
||||
import os
|
||||
import threading
|
||||
import shutil
|
||||
import sqlite3
|
||||
from collections import Counter
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
@@ -103,7 +101,7 @@ def _read_state_db(state_db_path: Path | None) -> dict[str, dict]:
|
||||
if state_db_path is None or not state_db_path.exists():
|
||||
return {}
|
||||
try:
|
||||
with closing(sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True)) as conn:
|
||||
with sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
tables = {row[0] for row in conn.execute("select name from sqlite_master where type='table'")}
|
||||
if "sessions" not in tables:
|
||||
@@ -389,27 +387,9 @@ def audit_session_discoverability(
|
||||
|
||||
|
||||
def _atomic_write_json(path: Path, payload) -> None:
|
||||
# Temp name includes the thread id, not just the pid: WebUI runs a
|
||||
# ThreadingHTTPServer, so two request threads reconciling the same _index.json
|
||||
# would otherwise collide on one `<name>.tmp.<pid>` path and truncate each
|
||||
# other's temp before its os.replace. fsync before the rename so a power loss
|
||||
# can't leave a zero-length/garbage file where the durable JSON should be; a
|
||||
# finally drops the temp if anything fails. Mode matches the prior write_text
|
||||
# (umask default) since the plain open() respects the umask.
|
||||
text = json.dumps(payload, ensure_ascii=False, indent=2)
|
||||
tmp = path.with_name(f".{path.name}.{os.getpid()}.{threading.get_ident()}.tmp")
|
||||
try:
|
||||
with open(tmp, "w", encoding="utf-8") as f:
|
||||
f.write(text)
|
||||
f.flush()
|
||||
os.fsync(f.fileno())
|
||||
os.replace(tmp, path)
|
||||
except BaseException:
|
||||
try:
|
||||
os.unlink(tmp)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
tmp = path.with_suffix(path.suffix + f".tmp.{os.getpid()}")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
os.replace(tmp, path)
|
||||
|
||||
|
||||
def _backup_file(path: Path, backup_dir: Path, backed_up: dict[Path, str]) -> str | None:
|
||||
@@ -502,7 +482,7 @@ def _materialize_sidecar_from_state_db(session_dir: Path, state_db_path: Path |
|
||||
payload = _state_db_row_to_sidecar(row)
|
||||
_backup_file(state_db_path, backup_dir, backed_up)
|
||||
session_dir.mkdir(parents=True, exist_ok=True)
|
||||
tmp = target.with_suffix(target.suffix + f".tmp.{os.getpid()}.{threading.get_ident()}")
|
||||
tmp = target.with_suffix(target.suffix + f".tmp.{os.getpid()}")
|
||||
tmp.write_text(json.dumps(payload, ensure_ascii=False, indent=2), encoding="utf-8")
|
||||
try:
|
||||
os.link(str(tmp), str(target))
|
||||
|
||||
@@ -1,297 +0,0 @@
|
||||
"""Self-contained HTML export for a hermes-webui session transcript.
|
||||
|
||||
Produces a single static HTML file (no external assets, no CDN) that renders
|
||||
the conversation the same way the WebUI does: Markdown, code blocks, tables,
|
||||
lists, blockquotes — all styled with an inlined dark theme matching the app.
|
||||
Double-click the file and it opens in any browser, offline.
|
||||
|
||||
Public entry point: render_session_html(session_dict) -> str
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
import html
|
||||
import re
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
try:
|
||||
from markdown_it import MarkdownIt
|
||||
_MD = (
|
||||
MarkdownIt("commonmark", {"html": False, "linkify": True, "typographer": False})
|
||||
.enable("table")
|
||||
.enable("strikethrough")
|
||||
)
|
||||
except Exception: # pragma: no cover - markdown_it always present in webui venv
|
||||
_MD = None
|
||||
|
||||
|
||||
def _neutralize_remote_images(rendered_html: str) -> str:
|
||||
"""Replace any <img> whose src isn't a data: URI with an inert placeholder.
|
||||
|
||||
This is the single chokepoint that keeps the export self-contained. The
|
||||
multimodal flattening in _content_to_text() handles structured image_url
|
||||
parts, but a *text* message body can carry Markdown image syntax —
|
||||
```` — which markdown_it renders
|
||||
into an active ``<img src="https://...">``. Opening the saved file would
|
||||
then fire a network request and leak a signed/private URL. Filtering here,
|
||||
after rendering, catches every path into the HTML (text Markdown images and
|
||||
any future source) regardless of how the <img> was produced. data: URIs are
|
||||
already embedded, render offline, and make no request, so they're kept.
|
||||
"""
|
||||
if not rendered_html or "<img" not in rendered_html:
|
||||
return rendered_html
|
||||
|
||||
def _repl(match: "re.Match[str]") -> str:
|
||||
tag = match.group(0)
|
||||
src_m = re.search(r'src\s*=\s*"([^"]*)"', tag) or re.search(
|
||||
r"src\s*=\s*'([^']*)'", tag
|
||||
)
|
||||
src = src_m.group(1) if src_m else ""
|
||||
if src.startswith("data:"):
|
||||
return tag # already embedded, offline-safe
|
||||
# Inert placeholder mirroring _content_to_text's remote-image handling.
|
||||
label = html.escape(src) if src else "image"
|
||||
return f"<code>[image: {label}]</code>"
|
||||
|
||||
return re.sub(r"<img\b[^>]*>", _repl, rendered_html, flags=re.IGNORECASE)
|
||||
|
||||
|
||||
def _render_markdown(text: str) -> str:
|
||||
"""Render Markdown to HTML. Falls back to escaped <pre> if the lib is missing."""
|
||||
if not text:
|
||||
return ""
|
||||
if _MD is None:
|
||||
return f"<pre>{html.escape(text)}</pre>"
|
||||
try:
|
||||
return _neutralize_remote_images(_MD.render(text))
|
||||
except Exception:
|
||||
return f"<pre>{html.escape(text)}</pre>"
|
||||
|
||||
|
||||
def _content_to_text(content: Any) -> str:
|
||||
"""Flatten message content (str or multimodal list) into Markdown text."""
|
||||
if isinstance(content, str):
|
||||
return content
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for c in content:
|
||||
if isinstance(c, dict):
|
||||
if c.get("type") == "text" or "text" in c:
|
||||
parts.append(str(c.get("text", "")))
|
||||
elif c.get("type") in ("image_url", "image"):
|
||||
url = ""
|
||||
if isinstance(c.get("image_url"), dict):
|
||||
url = c["image_url"].get("url", "")
|
||||
url = url or c.get("url", "")
|
||||
if url:
|
||||
# Keep the export self-contained and avoid leaking
|
||||
# private/signed URLs: only inline data: URIs (already
|
||||
# embedded, render offline). Remote http(s) images are
|
||||
# NOT rendered as <img> (that would fire a network
|
||||
# request on open) — show an inert placeholder + the
|
||||
# URL as plain text instead.
|
||||
if url.startswith("data:"):
|
||||
parts.append(f"")
|
||||
else:
|
||||
parts.append(f"`[image: {url}]`")
|
||||
else:
|
||||
parts.append(f"`[{c.get('type', 'content')}]`")
|
||||
else:
|
||||
parts.append(str(c))
|
||||
return "\n\n".join(p for p in parts if p)
|
||||
return str(content or "")
|
||||
|
||||
|
||||
def _fmt_ts(t: Any) -> str:
|
||||
try:
|
||||
return time.strftime("%Y-%m-%d %H:%M:%S", time.localtime(float(t)))
|
||||
except Exception:
|
||||
return ""
|
||||
|
||||
|
||||
_ROLE_LABELS = {
|
||||
"user": ("You", "role-user"),
|
||||
"assistant": ("Assistant", "role-assistant"),
|
||||
"system": ("System", "role-system"),
|
||||
"tool": ("Tool", "role-tool"),
|
||||
}
|
||||
|
||||
# Inlined theme matching the WebUI. Light is the default (:root); dark applies
|
||||
# when <html class="dark">. Palettes are copied from the WebUI's style.css gold
|
||||
# theme so the export follows whatever appearance the user has active.
|
||||
_CSS = """
|
||||
:root{--bg:#FEFCF7;--panel:#F3EEE3;--panel2:#FAF7F0;--border:#E0D8C8;
|
||||
--text:#1A1610;--muted:#5C5344;--accent:#B8860B;--user:#0288A8;--assistant:#3D8B40;
|
||||
--code-bg:#F5F0E5;--code-border:#E0D8C8;--code-text:#8b4513;
|
||||
--badge-user-bg:rgba(2,136,168,.12);--badge-user-text:#0288A8;
|
||||
--badge-assistant-bg:rgba(61,139,64,.12);--badge-assistant-text:#3D8B40;
|
||||
--badge-system-bg:rgba(92,83,68,.14);--badge-system-text:#5C5344;
|
||||
--badge-tool-bg:rgba(184,134,11,.14);--badge-tool-text:#8B6508;
|
||||
--row-stripe:rgba(0,0,0,.02);--subtle:rgba(0,0,0,.02);}
|
||||
:root.dark{--bg:#0D0D1A;--panel:#1A1A2E;--panel2:#141425;--border:#2A2A45;
|
||||
--text:#FFF8DC;--muted:#C0C0C0;--accent:#FFD700;--user:#4DD0E1;--assistant:#4CAF50;
|
||||
--code-bg:#1A1A2E;--code-border:#2A2A45;--code-text:#f0c27f;
|
||||
--badge-user-bg:rgba(77,208,225,.16);--badge-user-text:#4DD0E1;
|
||||
--badge-assistant-bg:rgba(76,175,80,.16);--badge-assistant-text:#56d364;
|
||||
--badge-system-bg:rgba(192,192,192,.16);--badge-system-text:#C0C0C0;
|
||||
--badge-tool-bg:rgba(255,191,0,.16);--badge-tool-text:#FFBF00;
|
||||
--row-stripe:rgba(255,255,255,.025);--subtle:rgba(255,255,255,.02);}
|
||||
*{box-sizing:border-box}
|
||||
html{-webkit-text-size-adjust:100%}
|
||||
body{margin:0;background:var(--bg);color:var(--text);
|
||||
font:15px/1.65 -apple-system,BlinkMacSystemFont,"Segoe UI","PingFang SC","Microsoft YaHei",Roboto,Helvetica,Arial,sans-serif;}
|
||||
.wrap{max-width:860px;margin:0 auto;padding:32px 20px 80px;}
|
||||
header.doc-head{border-bottom:1px solid var(--border);padding-bottom:20px;margin-bottom:28px;}
|
||||
header.doc-head h1{margin:0 0 10px;font-size:24px;font-weight:650;}
|
||||
.meta{color:var(--muted);font-size:13px;line-height:1.9;}
|
||||
.meta b{color:var(--text);font-weight:600;}
|
||||
.msg{margin:0 0 22px;border:1px solid var(--border);border-radius:12px;overflow:hidden;background:var(--panel);}
|
||||
.msg-head{display:flex;align-items:center;gap:10px;padding:10px 16px;background:var(--panel2);
|
||||
border-bottom:1px solid var(--border);font-size:13px;}
|
||||
.badge{font-weight:650;padding:2px 10px;border-radius:999px;font-size:12px;letter-spacing:.2px;}
|
||||
.role-user .badge{background:var(--badge-user-bg);color:var(--badge-user-text);}
|
||||
.role-assistant .badge{background:var(--badge-assistant-bg);color:var(--badge-assistant-text);}
|
||||
.role-system .badge{background:var(--badge-system-bg);color:var(--badge-system-text);}
|
||||
.role-tool .badge{background:var(--badge-tool-bg);color:var(--badge-tool-text);}
|
||||
.ts{color:var(--muted);margin-left:auto;font-size:12px;}
|
||||
.msg-body{padding:4px 18px 8px;}
|
||||
.msg-body>:first-child{margin-top:8px}.msg-body>:last-child{margin-bottom:8px}
|
||||
.msg-body p{margin:10px 0;}
|
||||
.msg-body h1,.msg-body h2,.msg-body h3,.msg-body h4{margin:18px 0 10px;line-height:1.3;font-weight:640;}
|
||||
.msg-body h1{font-size:21px}.msg-body h2{font-size:18px}.msg-body h3{font-size:16px}
|
||||
.msg-body a{color:var(--accent);text-decoration:none}.msg-body a:hover{text-decoration:underline}
|
||||
.msg-body ul,.msg-body ol{padding-left:24px;margin:10px 0;}
|
||||
.msg-body li{margin:4px 0;}
|
||||
.msg-body code{background:var(--code-bg);border:1px solid var(--code-border);border-radius:5px;color:var(--code-text);
|
||||
padding:.15em .4em;font-size:.88em;font-family:"SFMono-Regular",Consolas,"Liberation Mono",Menlo,monospace;}
|
||||
.msg-body pre{background:var(--code-bg);border:1px solid var(--code-border);border-radius:10px;
|
||||
padding:14px 16px;overflow:auto;margin:12px 0;}
|
||||
.msg-body pre code{background:none;border:0;padding:0;color:var(--text);font-size:13px;line-height:1.55;}
|
||||
.msg-body blockquote{border-left:3px solid var(--border);margin:12px 0;padding:2px 16px;color:var(--muted);}
|
||||
.msg-body table{border-collapse:collapse;margin:14px 0;width:100%;font-size:14px;display:block;overflow-x:auto;}
|
||||
.msg-body th,.msg-body td{border:1px solid var(--border);padding:8px 12px;text-align:left;}
|
||||
.msg-body th{background:var(--panel2);font-weight:640;}
|
||||
.msg-body tr:nth-child(even) td{background:var(--row-stripe);}
|
||||
.msg-body img{max-width:100%;border-radius:8px;}
|
||||
.msg-body hr{border:0;border-top:1px solid var(--border);margin:18px 0;}
|
||||
details.reasoning{margin:6px 0 4px;border:1px dashed var(--border);border-radius:8px;background:var(--subtle);}
|
||||
details.reasoning summary{cursor:pointer;padding:8px 14px;color:var(--muted);font-size:13px;user-select:none;}
|
||||
details.reasoning[open] summary{border-bottom:1px solid var(--border);}
|
||||
details.reasoning .reasoning-body{padding:4px 16px 10px;color:var(--muted);font-size:13.5px;}
|
||||
footer.doc-foot{margin-top:36px;padding-top:18px;border-top:1px solid var(--border);
|
||||
color:var(--muted);font-size:12px;text-align:center;}
|
||||
"""
|
||||
|
||||
|
||||
def _palette_to_css(palette: dict) -> str:
|
||||
"""Turn a {var-name: value} dict into a `:root{...}` override block.
|
||||
|
||||
Var names may be given with or without the leading `--`. Values are sanitised
|
||||
to a conservative charset (colors, numbers, a few CSS units/functions) so an
|
||||
untrusted palette can't break out of the style block.
|
||||
"""
|
||||
if not isinstance(palette, dict) or not palette:
|
||||
return ""
|
||||
decls = []
|
||||
for raw_name, raw_val in palette.items():
|
||||
name = str(raw_name).strip().lstrip("-")
|
||||
if not name or not re.fullmatch(r"[A-Za-z0-9-]+", name):
|
||||
continue
|
||||
val = str(raw_val).strip()
|
||||
# Allow hex/rgb/hsla colors, numbers, %, px, var(), color-mix(), commas, spaces.
|
||||
if not val or not re.fullmatch(r"[#A-Za-z0-9.,%()\-\s]+", val):
|
||||
continue
|
||||
if len(val) > 120:
|
||||
continue
|
||||
# Reject IE-only expression() — it evaluates JS in older IE and
|
||||
# contradicts the "no active code in <style>" guarantee.
|
||||
if re.search(r"expression\s*\(", val, re.IGNORECASE):
|
||||
continue
|
||||
decls.append(f"--{name}:{val};")
|
||||
if not decls:
|
||||
return ""
|
||||
# Use both :root and :root.dark so the captured palette wins regardless of
|
||||
# theme mode. _CSS defines :root.dark{…} at specificity (0,0,2,0) which
|
||||
# beats a plain :root{…} (0,0,1,0). By emitting both selectors the override
|
||||
# matches dark-mode specificity and source-order ensures it wins in light too.
|
||||
return ":root,:root.dark{" + "".join(decls) + "}"
|
||||
|
||||
|
||||
def render_session_html(session: dict, theme: str = "dark", palette: dict | None = None) -> str:
|
||||
"""Build a complete, self-contained HTML document for the session.
|
||||
|
||||
theme: "dark" or "light" — selects which inlined fallback palette is active.
|
||||
palette: optional {css-var: value} dict captured from the live WebUI
|
||||
(getComputedStyle of :root). When provided it is injected last so the
|
||||
export matches the user's exact active theme AND skin, not just the
|
||||
built-in dark/light fallback.
|
||||
"""
|
||||
html_class = ' class="dark"' if str(theme).lower() != "light" else ""
|
||||
palette_css = _palette_to_css(palette or {})
|
||||
title = (session.get("title") or "Hermes Conversation").strip()
|
||||
sid = session.get("session_id", "")
|
||||
model = session.get("model", "")
|
||||
provider = session.get("model_provider", "")
|
||||
created = _fmt_ts(session.get("created_at"))
|
||||
updated = _fmt_ts(session.get("updated_at"))
|
||||
messages = session.get("messages") or []
|
||||
# Skip system messages by default (usually long boilerplate); keep user/assistant/tool.
|
||||
visible = [m for m in messages if isinstance(m, dict) and m.get("role") != "system"]
|
||||
|
||||
blocks = []
|
||||
for m in visible:
|
||||
role = m.get("role", "")
|
||||
label, cls = _ROLE_LABELS.get(role, (role or "?", "role-system"))
|
||||
ts = _fmt_ts(m.get("timestamp"))
|
||||
ts_html = f'<span class="ts">{html.escape(ts)}</span>' if ts else ""
|
||||
body_md = _content_to_text(m.get("content"))
|
||||
body_html = _render_markdown(body_md)
|
||||
reasoning = m.get("reasoning")
|
||||
reasoning_html = ""
|
||||
if reasoning and isinstance(reasoning, str) and reasoning.strip():
|
||||
reasoning_html = (
|
||||
'<details class="reasoning"><summary>💭 Reasoning</summary>'
|
||||
f'<div class="reasoning-body">{_render_markdown(reasoning)}</div></details>'
|
||||
)
|
||||
blocks.append(
|
||||
f'<section class="msg {cls}">'
|
||||
f'<div class="msg-head"><span class="badge">{html.escape(label)}</span>{ts_html}</div>'
|
||||
f'<div class="msg-body">{reasoning_html}{body_html}</div>'
|
||||
f'</section>'
|
||||
)
|
||||
|
||||
meta_lines = []
|
||||
if sid:
|
||||
meta_lines.append(f"<div>Session: <b>{html.escape(sid)}</b></div>")
|
||||
if model:
|
||||
prov = f" · {html.escape(provider)}" if provider else ""
|
||||
meta_lines.append(f"<div>Model: <b>{html.escape(str(model))}</b>{prov}</div>")
|
||||
if created or updated:
|
||||
meta_lines.append(
|
||||
f"<div>Created: <b>{html.escape(created)}</b> · Updated: <b>{html.escape(updated)}</b></div>"
|
||||
)
|
||||
meta_lines.append(f"<div>Messages: <b>{len(visible)}</b></div>")
|
||||
meta_html = "".join(meta_lines)
|
||||
|
||||
exported = time.strftime("%Y-%m-%d %H:%M:%S")
|
||||
return f"""<!DOCTYPE html>
|
||||
<html lang="en"{html_class}>
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<title>{html.escape(title)}</title>
|
||||
<style>{_CSS}{palette_css}</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="wrap">
|
||||
<header class="doc-head">
|
||||
<h1>{html.escape(title)}</h1>
|
||||
<div class="meta">{meta_html}</div>
|
||||
</header>
|
||||
<main>
|
||||
{''.join(blocks)}
|
||||
</main>
|
||||
<footer class="doc-foot">Exported from Hermes WebUI on {exported}</footer>
|
||||
</div>
|
||||
</body>
|
||||
</html>"""
|
||||
+2
-108
@@ -45,56 +45,6 @@ _condition = threading.Condition(_lock)
|
||||
|
||||
_sessions: dict[str, dict] = {}
|
||||
|
||||
# Background commit threads spawned by fire-and-forget paths (e.g. POST /api/session/new).
|
||||
# Tracked so drain_all_on_shutdown() can join them before interpreter teardown,
|
||||
# preventing silent data loss when daemon threads are abandoned mid-commit.
|
||||
_background_commit_threads: set[threading.Thread] = set()
|
||||
_background_commit_threads_lock = threading.Lock()
|
||||
# Set once drain_all_on_shutdown() begins so no NEW background worker is started
|
||||
# after the drain has snapshotted the registry. A late arrival is not lost: its
|
||||
# uncommitted generation is still committed inline by the generation-drain loop
|
||||
# below (which re-snapshots _sessions each pass and commits with wait=True).
|
||||
_draining = False
|
||||
|
||||
|
||||
def _register_background_commit_thread(t: threading.Thread) -> bool:
|
||||
"""Register a fire-and-forget commit thread. Returns True if the caller
|
||||
should start it. Returns False when shutdown draining has already begun —
|
||||
the caller must NOT start a new worker in that window; the inline
|
||||
generation-drain will commit the pending work instead."""
|
||||
with _background_commit_threads_lock:
|
||||
if _draining:
|
||||
return False
|
||||
_background_commit_threads.add(t)
|
||||
return True
|
||||
|
||||
|
||||
def _unregister_background_commit_thread(t: threading.Thread) -> None:
|
||||
# A completed worker removes itself so the registry does not grow without
|
||||
# bound over the process lifetime; drain only needs threads still running.
|
||||
with _background_commit_threads_lock:
|
||||
_background_commit_threads.discard(t)
|
||||
|
||||
|
||||
def _drain_background_commit_threads(timeout: float = 5.0, deadline_fn=None) -> None:
|
||||
"""Join tracked fire-and-forget commit threads. When ``deadline_fn`` is given
|
||||
(returns the remaining overall shutdown budget in seconds), each join is
|
||||
clamped to that budget and the phase stops early once it is exhausted, so the
|
||||
join phase cannot overrun ``drain_all_on_shutdown``'s deadline."""
|
||||
with _background_commit_threads_lock:
|
||||
threads = list(_background_commit_threads)
|
||||
_background_commit_threads.clear()
|
||||
for t in threads:
|
||||
if not t.is_alive():
|
||||
continue
|
||||
if deadline_fn is None:
|
||||
t.join(timeout)
|
||||
continue
|
||||
remaining = deadline_fn()
|
||||
if remaining <= 0:
|
||||
return
|
||||
t.join(min(timeout, remaining))
|
||||
|
||||
|
||||
def _new_entry() -> dict:
|
||||
return {
|
||||
@@ -275,72 +225,16 @@ def commit_session_memory(session_id: str, agent=None, *, wait: bool = False, ti
|
||||
return True
|
||||
|
||||
|
||||
def drain_all_on_shutdown(deadline: float = 30.0) -> None:
|
||||
# Signal that draining has begun so no NEW background commit worker is
|
||||
# started after this point (see _register_background_commit_thread). Any
|
||||
# already-registered worker is joined below; a caller that loses the race
|
||||
# (registered after our snapshot, or refused a start) still has its pending
|
||||
# generation committed by the inline loop.
|
||||
global _draining
|
||||
with _background_commit_threads_lock:
|
||||
_draining = True
|
||||
|
||||
# Overall wall-clock budget so a stuck worker holding in_flight cannot hang
|
||||
# the shutdown path forever. Start the clock BEFORE the join phase so the
|
||||
# join and the inline drain share one budget. The inline commit runs in a
|
||||
# daemon worker joined only for the remaining budget — otherwise a provider
|
||||
# wedged inside commit_memory_session() (whether we are waiting on an
|
||||
# existing in_flight commit, OR the inline drain owns the segment and calls
|
||||
# the provider directly) would block indefinitely and the deadline check
|
||||
# would never run again.
|
||||
started = time.monotonic()
|
||||
|
||||
def _remaining() -> float:
|
||||
return deadline - (time.monotonic() - started)
|
||||
|
||||
def _commit_within_budget(session_id: str, budget: float) -> bool:
|
||||
"""Run one commit in a daemon worker, join at most `budget` seconds.
|
||||
Returns True only if the commit completed within budget AND made
|
||||
progress. On timeout the daemon worker is left running (it will finish
|
||||
or die at interpreter exit; either way it mutates generation state only
|
||||
under _condition, so nothing is corrupted) and we report no progress so
|
||||
the drain can honor its deadline instead of hanging on a wedged
|
||||
provider."""
|
||||
holder = {"ok": False}
|
||||
|
||||
def _run():
|
||||
holder["ok"] = commit_session_memory(session_id, wait=True)
|
||||
|
||||
worker = threading.Thread(target=_run, daemon=True, name=f"drain-commit-{session_id}")
|
||||
worker.start()
|
||||
worker.join(budget)
|
||||
return holder["ok"] if not worker.is_alive() else False
|
||||
|
||||
# Drain in-flight background commit threads first (fire-and-forget from
|
||||
# POST /api/session/new) so their commit work completes before we drain any
|
||||
# remaining uncommitted generations inline. Each join consumes the shared
|
||||
# budget so it cannot overrun the overall deadline.
|
||||
_drain_background_commit_threads(deadline_fn=_remaining)
|
||||
|
||||
def drain_all_on_shutdown() -> None:
|
||||
while True:
|
||||
with _lock:
|
||||
snapshot = [sid for sid, entry in _sessions.items() if entry["generation"] > entry["committed_generation"]]
|
||||
if not snapshot:
|
||||
return
|
||||
remaining = _remaining()
|
||||
if remaining <= 0:
|
||||
logger.warning("drain_all_on_shutdown: deadline hit with uncommitted sessions: %s", sorted(snapshot))
|
||||
return
|
||||
|
||||
made_progress = False
|
||||
for sid in snapshot:
|
||||
budget = _remaining()
|
||||
if budget <= 0:
|
||||
logger.warning("drain_all_on_shutdown: deadline hit with uncommitted sessions: %s", sorted(snapshot))
|
||||
return
|
||||
# Budget-bounded (covers both the wait-for-in_flight path AND the
|
||||
# unbounded provider call when the inline drain owns the segment).
|
||||
if _commit_within_budget(sid, budget):
|
||||
if commit_session_memory(sid, wait=True):
|
||||
made_progress = True
|
||||
if not made_progress:
|
||||
logger.debug("drain_all_on_shutdown: stopped with uncommitted sessions: %s", sorted(snapshot))
|
||||
|
||||
+2
-897
@@ -6,553 +6,17 @@ Behavior parity reference: gateway/run.py:_handle_*_command in
|
||||
the hermes-agent repo.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
import json
|
||||
import logging
|
||||
import uuid
|
||||
import copy
|
||||
import hashlib
|
||||
import math
|
||||
from dataclasses import dataclass
|
||||
from contextlib import nullcontext
|
||||
from bisect import bisect_left
|
||||
from typing import Any
|
||||
|
||||
from api.config import LOCK, _get_session_agent_lock
|
||||
from api.models import get_session, SESSIONS
|
||||
from api.agent_sessions import normalize_agent_session_source
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
AUTO_TITLE_LABELS = {'untitled', 'new chat'}
|
||||
|
||||
|
||||
class RegenerationUnavailable(Exception):
|
||||
def __init__(self, code: str, status: int = 409, message: str | None = None):
|
||||
super().__init__(message or code)
|
||||
self.code = code
|
||||
self.status = status
|
||||
|
||||
|
||||
def _regeneration_source_class(value):
|
||||
raw = str(value or "").strip().lower()
|
||||
if not raw:
|
||||
return ""
|
||||
if raw == "fork":
|
||||
return "fork"
|
||||
normalized = normalize_agent_session_source(raw).get("session_source")
|
||||
return str(normalized or raw).strip().lower()
|
||||
|
||||
|
||||
def _regeneration_source_allowed(value):
|
||||
return _regeneration_source_class(value) in {"webui", "fork"}
|
||||
|
||||
|
||||
def _selected_regeneration_turn_owned(session, row) -> bool:
|
||||
"""Accept only a final row whose provenance proves WebUI ownership."""
|
||||
if getattr(session, "read_only", False) or not isinstance(row, dict):
|
||||
return False
|
||||
session_source = _regeneration_source_class(getattr(session, "session_source", None))
|
||||
imported_session = bool(
|
||||
getattr(session, "is_cli_session", False)
|
||||
or session_source not in {"", "webui", "fork"}
|
||||
)
|
||||
raw_sources = (
|
||||
getattr(session, "raw_source", None),
|
||||
getattr(session, "source_tag", None),
|
||||
)
|
||||
row_source = row.get("_source") or row.get("source")
|
||||
if session_source == "fork":
|
||||
if any(source and not _regeneration_source_allowed(source) for source in raw_sources):
|
||||
return False
|
||||
if row_source and not _regeneration_source_allowed(row_source):
|
||||
return False
|
||||
return bool(
|
||||
getattr(session, "parent_session_id", None)
|
||||
and row.get("_fork_child_turn") == getattr(session, "session_id", None)
|
||||
)
|
||||
if imported_session:
|
||||
token = row.get("_active_turn_token")
|
||||
if not isinstance(token, str) or not token.strip() or ":" not in token:
|
||||
return False
|
||||
stream_id, started_at = token.rsplit(":", 1)
|
||||
try:
|
||||
started = float(started_at)
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
from api.process_event_utils import build_active_turn_token
|
||||
if not math.isfinite(started) or started <= 0:
|
||||
return False
|
||||
if build_active_turn_token(stream_id, started) != token:
|
||||
return False
|
||||
else:
|
||||
if any(source and not _regeneration_source_allowed(source) for source in raw_sources):
|
||||
return False
|
||||
if row_source and not _regeneration_source_allowed(row_source):
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegenerationTurn:
|
||||
user_index: int
|
||||
assistant_index: int
|
||||
message: dict
|
||||
message_text: str
|
||||
attachments: list
|
||||
source: str
|
||||
message_count: int
|
||||
revision: str
|
||||
row_digest: str
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class RegenerationPlan:
|
||||
canonical_rows: list
|
||||
canonical_context: list
|
||||
turn: RegenerationTurn
|
||||
revision: str
|
||||
row_digest: str
|
||||
message_count: int
|
||||
truncation_boundary: int
|
||||
|
||||
|
||||
def plan_regeneration(session, *, expected_revision=None, lock_held=False):
|
||||
"""Prepare one canonical display/context pair for a locked regeneration."""
|
||||
lock_context = nullcontext() if lock_held else _get_session_agent_lock(session.session_id)
|
||||
with lock_context:
|
||||
rows, context = regeneration_state(session, use_sidecar=True)
|
||||
revision = regeneration_revision_for(rows, session=session, context=context)
|
||||
if expected_revision is not None and expected_revision != revision:
|
||||
raise RegenerationUnavailable("stale_regeneration_revision")
|
||||
turn = resolve_regeneration_turn(
|
||||
rows, session=session, expected_revision=revision,
|
||||
lock_held=True, context=context,
|
||||
)
|
||||
return RegenerationPlan(
|
||||
canonical_rows=copy.deepcopy(rows),
|
||||
canonical_context=copy.deepcopy(context),
|
||||
turn=turn,
|
||||
revision=revision,
|
||||
row_digest=turn.row_digest,
|
||||
message_count=len(rows),
|
||||
truncation_boundary=turn.user_index + 1,
|
||||
)
|
||||
|
||||
|
||||
def apply_regeneration_plan(
|
||||
session,
|
||||
plan: RegenerationPlan,
|
||||
*,
|
||||
return_context_user: bool = False,
|
||||
):
|
||||
"""Install the prepared pair and truncate it without a second authority read."""
|
||||
def _result(success, context_user=None):
|
||||
return (success, context_user) if return_context_user else success
|
||||
|
||||
if not isinstance(plan, RegenerationPlan):
|
||||
return _result(False)
|
||||
rows = copy.deepcopy(plan.canonical_rows)
|
||||
context = copy.deepcopy(plan.canonical_context)
|
||||
if len(rows) != plan.message_count or plan.truncation_boundary != plan.turn.user_index + 1:
|
||||
return _result(False)
|
||||
if regeneration_revision_for(rows, session=session, context=context) != plan.revision:
|
||||
return _result(False)
|
||||
session.messages = rows
|
||||
session.context_messages = context
|
||||
current = session.messages[plan.turn.user_index]
|
||||
if not isinstance(current, dict) or current.get("role") != "user":
|
||||
return _result(False)
|
||||
truncate_session_at_keep(session, plan.truncation_boundary)
|
||||
prepared_context, context_boundary_index = truncate_context_for_display_keep(
|
||||
context,
|
||||
rows,
|
||||
plan.truncation_boundary,
|
||||
return_boundary_index=True,
|
||||
)
|
||||
session.context_messages = prepared_context if prepared_context or not context else context[: plan.truncation_boundary]
|
||||
retained_context_user = None
|
||||
if context_boundary_index is not None:
|
||||
for context_row in reversed(session.context_messages[: context_boundary_index + 1]):
|
||||
if isinstance(context_row, dict) and context_row.get("role") == "user":
|
||||
retained_context_user = context_row
|
||||
break
|
||||
return _result(True, retained_context_user)
|
||||
|
||||
|
||||
def snapshot_regeneration_state(session):
|
||||
return copy.deepcopy(session.__dict__)
|
||||
|
||||
|
||||
def restore_regeneration_state(session, snapshot):
|
||||
session.__dict__.clear()
|
||||
session.__dict__.update(copy.deepcopy(snapshot))
|
||||
|
||||
|
||||
def regeneration_revision_for(rows, *, session=None, context=None) -> str:
|
||||
"""Hash the canonical writable transcript and its aligned context."""
|
||||
payload = json.dumps(
|
||||
{
|
||||
"session_id": str(getattr(session, "session_id", "") or "") if session is not None else "",
|
||||
"messages": list(rows or []),
|
||||
"context_messages": list(context or []),
|
||||
"truncation_watermark": getattr(session, "truncation_watermark", None) if session is not None else None,
|
||||
"truncation_boundary": getattr(session, "truncation_boundary", None) if session is not None else None,
|
||||
},
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
)
|
||||
return hashlib.sha256(payload.encode("utf-8")).hexdigest()
|
||||
|
||||
|
||||
def regeneration_transcript(session, *, state_messages=None):
|
||||
"""Return the state.db-reconciled transcript used by every authority consumer."""
|
||||
if state_messages is None:
|
||||
return regeneration_state(session)[0]
|
||||
from api.models import reconciled_state_db_messages_for_session
|
||||
return reconciled_state_db_messages_for_session(session, state_messages=state_messages)
|
||||
|
||||
|
||||
def regeneration_context(session):
|
||||
return regeneration_state(session)[1]
|
||||
|
||||
|
||||
_REGENERATION_SIDECAR_ANCHOR_BUDGET = 200
|
||||
|
||||
|
||||
def _sidecar_regeneration_read_floor(session):
|
||||
"""Return a state.db tail-read floor anchored by the already-loaded sidecar.
|
||||
|
||||
#6826: regenerating a large session must not re-materialize the full
|
||||
state.db transcript (a >1min stall on big sessions). When the in-memory
|
||||
sidecar is a usable reconciliation base — append-only session with no
|
||||
active truncation markers and timestamped rows — return the timestamp
|
||||
floor for a bounded ``since_timestamp`` tail read. Rows at/after the
|
||||
floor (including any gateway/server-applied tail the sidecar has not seen
|
||||
yet) are re-read and merged, and rows the sidecar already carries are
|
||||
deduplicated by the append-only merge, so the #6611 reconciliation
|
||||
authority is preserved on the fast path.
|
||||
|
||||
Returns ``None`` when the sidecar cannot anchor a tail read; callers then
|
||||
fall back to the full state.db read (unchanged behavior).
|
||||
"""
|
||||
if getattr(session, "truncation_watermark", None) not in (None, ""):
|
||||
return None
|
||||
if getattr(session, "truncation_boundary", None) not in (None, ""):
|
||||
return None
|
||||
messages = getattr(session, "messages", None)
|
||||
if not isinstance(messages, list) or not messages:
|
||||
return None
|
||||
from api.models import _message_timestamp_as_float
|
||||
|
||||
timestamps = [_message_timestamp_as_float(message) for message in messages]
|
||||
if any(timestamp is None for timestamp in timestamps):
|
||||
return None
|
||||
# Conservative anchor: re-read a bounded tip window so sub-second/clock
|
||||
# drift near the sidecar tip cannot hide a concurrently appended state.db
|
||||
# row, while the raw read stays tiny for huge sessions.
|
||||
return min(timestamps[-_REGENERATION_SIDECAR_ANCHOR_BUDGET:])
|
||||
|
||||
|
||||
def _bounded_tail_snapshot_if_safe(session, read_floor):
|
||||
"""Return the bounded tail rows ONLY when it is provably identical to the
|
||||
full read; otherwise None (caller must fall back to the full read).
|
||||
|
||||
#6826 r3: the skipped state.db prefix (rows older than the floor) must be
|
||||
represented identically in the sidecar — same count AND same ordered
|
||||
visible identity — and the bounded tail must not repeat any skipped key
|
||||
(occurrence-count collision: a new tail turn repeating an older prompt
|
||||
would be mistaken for the old sidecar duplicate and dropped). The prefix
|
||||
proof and the tail data come from ONE read transaction (no TOCTOU).
|
||||
|
||||
Any mismatch, missing database, or uncertainty returns None, so the #6611
|
||||
regeneration authority never operates on an unreconciled view.
|
||||
"""
|
||||
sid = getattr(session, "session_id", None)
|
||||
if not sid:
|
||||
return None
|
||||
profile = getattr(session, "profile", None)
|
||||
from api.models import (
|
||||
_session_message_visible_key,
|
||||
get_state_db_regeneration_tail_snapshot,
|
||||
)
|
||||
|
||||
snap = get_state_db_regeneration_tail_snapshot(sid, read_floor, profile=profile)
|
||||
if snap is None:
|
||||
return None # cannot obtain a stable single-snapshot → full read
|
||||
# Compression-anchor coverage: if the anchor predates the floor the bounded
|
||||
# read can drop compacted-tail context rows (display may still match).
|
||||
anchor = getattr(session, "compression_anchor_message_key", None)
|
||||
if isinstance(anchor, dict):
|
||||
try:
|
||||
anchor_ts = float(anchor.get("ts"))
|
||||
except (TypeError, ValueError):
|
||||
anchor_ts = None
|
||||
if anchor_ts is None or anchor_ts < read_floor:
|
||||
return None
|
||||
prefix = snap["prefix"]
|
||||
if prefix.get("count") == 0 and prefix.get("null_timestamp_count") == 0:
|
||||
# Empty skipped prefix: the bounded read already covers every row.
|
||||
return snap["tail"]
|
||||
# Non-empty skipped prefix: prove identical ordered visible identity.
|
||||
sidecar_keys = []
|
||||
for message in getattr(session, "messages", None) or []:
|
||||
if not isinstance(message, dict):
|
||||
continue
|
||||
try:
|
||||
ts = float(message.get("timestamp"))
|
||||
except (TypeError, ValueError):
|
||||
ts = None
|
||||
if ts is not None and ts < read_floor:
|
||||
key = _session_message_visible_key(message)
|
||||
if key is None:
|
||||
return None
|
||||
sidecar_keys.append(key)
|
||||
if list(snap["prefix_keys"]) != sidecar_keys:
|
||||
return None # mismatch → full read
|
||||
# Occurrence-count collision (#6826 r3 #1): if any bounded-tail key ALSO
|
||||
# occurs in the skipped prefix, the reconciler may drop the repeated tail
|
||||
# row — fall back conservatively.
|
||||
prefix_key_set = set(snap["prefix_keys"])
|
||||
for key in snap["tail_keys"]:
|
||||
if key in prefix_key_set:
|
||||
return None
|
||||
# In-tail duplicates (#6826 r5): a repeated message wholly inside the
|
||||
# bounded tail makes the reconciler's context dedup diverge from the full
|
||||
# read (display may still match) → refuse the bounded path.
|
||||
if len(snap["tail_keys"]) != len(set(snap["tail_keys"])):
|
||||
return None
|
||||
return snap["tail"]
|
||||
|
||||
|
||||
def regeneration_state(session, *, use_sidecar=False):
|
||||
"""Read one immutable state.db snapshot and reconcile both transcript views.
|
||||
|
||||
``use_sidecar=True`` (#6826) anchors the state.db read to the already
|
||||
loaded in-memory sidecar: only a bounded tail (``since_timestamp`` floor)
|
||||
is re-read instead of the full transcript, and both views still route
|
||||
through :func:`reconciled_state_db_messages_for_session`, so the #6611
|
||||
reconciliation authority (recovered display/context pair survives local
|
||||
and gateway apply) is preserved on the fast path.
|
||||
|
||||
The bounded tail is only trusted when
|
||||
:func:`_bounded_tail_snapshot_if_safe` proves the skipped state.db prefix
|
||||
is identical in the sidecar (count + ordered visible identity + no
|
||||
occurrence collision + compression anchor coverage), and the tail rows
|
||||
come from the SAME single read transaction as the proof (no TOCTOU);
|
||||
otherwise the read falls back to the full transcript.
|
||||
"""
|
||||
from api.models import (
|
||||
get_state_db_session_messages,
|
||||
reconciled_state_db_messages_for_session,
|
||||
)
|
||||
|
||||
bounded_tail = None
|
||||
if use_sidecar:
|
||||
read_floor = _sidecar_regeneration_read_floor(session)
|
||||
if read_floor is not None:
|
||||
bounded_tail = _bounded_tail_snapshot_if_safe(session, read_floor)
|
||||
if bounded_tail is not None:
|
||||
state_messages = bounded_tail
|
||||
else:
|
||||
state_messages = get_state_db_session_messages(
|
||||
getattr(session, "session_id", None),
|
||||
profile=getattr(session, "profile", None),
|
||||
)
|
||||
return (
|
||||
reconciled_state_db_messages_for_session(
|
||||
session,
|
||||
state_messages=state_messages,
|
||||
),
|
||||
reconciled_state_db_messages_for_session(
|
||||
session,
|
||||
prefer_context=True,
|
||||
state_messages=state_messages,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def regeneration_revision(session) -> str:
|
||||
rows, context = regeneration_state(session, use_sidecar=True)
|
||||
return regeneration_revision_for(
|
||||
rows,
|
||||
session=session,
|
||||
context=context,
|
||||
)
|
||||
|
||||
|
||||
def regeneration_authority(
|
||||
session,
|
||||
rows=None,
|
||||
*,
|
||||
context=None,
|
||||
full_transcript=True,
|
||||
canonical_state=None,
|
||||
):
|
||||
"""Mint a revision only for a complete, writable, canonical transcript."""
|
||||
if not full_transcript:
|
||||
return None
|
||||
if getattr(session, "active_stream_id", None) or getattr(session, "pending_user_message", None):
|
||||
return None
|
||||
canonical_rows, canonical_context = canonical_state or regeneration_state(session)
|
||||
rows = list(canonical_rows if rows is None else rows)
|
||||
if not rows:
|
||||
return None
|
||||
if rows != canonical_rows:
|
||||
return None
|
||||
if context is not None and list(context or []) != canonical_context:
|
||||
return None
|
||||
try:
|
||||
resolve_regeneration_turn(
|
||||
canonical_rows,
|
||||
session=session,
|
||||
context=canonical_context,
|
||||
)
|
||||
except RegenerationUnavailable:
|
||||
return None
|
||||
return regeneration_revision_for(
|
||||
canonical_rows,
|
||||
session=session,
|
||||
context=canonical_context,
|
||||
)
|
||||
|
||||
|
||||
def resolve_regeneration_turn(
|
||||
rows,
|
||||
*,
|
||||
session=None,
|
||||
expected_revision=None,
|
||||
lock_held=False,
|
||||
context=None,
|
||||
):
|
||||
"""Select the current session's final complete local exchange under its lock."""
|
||||
legacy_session_call = session is None and not isinstance(rows, (list, tuple))
|
||||
legacy_context = None
|
||||
if legacy_session_call:
|
||||
session = rows
|
||||
rows, legacy_context = regeneration_state(session)
|
||||
lock_context = (
|
||||
_get_session_agent_lock(session.session_id)
|
||||
if legacy_session_call and not lock_held
|
||||
else nullcontext()
|
||||
)
|
||||
with lock_context:
|
||||
rows = list(rows or [])
|
||||
if context is None:
|
||||
context = legacy_context
|
||||
if context is None:
|
||||
_, context = regeneration_state(session)
|
||||
context = list(context)
|
||||
revision = regeneration_revision_for(rows, session=session, context=context)
|
||||
if expected_revision is not None and expected_revision != revision:
|
||||
raise RegenerationUnavailable("stale_regeneration_revision")
|
||||
if getattr(session, "active_stream_id", None):
|
||||
raise RegenerationUnavailable("session_active")
|
||||
if getattr(session, "pending_user_message", None):
|
||||
raise RegenerationUnavailable("session_active")
|
||||
assistant_index = next(
|
||||
(
|
||||
index
|
||||
for index in range(len(rows) - 1, -1, -1)
|
||||
if isinstance(rows[index], dict)
|
||||
and rows[index].get("role") == "assistant"
|
||||
and _assistant_message_has_final_visible_text(rows[index])
|
||||
),
|
||||
None,
|
||||
)
|
||||
if assistant_index is not None:
|
||||
index = next(
|
||||
(
|
||||
candidate
|
||||
for candidate in range(assistant_index - 1, -1, -1)
|
||||
if isinstance(rows[candidate], dict)
|
||||
and rows[candidate].get("role") == "user"
|
||||
),
|
||||
None,
|
||||
)
|
||||
else:
|
||||
index = None
|
||||
if index is not None:
|
||||
if any(
|
||||
isinstance(row, dict) and row.get("role") == "user"
|
||||
for row in rows[assistant_index + 1:]
|
||||
):
|
||||
raise RegenerationUnavailable("no_regenerable_turn", 400)
|
||||
if any(
|
||||
isinstance(row, dict) and row.get("role") in {"assistant", "tool"}
|
||||
for row in rows[assistant_index + 1:]
|
||||
):
|
||||
raise RegenerationUnavailable("no_regenerable_turn", 400)
|
||||
row = rows[index]
|
||||
if not _selected_regeneration_turn_owned(session, row):
|
||||
raise RegenerationUnavailable("regeneration_read_only", 403)
|
||||
content = _extract_text(row.get("content", ""))
|
||||
if content:
|
||||
row_digest = hashlib.sha256(
|
||||
json.dumps(
|
||||
row,
|
||||
sort_keys=True,
|
||||
separators=(",", ":"),
|
||||
default=str,
|
||||
).encode("utf-8")
|
||||
).hexdigest()
|
||||
return RegenerationTurn(
|
||||
index,
|
||||
assistant_index,
|
||||
copy.deepcopy(row),
|
||||
content,
|
||||
copy.deepcopy(row.get("attachments") or []),
|
||||
str(row.get("_source") or "webui"),
|
||||
len(rows),
|
||||
revision,
|
||||
row_digest,
|
||||
)
|
||||
raise RegenerationUnavailable("no_regenerable_turn", 400)
|
||||
|
||||
|
||||
def _assistant_message_has_final_visible_text(message) -> bool:
|
||||
from api.streaming import _assistant_message_has_final_visible_text as _has_final_text
|
||||
|
||||
return _has_final_text(message)
|
||||
|
||||
|
||||
def _live_active_stream_id(session) -> str | None:
|
||||
"""Return session.active_stream_id ONLY if that stream is live in THIS
|
||||
process; else None.
|
||||
|
||||
After a restart/crash the persisted active_stream_id survives in the
|
||||
session JSON but the in-memory STREAMS / ACTIVE_RUNS that actually drive a
|
||||
live turn were wiped. Exposing that dead id (e.g. via /api/session/status to
|
||||
the hidden-tab poller) would make a client attach its renderer to a stream
|
||||
that never emits — a permanent fake "thinking" state. Liveness test mirrors
|
||||
routes._clear_stale_stream_state: live iff present in STREAMS (open SSE
|
||||
channel) or ACTIVE_RUNS (worker bookkeeping) — except that a
|
||||
``phase="cancelling"`` run is excluded on both paths, because the worker may
|
||||
still be unwinding while the client already reached a terminal state for
|
||||
that stream.
|
||||
"""
|
||||
stream_id = getattr(session, 'active_stream_id', None)
|
||||
if not stream_id:
|
||||
return None
|
||||
try:
|
||||
from api import config as _cfg
|
||||
with _cfg.ACTIVE_RUNS_LOCK:
|
||||
_active_run_present = stream_id in (_cfg.ACTIVE_RUNS or {})
|
||||
_active_run_entry = (_cfg.ACTIVE_RUNS or {}).get(stream_id)
|
||||
if _active_run_present and not _cfg.active_run_is_attachable(_active_run_entry):
|
||||
return None
|
||||
with _cfg.STREAMS_LOCK:
|
||||
if stream_id in _cfg.STREAMS:
|
||||
return stream_id
|
||||
if _active_run_present:
|
||||
return stream_id
|
||||
except Exception:
|
||||
# On any introspection failure, fail SAFE (report no live stream) rather
|
||||
# than surfacing a possibly-stale id.
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def session_has_manual_title(session) -> bool:
|
||||
"""Return whether adaptive title refresh should leave this title alone."""
|
||||
return getattr(session, 'manual_title', False) is True
|
||||
@@ -603,336 +67,6 @@ def _truncation_watermark_for(messages):
|
||||
return 0.0
|
||||
|
||||
|
||||
def _stamp_intentional_shrink_generation(session, old_message_count: int, new_message_count: int) -> bool:
|
||||
"""Stamp a new generation only when the visible message list shrinks."""
|
||||
if new_message_count >= old_message_count:
|
||||
return False
|
||||
session.intentional_shrink_generation = uuid.uuid4().hex
|
||||
return True
|
||||
|
||||
|
||||
def truncate_context_for_display_keep(
|
||||
context_messages: list | None,
|
||||
full_messages: list | None,
|
||||
keep: int,
|
||||
*,
|
||||
return_boundary_index: bool = False,
|
||||
) -> list:
|
||||
"""Align model context with display prefix ``full_messages[:keep]``."""
|
||||
def _result(rows, boundary_index=None):
|
||||
return (rows, boundary_index) if return_boundary_index else rows
|
||||
|
||||
if keep <= 0:
|
||||
return _result([])
|
||||
ctx = context_messages if isinstance(context_messages, list) else []
|
||||
msgs = full_messages if isinstance(full_messages, list) else []
|
||||
if not ctx:
|
||||
return _result([])
|
||||
if len(msgs) == 0:
|
||||
return _result([])
|
||||
# Only the perfectly-parallel case (display and context row-for-row) can be
|
||||
# sliced at the raw display index. When the two arrays differ in length —
|
||||
# in EITHER direction — they have diverged and need alignment:
|
||||
# * context LONGER than display → an injected summary/system prefix, etc.
|
||||
# * context SHORTER than display → large-session context trimming dropped
|
||||
# turns from the model context that the display still shows.
|
||||
# The shorter-context case is the one that broke forked large sessions: the
|
||||
# old ``len(ctx) <= len(msgs)`` guard short-circuited to ``ctx[:keep]``,
|
||||
# slicing the shorter context at the display index (landing mid-turn, e.g.
|
||||
# on an assistant tool_call whose result was past the cut). Fall through to
|
||||
# the signature matcher for both divergent cases so the cut lands on a real
|
||||
# turn boundary. Any residual dangling tool_use in the persisted context is
|
||||
# made wire-safe on the send path (streaming: ``_sanitize_messages_for_api``
|
||||
# strips unanswered tool_calls; gateway: it forwards no tool_calls/tool rows
|
||||
# at all), so we do not re-do that trimming here.
|
||||
if len(ctx) == len(msgs):
|
||||
return _result(ctx[:keep], min(keep, len(ctx)) - 1)
|
||||
|
||||
def _row_signature(row: Any) -> tuple[str, ...] | None:
|
||||
if not isinstance(row, dict):
|
||||
return None
|
||||
tool_calls = row.get('tool_calls')
|
||||
tool_calls_sig = json.dumps(tool_calls, sort_keys=True, default=str) if tool_calls else ''
|
||||
return (
|
||||
str(row.get('role') or ''),
|
||||
str(row.get('content') or ''),
|
||||
str(row.get('tool_call_id') or ''),
|
||||
str(row.get('tool_use_id') or ''),
|
||||
str(row.get('tool_name') or row.get('name') or ''),
|
||||
tool_calls_sig,
|
||||
)
|
||||
|
||||
# Materialize signatures once. The matcher deliberately keeps the original
|
||||
# rows in ``ctx``; these records are only an alignment index. A signature
|
||||
# failure is deferred because the old matcher may return before reaching it.
|
||||
context_records = []
|
||||
deferred_signature_positions: list[int] = []
|
||||
for idx, row in enumerate(ctx):
|
||||
try:
|
||||
row_signature = _row_signature(row)
|
||||
except Exception:
|
||||
row_signature = None
|
||||
deferred_signature_positions.append(idx)
|
||||
context_records.append((row, row_signature))
|
||||
message_signatures = [_row_signature(message) for message in msgs]
|
||||
id_positions: dict[Any, list[int]] = {}
|
||||
signature_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
signature_no_id_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
signature_no_timestamp_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
signature_no_id_no_timestamp_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
signature_timestamp_positions: dict[
|
||||
tuple[tuple[str, ...], Any], list[int]
|
||||
] = {}
|
||||
signature_timestamp_no_id_positions: dict[
|
||||
tuple[tuple[str, ...], Any], list[int]
|
||||
] = {}
|
||||
unsafe_id_positions: list[int] = []
|
||||
unsafe_timestamp_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
unsafe_timestamp_no_id_positions: dict[tuple[str, ...], list[int]] = {}
|
||||
|
||||
def _safe_raw_value(value: Any) -> bool:
|
||||
# Keep dict lookup semantics aligned with the old explicit ``==`` scan:
|
||||
# only ordinary built-in metadata values may use the raw-value indexes.
|
||||
# In particular, custom objects and non-reflexive NaN values can make a
|
||||
# dict find a key that the old equality check rejected.
|
||||
if value is None:
|
||||
return True
|
||||
if type(value) not in (str, int, float):
|
||||
return False
|
||||
try:
|
||||
hash(value)
|
||||
return value == value
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
for idx, (context_row, context_sig) in enumerate(context_records):
|
||||
if context_sig is not None:
|
||||
signature_positions.setdefault(context_sig, []).append(idx)
|
||||
if not isinstance(context_row, dict):
|
||||
continue
|
||||
context_id = context_row.get('id')
|
||||
context_ts = context_row.get('timestamp')
|
||||
if context_sig is not None:
|
||||
if context_id is None:
|
||||
signature_no_id_positions.setdefault(context_sig, []).append(idx)
|
||||
if context_ts is None:
|
||||
signature_no_timestamp_positions.setdefault(context_sig, []).append(idx)
|
||||
if context_id is None and context_ts is None:
|
||||
signature_no_id_no_timestamp_positions.setdefault(
|
||||
context_sig, []
|
||||
).append(idx)
|
||||
if context_id is not None and _safe_raw_value(context_id):
|
||||
id_positions.setdefault(context_id, []).append(idx)
|
||||
elif context_id is not None:
|
||||
unsafe_id_positions.append(idx)
|
||||
if (
|
||||
context_sig is not None
|
||||
and context_ts is not None
|
||||
and _safe_raw_value(context_ts)
|
||||
):
|
||||
timestamp_key = (context_sig, context_ts)
|
||||
signature_timestamp_positions.setdefault(timestamp_key, []).append(idx)
|
||||
if context_id is None:
|
||||
signature_timestamp_no_id_positions.setdefault(
|
||||
timestamp_key, []
|
||||
).append(idx)
|
||||
elif context_sig is not None and context_ts is not None:
|
||||
unsafe_timestamp_positions.setdefault(context_sig, []).append(idx)
|
||||
if context_id is None:
|
||||
unsafe_timestamp_no_id_positions.setdefault(
|
||||
context_sig, []
|
||||
).append(idx)
|
||||
|
||||
def _first_at_or_after(positions: list[int] | None, start_idx: int) -> int | None:
|
||||
if not positions:
|
||||
return None
|
||||
offset = bisect_left(positions, start_idx)
|
||||
return positions[offset] if offset < len(positions) else None
|
||||
|
||||
def _lazy_first_match_from(
|
||||
message: Any,
|
||||
start_idx: int,
|
||||
) -> tuple[int | None, int | None]:
|
||||
"""Match exactly as the original ordered scan did."""
|
||||
msg_sig = _row_signature(message)
|
||||
if msg_sig is None:
|
||||
return None, None
|
||||
weak_matches: list[int] = []
|
||||
for idx in range(start_idx, len(ctx)):
|
||||
context_row = ctx[idx]
|
||||
context_sig = _row_signature(context_row)
|
||||
if context_sig is None or not isinstance(context_row, dict):
|
||||
continue
|
||||
context_id = context_row.get('id')
|
||||
msg_id = message.get('id')
|
||||
if context_id is not None and msg_id is not None:
|
||||
if context_id == msg_id:
|
||||
return idx, None
|
||||
continue
|
||||
if context_sig != msg_sig:
|
||||
continue
|
||||
context_ts = context_row.get('timestamp')
|
||||
msg_ts = message.get('timestamp')
|
||||
if context_ts is not None and msg_ts is not None:
|
||||
if context_ts == msg_ts:
|
||||
return idx, None
|
||||
continue
|
||||
weak_matches.append(idx)
|
||||
if len(weak_matches) > 1:
|
||||
return None, weak_matches[0]
|
||||
return (weak_matches[0], None) if len(weak_matches) == 1 else (None, None)
|
||||
|
||||
def _first_match_from(
|
||||
message_idx: int,
|
||||
message: Any,
|
||||
start_idx: int,
|
||||
) -> tuple[int | None, int | None]:
|
||||
msg_sig = message_signatures[message_idx]
|
||||
if msg_sig is None:
|
||||
return None, None
|
||||
msg_id = message.get('id')
|
||||
msg_ts = message.get('timestamp')
|
||||
deferred_reachable = _first_at_or_after(
|
||||
deferred_signature_positions, start_idx
|
||||
) is not None
|
||||
unsafe_id_reachable = (
|
||||
msg_id is not None
|
||||
and _first_at_or_after(unsafe_id_positions, start_idx) is not None
|
||||
)
|
||||
unsafe_timestamp_candidates = (
|
||||
unsafe_timestamp_no_id_positions.get(msg_sig, [])
|
||||
if msg_id is not None
|
||||
else unsafe_timestamp_positions.get(msg_sig, [])
|
||||
)
|
||||
unsafe_timestamp_reachable = (
|
||||
msg_ts is not None
|
||||
and _first_at_or_after(unsafe_timestamp_candidates, start_idx) is not None
|
||||
)
|
||||
if not _safe_raw_value(msg_id) or not _safe_raw_value(msg_ts):
|
||||
return _lazy_first_match_from(message, start_idx)
|
||||
if deferred_reachable or unsafe_id_reachable or unsafe_timestamp_reachable:
|
||||
return _lazy_first_match_from(message, start_idx)
|
||||
|
||||
exact_positions: list[int] = []
|
||||
if msg_id is not None:
|
||||
id_idx = _first_at_or_after(id_positions.get(msg_id), start_idx)
|
||||
if id_idx is not None:
|
||||
exact_positions.append(id_idx)
|
||||
if msg_ts is not None:
|
||||
timestamp_key = (msg_sig, msg_ts)
|
||||
if msg_id is not None:
|
||||
timestamp_positions = signature_timestamp_no_id_positions.get(
|
||||
timestamp_key, []
|
||||
)
|
||||
else:
|
||||
timestamp_positions = signature_timestamp_positions.get(
|
||||
timestamp_key, []
|
||||
)
|
||||
timestamp_idx = _first_at_or_after(timestamp_positions, start_idx)
|
||||
if timestamp_idx is not None:
|
||||
exact_positions.append(timestamp_idx)
|
||||
exact_idx = min(exact_positions, default=None)
|
||||
|
||||
if msg_id is not None and msg_ts is not None:
|
||||
weak_candidates = signature_no_id_no_timestamp_positions.get(msg_sig)
|
||||
elif msg_id is not None:
|
||||
weak_candidates = signature_no_id_positions.get(msg_sig)
|
||||
elif msg_ts is not None:
|
||||
weak_candidates = signature_no_timestamp_positions.get(msg_sig)
|
||||
else:
|
||||
weak_candidates = signature_positions.get(msg_sig)
|
||||
weak_start = bisect_left(weak_candidates, start_idx) if weak_candidates else 0
|
||||
weak_positions = weak_candidates[weak_start:weak_start + 2] if weak_candidates else []
|
||||
second_weak_idx = weak_positions[1] if len(weak_positions) > 1 else None
|
||||
if second_weak_idx is not None and (
|
||||
exact_idx is None or second_weak_idx < exact_idx
|
||||
):
|
||||
return None, weak_positions[0]
|
||||
if exact_idx is not None:
|
||||
return exact_idx, None
|
||||
return (weak_positions[0], None) if len(weak_positions) == 1 else (None, None)
|
||||
|
||||
matches = [None] * len(msgs)
|
||||
ambiguous_matches = [None] * len(msgs)
|
||||
next_ctx_idx = 0
|
||||
for msg_idx, message in enumerate(msgs):
|
||||
match_idx, ambiguous_idx = _first_match_from(msg_idx, message, next_ctx_idx)
|
||||
matches[msg_idx] = match_idx
|
||||
ambiguous_matches[msg_idx] = ambiguous_idx
|
||||
if match_idx is not None:
|
||||
next_ctx_idx = match_idx + 1
|
||||
|
||||
# Cut at the first unkept display turn, or fallback to the last kept turn
|
||||
# if the boundary is not directly alignable.
|
||||
if keep < len(msgs):
|
||||
last_kept = None
|
||||
if keep > 0:
|
||||
last_kept = matches[keep - 1]
|
||||
first_unkept = matches[keep]
|
||||
if first_unkept is not None:
|
||||
if (
|
||||
last_kept is not None
|
||||
and isinstance(msgs[keep - 1], dict)
|
||||
and msgs[keep - 1].get('role') == 'user'
|
||||
):
|
||||
return _result(ctx[:last_kept + 1], last_kept)
|
||||
return _result(ctx[:first_unkept], first_unkept - 1)
|
||||
if last_kept is not None:
|
||||
ambiguous_first_unkept = ambiguous_matches[keep]
|
||||
if (
|
||||
ambiguous_first_unkept is not None
|
||||
and isinstance(msgs[keep - 1], dict)
|
||||
and msgs[keep - 1].get('role') != 'user'
|
||||
):
|
||||
return _result(ctx[:ambiguous_first_unkept], ambiguous_first_unkept - 1)
|
||||
return _result(ctx[:last_kept + 1], last_kept)
|
||||
|
||||
# Both boundary rows were ambiguous/unmatched (common in large sessions
|
||||
# where context rows have lost their id/timestamp so the matcher can't
|
||||
# disambiguate structurally-identical rows). Only for the shorter-context
|
||||
# case: cut just past the LAST display row in the kept prefix that
|
||||
# resolved to a context index — preferring an exact match but accepting
|
||||
# an ambiguous (weak) one, mirroring how the sibling branches above fold
|
||||
# ``ambiguous_matches`` into the boundary. Accepting the weak match keeps
|
||||
# the forked boundary turn's own context (often exactly that ambiguous
|
||||
# row) instead of dropping back to an earlier exact match. It still errs
|
||||
# toward UNDER-keeping rather than slicing at the raw display index,
|
||||
# which would over-keep and mis-attribute later context rows to the kept
|
||||
# display turns. The context-longer case (injected summary prefix) is
|
||||
# left to the #5096 fallback below, which preserves that prefix.
|
||||
if len(ctx) < len(msgs):
|
||||
for i in range(keep - 1, -1, -1):
|
||||
resolved = matches[i] if matches[i] is not None else ambiguous_matches[i]
|
||||
if resolved is not None:
|
||||
return _result(ctx[:resolved + 1], resolved)
|
||||
|
||||
# Final fallback preserves #5096 behavior when alignment is unreliable
|
||||
# (no display row resolved to a context index, or keep >= len(msgs)).
|
||||
prefix_len = max(0, len(ctx) - len(msgs))
|
||||
prefix = ctx[:prefix_len]
|
||||
suffix = ctx[prefix_len:]
|
||||
result = prefix + suffix[:keep]
|
||||
return _result(result, len(result) - 1 if result else None)
|
||||
|
||||
|
||||
def truncate_session_at_keep(session, keep: int) -> tuple[int, int]:
|
||||
"""Truncate display + context; set watermark/boundary. Returns old counts."""
|
||||
full_messages = list(session.messages or [])
|
||||
old_msg_count = len(full_messages)
|
||||
old_ctx_count = len(getattr(session, 'context_messages', None) or [])
|
||||
session.messages = full_messages[:keep]
|
||||
_stamp_intentional_shrink_generation(session, old_msg_count, len(session.messages))
|
||||
if isinstance(getattr(session, 'context_messages', None), list):
|
||||
session.context_messages = truncate_context_for_display_keep(
|
||||
session.context_messages,
|
||||
full_messages,
|
||||
keep,
|
||||
)
|
||||
session.truncation_watermark = _truncation_watermark_for(session.messages)
|
||||
session.truncation_boundary = session.truncation_watermark
|
||||
return old_msg_count, old_ctx_count
|
||||
|
||||
|
||||
def retry_last(session_id: str) -> dict[str, Any]:
|
||||
"""Truncate the session to before the last user message, return its text.
|
||||
|
||||
@@ -981,7 +115,6 @@ def retry_last(session_id: str) -> dict[str, Any]:
|
||||
last_user_text = _extract_text(history[last_user_idx].get('content', ''))
|
||||
removed_count = len(history) - last_user_idx
|
||||
s.messages = history[:last_user_idx]
|
||||
_stamp_intentional_shrink_generation(s, len(history), len(s.messages))
|
||||
s.truncation_watermark = _truncation_watermark_for(s.messages)
|
||||
# Persist the original truncate cutoff so empty-sidecar recovery
|
||||
# can distinguish legitimate prefix from deleted suffix.
|
||||
@@ -1025,7 +158,6 @@ def undo_last(session_id: str) -> dict[str, Any]:
|
||||
removed_text = _extract_text(history[last_user_idx].get('content', ''))
|
||||
removed_count = len(history) - last_user_idx
|
||||
s.messages = history[:last_user_idx]
|
||||
_stamp_intentional_shrink_generation(s, len(history), len(s.messages))
|
||||
s.truncation_watermark = _truncation_watermark_for(s.messages)
|
||||
# Persist the original truncate cutoff.
|
||||
s.truncation_boundary = s.truncation_watermark
|
||||
@@ -1070,22 +202,6 @@ def session_status(session_id: str) -> dict[str, Any]:
|
||||
'created_at': s.created_at,
|
||||
'updated_at': s.updated_at,
|
||||
'agent_running': bool(getattr(s, 'active_stream_id', None)),
|
||||
# Expose the stream id itself (not just the agent_running bool) so a
|
||||
# hidden-tab poller can attach the live renderer to a server-initiated
|
||||
# turn (self-wake / cron / restart hook) without opening the persistent
|
||||
# per-session SSE while the tab is hidden. See messages.js hidden-tab
|
||||
# active-stream poll. Additive field — existing consumers ignore it.
|
||||
#
|
||||
# CRITICAL: only expose a stream id that is actually LIVE in this
|
||||
# process. After a restart/crash the persisted active_stream_id is stale
|
||||
# (the in-memory STREAMS/ACTIVE_RUNS were wiped) — handing that dead id
|
||||
# to the poller would make it attach a renderer to a stream that never
|
||||
# produces tokens (a permanent fake "thinking" state). Mirror
|
||||
# _clear_stale_stream_state's liveness test: a stream counts as live
|
||||
# only if it's in STREAMS (SSE channel open) or ACTIVE_RUNS (worker
|
||||
# bookkeeping). Otherwise report None so the poller waits for a REAL
|
||||
# server_turn_started instead of latching a ghost.
|
||||
'active_stream_id': _live_active_stream_id(s),
|
||||
'input_tokens': inp,
|
||||
'output_tokens': out,
|
||||
'total_tokens': inp + out,
|
||||
@@ -1120,18 +236,7 @@ def _extract_text(content: Any) -> str:
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for p in content:
|
||||
if not isinstance(p, dict):
|
||||
continue
|
||||
part_type = str(p.get('type') or '').lower()
|
||||
if part_type not in ('', 'text', 'input_text', 'output_text'):
|
||||
continue
|
||||
part_text = (
|
||||
p.get('text')
|
||||
or p.get('content')
|
||||
or p.get('input_text')
|
||||
or p.get('output_text')
|
||||
or ''
|
||||
)
|
||||
parts.append(str(part_text))
|
||||
if isinstance(p, dict) and p.get('type') == 'text':
|
||||
parts.append(p.get('text', ''))
|
||||
return ' '.join(parts)
|
||||
return str(content)
|
||||
|
||||
+3
-300
@@ -29,11 +29,9 @@ import argparse
|
||||
import json
|
||||
import logging
|
||||
import os
|
||||
import re
|
||||
import shutil
|
||||
import sqlite3
|
||||
import threading
|
||||
from contextlib import closing
|
||||
from pathlib import Path
|
||||
|
||||
from api.turn_journal import (
|
||||
@@ -45,24 +43,6 @@ from api.turn_journal import (
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_INTENTIONAL_SHRINK_GENERATION_RE = re.compile(
|
||||
r"^[0-9a-f]{12}4[0-9a-f]{3}[89ab][0-9a-f]{15}$"
|
||||
)
|
||||
|
||||
|
||||
def _is_valid_intentional_shrink_generation(value) -> bool:
|
||||
"""Return whether *value* has the exact ``uuid.uuid4().hex`` shape.
|
||||
|
||||
``session_ops`` stamps generations with ``uuid.uuid4().hex``. Recovery
|
||||
must not treat arbitrary non-empty strings as provenance, because a
|
||||
malformed live/backup marker must leave the original restore safeguard
|
||||
enabled.
|
||||
"""
|
||||
return (
|
||||
isinstance(value, str)
|
||||
and _INTENTIONAL_SHRINK_GENERATION_RE.fullmatch(value) is not None
|
||||
)
|
||||
|
||||
|
||||
def _msg_count(p: Path) -> int:
|
||||
"""Return the number of messages in a session JSON file, or -1 on read/parse error.
|
||||
@@ -231,108 +211,6 @@ def _backup_predates_intentional_shrink(session_path: Path, bak_path: Path) -> b
|
||||
return bak_ctx_len > live_ctx_len
|
||||
|
||||
|
||||
def _session_records_clear_sentinel(session_path: Path, bak_path: Path) -> bool:
|
||||
"""Return True when the live sidecar records a provenanced clear sentinel.
|
||||
|
||||
The live sidecar must carry the explicit /api/session/clear marker, and
|
||||
the backup must not carry the same marker. Same-generation backups stay
|
||||
recoverable; unreadable or partial matches fail open.
|
||||
"""
|
||||
try:
|
||||
data = json.loads(session_path.read_text(encoding='utf-8'))
|
||||
bak = json.loads(bak_path.read_text(encoding='utf-8'))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
if not isinstance(data, dict) or not isinstance(bak, dict):
|
||||
return False
|
||||
clear_generation = data.get('clear_generation')
|
||||
if not isinstance(clear_generation, str) or not clear_generation:
|
||||
return False
|
||||
if bak.get('clear_generation') == clear_generation:
|
||||
return False
|
||||
expected = {
|
||||
'messages': [],
|
||||
'context_messages': [],
|
||||
'truncation_watermark': 0.0,
|
||||
'truncation_boundary': 0.0,
|
||||
'active_stream_id': None,
|
||||
'pending_user_message': None,
|
||||
'pending_attachments': [],
|
||||
'pending_started_at': None,
|
||||
'pending_user_source': None,
|
||||
}
|
||||
for key, value in expected.items():
|
||||
if key not in data or data.get(key) != value:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _live_supersedes_backup_by_clear_generation(session_path: Path, bak_path: Path) -> bool:
|
||||
"""Return True when the live sidecar provably post-dates the backup via a
|
||||
clear sentinel even though the user has since sent NEW messages.
|
||||
|
||||
Scope: this handles ONLY the post-clear-message case that the exact-empty
|
||||
sentinel (_session_records_clear_sentinel) can't. After /api/session/clear
|
||||
stamps a unique ``clear_generation`` and resets the truncation boundary to
|
||||
0.0, a pre-clear ``.json.bak`` is stale — restoring it would resurrect
|
||||
cleared history on top of the post-clear message. We require: live carries a
|
||||
``clear_generation`` the backup lacks, live has a NON-EMPTY transcript, and
|
||||
the live boundary still shows the clear reset (watermark == boundary == 0.0).
|
||||
Empty clear-shaped sidecars stay governed by the exact-empty sentinel and its
|
||||
existing recovery semantics. Same-generation backups stay recoverable;
|
||||
unreadable/partial reads fail open (return False -> normal recovery), so a
|
||||
genuine crash-loss is never suppressed.
|
||||
"""
|
||||
try:
|
||||
data = json.loads(session_path.read_text(encoding='utf-8'))
|
||||
bak = json.loads(bak_path.read_text(encoding='utf-8'))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
if not isinstance(data, dict) or not isinstance(bak, dict):
|
||||
return False
|
||||
clear_generation = data.get('clear_generation')
|
||||
if not isinstance(clear_generation, str) or not clear_generation:
|
||||
return False
|
||||
if bak.get('clear_generation') == clear_generation:
|
||||
return False
|
||||
live_messages = data.get('messages')
|
||||
if not isinstance(live_messages, list) or len(live_messages) == 0:
|
||||
return False
|
||||
if data.get('truncation_watermark') != 0.0 or data.get('truncation_boundary') != 0.0:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _session_records_intentional_message_shrink(session_path: Path, bak_path: Path) -> bool:
|
||||
"""Return True when a live shrink generation supersedes an older backup.
|
||||
|
||||
The marker is deliberately narrow: an unreadable, malformed, or otherwise
|
||||
invalid marker must leave the existing data-loss recovery path enabled.
|
||||
"""
|
||||
try:
|
||||
live = json.loads(session_path.read_text(encoding='utf-8'))
|
||||
bak = json.loads(bak_path.read_text(encoding='utf-8'))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
if not isinstance(live, dict) or not isinstance(bak, dict):
|
||||
return False
|
||||
live_generation = live.get('intentional_shrink_generation')
|
||||
if not _is_valid_intentional_shrink_generation(live_generation):
|
||||
return False
|
||||
if 'intentional_shrink_generation' not in bak:
|
||||
return True
|
||||
backup_generation = bak.get('intentional_shrink_generation')
|
||||
# Older backups persist the field as null (or omit it entirely) because
|
||||
# the live session had not yet passed through a session_ops shrink. Keep
|
||||
# that legacy provenance behavior; malformed non-empty values below must
|
||||
# fail open to the original restore path.
|
||||
if backup_generation is None:
|
||||
return True
|
||||
if not _is_valid_intentional_shrink_generation(backup_generation):
|
||||
return False
|
||||
return backup_generation != live_generation
|
||||
|
||||
|
||||
def inspect_session_recovery_status(session_path: Path) -> dict:
|
||||
"""Return a status dict describing whether recovery is recommended.
|
||||
|
||||
@@ -354,17 +232,6 @@ def inspect_session_recovery_status(session_path: Path) -> dict:
|
||||
}
|
||||
bak_count = _msg_count(bak_path)
|
||||
if bak_count > live_count:
|
||||
if (
|
||||
_session_records_clear_sentinel(session_path, bak_path)
|
||||
or _live_supersedes_backup_by_clear_generation(session_path, bak_path)
|
||||
):
|
||||
return {
|
||||
"session_id": session_path.stem,
|
||||
"live_messages": live_count,
|
||||
"bak_messages": bak_count,
|
||||
"recommend": "no_action",
|
||||
"intentional_clear_truncate": True,
|
||||
}
|
||||
if (
|
||||
_session_records_intentional_compress_shrink(session_path)
|
||||
and _backup_predates_intentional_shrink(session_path, bak_path)
|
||||
@@ -376,14 +243,6 @@ def inspect_session_recovery_status(session_path: Path) -> dict:
|
||||
"recommend": "no_action",
|
||||
"intentional_compress_shrink": True,
|
||||
}
|
||||
if _session_records_intentional_message_shrink(session_path, bak_path):
|
||||
return {
|
||||
"session_id": session_path.stem,
|
||||
"live_messages": live_count,
|
||||
"bak_messages": bak_count,
|
||||
"recommend": "no_action",
|
||||
"intentional_message_shrink": True,
|
||||
}
|
||||
return {
|
||||
"session_id": session_path.stem,
|
||||
"live_messages": live_count,
|
||||
@@ -439,7 +298,7 @@ def _state_db_has_session(session_id: str, state_db_path: Path | None) -> bool:
|
||||
if state_db_path is None or not state_db_path.exists():
|
||||
return True
|
||||
try:
|
||||
with closing(sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True)) as conn:
|
||||
with sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True) as conn:
|
||||
cur = conn.execute(
|
||||
"select 1 from sqlite_master where type='table' and name='sessions'"
|
||||
)
|
||||
@@ -471,18 +330,6 @@ def _orphaned_backup_live_paths(
|
||||
if _msg_count(bak_path) < 0:
|
||||
continue
|
||||
session_id = live_path.stem
|
||||
# A WebUI session the user deleted must not be resurrected from its
|
||||
# surviving .bak on the next boot (#5498). Use the DURABLE tombstone
|
||||
# only — not the _index.json heuristic — so a genuine crash that loses
|
||||
# the sidecar while its index entry survives is still restored (the
|
||||
# crash-recovery case this helper exists for).
|
||||
if _durable_tombstone_marks_deleted_webui_session(session_dir, session_id):
|
||||
logger.info(
|
||||
"recover_all_sessions_on_startup: skipped orphan backup %s; "
|
||||
"session is tombstoned as a deleted WebUI session",
|
||||
bak_path.name,
|
||||
)
|
||||
continue
|
||||
if not _state_db_has_session(session_id, state_db_path):
|
||||
logger.info(
|
||||
"recover_all_sessions_on_startup: skipped orphan backup %s; "
|
||||
@@ -504,7 +351,7 @@ def _read_state_db_missing_sidecar_rows(
|
||||
if state_db_path is None or not state_db_path.exists():
|
||||
return []
|
||||
try:
|
||||
with closing(sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True)) as conn:
|
||||
with sqlite3.connect(f"file:{state_db_path}?mode=ro", uri=True) as conn:
|
||||
conn.row_factory = sqlite3.Row
|
||||
session_cols = {row[1] for row in conn.execute("PRAGMA table_info(sessions)").fetchall()}
|
||||
message_cols = {row[1] for row in conn.execute("PRAGMA table_info(messages)").fetchall()}
|
||||
@@ -536,17 +383,6 @@ def _read_state_db_missing_sidecar_rows(
|
||||
sid = str(data.get('id') or '').strip()
|
||||
if not sid or (session_dir / f"{sid}.json").exists():
|
||||
continue
|
||||
# Only the DURABLE delete tombstone suppresses state.db sidecar
|
||||
# repair. The _index.json heuristic must NOT gate this path: a
|
||||
# genuine crash that loses the sidecar while its index entry
|
||||
# survives (no durable tombstone) is exactly the case this repair
|
||||
# exists for, and on master it materialized the sidecar. Using
|
||||
# _marks_deleted_webui_session() here (which ORs in the index
|
||||
# heuristic) wrongly classified that crash as a delete and
|
||||
# stopped recovery (#5504 Codex/Opus finding).
|
||||
tombstoned = _durable_tombstone_marks_deleted_webui_session(session_dir, sid)
|
||||
if tombstoned and not include_empty:
|
||||
continue
|
||||
message_rows: list[dict] = []
|
||||
if {'session_id', 'role', 'content'}.issubset(message_cols):
|
||||
order = "timestamp, id" if 'timestamp' in message_cols and 'id' in message_cols else "rowid"
|
||||
@@ -566,8 +402,6 @@ def _read_state_db_missing_sidecar_rows(
|
||||
continue
|
||||
data['messages'] = message_rows
|
||||
data['_state_db_empty_messages'] = not message_rows
|
||||
if tombstoned:
|
||||
data['_state_db_deleted_webui_tombstone'] = True
|
||||
rows.append(data)
|
||||
return rows
|
||||
except Exception as exc:
|
||||
@@ -594,9 +428,6 @@ def _state_db_row_to_sidecar(row: dict) -> dict:
|
||||
messages = row.get('messages') if isinstance(row.get('messages'), list) else []
|
||||
last_ts = messages[-1].get('timestamp') if messages and isinstance(messages[-1], dict) else started_at
|
||||
workspace_value = row.get('workspace') or ''
|
||||
compression_recovery = row.get('compression_recovery')
|
||||
if not isinstance(compression_recovery, dict):
|
||||
compression_recovery = {}
|
||||
return {
|
||||
'session_id': row.get('id'),
|
||||
'title': row.get('title') or 'Recovered WebUI Session',
|
||||
@@ -628,10 +459,6 @@ def _state_db_row_to_sidecar(row: dict) -> dict:
|
||||
'context_length': None,
|
||||
'threshold_tokens': None,
|
||||
'last_prompt_tokens': None,
|
||||
'compression_recovery': compression_recovery,
|
||||
'recommended_recovery_action': row.get('recommended_recovery_action') or None,
|
||||
'compression_recovery_source_session_id': row.get('compression_recovery_source_session_id') or None,
|
||||
'compression_recovery_action': row.get('compression_recovery_action') or None,
|
||||
'gateway_routing': None,
|
||||
'gateway_routing_history': [],
|
||||
'llm_title_generated': False,
|
||||
@@ -738,79 +565,6 @@ def _read_index_session_ids(index_path: Path) -> set[str]:
|
||||
return ids
|
||||
|
||||
|
||||
def _index_marks_deleted_webui_session(session_dir: Path, sid: str) -> bool:
|
||||
"""Return True when _index.json has a WebUI-like entry whose sidecar is missing.
|
||||
|
||||
This is a delete-route heuristic for cases where index pruning and durable
|
||||
tombstone recording both failed; it can also match other sidecar-loss modes.
|
||||
"""
|
||||
if not sid or (session_dir / f"{sid}.json").exists():
|
||||
return False
|
||||
index_path = session_dir / '_index.json'
|
||||
if not index_path.exists():
|
||||
return False
|
||||
try:
|
||||
data = json.loads(index_path.read_text(encoding='utf-8'))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
if not isinstance(data, list):
|
||||
return False
|
||||
for entry in data:
|
||||
if not isinstance(entry, dict) or entry.get('session_id') != sid:
|
||||
continue
|
||||
srcs = [
|
||||
str(entry.get('source_tag') or '').strip().lower(),
|
||||
str(entry.get('raw_source') or '').strip().lower(),
|
||||
str(entry.get('session_source') or '').strip().lower(),
|
||||
]
|
||||
explicit = [src for src in srcs if src]
|
||||
if any(src in ('webui', 'fork') for src in explicit):
|
||||
return True
|
||||
if explicit:
|
||||
return False
|
||||
is_cli = entry.get('is_cli_session') is True
|
||||
is_read_only = bool(entry.get('read_only') or entry.get('is_read_only'))
|
||||
return not (is_cli or is_read_only)
|
||||
return False
|
||||
|
||||
|
||||
def _durable_tombstone_marks_deleted_webui_session(session_dir: Path, sid: str) -> bool:
|
||||
"""Return True when the durable WebUI delete tombstone contains sid."""
|
||||
if not sid or (session_dir / f"{sid}.json").exists():
|
||||
return False
|
||||
try:
|
||||
from api import models as _models
|
||||
|
||||
if Path(_models.SESSION_DIR).resolve() == session_dir.resolve():
|
||||
return sid in _models._load_webui_deleted_session_tombstone()
|
||||
except Exception:
|
||||
pass
|
||||
tombstone_path = session_dir / '_deleted_webui_sessions.json'
|
||||
try:
|
||||
raw = json.loads(tombstone_path.read_text(encoding='utf-8'))
|
||||
except (OSError, json.JSONDecodeError, ValueError):
|
||||
return False
|
||||
if not isinstance(raw, dict):
|
||||
return False
|
||||
try:
|
||||
version = int(raw.get('version', 0))
|
||||
except (TypeError, ValueError):
|
||||
return False
|
||||
if version != 1:
|
||||
return False
|
||||
ids = raw.get('ids')
|
||||
if not isinstance(ids, list):
|
||||
return False
|
||||
return sid in {str(value).strip() for value in ids if str(value or '').strip()}
|
||||
|
||||
|
||||
def _marks_deleted_webui_session(session_dir: Path, sid: str) -> bool:
|
||||
return (
|
||||
_index_marks_deleted_webui_session(session_dir, sid)
|
||||
or _durable_tombstone_marks_deleted_webui_session(session_dir, sid)
|
||||
)
|
||||
|
||||
|
||||
def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None) -> dict:
|
||||
"""Read-only audit of session recovery state.
|
||||
|
||||
@@ -829,16 +583,6 @@ def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None)
|
||||
items: list[dict] = []
|
||||
live_paths = sorted(p for p in session_dir.glob('*.json') if not p.name.startswith('_'))
|
||||
live_ids = {p.stem for p in live_paths}
|
||||
state_db_missing_rows = _read_state_db_missing_sidecar_rows(
|
||||
session_dir,
|
||||
state_db_path,
|
||||
include_empty=True,
|
||||
)
|
||||
state_db_deleted_webui_ids = {
|
||||
str(row.get('id') or '')
|
||||
for row in state_db_missing_rows
|
||||
if row.get('_state_db_deleted_webui_tombstone')
|
||||
}
|
||||
|
||||
for live_path in live_paths:
|
||||
status = inspect_session_recovery_status(live_path)
|
||||
@@ -852,12 +596,6 @@ def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None)
|
||||
status.get('bak_messages', -1),
|
||||
))
|
||||
|
||||
# Track sids already classified as deleted-webui-tombstone from the orphan
|
||||
# .bak branch below, so the later state_db_missing_rows loop doesn't emit a
|
||||
# duplicate audit item for the same sid (both a surviving .bak and a state.db
|
||||
# row can exist for one deleted session). (#5504)
|
||||
_bak_tombstoned_ids: set[str] = set()
|
||||
|
||||
for bak_path in sorted(session_dir.glob('*.json.bak')):
|
||||
live_path = bak_path.with_suffix('')
|
||||
if live_path.exists() or live_path.name.startswith('_'):
|
||||
@@ -868,21 +606,6 @@ def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None)
|
||||
items.append(_new_audit_item(
|
||||
session_id, "malformed_orphan_backup", "unsafe_to_repair", "manual_review", -1, bak_messages
|
||||
))
|
||||
elif _durable_tombstone_marks_deleted_webui_session(session_dir, session_id):
|
||||
# The user deleted this WebUI session; its surviving .bak must NOT be
|
||||
# reported as repairable. DURABLE tombstone only — the _index.json
|
||||
# heuristic must not suppress a genuine crash whose index survived
|
||||
# (that case is legitimately repairable). Matches the startup-recovery
|
||||
# skip + the state.db recovery path (#5504 Codex/Opus finding).
|
||||
_bak_tombstoned_ids.add(session_id)
|
||||
items.append(_new_audit_item(
|
||||
session_id,
|
||||
"state_db_deleted_webui_tombstone",
|
||||
"unsafe_to_repair",
|
||||
"deleted_session_skipped",
|
||||
-1,
|
||||
bak_messages,
|
||||
))
|
||||
elif _state_db_has_session(session_id, state_db_path):
|
||||
items.append(_new_audit_item(
|
||||
session_id, "orphan_backup", "repairable", "restore_from_bak", -1, bak_messages
|
||||
@@ -901,11 +624,6 @@ def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None)
|
||||
if index_path.exists():
|
||||
index_ids = _read_index_session_ids(index_path)
|
||||
for session_id in sorted(index_ids - live_ids):
|
||||
if (
|
||||
session_id in state_db_deleted_webui_ids
|
||||
or _durable_tombstone_marks_deleted_webui_session(session_dir, session_id)
|
||||
):
|
||||
continue
|
||||
items.append(_new_audit_item(
|
||||
session_id, "index_missing_file", "repairable", "rebuild_index"
|
||||
))
|
||||
@@ -915,23 +633,8 @@ def audit_session_recovery(session_dir: Path, state_db_path: Path | None = None)
|
||||
_msg_count(session_dir / f"{session_id}.json"), -1,
|
||||
))
|
||||
|
||||
for row in state_db_missing_rows:
|
||||
for row in _read_state_db_missing_sidecar_rows(session_dir, state_db_path, include_empty=True):
|
||||
sid = str(row.get('id') or '')
|
||||
if sid in _bak_tombstoned_ids:
|
||||
# Already emitted a deleted-webui-tombstone audit item from the
|
||||
# orphan .bak branch above (surviving .bak + state.db row both exist
|
||||
# for this one deleted session) — don't double-count. (#5504)
|
||||
continue
|
||||
if row.get('_state_db_deleted_webui_tombstone'):
|
||||
items.append(_new_audit_item(
|
||||
sid,
|
||||
"state_db_deleted_webui_tombstone",
|
||||
"unsafe_to_repair",
|
||||
"deleted_session_skipped",
|
||||
-1,
|
||||
-1,
|
||||
))
|
||||
continue
|
||||
if row.get('_state_db_empty_messages'):
|
||||
items.append(_new_audit_item(
|
||||
sid,
|
||||
|
||||
-523
@@ -1,523 +0,0 @@
|
||||
"""
|
||||
Hermes Web UI -- public read-only share snapshots.
|
||||
|
||||
Stores a sanitized, immutable snapshot of a conversation under STATE_DIR/shares.
|
||||
The snapshot is intentionally narrower than a full session export so public
|
||||
links do not leak local workspace paths, profile details, or raw tool payloads.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import html
|
||||
import io
|
||||
import json
|
||||
import logging
|
||||
import mimetypes
|
||||
import os
|
||||
import re
|
||||
import secrets
|
||||
import tempfile
|
||||
import threading
|
||||
import time
|
||||
import xml.etree.ElementTree as ET
|
||||
from pathlib import Path
|
||||
|
||||
from api.config import STATE_DIR
|
||||
from api.helpers import redact_session_data
|
||||
# _redact_fn_cached is the ALWAYS-ON credential redactor (agent redactor with
|
||||
# force=True + local fallback regex). Unlike redact_session_data it does NOT
|
||||
# consult the user-toggleable api_redact_enabled setting — a public share is a
|
||||
# hard safety boundary that must redact credentials even if the operator turned
|
||||
# API-response redaction off.
|
||||
from api.helpers import _redact_fn_cached as _force_redact_credentials
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
SHARES_DIR = STATE_DIR / "shares"
|
||||
_SHARE_LOCK = threading.Lock()
|
||||
|
||||
|
||||
def _ensure_share_dir() -> None:
|
||||
SHARES_DIR.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
|
||||
def _share_path(token: str) -> Path:
|
||||
token = str(token or "").strip()
|
||||
if not token:
|
||||
raise ValueError("share token is required")
|
||||
if not token.replace("-", "").replace("_", "").isalnum():
|
||||
raise ValueError("invalid share token")
|
||||
return SHARES_DIR / f"{token}.json"
|
||||
|
||||
|
||||
def _write_json_atomic(path: Path, payload: dict) -> None:
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
fd, tmp_name = tempfile.mkstemp(
|
||||
dir=str(path.parent),
|
||||
prefix=f"{path.stem}.",
|
||||
suffix=".tmp",
|
||||
text=True,
|
||||
)
|
||||
try:
|
||||
with os.fdopen(fd, "w", encoding="utf-8") as fh:
|
||||
json.dump(payload, fh, ensure_ascii=False, indent=2)
|
||||
fh.flush()
|
||||
os.fsync(fh.fileno())
|
||||
os.replace(tmp_name, path)
|
||||
except Exception:
|
||||
try:
|
||||
os.unlink(tmp_name)
|
||||
except OSError:
|
||||
pass
|
||||
raise
|
||||
|
||||
|
||||
def _share_message_text(message: dict) -> str:
|
||||
content = message.get("content") if isinstance(message, dict) else ""
|
||||
if isinstance(content, list):
|
||||
parts = []
|
||||
for item in content:
|
||||
if not isinstance(item, dict):
|
||||
# Non-dict list items (e.g. nested structures) are NOT plain text —
|
||||
# never stringify them into the public snapshot.
|
||||
continue
|
||||
if item.get("type") == "text":
|
||||
# Only append genuine string text — a dict-valued "text" (possible
|
||||
# via /api/session/import) must NOT be str()'d into the public
|
||||
# snapshot (that would publish structured/tool payload verbatim).
|
||||
_t = item.get("text")
|
||||
if isinstance(_t, str):
|
||||
parts.append(_t)
|
||||
return "".join(parts).strip()
|
||||
if isinstance(content, str):
|
||||
return content.strip()
|
||||
# A dict/other structured content (e.g. a tool-result object) is NOT shareable
|
||||
# text — do NOT str() it (that would publish raw structured/tool payload).
|
||||
return ""
|
||||
|
||||
|
||||
def _redact_share_paths(text: str, extra_paths) -> str:
|
||||
"""Strip known local session/workspace/home paths out of public-share text.
|
||||
|
||||
A workspace path or Hermes home can be embedded inside message prose (an
|
||||
agent quoting a file path, a traceback, etc.). Redact the concrete local
|
||||
paths so a public share never discloses the operator's filesystem layout.
|
||||
"""
|
||||
if not isinstance(text, str) or not text:
|
||||
return text
|
||||
for p in extra_paths:
|
||||
if not p:
|
||||
continue
|
||||
p = str(p).strip()
|
||||
if len(p) >= 4 and p in text:
|
||||
text = text.replace(p, "[redacted-path]")
|
||||
return text
|
||||
|
||||
|
||||
# Regex matching local MEDIA:<path> references — same pattern that
|
||||
# _inlineMediaHtmlForRef() in ui.js handles when rendering messages.
|
||||
# Excludes MEDIA: followed by http/https URLs so external images pass
|
||||
# through unchanged. file:// references are NOT matched here — they are
|
||||
# always rejected at the public-share boundary (absolute, un-scoped).
|
||||
_SHARE_MEDIA_RE = re.compile(
|
||||
r"MEDIA:(?!https?://)([^\s\)\]>]+)"
|
||||
)
|
||||
|
||||
# Max size (in bytes) for files we'll embed as base64 in a share snapshot.
|
||||
_SHARE_EMBED_MAX_BYTES = 512 * 1024 # 512 KiB
|
||||
|
||||
# Only these image MIME types may be embedded in public shares.
|
||||
# Non-image files and SVG are NEVER embedded — embedding arbitrary file
|
||||
# bytes circumvents the credential-redaction boundary that protects
|
||||
# message prose, and a public share is not a file-transfer service.
|
||||
# SVG is excluded because it is the only text-bearing type in this set;
|
||||
# agent-authored SVGs can carry credentials in their text content which
|
||||
# _redact_share_paths (which only touches message prose, not embedded
|
||||
# bytes) cannot reach.
|
||||
_SHARE_ALLOWED_MIME_TYPES: frozenset[str] = frozenset({
|
||||
"image/png",
|
||||
"image/jpeg",
|
||||
"image/gif",
|
||||
"image/webp",
|
||||
})
|
||||
|
||||
# SVG namespace URI used during sanitisation.
|
||||
_SVG_NS = "http://www.w3.org/2000/svg"
|
||||
|
||||
# Pattern matching on* event-handler attributes.
|
||||
_ON_ATTR_RE = re.compile(r"^on\w+$", re.IGNORECASE)
|
||||
|
||||
# Dangerous href/xlink:href schemes.
|
||||
_DANGEROUS_HREF_RE = re.compile(r"^\s*javascript\s*:", re.IGNORECASE)
|
||||
|
||||
# Static placeholder emitted when a media reference cannot be embedded.
|
||||
_PLACEHOLDER = "[*Local attachment omitted from public share*]"
|
||||
|
||||
# Magic byte signatures for allowed image formats — content-based validation
|
||||
# that catches mismatched extensions (e.g. a .png that is actually a script).
|
||||
# SVG is excluded here because it is validated by XML parsing in
|
||||
# _sanitize_svg_bytes.
|
||||
_IMAGE_MAGIC: dict[str, bytes] = {
|
||||
"image/png": b"\x89PNG\r\n\x1a\n",
|
||||
"image/jpeg": b"\xff\xd8\xff",
|
||||
"image/gif": b"GIF8",
|
||||
"image/webp": b"RIFF",
|
||||
}
|
||||
# Offset for WebP magic: "RIFF" at 0, file size at 4, "WEBP" at 8.
|
||||
_WEBP_MAGIC_OFFSET = 8
|
||||
_WEBP_MAGIC = b"WEBP"
|
||||
|
||||
|
||||
def _check_image_magic(data: bytes, mime_type: str) -> bool:
|
||||
"""Verify *data* header bytes match the expected magic for *mime_type*.
|
||||
|
||||
Returns ``True`` if the content is consistent with the claimed type.
|
||||
SVG is exempt because it is validated structurally by
|
||||
:func:`_sanitize_svg_bytes`.
|
||||
"""
|
||||
if mime_type == "image/svg+xml":
|
||||
return True
|
||||
magic = _IMAGE_MAGIC.get(mime_type)
|
||||
if magic is None:
|
||||
return False
|
||||
if not data.startswith(magic):
|
||||
return False
|
||||
# Extra check for WebP: "WEBP" at offset 8.
|
||||
if mime_type == "image/webp":
|
||||
if len(data) < 12 or data[_WEBP_MAGIC_OFFSET:_WEBP_MAGIC_OFFSET + 4] != _WEBP_MAGIC:
|
||||
return False
|
||||
return True
|
||||
|
||||
|
||||
def _sanitize_svg_bytes(data: bytes) -> bytes:
|
||||
"""Strip script elements, on* handlers, and javascript: hrefs from SVG.
|
||||
|
||||
SVG images served via ``<img src="data:image/svg+xml;base64,…">`` are
|
||||
sandboxed by modern browsers and script execution is blocked. However,
|
||||
a sufficiently determined adversary with an older or exotic client may
|
||||
still extract credentials embedded in the SVG, so we strip the unsafe
|
||||
content at the server before it ever reaches a share page.
|
||||
|
||||
Returns sanitised SVG bytes on success, or the original *data* unchanged
|
||||
if the content cannot be parsed as XML (fail-closed).
|
||||
"""
|
||||
try:
|
||||
ET.register_namespace("", _SVG_NS)
|
||||
root = ET.fromstring(data.decode("utf-8", errors="replace"))
|
||||
except ET.ParseError:
|
||||
# Not valid XML — cannot sanitise safely. Return a minimal empty SVG
|
||||
# so the <img> renders nothing rather than embedding un-sanitised bytes.
|
||||
return b'<svg xmlns="http://www.w3.org/2000/svg"/>'
|
||||
|
||||
# Walk the tree depth-first, stripping on* attrs, dangerous hrefs,
|
||||
# and removing <script> children.
|
||||
def _walk(elem: ET.Element) -> None:
|
||||
for attr_name in list(elem.attrib):
|
||||
if _ON_ATTR_RE.match(attr_name):
|
||||
del elem.attrib[attr_name]
|
||||
elif attr_name in ("href", "xlink:href", "{http://www.w3.org/1999/xlink}href"):
|
||||
val = elem.attrib[attr_name]
|
||||
if _DANGEROUS_HREF_RE.match(val):
|
||||
del elem.attrib[attr_name]
|
||||
|
||||
for child in list(elem):
|
||||
tag = child.tag.split("}", 1)[-1] if "}" in child.tag else child.tag
|
||||
if tag == "script":
|
||||
elem.remove(child)
|
||||
else:
|
||||
_walk(child)
|
||||
|
||||
_walk(root)
|
||||
|
||||
buf = io.BytesIO()
|
||||
tree = ET.ElementTree(root)
|
||||
tree.write(buf, encoding="utf-8", xml_declaration=False)
|
||||
return buf.getvalue()
|
||||
|
||||
|
||||
def _embed_share_media(text: str, *, allowed_roots: tuple[Path, ...] = ()) -> str:
|
||||
"""Find local MEDIA: references and replace them with inline <img> tags.
|
||||
|
||||
Only relative paths that resolve inside at least one of *allowed_roots*
|
||||
are honoured. Absolute paths, ``file://`` URIs, paths that traverse
|
||||
outside the allowed directories via ``..`` or symlinks, non-image MIME
|
||||
types, and files larger than ``_SHARE_EMBED_MAX_BYTES`` are all replaced
|
||||
with a static placeholder — no file content leaves the server.
|
||||
|
||||
This runs BEFORE :func:`_redact_share_paths` so the concrete file path
|
||||
is still available for the allowed-roots check.
|
||||
"""
|
||||
if not isinstance(text, str) or not text:
|
||||
return text
|
||||
|
||||
allowed = tuple(Path(r).resolve() for r in allowed_roots if r)
|
||||
|
||||
def _resolve_against_roots(raw: str) -> Path | None:
|
||||
"""Resolve *raw* against each allowed root, returning the first valid
|
||||
absolute Path that lives inside one of them, or ``None``.
|
||||
|
||||
- ``file://`` is always rejected (absolute, un-scoped).
|
||||
- Absolute paths (``/…``, ``~…``) are resolved as-is and checked
|
||||
against the allowed-roots allow-list via ``is_relative_to()``.
|
||||
- Relative paths are joined with each allowed root in turn so they
|
||||
don't silently anchor to the server's process CWD.
|
||||
"""
|
||||
if raw.startswith("file://"):
|
||||
return None
|
||||
|
||||
# --- Absolute paths: resolve as-is, then allow-list check ------------
|
||||
if raw.startswith("/") or raw.startswith("~"):
|
||||
try:
|
||||
p = Path(raw).expanduser().resolve(strict=False)
|
||||
except (OSError, ValueError, RuntimeError):
|
||||
return None
|
||||
if not allowed or not any(p.is_relative_to(r) for r in allowed):
|
||||
return None
|
||||
return p if p.is_file() else None
|
||||
|
||||
# --- Relative paths: try each allowed root as the anchor -------------
|
||||
for root in allowed:
|
||||
try:
|
||||
candidate = (root / raw).resolve(strict=False)
|
||||
except (OSError, ValueError, RuntimeError):
|
||||
continue
|
||||
# Path traversal guard: resolved path must still be under the root.
|
||||
if not candidate.is_relative_to(root):
|
||||
continue
|
||||
if candidate.is_file():
|
||||
return candidate
|
||||
return None
|
||||
|
||||
def _replace_ref(m: re.Match) -> str:
|
||||
raw = (m.group(1) or "").strip()
|
||||
if not raw:
|
||||
return m.group(0)
|
||||
|
||||
# --- Resolve and validate against allowed roots -----------------------
|
||||
p = _resolve_against_roots(raw)
|
||||
if p is None:
|
||||
return _PLACEHOLDER
|
||||
|
||||
# --- Size guard -------------------------------------------------------
|
||||
try:
|
||||
size = p.stat().st_size
|
||||
except OSError:
|
||||
return _PLACEHOLDER
|
||||
|
||||
if size > _SHARE_EMBED_MAX_BYTES:
|
||||
return _PLACEHOLDER
|
||||
|
||||
# --- MIME allow-list (images only) ------------------------------------
|
||||
mime_type, _ = mimetypes.guess_type(str(p))
|
||||
if not mime_type or mime_type not in _SHARE_ALLOWED_MIME_TYPES:
|
||||
return _PLACEHOLDER
|
||||
|
||||
# --- Embed as base64 <img> -------------------------------------------
|
||||
try:
|
||||
data = p.read_bytes()
|
||||
# Content-based MIME validation: verify the actual file header
|
||||
# matches the claimed MIME type — catches extension-spoofed files
|
||||
# (e.g. a script renamed to .png).
|
||||
if not _check_image_magic(data, mime_type):
|
||||
return _PLACEHOLDER
|
||||
# Sanitise SVG content before embedding — SVG can carry
|
||||
# <script> elements and on* event handlers that could leak
|
||||
# credentials in the context of a public share page.
|
||||
if mime_type == "image/svg+xml":
|
||||
data = _sanitize_svg_bytes(data)
|
||||
b64 = base64.b64encode(data).decode("ascii")
|
||||
# HTML-escape the filename so a crafted name like
|
||||
# '"><script>alert(1)</script>' cannot break out of the
|
||||
# attribute and inject script into the share page.
|
||||
safe_name = html.escape(p.name, quote=True)
|
||||
return (
|
||||
f'<img src="data:{mime_type};base64,{b64}"'
|
||||
f' class="msg-media-img" alt="{safe_name}"'
|
||||
f' loading="lazy">'
|
||||
)
|
||||
except (OSError, MemoryError):
|
||||
return _PLACEHOLDER
|
||||
|
||||
return _SHARE_MEDIA_RE.sub(_replace_ref, text)
|
||||
|
||||
|
||||
def _sanitize_message(message: dict, *, redact_paths=(), allowed_roots: tuple[Path, ...] = ()) -> dict | None:
|
||||
if not isinstance(message, dict):
|
||||
return None
|
||||
role = str(message.get("role") or "").strip().lower()
|
||||
if role not in {"user", "assistant"}:
|
||||
return None
|
||||
text = _share_message_text(message)
|
||||
if not text:
|
||||
return None
|
||||
# ALWAYS-ON hardening for the public boundary, independent of any setting:
|
||||
# (1) force credential redaction, (2) embed allowed local media,
|
||||
# (3) strip known local paths.
|
||||
text = _force_redact_credentials(text)
|
||||
# Embed local media BEFORE path redaction so the concrete path is still
|
||||
# available for file reads. MEDIA: references become self-contained data
|
||||
# URIs — or a static placeholder if the path is outside the allowed roots.
|
||||
text = _embed_share_media(text, allowed_roots=allowed_roots)
|
||||
text = _redact_share_paths(text, redact_paths)
|
||||
if not text.strip():
|
||||
return None
|
||||
sanitized = {
|
||||
"role": role,
|
||||
"content": text,
|
||||
}
|
||||
ts = message.get("timestamp")
|
||||
if isinstance(ts, (int, float)):
|
||||
sanitized["timestamp"] = ts
|
||||
return sanitized
|
||||
|
||||
|
||||
def _public_share_payload(payload: dict) -> dict:
|
||||
messages = payload.get("messages")
|
||||
if not isinstance(messages, list):
|
||||
messages = []
|
||||
public = {
|
||||
"title": str(payload.get("title") or "Untitled"),
|
||||
"messages": messages,
|
||||
"message_count": int(payload.get("message_count") or len(messages)),
|
||||
}
|
||||
created_at = payload.get("created_at")
|
||||
updated_at = payload.get("updated_at")
|
||||
if isinstance(created_at, (int, float)):
|
||||
public["created_at"] = created_at
|
||||
if isinstance(updated_at, (int, float)):
|
||||
public["updated_at"] = updated_at
|
||||
return public
|
||||
|
||||
|
||||
def build_share_snapshot(session) -> dict:
|
||||
raw_dict = getattr(session, "__dict__", {}) or {}
|
||||
# redact_session_data respects the api_redact_enabled setting; keep it as a
|
||||
# first pass, but the per-message sanitizer below applies ALWAYS-ON credential
|
||||
# + path redaction that does NOT depend on that setting (the public boundary
|
||||
# must hold even if the operator disabled api_redact_enabled).
|
||||
safe_session = redact_session_data(raw_dict)
|
||||
# Concrete local paths to scrub from any message prose / title.
|
||||
redact_paths = []
|
||||
for key in ("workspace", "worktree_path", "worktree_repo_root"):
|
||||
val = raw_dict.get(key)
|
||||
if val:
|
||||
redact_paths.append(str(val))
|
||||
try:
|
||||
from api.profiles import get_active_hermes_home
|
||||
redact_paths.append(str(get_active_hermes_home()))
|
||||
except Exception:
|
||||
pass
|
||||
try:
|
||||
redact_paths.append(str(Path.home()))
|
||||
except Exception:
|
||||
pass
|
||||
# Collect allowed roots for _embed_share_media: only files inside the
|
||||
# session workspace or the attachments root may be embedded. This is
|
||||
# the hard security boundary that prevents arbitrary file reads through
|
||||
# crafted MEDIA: references in message text.
|
||||
_allowed_roots: list[Path] = []
|
||||
_ws = raw_dict.get("workspace")
|
||||
if _ws and isinstance(_ws, str) and _ws.strip():
|
||||
_allowed_roots.append(Path(_ws.strip()))
|
||||
try:
|
||||
from api.upload import _attachment_root
|
||||
_allowed_roots.append(_attachment_root())
|
||||
except Exception:
|
||||
pass
|
||||
_allowed_roots_tuple: tuple[Path, ...] = tuple(_allowed_roots)
|
||||
safe_messages = []
|
||||
for raw in safe_session.get("messages") or []:
|
||||
sanitized = _sanitize_message(
|
||||
raw, redact_paths=redact_paths, allowed_roots=_allowed_roots_tuple,
|
||||
)
|
||||
if sanitized:
|
||||
safe_messages.append(sanitized)
|
||||
if not safe_messages:
|
||||
raise ValueError("This conversation has no shareable messages yet.")
|
||||
# Only accept a genuine string title — a dict-valued title (possible via
|
||||
# /api/session/import) must not be str()'d into the public snapshot.
|
||||
_raw_title = safe_session.get("title")
|
||||
_raw_title = _raw_title if isinstance(_raw_title, str) else "Untitled"
|
||||
title = _force_redact_credentials(_raw_title or "Untitled")
|
||||
title = _redact_share_paths(title, redact_paths) or "Untitled"
|
||||
return {
|
||||
"title": title,
|
||||
"messages": safe_messages,
|
||||
"message_count": len(safe_messages),
|
||||
}
|
||||
|
||||
|
||||
def create_or_refresh_share(session) -> dict:
|
||||
snapshot = build_share_snapshot(session)
|
||||
with _SHARE_LOCK:
|
||||
_ensure_share_dir()
|
||||
existing_token = str(getattr(session, "share_token", "") or "").strip()
|
||||
token = existing_token or secrets.token_urlsafe(18)
|
||||
now = time.time()
|
||||
payload = {
|
||||
"token": token,
|
||||
"source_session_id": str(getattr(session, "session_id", "") or ""),
|
||||
"title": snapshot["title"],
|
||||
"messages": snapshot["messages"],
|
||||
"message_count": snapshot["message_count"],
|
||||
"created_at": now,
|
||||
"updated_at": now,
|
||||
"revoked_at": None,
|
||||
}
|
||||
path = _share_path(token)
|
||||
if path.exists():
|
||||
try:
|
||||
existing = json.loads(path.read_text(encoding="utf-8"))
|
||||
if isinstance(existing, dict):
|
||||
payload["created_at"] = existing.get("created_at") or now
|
||||
except Exception:
|
||||
logger.debug("Ignoring malformed share snapshot at %s", path, exc_info=True)
|
||||
_write_json_atomic(path, payload)
|
||||
return {
|
||||
"share_token": token,
|
||||
"share_title": payload["title"],
|
||||
"share_message_count": payload["message_count"],
|
||||
"share_created_at": payload["created_at"],
|
||||
"share_updated_at": payload["updated_at"],
|
||||
}
|
||||
|
||||
|
||||
def load_share(token: str) -> dict | None:
|
||||
try:
|
||||
path = _share_path(token)
|
||||
except ValueError:
|
||||
return None
|
||||
if not path.exists():
|
||||
return None
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
logger.warning("Failed to read share snapshot %s", path, exc_info=True)
|
||||
return None
|
||||
if not isinstance(payload, dict):
|
||||
return None
|
||||
if payload.get("revoked_at"):
|
||||
return None
|
||||
return _public_share_payload(payload)
|
||||
|
||||
|
||||
def revoke_share(session) -> bool:
|
||||
token = str(getattr(session, "share_token", "") or "").strip()
|
||||
if not token:
|
||||
return False
|
||||
with _SHARE_LOCK:
|
||||
try:
|
||||
path = _share_path(token)
|
||||
except ValueError:
|
||||
return False
|
||||
if path.exists():
|
||||
try:
|
||||
payload = json.loads(path.read_text(encoding="utf-8"))
|
||||
except Exception:
|
||||
payload = {}
|
||||
if not isinstance(payload, dict):
|
||||
payload = {}
|
||||
payload["revoked_at"] = time.time()
|
||||
_write_json_atomic(path, payload)
|
||||
return True
|
||||
+2
-65
@@ -134,9 +134,7 @@ def sync_session_start(session_id: str, model=None, profile: Optional[str] = Non
|
||||
|
||||
def sync_session_usage(session_id: str, input_tokens: int=0, output_tokens: int=0,
|
||||
estimated_cost=None, model=None, title: Optional[str] = None,
|
||||
message_count: Optional[int] = None, profile: Optional[str] = None,
|
||||
cache_read_tokens: int = 0, cache_write_tokens: int = 0,
|
||||
api_call_count: Optional[int] = None) -> None:
|
||||
message_count: Optional[int] = None, profile: Optional[str] = None) -> None:
|
||||
"""Update token usage and title for a WebUI session in state.db.
|
||||
Called after each turn completes. Uses absolute=True to set totals
|
||||
(the WebUI Session already accumulates across turns).
|
||||
@@ -155,18 +153,11 @@ def sync_session_usage(session_id: str, input_tokens: int=0, output_tokens: int=
|
||||
try:
|
||||
# Ensure session exists first (idempotent)
|
||||
db.ensure_session(session_id=session_id, source='webui', model=model)
|
||||
# Set absolute token counts. WebUI's sidecar already accumulates
|
||||
# input/output/cache totals across turns, so mirror the same absolute
|
||||
# values into state.db. Omitting cache counters makes insights/reporting
|
||||
# show false 0% hit rates even when the live stream and sidecar saw warm
|
||||
# prefix reads.
|
||||
# Set absolute token counts
|
||||
db.update_token_counts(
|
||||
session_id=session_id,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
cache_read_tokens=cache_read_tokens,
|
||||
cache_write_tokens=cache_write_tokens,
|
||||
api_call_count=api_call_count,
|
||||
estimated_cost_usd=estimated_cost,
|
||||
model=model,
|
||||
absolute=True,
|
||||
@@ -195,57 +186,3 @@ def sync_session_usage(session_id: str, input_tokens: int=0, output_tokens: int=
|
||||
db.close()
|
||||
except Exception:
|
||||
logger.debug("Failed to close state.db")
|
||||
|
||||
|
||||
def sync_session_title(session_id: str, title: str, profile: Optional[str] = None) -> None:
|
||||
"""Sync an auto-generated title to state.db (not gated by sync_to_insights).
|
||||
|
||||
Background title generation writes the title to the WebUI sidecar JSON but
|
||||
not to hermes-agent's state.db, so ``hermes sessions list`` shows blank
|
||||
titles for WebUI sessions. This function bridges that gap and is called
|
||||
from the background title update/refresh paths after a title is persisted.
|
||||
|
||||
Uses ``set_auto_title`` (LLM provenance) so it will only populate a row that
|
||||
is NULL or holds a lower-authority auto-title, and never overwrites a manual
|
||||
rename made via CLI/Gateway/TUI (``set_auto_title`` returns ``False``,
|
||||
untouched, when a higher-authority title holds the row). This means title
|
||||
refreshes (where state.db already holds the initial auto-title) are
|
||||
effectively no-ops at the state.db layer -- acceptable because the primary
|
||||
goal is ensuring ``hermes sessions list`` is not blank.
|
||||
|
||||
On a title collision (two sessions with the same auto-title), the title is
|
||||
de-duplicated via ``get_next_title_in_lineage`` (e.g. "My Session" ->
|
||||
"My Session #2") and retried, so the second session is never left blank.
|
||||
"""
|
||||
if not title:
|
||||
return
|
||||
db = _get_state_db(profile=profile)
|
||||
if not db:
|
||||
return
|
||||
try:
|
||||
# Ensure the session row exists (idempotent) so the UPDATE has a target.
|
||||
db.ensure_session(session_id=session_id, source='webui')
|
||||
# hermes-agent's SessionDB.set_auto_title_if_empty was renamed to
|
||||
# set_auto_title(session_id, title, *, source) in the state-module
|
||||
# split (agent commit 53db597201, released v2026.9.7). set_auto_title
|
||||
# preserves the same "only populate NULL / never clobber a manual
|
||||
# rename" semantics (returns False, untouched, when a higher-authority
|
||||
# title holds the row) and requires an explicit auto source.
|
||||
_llm_source = getattr(db, "TITLE_SOURCE_LLM", "llm")
|
||||
try:
|
||||
db.set_auto_title(session_id, title, source=_llm_source)
|
||||
except ValueError:
|
||||
# state.db enforces uniqueness on sessions.title, so a byte-identical
|
||||
# auto-title generated for two sessions raises ValueError here. Derive
|
||||
# a de-duplicated variant (e.g. "My Session" -> "My Session #2") and
|
||||
# retry instead of leaving the second row blank (#6964).
|
||||
alt = db.get_next_title_in_lineage(title)
|
||||
if alt and alt != title:
|
||||
db.set_auto_title(session_id, alt, source=_llm_source)
|
||||
except Exception:
|
||||
logger.debug("Failed to sync session title to state.db for %s", session_id)
|
||||
finally:
|
||||
try:
|
||||
db.close()
|
||||
except Exception:
|
||||
logger.debug("Failed to close state.db")
|
||||
|
||||
+510
-4428
File diff suppressed because it is too large
Load Diff
@@ -1,18 +0,0 @@
|
||||
"""Dependency-light helpers for launching child processes consistently."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
|
||||
def windows_hide_flags() -> int:
|
||||
"""Hide a short-lived console child on Win32 and remain a POSIX no-op.
|
||||
|
||||
``CREATE_NO_WINDOW`` keeps captured stdout and stderr connected, unlike
|
||||
detaching the process. Passing ``0`` elsewhere preserves the subprocess
|
||||
default. See #5692.
|
||||
"""
|
||||
if sys.platform == "win32":
|
||||
return getattr(subprocess, "CREATE_NO_WINDOW", 0)
|
||||
return 0
|
||||
@@ -159,113 +159,6 @@ def _safe_error(metric: str, exc: Exception) -> dict[str, str]:
|
||||
return {"metric": metric, "code": type(exc).__name__}
|
||||
|
||||
|
||||
def _zero_webui_runtime_payload() -> dict[str, Any]:
|
||||
return {
|
||||
"sessions": {"available": False, "resident": 0, "cap": 0},
|
||||
"streams": {
|
||||
"available": False, "active": 0, "agent_instances": 0,
|
||||
"subscribers": 0, "offline_buffered_events": 0,
|
||||
"offline_dropped_events": 0, "subscriber_dropped_events": 0,
|
||||
"unavailable_channels": 0,
|
||||
},
|
||||
"session_list_cache": {
|
||||
"available": False, "entries": 0, "inflight_rebuilds": 0, "cap": 0,
|
||||
},
|
||||
"models_cache": {
|
||||
"available": False, "groups": 0, "models": 0, "age_seconds": None,
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def _safe_runtime_int(value: Any) -> int:
|
||||
try:
|
||||
return max(0, int(value))
|
||||
except (TypeError, ValueError):
|
||||
return 0
|
||||
|
||||
|
||||
def _webui_runtime_payload(errors: list[dict[str, str]]) -> dict[str, Any]:
|
||||
runtime = _zero_webui_runtime_payload()
|
||||
# The three owner modules stay lazily imported so this small diagnostics
|
||||
# module cannot create an eager import cycle. The import-lock wait that
|
||||
# implies is only reachable while a first import is in flight, and
|
||||
# api/routes.py imports all three at load, before any request can dispatch.
|
||||
try:
|
||||
from api import config
|
||||
except Exception as exc:
|
||||
config = None
|
||||
config_import_error = exc
|
||||
else:
|
||||
config_import_error = None
|
||||
try:
|
||||
from api import route_session_list_cache
|
||||
except Exception as exc:
|
||||
route_session_list_cache = None
|
||||
cache_import_error = exc
|
||||
else:
|
||||
cache_import_error = None
|
||||
try:
|
||||
from api import streaming
|
||||
except Exception as exc:
|
||||
streaming = None
|
||||
streaming_import_error = exc
|
||||
else:
|
||||
streaming_import_error = None
|
||||
|
||||
# Collect the config owner exactly once per payload: two calls would double
|
||||
# the blocking edge and let `sessions` and `models_cache` come from two
|
||||
# different snapshots of the same process.
|
||||
config_snapshot: Any = None
|
||||
config_error = config_import_error
|
||||
if config is not None and config_error is None:
|
||||
try:
|
||||
config_snapshot = config.get_runtime_diagnostics_snapshot()
|
||||
except Exception as exc:
|
||||
config_error = exc
|
||||
|
||||
def _config_section(section: str):
|
||||
def project():
|
||||
if not isinstance(config_snapshot, dict):
|
||||
raise TypeError("runtime_snapshot_invalid")
|
||||
return config_snapshot[section]
|
||||
|
||||
return project
|
||||
|
||||
sources = {
|
||||
"sessions": (_config_section("sessions"), config_error),
|
||||
"models_cache": (_config_section("models_cache"), config_error),
|
||||
"streams": (
|
||||
(lambda: streaming.get_stream_runtime_snapshot())
|
||||
if streaming is not None else None,
|
||||
streaming_import_error,
|
||||
),
|
||||
"session_list_cache": (
|
||||
(lambda: route_session_list_cache.get_session_list_cache_snapshot())
|
||||
if route_session_list_cache is not None else None,
|
||||
cache_import_error,
|
||||
),
|
||||
}
|
||||
for name, (collect, import_error) in sources.items():
|
||||
try:
|
||||
if import_error is not None:
|
||||
raise import_error
|
||||
source = collect()
|
||||
if not isinstance(source, dict):
|
||||
raise TypeError("runtime_snapshot_invalid")
|
||||
target = runtime[name]
|
||||
for key in target:
|
||||
if key == "available":
|
||||
target[key] = bool(source.get(key, False))
|
||||
elif key == "age_seconds":
|
||||
age = source.get(key)
|
||||
target[key] = None if age is None else max(0.0, float(age))
|
||||
else:
|
||||
target[key] = _safe_runtime_int(source.get(key, 0))
|
||||
except Exception as exc:
|
||||
errors.append(_safe_error(f"webui_runtime.{name}", exc))
|
||||
return runtime
|
||||
|
||||
|
||||
def build_system_health_payload() -> dict[str, Any]:
|
||||
metrics: dict[str, Any] = {"cpu": None, "memory": None, "disk": None}
|
||||
errors: list[dict[str, str]] = []
|
||||
@@ -289,15 +182,6 @@ def build_system_health_payload() -> dict[str, Any]:
|
||||
except Exception as exc:
|
||||
errors.append(_safe_error(name, exc))
|
||||
|
||||
try:
|
||||
runtime = _webui_runtime_payload(errors)
|
||||
except Exception as exc:
|
||||
# Terminal fail-open: an unexpected failure in the runtime compositor
|
||||
# itself must not fail the whole authenticated response, which returned
|
||||
# host metrics before this subtree existed.
|
||||
runtime = _zero_webui_runtime_payload()
|
||||
errors.append(_safe_error("webui_runtime", exc))
|
||||
|
||||
available = any(metrics[name] is not None for name in metrics)
|
||||
status = "ok" if available and not errors else "partial" if available else "unavailable"
|
||||
return {
|
||||
@@ -307,6 +191,5 @@ def build_system_health_payload() -> dict[str, Any]:
|
||||
"cpu": metrics["cpu"],
|
||||
"memory": metrics["memory"],
|
||||
"disk": metrics["disk"],
|
||||
"webui_runtime": runtime,
|
||||
"errors": errors,
|
||||
}
|
||||
|
||||
+24
-337
@@ -11,7 +11,6 @@ from __future__ import annotations
|
||||
import errno
|
||||
import atexit
|
||||
import codecs
|
||||
import collections
|
||||
import os
|
||||
import queue
|
||||
import shutil
|
||||
@@ -54,12 +53,6 @@ def _safe_close_fd(fd: int) -> None:
|
||||
pass
|
||||
|
||||
|
||||
# Bounds both the replay backlog and each subscriber's live queue: how many
|
||||
# output chunks are buffered before the oldest is dropped. One value so the
|
||||
# catch-up backlog and the per-viewer queue stay in lockstep.
|
||||
_OUTPUT_BUFFER_MAXLEN = 2000
|
||||
|
||||
|
||||
@dataclass
|
||||
class TerminalSession:
|
||||
session_id: str
|
||||
@@ -68,105 +61,30 @@ class TerminalSession:
|
||||
master_fd: int
|
||||
rows: int = 24
|
||||
cols: int = 80
|
||||
# Output is fanned out to every attached viewer. Each SSE consumer
|
||||
# subscribe()s its own queue; put_output broadcasts to all of them plus a
|
||||
# bounded backlog that a newly attaching consumer replays. Two tabs/windows
|
||||
# on the same session therefore each get the FULL byte stream — previously a
|
||||
# single shared queue was read destructively, so two consumers split the
|
||||
# output between them and only one saw terminal_closed.
|
||||
_subscribers: list = field(default_factory=list)
|
||||
_backlog: collections.deque = field(
|
||||
default_factory=lambda: collections.deque(maxlen=_OUTPUT_BUFFER_MAXLEN)
|
||||
)
|
||||
_sub_lock: threading.Lock = field(default_factory=threading.Lock)
|
||||
_next_output_seq: int = 1
|
||||
output: queue.Queue = field(default_factory=lambda: queue.Queue(maxsize=2000))
|
||||
closed: threading.Event = field(default_factory=threading.Event)
|
||||
reader: threading.Thread | None = None
|
||||
# Serializes fd-touching ops (os.write, resize ioctl) against os.close, so a
|
||||
# write can never land on a master_fd that was closed and whose number was
|
||||
# already recycled by a concurrent openpty — that would inject the user's
|
||||
# keystrokes into a foreign fd. Holders re-check ``closed`` under this lock
|
||||
# and bail if the terminal has been torn down.
|
||||
io_lock: threading.Lock = field(default_factory=threading.Lock)
|
||||
# Wall-clock of the last input written or output produced. Drives which
|
||||
# terminal the cap evicts first (least-recently-active): a shell abandoned
|
||||
# at its prompt has neither, so it sorts oldest and is evicted before an
|
||||
# actively used one.
|
||||
last_activity: float = field(default_factory=time.time)
|
||||
# Wall-clock of when the terminal last had zero attached viewers, or None
|
||||
# while at least one is attached. The reaper closes a terminal that has been
|
||||
# unwatched for longer than the idle grace: a client that drops its output
|
||||
# stream without POSTing /api/terminal/close (tab close, crash, network drop)
|
||||
# otherwise leaves the shell running forever (no PDEATHSIG). A terminal is
|
||||
# born unwatched, so a spawn nobody ever attaches to is reaped too. The grace
|
||||
# spans transient reconnects (a tab refresh re-attaches and clears it).
|
||||
unwatched_since: float | None = field(default_factory=time.time)
|
||||
|
||||
def is_alive(self) -> bool:
|
||||
return not self.closed.is_set() and self.proc.poll() is None
|
||||
|
||||
def subscribe(self, after_seq: int | None = None) -> queue.Queue:
|
||||
"""Attach a viewer: return a queue seeded with the current backlog and
|
||||
registered to receive all subsequent output.
|
||||
|
||||
A reconnecting EventSource supplies its last received sequence so only
|
||||
newer backlog entries are replayed. A new viewer leaves ``after_seq``
|
||||
unset and receives the full bounded backlog.
|
||||
"""
|
||||
q: queue.Queue = queue.Queue(maxsize=_OUTPUT_BUFFER_MAXLEN)
|
||||
with self._sub_lock:
|
||||
for item in self._backlog:
|
||||
if after_seq is None or item[0] > after_seq:
|
||||
q.put_nowait(item)
|
||||
self._subscribers.append(q)
|
||||
self.unwatched_since = None # a viewer is attached
|
||||
return q
|
||||
|
||||
def unsubscribe(self, q: queue.Queue) -> None:
|
||||
with self._sub_lock:
|
||||
try:
|
||||
self._subscribers.remove(q)
|
||||
except ValueError:
|
||||
pass
|
||||
if not self._subscribers:
|
||||
self.unwatched_since = time.time()
|
||||
|
||||
def put_output(self, event: str, payload: dict) -> None:
|
||||
self.last_activity = time.time()
|
||||
with self._sub_lock:
|
||||
item = (self._next_output_seq, event, payload)
|
||||
self._next_output_seq += 1
|
||||
self._backlog.append(item)
|
||||
# Keep sequence assignment, backlog append, and non-blocking fanout in
|
||||
# one publication order. Releasing this lock before fanout lets two
|
||||
# producers enqueue seq N+1 before seq N to a live subscriber.
|
||||
for q in self._subscribers:
|
||||
try:
|
||||
q.put_nowait(item)
|
||||
except queue.Full:
|
||||
# Slow viewer: drop its oldest chunk to stay responsive.
|
||||
# Isolated per subscriber, so one lagging tab can't starve
|
||||
# another; all queue operations remain non-blocking.
|
||||
try:
|
||||
q.get_nowait()
|
||||
except queue.Empty:
|
||||
pass
|
||||
try:
|
||||
q.put_nowait(item)
|
||||
except queue.Full:
|
||||
pass
|
||||
try:
|
||||
self.output.put_nowait((event, payload))
|
||||
except queue.Full:
|
||||
# Keep the terminal responsive by dropping the oldest queued chunk.
|
||||
try:
|
||||
self.output.get_nowait()
|
||||
except queue.Empty:
|
||||
pass
|
||||
try:
|
||||
self.output.put_nowait((event, payload))
|
||||
except queue.Full:
|
||||
pass
|
||||
|
||||
|
||||
_TERMINALS: dict[str, TerminalSession] = {}
|
||||
_LOCK = threading.RLock()
|
||||
# Hard cap on concurrently live embedded terminals. Each holds a shell process,
|
||||
# a pty master fd, and a reader thread; a client that drops its output stream
|
||||
# without POSTing /api/terminal/close (tab close, crash, network drop) leaves
|
||||
# the shell running (no PDEATHSIG — see the note below), so without a ceiling
|
||||
# these accumulate over a long uptime toward fd/thread exhaustion (#4633). The
|
||||
# cap evicts the least-recently-active terminal to make room. Generous enough
|
||||
# that real interactive use never trips it.
|
||||
_MAX_TERMINALS = 32
|
||||
_spawn_queue: queue.Queue = queue.Queue()
|
||||
_spawn_supervisor_started = False
|
||||
_spawn_supervisor_lock = threading.Lock()
|
||||
@@ -174,19 +92,6 @@ _spawn_supervisor_thread: threading.Thread | None = None
|
||||
_terminal_descendant_reaper_lock = threading.Lock()
|
||||
_TERMINAL_DESCENDANT_REAPER_LIMIT = 64
|
||||
|
||||
# Idle-terminal reaper: proactively close terminals whose viewers have all gone
|
||||
# away, instead of leaving an abandoned shell running until the cap evicts it.
|
||||
# A terminal unwatched (zero attached output streams) for longer than the grace
|
||||
# is closed; the grace spans a tab refresh / brief network drop so a real
|
||||
# reconnect keeps the session. Dead-process terminals are swept too as a
|
||||
# belt-and-suspenders for the reader-loop retire.
|
||||
_TERMINAL_IDLE_GRACE_SECONDS = 900 # 15 min unwatched -> reap
|
||||
_TERMINAL_REAPER_INTERVAL_SECONDS = 60
|
||||
_terminal_reaper_started = False
|
||||
_terminal_reaper_lock = threading.Lock()
|
||||
_terminal_reaper_thread: threading.Thread | None = None
|
||||
_terminal_reaper_stop = threading.Event()
|
||||
|
||||
|
||||
@dataclass
|
||||
class _SpawnRequest:
|
||||
@@ -373,27 +278,15 @@ def _reader_loop(term: TerminalSession) -> None:
|
||||
code = term.proc.poll()
|
||||
_reap_terminal_descendants(term.proc.pid)
|
||||
term.put_output("terminal_closed", {"exit_code": code})
|
||||
# The shell has exited (or its pty broke): retire the session so its
|
||||
# master fd and _TERMINALS entry are released. Previously only an
|
||||
# explicit close_terminal() / restart / atexit did this, so a shell that
|
||||
# exited on its own (user typed `exit`, process died) leaked its master
|
||||
# fd and dict entry for the rest of the WebUI's uptime. ``expected=term``
|
||||
# makes this a no-op if a restart already replaced the entry with a new
|
||||
# terminal for the same session id.
|
||||
close_terminal(term.session_id, expected=term)
|
||||
|
||||
|
||||
def _set_size(term: TerminalSession, rows: int, cols: int) -> None:
|
||||
term.rows = max(8, min(int(rows or term.rows or 24), 80))
|
||||
term.cols = max(20, min(int(cols or term.cols or 80), 240))
|
||||
# The ioctl touches master_fd, so guard it against a concurrent close (and
|
||||
# the fd-number recycling that can follow) with the same io_lock as writes.
|
||||
with term.io_lock:
|
||||
if not term.closed.is_set():
|
||||
try:
|
||||
fcntl.ioctl(term.master_fd, termios.TIOCSWINSZ, _winsize(term.rows, term.cols))
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
fcntl.ioctl(term.master_fd, termios.TIOCSWINSZ, _winsize(term.rows, term.cols))
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
if term.proc.poll() is None:
|
||||
os.killpg(term.proc.pid, signal.SIGWINCH)
|
||||
@@ -401,138 +294,6 @@ def _set_size(term: TerminalSession, rows: int, cols: int) -> None:
|
||||
pass
|
||||
|
||||
|
||||
def _enforce_terminal_cap(*, exclude_sid: str | None = None) -> None:
|
||||
"""Evict terminals until there is room under ``_MAX_TERMINALS``.
|
||||
|
||||
Picks a victim under the lock but closes it *outside* the lock (close_terminal
|
||||
may spend up to a couple of seconds killing/waiting on the shell). Prefers a
|
||||
dead-process terminal, else the least-recently-active one — an abandoned
|
||||
shell idle at its prompt sorts oldest and goes first. ``exclude_sid`` is the
|
||||
session about to reuse/replace its own entry, so it never evicts itself.
|
||||
"""
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
return
|
||||
# Bounded loop: at most the current population; guards against a pathological
|
||||
# spin if close_terminal somehow can't remove an entry.
|
||||
for _ in range(_MAX_TERMINALS + 1):
|
||||
victim_sid = None
|
||||
victim_term = None
|
||||
with _LOCK:
|
||||
# Reuse/restart of an existing sid replaces in place — no growth.
|
||||
if exclude_sid in _TERMINALS:
|
||||
return
|
||||
if len(_TERMINALS) < _MAX_TERMINALS:
|
||||
return
|
||||
candidates = [
|
||||
(sid, term) for sid, term in _TERMINALS.items() if sid != exclude_sid
|
||||
]
|
||||
if not candidates:
|
||||
return
|
||||
dead = [(sid, term) for sid, term in candidates if not term.is_alive()]
|
||||
victim_sid, victim_term = (
|
||||
dead[0] if dead else min(candidates, key=lambda kv: kv[1].last_activity)
|
||||
)
|
||||
# ``expected=victim_term`` so that if this sid was restarted/replaced in
|
||||
# the gap between picking it and closing it, we don't tear down the new
|
||||
# (possibly active) terminal — symmetric with the reader-loop retire.
|
||||
close_terminal(victim_sid, expected=victim_term)
|
||||
|
||||
|
||||
def _terminals_to_reap(now: float) -> list[tuple[str, TerminalSession]]:
|
||||
"""Return (sid, term) pairs the reaper should close: a dead process, or a
|
||||
terminal unwatched for longer than the idle grace. Pure/snapshotted under
|
||||
the lock so it can be unit-tested without threads."""
|
||||
victims = []
|
||||
with _LOCK:
|
||||
for sid, term in _TERMINALS.items():
|
||||
if not term.is_alive():
|
||||
victims.append((sid, term))
|
||||
continue
|
||||
unwatched = term.unwatched_since
|
||||
if unwatched is not None and (now - unwatched) >= _TERMINAL_IDLE_GRACE_SECONDS:
|
||||
victims.append((sid, term))
|
||||
return victims
|
||||
|
||||
|
||||
def _claim_reap_victim(sid: str, term: TerminalSession, now: float) -> TerminalSession | None:
|
||||
"""Atomically remove *term* from the registry, or refuse.
|
||||
|
||||
``_terminals_to_reap`` snapshots victims and then releases ``_LOCK``, so by
|
||||
the time we get here a viewer may have reconnected: ``subscribe()`` appends
|
||||
to ``_subscribers`` and clears ``unwatched_since`` under the terminal's
|
||||
``_sub_lock``, which the selection pass never held. Object identity alone —
|
||||
what ``close_terminal(expected=…)`` checks — is still true in that case, so
|
||||
the reaper would kill a terminal that now has a live viewer.
|
||||
|
||||
The claim therefore re-establishes the *whole* selection predicate while
|
||||
holding both locks, and takes the entry out of ``_TERMINALS`` in the same
|
||||
critical section. A concurrent ``attach_terminal()`` acquires the same two
|
||||
locks in the same order, so exactly one of the two wins: either the viewer
|
||||
is attached (and we refuse) or the entry is already gone (and the attach
|
||||
reports "not running").
|
||||
|
||||
Returns the claimed terminal — the caller owns its teardown — or ``None``.
|
||||
"""
|
||||
with _LOCK:
|
||||
if _TERMINALS.get(sid) is not term:
|
||||
return None
|
||||
# A dead process is reaped unconditionally: it cannot come back to life,
|
||||
# and an attached viewer only means someone is watching a corpse.
|
||||
if term.is_alive():
|
||||
with term._sub_lock:
|
||||
if term._subscribers:
|
||||
return None
|
||||
unwatched = term.unwatched_since
|
||||
if unwatched is None:
|
||||
return None
|
||||
if (now - unwatched) < _TERMINAL_IDLE_GRACE_SECONDS:
|
||||
return None
|
||||
del _TERMINALS[sid]
|
||||
return term
|
||||
|
||||
|
||||
def _reap_idle_terminals(now: float) -> int:
|
||||
"""Close every terminal selected by ``_terminals_to_reap`` that is *still*
|
||||
idle when claimed. Returns the count closed."""
|
||||
reaped = 0
|
||||
for sid, term in _terminals_to_reap(now):
|
||||
claimed = _claim_reap_victim(sid, term, now)
|
||||
if claimed is None:
|
||||
continue
|
||||
# Process/fd teardown runs after both locks are released: killpg + wait
|
||||
# can take seconds and must not block attaches or spawns.
|
||||
_teardown_terminal(claimed)
|
||||
reaped += 1
|
||||
return reaped
|
||||
|
||||
|
||||
def _terminal_reaper_loop() -> None:
|
||||
while not _terminal_reaper_stop.wait(_TERMINAL_REAPER_INTERVAL_SECONDS):
|
||||
try:
|
||||
# Wall-clock, consistent with unwatched_since / last_activity.
|
||||
_reap_idle_terminals(time.time())
|
||||
except Exception:
|
||||
# Never let a transient error kill the reaper thread.
|
||||
pass
|
||||
|
||||
|
||||
def _ensure_terminal_reaper() -> None:
|
||||
global _terminal_reaper_started, _terminal_reaper_thread
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
return
|
||||
with _terminal_reaper_lock:
|
||||
if (
|
||||
_terminal_reaper_started
|
||||
and _terminal_reaper_thread is not None
|
||||
and getattr(_terminal_reaper_thread, "is_alive", lambda: False)()
|
||||
):
|
||||
return
|
||||
thread = threading.Thread(target=_terminal_reaper_loop, daemon=True)
|
||||
thread.start()
|
||||
_terminal_reaper_thread = thread
|
||||
_terminal_reaper_started = True
|
||||
|
||||
|
||||
def start_terminal(session_id: str, workspace: Path, rows: int = 24, cols: int = 80, restart: bool = False) -> TerminalSession:
|
||||
"""Start or return the embedded terminal for a WebUI session."""
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
@@ -544,12 +305,6 @@ def start_terminal(session_id: str, workspace: Path, rows: int = 24, cols: int =
|
||||
if not Path(cwd).is_dir():
|
||||
raise ValueError("workspace is not a directory")
|
||||
|
||||
# Enforce the cap before spawning. Done outside the main lock below (the
|
||||
# eviction's process teardown must not run while _LOCK is held for the whole
|
||||
# spawn), and skipped for a same-sid reuse/restart, which replaces rather
|
||||
# than adds an entry.
|
||||
_enforce_terminal_cap(exclude_sid=sid)
|
||||
|
||||
with _LOCK:
|
||||
current = _TERMINALS.get(sid)
|
||||
if current and current.is_alive() and not restart and current.workspace == cwd:
|
||||
@@ -594,7 +349,6 @@ def start_terminal(session_id: str, workspace: Path, rows: int = 24, cols: int =
|
||||
}
|
||||
)
|
||||
_ensure_spawn_supervisor()
|
||||
_ensure_terminal_reaper()
|
||||
_spawn_queue.put(request)
|
||||
try:
|
||||
if not request.done.wait(timeout=5.0):
|
||||
@@ -642,56 +396,13 @@ def get_terminal(session_id: str) -> TerminalSession | None:
|
||||
return term
|
||||
|
||||
|
||||
def attach_terminal(
|
||||
session_id: str, after_seq: int | None = None
|
||||
) -> tuple[TerminalSession, queue.Queue] | None:
|
||||
"""Attach a viewer atomically against the idle reaper.
|
||||
|
||||
``get_terminal()`` followed by ``term.subscribe()`` leaves a window: the
|
||||
reaper can claim and tear the terminal down in between, so the viewer ends
|
||||
up subscribed to a corpse and the caller reports a live stream that will
|
||||
never produce output. Doing the lookup and the subscribe inside the same
|
||||
``_LOCK`` section — the same lock, in the same order, that
|
||||
``_claim_reap_victim`` takes — makes the two mutually exclusive:
|
||||
|
||||
* attach wins → ``_subscribers`` is non-empty and ``unwatched_since`` is
|
||||
``None`` before the reaper can revalidate, so the reap is refused;
|
||||
* reap wins → the entry is already out of ``_TERMINALS``, so this returns
|
||||
``None`` and the route answers "terminal not running" instead of hanging.
|
||||
|
||||
Registration is the authority, deliberately: both the reaper and
|
||||
``close_terminal()`` remove the entry *before* tearing the terminal down, so
|
||||
"still in ``_TERMINALS``" is exactly the condition that cannot race. A
|
||||
terminal that is registered but already flagged ``closed`` (its shell exited
|
||||
and the reader loop has not retired it yet) still attaches, so the viewer
|
||||
receives the ``terminal_closed`` event instead of a bare 404.
|
||||
|
||||
Returns ``(term, queue)`` or ``None``.
|
||||
"""
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
return None
|
||||
sid = str(session_id or "")
|
||||
with _LOCK:
|
||||
term = _TERMINALS.get(sid)
|
||||
if term is None:
|
||||
return None
|
||||
return term, term.subscribe(after_seq=after_seq)
|
||||
|
||||
|
||||
def write_terminal(session_id: str, data: str) -> None:
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
raise NotImplementedError("Embedded terminal is not supported on Windows")
|
||||
term = get_terminal(session_id)
|
||||
if not term or not term.is_alive():
|
||||
raise KeyError("terminal not running")
|
||||
# Re-check ``closed`` under io_lock and write while holding it, so the fd
|
||||
# can't be closed (and its number recycled by another openpty) between the
|
||||
# check and the write — which would inject this input into a foreign fd.
|
||||
with term.io_lock:
|
||||
if term.closed.is_set():
|
||||
raise KeyError("terminal not running")
|
||||
os.write(term.master_fd, str(data or "").encode("utf-8", errors="replace"))
|
||||
term.last_activity = time.time()
|
||||
os.write(term.master_fd, str(data or "").encode("utf-8", errors="replace"))
|
||||
|
||||
|
||||
def resize_terminal(session_id: str, rows: int, cols: int) -> None:
|
||||
@@ -703,36 +414,14 @@ def resize_terminal(session_id: str, rows: int, cols: int) -> None:
|
||||
_set_size(term, rows, cols)
|
||||
|
||||
|
||||
def close_terminal(session_id: str, *, expected: TerminalSession | None = None) -> bool:
|
||||
"""Tear down the terminal for *session_id*: kill the shell, close the pty
|
||||
master fd, reap descendants, and drop the ``_TERMINALS`` entry.
|
||||
|
||||
``expected`` guards the retire-from-reader-loop path: only act if the live
|
||||
entry is still that exact terminal, so an old reader thread finishing after
|
||||
a restart cannot tear down the *new* terminal that replaced it (the old
|
||||
one's fd was already closed by the restart's own close_terminal call).
|
||||
"""
|
||||
def close_terminal(session_id: str) -> bool:
|
||||
if not _TERMINAL_SUPPORTED:
|
||||
return False
|
||||
sid = str(session_id or "")
|
||||
with _LOCK:
|
||||
if expected is not None and _TERMINALS.get(sid) is not expected:
|
||||
return False
|
||||
term = _TERMINALS.pop(sid, None)
|
||||
if not term:
|
||||
return False
|
||||
_teardown_terminal(term)
|
||||
return True
|
||||
|
||||
|
||||
def _teardown_terminal(term: TerminalSession) -> None:
|
||||
"""Kill the shell, close the pty master fd and reap descendants.
|
||||
|
||||
Split out of ``close_terminal`` so a caller that has already claimed the
|
||||
registry entry (the idle reaper) can run the teardown without re-entering
|
||||
the registry lookup — and, importantly, without holding any lock while
|
||||
``killpg``/``wait`` block for up to ~2.5s.
|
||||
"""
|
||||
term.closed.set()
|
||||
try:
|
||||
if term.proc.poll() is None:
|
||||
@@ -752,14 +441,12 @@ def _teardown_terminal(term: TerminalSession) -> None:
|
||||
except (subprocess.TimeoutExpired, ProcessLookupError):
|
||||
pass
|
||||
finally:
|
||||
# ``closed`` is already set above, so a writer/resizer blocked on io_lock
|
||||
# will see it and bail rather than touch the fd we are about to close.
|
||||
with term.io_lock:
|
||||
try:
|
||||
os.close(term.master_fd)
|
||||
except OSError:
|
||||
pass
|
||||
try:
|
||||
os.close(term.master_fd)
|
||||
except OSError:
|
||||
pass
|
||||
_reap_terminal_descendants(term.proc.pid)
|
||||
return True
|
||||
|
||||
|
||||
def close_all_terminals() -> None:
|
||||
|
||||
+169
-1008
File diff suppressed because it is too large
Load Diff
+10
-89
@@ -4,13 +4,13 @@ Hermes Web UI -- File upload: multipart parser and upload handler.
|
||||
import mimetypes
|
||||
import os
|
||||
import re as _re
|
||||
import email.parser
|
||||
import tempfile
|
||||
from pathlib import Path
|
||||
|
||||
from api.config import MAX_UPLOAD_BYTES, STATE_DIR
|
||||
from api.helpers import j
|
||||
from api.helpers import j, bad
|
||||
from api.models import get_session
|
||||
from api.profiles import _profiles_match, get_active_profile_name as _get_active_profile_name
|
||||
from api.workspace import (
|
||||
safe_resolve_ws,
|
||||
resolve_trusted_workspace,
|
||||
@@ -74,6 +74,7 @@ def parse_multipart(rfile, content_type, content_length) -> tuple:
|
||||
fields = {}
|
||||
files = {}
|
||||
delimiter = b'--' + boundary
|
||||
end_marker = b'--' + boundary + b'--'
|
||||
parts = raw.split(delimiter)
|
||||
for part in parts[1:]:
|
||||
stripped = part.lstrip(b'\r\n')
|
||||
@@ -122,8 +123,8 @@ def _attachment_root() -> Path:
|
||||
return (STATE_DIR / 'attachments').resolve()
|
||||
|
||||
|
||||
def _upload_destination(session_id: str, safe_name: str, dest_dir: Path | None = None) -> Path:
|
||||
dest_dir = dest_dir if dest_dir is not None else _session_attachment_dir(session_id)
|
||||
def _upload_destination(session_id: str, safe_name: str) -> Path:
|
||||
dest_dir = _session_attachment_dir(session_id)
|
||||
dest_dir.mkdir(parents=True, exist_ok=True)
|
||||
dest = (dest_dir / safe_name).resolve()
|
||||
if not dest.is_relative_to(dest_dir):
|
||||
@@ -149,60 +150,6 @@ def _session_attachment_dir(session_id: str, *, root: Path | None = None) -> Pat
|
||||
return dest_dir
|
||||
|
||||
|
||||
def _session_visible_to_active_profile(session) -> bool:
|
||||
"""Return whether an upload target session belongs to the active profile."""
|
||||
session_profile = getattr(session, 'profile', None)
|
||||
if not isinstance(session_profile, str):
|
||||
session_profile = None
|
||||
return _profiles_match(session_profile, _get_active_profile_name())
|
||||
|
||||
|
||||
def _reject_invisible_session(handler, session) -> bool:
|
||||
if _session_visible_to_active_profile(session):
|
||||
return False
|
||||
j(handler, {'error': 'Session not found'}, status=404)
|
||||
return True
|
||||
|
||||
|
||||
def _write_office_upload_sidecar(workspace: Path, dest: Path, file_bytes: bytes) -> dict | None:
|
||||
"""Write a Markdown preview sidecar for supported Office uploads."""
|
||||
if dest.suffix.lower() not in {'.docx', '.xlsx', '.pptx'}:
|
||||
return None
|
||||
|
||||
error_message = 'Office sidecar extraction failed'
|
||||
sidecar = dest.with_name(f'{dest.name}.md')
|
||||
sidecar_path = sidecar.resolve()
|
||||
created_sidecar = False
|
||||
try:
|
||||
from api.office_documents import preview_office_document
|
||||
|
||||
preview = preview_office_document(dest.name, file_bytes)
|
||||
if not sidecar_path.is_relative_to(workspace.resolve()):
|
||||
raise ValueError('Invalid sidecar destination')
|
||||
sidecar_fd = open_anchored_create_fd(workspace, sidecar_path)
|
||||
created_sidecar = True
|
||||
with os.fdopen(sidecar_fd, 'w', encoding='utf-8', closefd=True) as sidecar_file:
|
||||
sidecar_file.write(str(preview.get('content') or ''))
|
||||
return {
|
||||
'filename': sidecar_path.name,
|
||||
'path': str(sidecar_path),
|
||||
'size': sidecar_path.stat().st_size,
|
||||
'preview_kind': preview.get('preview_kind'),
|
||||
'office_format': preview.get('office_format'),
|
||||
}
|
||||
except FileExistsError:
|
||||
return {'error': error_message}
|
||||
except Exception:
|
||||
if created_sidecar:
|
||||
try:
|
||||
unlink_anchored(workspace, sidecar_path)
|
||||
except FileNotFoundError:
|
||||
pass
|
||||
except Exception:
|
||||
pass
|
||||
return {'error': error_message}
|
||||
|
||||
|
||||
def handle_upload(handler):
|
||||
import traceback as _tb
|
||||
try:
|
||||
@@ -221,23 +168,9 @@ def handle_upload(handler):
|
||||
s = get_session(session_id)
|
||||
except KeyError:
|
||||
return j(handler, {'error': 'Session not found'}, status=404)
|
||||
if _reject_invisible_session(handler, s):
|
||||
return True
|
||||
safe_name = _sanitize_upload_name(filename)
|
||||
dest_dir = _session_attachment_dir(session_id)
|
||||
dest = _upload_destination(session_id, safe_name, dest_dir)
|
||||
# #3398-style TOCTOU hardening, mirrored from the workspace upload
|
||||
# path: O_CREAT|O_EXCL|O_NOFOLLOW anchored open so a raced duplicate
|
||||
# cannot be silently overwritten and a raced symlink subpath cannot
|
||||
# redirect the write outside the attachment dir.
|
||||
try:
|
||||
_wfd = open_anchored_create_fd(dest_dir, dest)
|
||||
except FileExistsError:
|
||||
return j(handler, {'error': f'Upload destination already exists: {safe_name}'}, status=409)
|
||||
except (ValueError, OSError):
|
||||
return j(handler, {'error': 'Upload destination rejected'}, status=403)
|
||||
with os.fdopen(_wfd, 'wb', closefd=True) as _wfh:
|
||||
_wfh.write(file_bytes)
|
||||
dest = _upload_destination(session_id, safe_name)
|
||||
dest.write_bytes(file_bytes)
|
||||
mime = mimetypes.guess_type(safe_name)[0] or 'application/octet-stream'
|
||||
return j(handler, {
|
||||
'filename': dest.name,
|
||||
@@ -259,7 +192,7 @@ def extract_archive(file_bytes: bytes, filename: str, workspace: Path):
|
||||
Returns a dict with ``extracted`` (int), ``files`` (list[str]).
|
||||
Raises ValueError on zip-slip or unsupported format.
|
||||
"""
|
||||
import zipfile, tarfile, io, os
|
||||
import zipfile, tarfile, io, os, shutil
|
||||
|
||||
cap = _max_extracted_bytes()
|
||||
name = Path(filename).name
|
||||
@@ -411,8 +344,6 @@ def handle_upload_extract(handler):
|
||||
s = get_session(session_id)
|
||||
except KeyError:
|
||||
return j(handler, {'error': 'Session not found'}, status=404)
|
||||
if _reject_invisible_session(handler, s):
|
||||
return True
|
||||
session_dir = _session_attachment_dir(session_id)
|
||||
session_dir.mkdir(parents=True, exist_ok=True)
|
||||
result = extract_archive(file_bytes, filename, session_dir)
|
||||
@@ -632,16 +563,9 @@ def handle_workspace_upload(handler):
|
||||
session = get_session(session_id)
|
||||
except KeyError:
|
||||
return j(handler, {'error': 'Session not found'}, status=404)
|
||||
if _reject_invisible_session(handler, session):
|
||||
return True
|
||||
|
||||
# Resolve workspace root using the session profile, not the ambient request profile.
|
||||
try:
|
||||
workspace = resolve_trusted_workspace(
|
||||
session.workspace, profile=getattr(session, "profile", None)
|
||||
)
|
||||
except TypeError:
|
||||
workspace = resolve_trusted_workspace(session.workspace)
|
||||
# Resolve workspace root from session
|
||||
workspace = resolve_trusted_workspace(session.workspace)
|
||||
|
||||
# Resolve target subdirectory within workspace
|
||||
target_dir = safe_resolve_ws(workspace, subpath) if subpath else workspace
|
||||
@@ -762,7 +686,6 @@ def handle_workspace_upload(handler):
|
||||
})
|
||||
continue
|
||||
|
||||
sidecar = _write_office_upload_sidecar(workspace, dest, file_bytes)
|
||||
results.append({
|
||||
'filename': dest.name,
|
||||
'path': str(dest),
|
||||
@@ -770,8 +693,6 @@ def handle_workspace_upload(handler):
|
||||
'mime': mime,
|
||||
'is_image': mime.startswith('image/'),
|
||||
'extracted': False,
|
||||
**({'sidecar': sidecar} if sidecar and 'error' not in sidecar else {}),
|
||||
**({'sidecar_error': sidecar['error']} if sidecar and 'error' in sidecar else {}),
|
||||
})
|
||||
|
||||
if len(results) == 1:
|
||||
|
||||
@@ -51,10 +51,6 @@ _METADATA_FIELDS = frozenset(
|
||||
"context_length",
|
||||
"threshold_tokens",
|
||||
"last_prompt_tokens",
|
||||
"compression_recovery",
|
||||
"recommended_recovery_action",
|
||||
"compression_recovery_source_session_id",
|
||||
"compression_recovery_action",
|
||||
"truncation_watermark",
|
||||
"truncation_boundary",
|
||||
"gateway_routing",
|
||||
|
||||
+63
-439
@@ -12,16 +12,13 @@ import json
|
||||
import logging
|
||||
import os
|
||||
import posixpath
|
||||
import re
|
||||
import secrets
|
||||
import shutil
|
||||
import stat
|
||||
import subprocess
|
||||
import sys
|
||||
import concurrent.futures
|
||||
import threading
|
||||
import time
|
||||
from collections.abc import Callable
|
||||
from pathlib import Path, PurePosixPath
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
@@ -36,19 +33,12 @@ from api.config import (
|
||||
DEFAULT_WORKSPACE as _BOOT_DEFAULT_WORKSPACE,
|
||||
MAX_FILE_BYTES, IMAGE_EXTS, MD_EXTS
|
||||
)
|
||||
from api.subprocess_utils import windows_hide_flags
|
||||
|
||||
|
||||
# ── Profile-aware path resolution ───────────────────────────────────────────
|
||||
|
||||
# Logical profile-name grammar — mirrors api.profiles._PROFILE_ID_RE. Kept as
|
||||
# a local copy so workspace-layer validation does not import profiles at module
|
||||
# load time (profiles may not be importable in every embedding context).
|
||||
_PROFILE_NAME_RE = re.compile(r'^[a-z0-9][a-z0-9_-]{0,63}$')
|
||||
|
||||
|
||||
def _profile_state_dir(profile: str | Path | None = None) -> Path:
|
||||
"""Return the webui_state directory for the active or given profile.
|
||||
def _profile_state_dir() -> Path:
|
||||
"""Return the webui_state directory for the active profile.
|
||||
|
||||
For the default profile, returns the global STATE_DIR (respects
|
||||
HERMES_WEBUI_STATE_DIR env var for test isolation).
|
||||
@@ -56,28 +46,6 @@ def _profile_state_dir(profile: str | Path | None = None) -> Path:
|
||||
"""
|
||||
try:
|
||||
from api.profiles import get_active_profile_name, get_active_hermes_home
|
||||
if profile is not None:
|
||||
# Literal-"default" STATE routing (#7168 re-gate round 7): the
|
||||
# default profile's workspace state always lives in the global
|
||||
# state files, even when isolated mode pins the default home at
|
||||
# <base>/profiles/default. The round-6 resolver change made
|
||||
# _resolve_profile_home_param("default") return that pinned home,
|
||||
# so the canonical-home check below sent explicit
|
||||
# profile="default" state reads/writes to {pinned}/webui_state/
|
||||
# while ambient calls kept using the global dir — splitting saved
|
||||
# workspaces between two authorities. Config and workspace PATH
|
||||
# resolution still use the pinned home via
|
||||
# _resolve_profile_home_param; only this state-file tier stays
|
||||
# global for the logical string "default".
|
||||
if isinstance(profile, str) and profile.strip() == 'default':
|
||||
return _GLOBAL_WS_FILE.parent
|
||||
profile_home = _resolve_profile_home_param(profile)
|
||||
if not _is_default_profile_home(profile_home):
|
||||
d = profile_home / 'webui_state'
|
||||
d.mkdir(parents=True, exist_ok=True)
|
||||
return d
|
||||
return _GLOBAL_WS_FILE.parent
|
||||
|
||||
name = get_active_profile_name()
|
||||
if name and name != 'default':
|
||||
d = get_active_hermes_home() / 'webui_state'
|
||||
@@ -88,41 +56,14 @@ def _profile_state_dir(profile: str | Path | None = None) -> Path:
|
||||
return _GLOBAL_WS_FILE.parent
|
||||
|
||||
|
||||
def _workspaces_file(profile: str | Path | None = None) -> Path:
|
||||
"""Return the workspaces.json path for the active or given profile."""
|
||||
return _profile_state_dir(profile=profile) / 'workspaces.json'
|
||||
def _workspaces_file() -> Path:
|
||||
"""Return the workspaces.json path for the active profile."""
|
||||
return _profile_state_dir() / 'workspaces.json'
|
||||
|
||||
|
||||
def _last_workspace_file(profile: str | Path | None = None) -> Path:
|
||||
"""Return the last_workspace.txt path for the active or given profile."""
|
||||
return _profile_state_dir(profile=profile) / 'last_workspace.txt'
|
||||
|
||||
|
||||
def _workspaces_file_for_profile(profile: str | Path | None = None) -> Path | None:
|
||||
"""Profile-scoped workspaces.json path, or None for an INVALID profile.
|
||||
|
||||
``None`` is the fail-closed contract (#7168 re-gate round 4): a malformed
|
||||
profile name must not be clamped onto the default/global state files, so
|
||||
callers treat it as "no readable/writable profile-local state".
|
||||
"""
|
||||
try:
|
||||
return _workspaces_file(profile=profile) if profile is not None else _workspaces_file()
|
||||
except TypeError:
|
||||
return _workspaces_file()
|
||||
except ValueError:
|
||||
logger.debug("Ignoring invalid profile name %r for workspaces file", profile)
|
||||
return None
|
||||
|
||||
|
||||
def _last_workspace_file_for_profile(profile: str | Path | None = None) -> Path | None:
|
||||
"""Profile-scoped last_workspace.txt path, or None for an INVALID profile."""
|
||||
try:
|
||||
return _last_workspace_file(profile=profile) if profile is not None else _last_workspace_file()
|
||||
except TypeError:
|
||||
return _last_workspace_file()
|
||||
except ValueError:
|
||||
logger.debug("Ignoring invalid profile name %r for last-workspace file", profile)
|
||||
return None
|
||||
def _last_workspace_file() -> Path:
|
||||
"""Return the last_workspace.txt path for the active profile."""
|
||||
return _profile_state_dir() / 'last_workspace.txt'
|
||||
|
||||
|
||||
def _expanduser_path(path: str | Path) -> Path:
|
||||
@@ -157,11 +98,8 @@ def _expanduser_path(path: str | Path) -> Path:
|
||||
return Path(raw)
|
||||
|
||||
|
||||
def _resolve_path(path: str | Path, profile: str | Path | None = None) -> Path:
|
||||
"""Resolve *path* after env-aware home expansion, preserving remote POSIX paths without raising."""
|
||||
remote_candidate = _remote_terminal_workspace_candidate(path, profile=profile)
|
||||
if remote_candidate is not None:
|
||||
return remote_candidate
|
||||
def _resolve_path(path: str | Path) -> Path:
|
||||
"""Resolve *path* after env-aware home expansion, without raising."""
|
||||
return _safe_resolve(_expanduser_path(path))
|
||||
|
||||
|
||||
@@ -207,75 +145,12 @@ def _is_remote_terminal_backend(terminal_cfg: dict | None) -> bool:
|
||||
return backend not in ('', 'local')
|
||||
|
||||
|
||||
def _resolve_profile_home_param(profile: str | Path | None) -> Path:
|
||||
"""Resolve a profile parameter (name string, directory path string, or Path) to a profile home Path.
|
||||
|
||||
Logical profile names are validated strictly (#7168 re-gate round 4): a
|
||||
name that fails the profile-id grammar raises ValueError instead of being
|
||||
silently clamped onto the default home — the old clamp let a malformed
|
||||
name such as ``"bad name"`` read/write the DEFAULT profile's state.
|
||||
Round 5 tightens the grammar gate: a STRING profile value is strictly a
|
||||
logical profile id and is NEVER treated as a path-shaped home — the old
|
||||
``"/" in raw`` branch resolved any slash-bearing string directly, so a
|
||||
malformed value like ``"../evil"`` bypassed validation entirely and could
|
||||
read/overwrite an arbitrary ``webui_state/last_workspace.txt``
|
||||
(#7168 re-gate round 5, path traversal on the profile-isolation boundary).
|
||||
Round 6 closes the last isolation hole in this resolver: the logical
|
||||
string ``"default"`` used to short-circuit to ``_DEFAULT_HERMES_HOME``
|
||||
before reaching ``get_hermes_home_for_profile()``, bypassing the
|
||||
isolated-mode clamp in ``api.profiles._resolve_profile_home_for_name``
|
||||
(#7168 re-gate round 6). In an isolated deployment pinned at
|
||||
``<base>/profiles/default``, a session created with ``profile="default"``
|
||||
therefore resolved workspace/config from the BASE root home instead of
|
||||
the pinned one. The name now flows through the same delegated path as
|
||||
every other logical id; literal-default routing to the global state
|
||||
files is retained in ``_profile_state_dir``/``get_last_workspace`` via
|
||||
canonical ``_is_default_profile_home`` identity.
|
||||
An explicit home directory is expressed as a ``Path`` object (callers such
|
||||
as streaming's legacy ``_profile_home`` fallback wrap their home strings
|
||||
in ``Path``); Path values are honored and canonicalized so identity
|
||||
comparisons never depend on lexical spelling (e.g. a symlink alias of the
|
||||
default home must compare equal to it).
|
||||
"""
|
||||
if profile is None or str(profile).strip() == "":
|
||||
from api.profiles import get_active_hermes_home
|
||||
return get_active_hermes_home()
|
||||
|
||||
raw = str(profile).strip()
|
||||
|
||||
if isinstance(profile, Path):
|
||||
return _safe_resolve(profile.expanduser())
|
||||
|
||||
# Strings are LOGICAL PROFILE IDS ONLY — no path-shaped strings, ever.
|
||||
if not _PROFILE_NAME_RE.fullmatch(raw):
|
||||
raise ValueError(f"invalid profile name: {raw!r}")
|
||||
|
||||
from api.profiles import get_hermes_home_for_profile
|
||||
return _safe_resolve(get_hermes_home_for_profile(raw))
|
||||
|
||||
|
||||
def _is_default_profile_home(profile_home: Path) -> bool:
|
||||
"""Canonical identity check against the root/default Hermes home.
|
||||
|
||||
Compares resolved paths so a symlink alias of _DEFAULT_HERMES_HOME is
|
||||
recognized as the default profile rather than treated as a foreign,
|
||||
lexically-different directory (#7168 re-gate round 4).
|
||||
"""
|
||||
def _remote_terminal_cwd() -> str | None:
|
||||
"""Return target-side terminal cwd for remote profiles, without local stat()."""
|
||||
try:
|
||||
from api.profiles import _DEFAULT_HERMES_HOME
|
||||
return _safe_resolve(profile_home) == _safe_resolve(_DEFAULT_HERMES_HOME)
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
|
||||
def _remote_terminal_cwd(profile: str | Path | None = None) -> str | None:
|
||||
"""Return target-side terminal cwd for a remote profile, without local stat()."""
|
||||
try:
|
||||
from api.config import get_config_for_profile_home
|
||||
|
||||
profile_home = _resolve_profile_home_param(profile)
|
||||
terminal_cfg = get_config_for_profile_home(profile_home).get('terminal', {})
|
||||
from api.config import get_config
|
||||
|
||||
terminal_cfg = get_config().get('terminal', {})
|
||||
if not _is_remote_terminal_backend(terminal_cfg):
|
||||
return None
|
||||
cwd = str(terminal_cfg.get('cwd') or '').strip()
|
||||
@@ -287,19 +162,9 @@ def _remote_terminal_cwd(profile: str | Path | None = None) -> str | None:
|
||||
return None
|
||||
|
||||
|
||||
def _remote_terminal_workspace_candidate(path: str | Path, profile: str | Path | None = None) -> Path | None:
|
||||
"""Return a non-stat'ed target-side Path when it is under terminal.cwd for the given profile.
|
||||
|
||||
Remote workspace paths live on the target host (e.g., remote SSH/Docker
|
||||
backend). For valid target-side POSIX paths under ``terminal.cwd``, the
|
||||
normalized target path is preserved as a ``Path`` object without invoking
|
||||
local host-filesystem resolution (avoiding host-specific firmlink rewriting
|
||||
such as macOS synthetic ``/home`` -> ``/System/Volumes/Data/home``).
|
||||
"""
|
||||
try:
|
||||
cwd = _remote_terminal_cwd(profile=profile) if profile is not None else _remote_terminal_cwd()
|
||||
except TypeError:
|
||||
cwd = _remote_terminal_cwd()
|
||||
def _remote_terminal_workspace_candidate(path: str | Path) -> Path | None:
|
||||
"""Return a non-stat'ed target-side Path when it is under terminal.cwd."""
|
||||
cwd = _remote_terminal_cwd()
|
||||
if not cwd:
|
||||
return None
|
||||
raw = _strip_surrounding_quotes(str(path)).strip()
|
||||
@@ -315,10 +180,10 @@ def _remote_terminal_workspace_candidate(path: str | Path, profile: str | Path |
|
||||
if _is_blocked_workspace_path(Path(normalized_raw), normalized_raw) or _is_blocked_workspace_path(Path(normalized_cwd), normalized_cwd):
|
||||
return None
|
||||
if posix_candidate == posix_base or _posix_is_within(posix_candidate, posix_base):
|
||||
return Path(normalized_raw)
|
||||
return _resolve_path(normalized_raw)
|
||||
return None
|
||||
candidate = _safe_resolve(_expanduser_path(raw))
|
||||
base = _safe_resolve(_expanduser_path(cwd))
|
||||
candidate = _resolve_path(raw)
|
||||
base = _resolve_path(cwd)
|
||||
if _is_blocked_workspace_path(candidate, raw) or _is_blocked_workspace_path(base, cwd):
|
||||
return None
|
||||
if candidate == base or _is_within(candidate, base):
|
||||
@@ -326,7 +191,7 @@ def _remote_terminal_workspace_candidate(path: str | Path, profile: str | Path |
|
||||
return None
|
||||
|
||||
|
||||
def _profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
def _profile_default_workspace() -> str:
|
||||
"""Read the profile's default workspace from its config.yaml.
|
||||
|
||||
Checks keys in priority order:
|
||||
@@ -342,9 +207,8 @@ def _profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
Falls back to the live DEFAULT_WORKSPACE from api.config.
|
||||
"""
|
||||
try:
|
||||
from api.config import get_config_for_profile_home
|
||||
profile_home = _resolve_profile_home_param(profile)
|
||||
cfg = get_config_for_profile_home(profile_home)
|
||||
from api.config import get_config
|
||||
cfg = get_config()
|
||||
terminal_cfg = cfg.get('terminal', {})
|
||||
remote_terminal = _is_remote_terminal_backend(terminal_cfg)
|
||||
# Explicit webui workspace keys first
|
||||
@@ -353,7 +217,7 @@ def _profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
if ws:
|
||||
if remote_terminal:
|
||||
return str(ws).strip()
|
||||
p = _resolve_path(str(ws), profile=profile)
|
||||
p = _resolve_path(str(ws))
|
||||
if remote_terminal or p.is_dir():
|
||||
return str(p)
|
||||
# Fall through to terminal.cwd — the agent's configured working directory
|
||||
@@ -362,7 +226,7 @@ def _profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
if cwd and str(cwd) not in ('.', ''):
|
||||
if remote_terminal:
|
||||
return str(cwd).strip()
|
||||
p = _resolve_path(str(cwd), profile=profile)
|
||||
p = _resolve_path(str(cwd))
|
||||
if remote_terminal or p.is_dir():
|
||||
return str(p)
|
||||
except (ImportError, Exception):
|
||||
@@ -370,17 +234,15 @@ def _profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
try:
|
||||
from api.config import DEFAULT_WORKSPACE as _LIVE_DEFAULT_WORKSPACE
|
||||
|
||||
return str(_resolve_path(_LIVE_DEFAULT_WORKSPACE, profile=profile))
|
||||
return str(_resolve_path(_LIVE_DEFAULT_WORKSPACE))
|
||||
except Exception:
|
||||
return str(_resolve_path(_BOOT_DEFAULT_WORKSPACE, profile=profile))
|
||||
return str(_resolve_path(_BOOT_DEFAULT_WORKSPACE))
|
||||
|
||||
|
||||
# ── Public API ──────────────────────────────────────────────────────────────
|
||||
|
||||
def _clean_workspace_list(workspaces: list, profile: str | Path | None = None) -> list:
|
||||
def _clean_workspace_list(workspaces: list) -> list:
|
||||
"""Sanitize a workspace list:
|
||||
- Preserve target-side remote terminal workspace paths (SSH/Docker) without
|
||||
resolving them against the local WebUI host filesystem.
|
||||
- Preserve saved paths even when they are currently missing or inaccessible;
|
||||
picker state must not be destroyed by a transient stat/permission failure.
|
||||
- Remove entries whose paths live inside another profile's directory
|
||||
@@ -396,11 +258,7 @@ def _clean_workspace_list(workspaces: list, profile: str | Path | None = None) -
|
||||
name = w.get('name', '')
|
||||
if not path:
|
||||
continue
|
||||
remote_cand = _remote_terminal_workspace_candidate(path, profile=profile)
|
||||
if remote_cand is not None:
|
||||
p = remote_cand
|
||||
else:
|
||||
p = _safe_resolve(_expanduser_path(path))
|
||||
p = _safe_resolve(_expanduser_path(path))
|
||||
# Skip paths inside a DIFFERENT profile's directory (cross-profile leak).
|
||||
# Allow paths inside the CURRENT profile's own directory (e.g. test workspaces
|
||||
# created under ~/.hermes/profiles/webui/webui-mvp-test/).
|
||||
@@ -409,14 +267,7 @@ def _clean_workspace_list(workspaces: list, profile: str | Path | None = None) -
|
||||
# p is under ~/.hermes/profiles/ — only skip if it's under a DIFFERENT profile
|
||||
try:
|
||||
from api.profiles import get_active_hermes_home
|
||||
if profile is not None:
|
||||
# Explicit profile wins: the list belongs to that profile,
|
||||
# so "own" is defined by the profile parameter, never by the
|
||||
# ambient home (loading profile A's list under ambient B must
|
||||
# not silently drop A's own workspaces).
|
||||
own_profile_dir = _resolve_profile_home_param(profile).resolve()
|
||||
else:
|
||||
own_profile_dir = get_active_hermes_home().resolve()
|
||||
own_profile_dir = get_active_hermes_home().resolve()
|
||||
p.relative_to(own_profile_dir)
|
||||
# p is under our own profile dir — keep it
|
||||
except (ValueError, Exception):
|
||||
@@ -483,12 +334,12 @@ def _migrate_global_workspaces() -> list:
|
||||
return []
|
||||
|
||||
|
||||
def load_workspaces(profile: str | Path | None = None) -> list:
|
||||
ws_file = _workspaces_file_for_profile(profile)
|
||||
if ws_file is not None and ws_file.exists():
|
||||
def load_workspaces() -> list:
|
||||
ws_file = _workspaces_file()
|
||||
if ws_file.exists():
|
||||
try:
|
||||
raw = json.loads(ws_file.read_text(encoding='utf-8'))
|
||||
cleaned = _clean_workspace_list(raw, profile=profile)
|
||||
cleaned = _clean_workspace_list(raw)
|
||||
if len(cleaned) != len(raw):
|
||||
# Persist the cleaned version so stale entries don't keep reappearing
|
||||
try:
|
||||
@@ -497,7 +348,7 @@ def load_workspaces(profile: str | Path | None = None) -> list:
|
||||
)
|
||||
except Exception:
|
||||
logger.debug("Failed to persist cleaned workspace list")
|
||||
return cleaned or [{'path': _profile_default_workspace(profile=profile), 'name': 'Home'}]
|
||||
return cleaned or [{'path': _profile_default_workspace(), 'name': 'Home'}]
|
||||
except Exception:
|
||||
logger.debug("Failed to load workspaces from %s", ws_file)
|
||||
# No profile-local file yet.
|
||||
@@ -505,11 +356,7 @@ def load_workspaces(profile: str | Path | None = None) -> list:
|
||||
# For NAMED profiles: always start clean with just their own workspace.
|
||||
try:
|
||||
from api.profiles import get_active_profile_name
|
||||
if profile is not None:
|
||||
profile_home = _resolve_profile_home_param(profile)
|
||||
is_default = _is_default_profile_home(profile_home)
|
||||
else:
|
||||
is_default = get_active_profile_name() in ('default', None)
|
||||
is_default = get_active_profile_name() in ('default', None)
|
||||
except ImportError:
|
||||
is_default = True
|
||||
if is_default:
|
||||
@@ -517,66 +364,17 @@ def load_workspaces(profile: str | Path | None = None) -> list:
|
||||
if migrated:
|
||||
return migrated
|
||||
# Fresh start: single entry from the profile's configured workspace, labeled "Home"
|
||||
return [{'path': _profile_default_workspace(profile=profile), 'name': 'Home'}]
|
||||
return [{'path': _profile_default_workspace(), 'name': 'Home'}]
|
||||
|
||||
|
||||
def save_workspaces(workspaces: list, profile: str | Path | None = None) -> None:
|
||||
ws_file = _workspaces_file_for_profile(profile)
|
||||
if ws_file is None:
|
||||
# Fail-closed: an invalid profile name must not write any state file
|
||||
# (it would land in the default profile's directory via the old clamp).
|
||||
raise ValueError(f"cannot save workspaces for invalid profile {profile!r}")
|
||||
def save_workspaces(workspaces: list) -> None:
|
||||
ws_file = _workspaces_file()
|
||||
ws_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
ws_file.write_text(json.dumps(workspaces, ensure_ascii=False, indent=2), encoding='utf-8')
|
||||
|
||||
|
||||
def get_profile_default_workspace(profile: str | Path | None = None) -> str:
|
||||
"""Resolve the ACTIVE PROFILE's default workspace, never the global file.
|
||||
|
||||
Like get_last_workspace() but WITHOUT the global ``_GLOBAL_LW_FILE``
|
||||
fallback: for a named profile that has not yet written its own
|
||||
profile-scoped ``last_workspace.txt``, that global fallback would leak the
|
||||
*global* last-workspace instead of the profile's configured workspace —
|
||||
which is exactly the #5169 bug (the composer chip on a blank new-chat page
|
||||
showing the wrong/global workspace for a named profile). Used by
|
||||
``GET /api/profile/active`` so a cold boot under a profile cookie reflects
|
||||
the profile's own configured working directory.
|
||||
|
||||
Priority: profile-scoped ``last_workspace.txt`` -> profile ``config.yaml``
|
||||
``workspace``/``default_workspace`` -> ``terminal.cwd`` -> process default.
|
||||
"""
|
||||
try:
|
||||
remote_cwd = _remote_terminal_cwd(profile=profile) if profile is not None else _remote_terminal_cwd()
|
||||
except TypeError:
|
||||
remote_cwd = _remote_terminal_cwd()
|
||||
|
||||
def _valid(raw: str) -> str | None:
|
||||
if not raw:
|
||||
return None
|
||||
if remote_cwd:
|
||||
if _remote_terminal_workspace_candidate(raw, profile=profile) is not None:
|
||||
return raw
|
||||
return None
|
||||
if Path(raw).is_dir():
|
||||
return raw
|
||||
return None
|
||||
|
||||
lw_file = _last_workspace_file_for_profile(profile)
|
||||
if lw_file is not None and lw_file.exists():
|
||||
try:
|
||||
p = _valid(lw_file.read_text(encoding='utf-8').strip())
|
||||
if p:
|
||||
return p
|
||||
except Exception:
|
||||
logger.debug("Failed to read profile last workspace from %s", lw_file)
|
||||
return _profile_default_workspace(profile=profile)
|
||||
|
||||
|
||||
def get_last_workspace(profile: str | Path | None = None) -> str:
|
||||
try:
|
||||
remote_cwd = _remote_terminal_cwd(profile=profile) if profile is not None else _remote_terminal_cwd()
|
||||
except TypeError:
|
||||
remote_cwd = _remote_terminal_cwd()
|
||||
def get_last_workspace() -> str:
|
||||
remote_cwd = _remote_terminal_cwd()
|
||||
|
||||
def valid_last_workspace(raw: str) -> str | None:
|
||||
if not raw:
|
||||
@@ -585,58 +383,35 @@ def get_last_workspace(profile: str | Path | None = None) -> str:
|
||||
# For remote/SSH profiles, last_workspace is target-side state. Do
|
||||
# not accept stale server-local paths merely because they exist on
|
||||
# the WebUI host; require the value to stay under terminal.cwd.
|
||||
if _remote_terminal_workspace_candidate(raw, profile=profile) is not None:
|
||||
if _remote_terminal_workspace_candidate(raw) is not None:
|
||||
return raw
|
||||
return None
|
||||
if Path(raw).is_dir():
|
||||
return raw
|
||||
return None
|
||||
|
||||
lw_file = _last_workspace_file_for_profile(profile)
|
||||
if lw_file is not None and lw_file.exists():
|
||||
lw_file = _last_workspace_file()
|
||||
if lw_file.exists():
|
||||
try:
|
||||
p = valid_last_workspace(lw_file.read_text(encoding='utf-8').strip())
|
||||
if p:
|
||||
return p
|
||||
except Exception:
|
||||
logger.debug("Failed to read last workspace from %s", lw_file)
|
||||
# Fallback: try global file — but ONLY for the root/default profile. A named
|
||||
# profile must never inherit another profile's last-workspace binding through
|
||||
# the legacy global state (#7168 re-gate round 3). Identity is canonical:
|
||||
# a symlink alias of the default home still counts as default, while a
|
||||
# malformed profile name fails validation and is denied the fallback
|
||||
# rather than being clamped onto the global file (#7168 re-gate round 4).
|
||||
_global_fallback_allowed = True # ambient / no explicit profile: historical behavior
|
||||
if profile is not None:
|
||||
if str(profile).strip() == 'default':
|
||||
_global_fallback_allowed = True
|
||||
else:
|
||||
try:
|
||||
_global_fallback_allowed = _is_default_profile_home(
|
||||
_resolve_profile_home_param(profile)
|
||||
)
|
||||
except Exception:
|
||||
# Conservative default: an unresolvable explicit profile is NOT the
|
||||
# root profile — deny the global fallback rather than leak.
|
||||
_global_fallback_allowed = False
|
||||
if _global_fallback_allowed and _GLOBAL_LW_FILE.exists():
|
||||
# Fallback: try global file
|
||||
if _GLOBAL_LW_FILE.exists():
|
||||
try:
|
||||
p = valid_last_workspace(_GLOBAL_LW_FILE.read_text(encoding='utf-8').strip())
|
||||
if p:
|
||||
return p
|
||||
except Exception:
|
||||
logger.debug("Failed to read global last workspace")
|
||||
return _profile_default_workspace(profile=profile)
|
||||
return _profile_default_workspace()
|
||||
|
||||
|
||||
def set_last_workspace(path: str, profile: str | Path | None = None) -> None:
|
||||
def set_last_workspace(path: str) -> None:
|
||||
try:
|
||||
lw_file = _last_workspace_file_for_profile(profile)
|
||||
if lw_file is None:
|
||||
# Fail-closed: an invalid profile name must not write the default
|
||||
# profile's last_workspace.txt (#7168 re-gate round 4).
|
||||
logger.debug("Refusing to set last workspace for invalid profile %r", profile)
|
||||
return
|
||||
lw_file = _last_workspace_file()
|
||||
lw_file.parent.mkdir(parents=True, exist_ok=True)
|
||||
lw_file.write_text(str(path), encoding='utf-8')
|
||||
except Exception:
|
||||
@@ -853,15 +628,7 @@ def _is_within(path: Path, root: Path) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
def _trusted_workspace_roots(profile: str | Path | None = None) -> list[Path]:
|
||||
"""Return the host directories workspace suggestions may traverse.
|
||||
|
||||
Saved-workspace roots follow the same trust rule as
|
||||
:func:`resolve_trusted_workspace`: with an explicit *profile*, only
|
||||
workspaces saved under THAT profile widen the boundary (plus the ambient
|
||||
home / boot-default carve-outs); ``None`` keeps the historical ambient /
|
||||
global saved-list behaviour.
|
||||
"""
|
||||
def _trusted_workspace_roots() -> list[Path]:
|
||||
roots: list[Path] = []
|
||||
|
||||
def add(candidate: str | Path | None) -> None:
|
||||
@@ -880,26 +647,24 @@ def _trusted_workspace_roots(profile: str | Path | None = None) -> list[Path]:
|
||||
|
||||
add(_home_path())
|
||||
add(_BOOT_DEFAULT_WORKSPACE)
|
||||
for w in load_workspaces(profile=profile):
|
||||
for w in load_workspaces():
|
||||
add(w.get("path"))
|
||||
roots.sort(key=lambda p: len(str(p)))
|
||||
return roots
|
||||
|
||||
|
||||
def list_workspace_suggestions(
|
||||
prefix: str = "", limit: int = 12, profile: str | Path | None = None
|
||||
) -> list[str]:
|
||||
def list_workspace_suggestions(prefix: str = "", limit: int = 12) -> list[str]:
|
||||
"""Return workspace path suggestions under trusted roots only.
|
||||
|
||||
Suggestions are limited to directories under one of:
|
||||
- Path.home()
|
||||
- the boot default workspace
|
||||
- already-saved workspace roots (scoped to *profile* when given)
|
||||
- already-saved workspace roots
|
||||
|
||||
Arbitrary system prefixes return an empty list rather than an error so the
|
||||
UI can safely autocomplete while the user types.
|
||||
"""
|
||||
roots = _trusted_workspace_roots(profile=profile)
|
||||
roots = _trusted_workspace_roots()
|
||||
if not roots:
|
||||
return []
|
||||
|
||||
@@ -995,7 +760,7 @@ def list_workspace_suggestions(
|
||||
return suggestions[:limit]
|
||||
|
||||
|
||||
def resolve_trusted_workspace(path: str | Path | None = None, profile: str | Path | None = None) -> Path:
|
||||
def resolve_trusted_workspace(path: str | Path | None = None) -> Path:
|
||||
"""Resolve and validate a workspace path.
|
||||
|
||||
A path is trusted if it satisfies at least one of:
|
||||
@@ -1017,12 +782,12 @@ def resolve_trusted_workspace(path: str | Path | None = None, profile: str | Pat
|
||||
trusted (it was validated at server startup).
|
||||
"""
|
||||
if path in (None, ""):
|
||||
return _resolve_path(_BOOT_DEFAULT_WORKSPACE, profile) if profile is not None else _resolve_path(_BOOT_DEFAULT_WORKSPACE)
|
||||
return _resolve_path(_BOOT_DEFAULT_WORKSPACE)
|
||||
|
||||
candidate = _resolve_path(path, profile) if profile is not None else _resolve_path(path)
|
||||
candidate = _resolve_path(path)
|
||||
|
||||
access_error = _workspace_access_error(candidate)
|
||||
remote_candidate = _remote_terminal_workspace_candidate(path, profile=profile)
|
||||
remote_candidate = _remote_terminal_workspace_candidate(path)
|
||||
if access_error:
|
||||
# For remote terminal profiles, workspace paths belong to the target
|
||||
# machine. Allow paths under terminal.cwd so session switching can
|
||||
@@ -1049,11 +814,6 @@ def resolve_trusted_workspace(path: str | Path | None = None, profile: str | Pat
|
||||
|
||||
# (B) Trusted if already in the saved workspace list — covers non-home installs
|
||||
try:
|
||||
saved = load_workspaces(profile=profile)
|
||||
saved_paths = {_resolve_path(w["path"], profile) for w in saved if w.get("path")}
|
||||
if candidate in saved_paths:
|
||||
return candidate
|
||||
except TypeError:
|
||||
saved = load_workspaces()
|
||||
saved_paths = {_resolve_path(w["path"]) for w in saved if w.get("path")}
|
||||
if candidate in saved_paths:
|
||||
@@ -1080,84 +840,6 @@ def resolve_trusted_workspace(path: str | Path | None = None, profile: str | Pat
|
||||
)
|
||||
|
||||
|
||||
def resolve_implicit_workspace_with_recovery(
|
||||
candidate: str | Path | None,
|
||||
fallback: str | Path | None | Callable[[], str | Path | None],
|
||||
profile: str | Path | None = None,
|
||||
) -> tuple[Path, bool]:
|
||||
"""Resolve an implicit workspace, recovering only a genuinely missing path.
|
||||
|
||||
The fallback still passes through :func:`resolve_trusted_workspace`. Existing
|
||||
but untrusted, inaccessible, or non-directory candidates are not recovery
|
||||
cases: their original validation error is preserved so fallback cannot widen
|
||||
the workspace trust boundary. When *profile* is given, both the trust
|
||||
resolution and the recovery fallback are scoped to that profile.
|
||||
"""
|
||||
try:
|
||||
if profile is not None:
|
||||
try:
|
||||
return resolve_trusted_workspace(candidate, profile=profile), False
|
||||
except TypeError:
|
||||
pass
|
||||
return resolve_trusted_workspace(candidate), False
|
||||
except ValueError as original_error:
|
||||
if candidate in (None, ""):
|
||||
raise original_error from None
|
||||
# Remote terminal workspaces live on the target host. Classify the
|
||||
# backend independently of terminal.cwd: remote backends may omit cwd,
|
||||
# set it to an empty string, or use ".". A failed host-local stat can
|
||||
# never prove target-side deletion. Config-read uncertainty also fails
|
||||
# closed by preserving the original validation error.
|
||||
try:
|
||||
from api.config import get_config, get_config_for_profile_home
|
||||
|
||||
if profile is not None:
|
||||
# Classify the backend from THIS profile's own config, never the
|
||||
# ambient one: a remote profile without terminal.cwd must still be
|
||||
# recognized as remote when loaded under a different ambient home.
|
||||
terminal_cfg = get_config_for_profile_home(
|
||||
_resolve_profile_home_param(profile)
|
||||
).get("terminal", {})
|
||||
else:
|
||||
terminal_cfg = get_config().get("terminal", {})
|
||||
except Exception:
|
||||
logger.debug("Failed to classify terminal backend for workspace recovery", exc_info=True)
|
||||
raise original_error from None
|
||||
if _is_remote_terminal_backend(terminal_cfg):
|
||||
raise original_error from None
|
||||
try:
|
||||
local_candidate = (
|
||||
_resolve_path(candidate, profile=profile)
|
||||
if profile is not None
|
||||
else _resolve_path(candidate)
|
||||
)
|
||||
local_candidate.stat()
|
||||
except FileNotFoundError:
|
||||
def _profile_bound_fallback():
|
||||
if profile is not None and callable(fallback):
|
||||
# Profile-bound getter FIRST (#7168 re-gate round 3): the
|
||||
# production call sites pass profile-aware getters such as
|
||||
# get_last_workspace, so binding the explicit profile must
|
||||
# take precedence over any zero-argument compatibility call,
|
||||
# which would read ambient/global state.
|
||||
try:
|
||||
return fallback(profile)
|
||||
except TypeError:
|
||||
pass
|
||||
return fallback() if callable(fallback) else fallback
|
||||
|
||||
fallback_value = _profile_bound_fallback()
|
||||
if profile is not None:
|
||||
try:
|
||||
return resolve_trusted_workspace(fallback_value, profile=profile), True
|
||||
except TypeError:
|
||||
pass
|
||||
return resolve_trusted_workspace(fallback_value), True
|
||||
except (OSError, RuntimeError, ValueError):
|
||||
raise original_error from None
|
||||
raise original_error from None
|
||||
|
||||
|
||||
|
||||
|
||||
def _strip_surrounding_quotes(path: str) -> str:
|
||||
@@ -1179,7 +861,7 @@ def _strip_surrounding_quotes(path: str) -> str:
|
||||
return s
|
||||
|
||||
|
||||
def validate_workspace_to_add(path: str, profile: str | Path | None = None) -> Path:
|
||||
def validate_workspace_to_add(path: str) -> Path:
|
||||
"""Validate a path for *adding* to the workspace list (less restrictive than resolve_trusted_workspace).
|
||||
|
||||
When a user explicitly adds a new workspace path, we trust their intent — they
|
||||
@@ -1194,10 +876,10 @@ def validate_workspace_to_add(path: str, profile: str | Path | None = None) -> P
|
||||
and users routinely paste those into the Add Space input.
|
||||
"""
|
||||
path = _strip_surrounding_quotes(path)
|
||||
candidate = _resolve_path(path, profile) if profile is not None else _resolve_path(path)
|
||||
candidate = _resolve_path(path)
|
||||
|
||||
access_error = _workspace_access_error(candidate)
|
||||
remote_candidate = _remote_terminal_workspace_candidate(path, profile=profile)
|
||||
remote_candidate = _remote_terminal_workspace_candidate(path)
|
||||
if access_error:
|
||||
# Remote terminal profiles validate workspace existence on the target
|
||||
# machine, not on the WebUI server. Permit target-side paths under
|
||||
@@ -1252,7 +934,6 @@ def safe_resolve_ws(root: Path, requested: str) -> Path:
|
||||
_DIR_FD_OK = os.open in getattr(os, "supports_dir_fd", set())
|
||||
_O_NOFOLLOW = getattr(os, "O_NOFOLLOW", 0)
|
||||
_O_DIRECTORY = getattr(os, "O_DIRECTORY", 0)
|
||||
_O_BINARY = getattr(os, "O_BINARY", 0)
|
||||
|
||||
|
||||
def open_anchored_fd(workspace: Path, target: Path, *, want_dir: bool) -> int:
|
||||
@@ -1274,11 +955,7 @@ def open_anchored_fd(workspace: Path, target: Path, *, want_dir: bool) -> int:
|
||||
# Windows / no openat: fall back to a plain pathname open. No new race
|
||||
# protection, but no regression vs the prior path-based behaviour, and
|
||||
# symlink creation needs admin on Windows anyway.
|
||||
flags = (
|
||||
os.O_RDONLY
|
||||
| (_O_DIRECTORY if want_dir else _O_BINARY)
|
||||
| _O_NOFOLLOW
|
||||
)
|
||||
flags = os.O_RDONLY | (_O_DIRECTORY if want_dir else 0) | _O_NOFOLLOW
|
||||
try:
|
||||
return os.open(str(target), flags)
|
||||
except OSError:
|
||||
@@ -1293,11 +970,7 @@ def open_anchored_fd(workspace: Path, target: Path, *, want_dir: bool) -> int:
|
||||
for i, part in enumerate(rel_parts):
|
||||
is_last = i == len(rel_parts) - 1
|
||||
want_directory = (not is_last) or want_dir
|
||||
flags = (
|
||||
os.O_RDONLY
|
||||
| _O_NOFOLLOW
|
||||
| (_O_DIRECTORY if want_directory else _O_BINARY)
|
||||
)
|
||||
flags = os.O_RDONLY | _O_NOFOLLOW | (_O_DIRECTORY if want_directory else 0)
|
||||
try:
|
||||
nfd = os.open(part, flags, dir_fd=fd)
|
||||
except OSError:
|
||||
@@ -1519,40 +1192,6 @@ def rename_anchored(root: Path, source: Path, dest: Path) -> None:
|
||||
os.close(src_parent_fd)
|
||||
|
||||
|
||||
def _birthtime_ns(lst) -> int | None:
|
||||
"""Return creation time in ns, or None when the platform lacks birthtime."""
|
||||
value = getattr(lst, 'st_birthtime_ns', None)
|
||||
if value is not None:
|
||||
return value
|
||||
value = getattr(lst, 'st_birthtime', None)
|
||||
if value is not None:
|
||||
return int(value * 1_000_000_000)
|
||||
if sys.platform == 'win32':
|
||||
return getattr(lst, 'st_ctime_ns', None)
|
||||
return None
|
||||
|
||||
|
||||
def _browser_timestamp_ns(value) -> str | None:
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
return str(int(value))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def serialize_workspace_entries_for_browser(entries: list[dict] | None) -> list[dict]:
|
||||
payload = []
|
||||
for entry in entries or []:
|
||||
item = dict(entry or {})
|
||||
if 'mtime_ns' in item:
|
||||
item['mtime_ns'] = _browser_timestamp_ns(item.get('mtime_ns'))
|
||||
if 'birthtime_ns' in item:
|
||||
item['birthtime_ns'] = _browser_timestamp_ns(item.get('birthtime_ns'))
|
||||
payload.append(item)
|
||||
return payload
|
||||
|
||||
|
||||
def list_dir(workspace: Path, rel: str='.'):
|
||||
target = safe_resolve_ws(workspace, rel)
|
||||
if not target.is_dir():
|
||||
@@ -1567,8 +1206,6 @@ def list_dir(workspace: Path, rel: str='.'):
|
||||
with follow_symlinks=False (else None); ``reachable`` is False when a
|
||||
follow_symlinks=True stat raised (broken target or symlink loop)."""
|
||||
if is_symlink:
|
||||
# Keep the transport rank aligned with _sort_key_de/_sort_key_p.
|
||||
workspace_sort_rank = 0
|
||||
if raw_link is None:
|
||||
return
|
||||
# A symlink whose follow-stat raised (ELOOP / broken target) can never
|
||||
@@ -1617,10 +1254,8 @@ def list_dir(workspace: Path, rel: str='.'):
|
||||
'path': display_path,
|
||||
'type': 'symlink',
|
||||
'is_dir': False,
|
||||
'workspace_sort_rank': workspace_sort_rank,
|
||||
'target_outside_workspace': True,
|
||||
'mtime_ns': mtime_ns,
|
||||
'birthtime_ns': _birthtime_ns(lstat_result) if lstat_result is not None else None,
|
||||
}
|
||||
entries.append(entry)
|
||||
else:
|
||||
@@ -1631,10 +1266,8 @@ def list_dir(workspace: Path, rel: str='.'):
|
||||
'type': 'symlink',
|
||||
'target': str(link_target),
|
||||
'is_dir': is_dir,
|
||||
'workspace_sort_rank': workspace_sort_rank,
|
||||
'target_outside_workspace': False,
|
||||
'mtime_ns': mtime_ns,
|
||||
'birthtime_ns': _birthtime_ns(lstat_result) if lstat_result is not None else None,
|
||||
}
|
||||
if not is_dir:
|
||||
try:
|
||||
@@ -1648,7 +1281,6 @@ def list_dir(workspace: Path, rel: str='.'):
|
||||
entry_path = rel + '/' + name
|
||||
if lstat_result is not None:
|
||||
is_file = stat.S_ISREG(lstat_result.st_mode)
|
||||
workspace_sort_rank = 2 if is_file else 1
|
||||
size = lstat_result.st_size if is_file else None
|
||||
mtime_ns = lstat_result.st_mtime_ns
|
||||
is_dir_entry = stat.S_ISDIR(lstat_result.st_mode)
|
||||
@@ -1656,15 +1288,12 @@ def list_dir(workspace: Path, rel: str='.'):
|
||||
size = None
|
||||
mtime_ns = None
|
||||
is_dir_entry = False
|
||||
workspace_sort_rank = 1
|
||||
entries.append({
|
||||
'name': name,
|
||||
'path': entry_path,
|
||||
'type': 'dir' if is_dir_entry else 'file',
|
||||
'size': size,
|
||||
'mtime_ns': mtime_ns,
|
||||
'birthtime_ns': _birthtime_ns(lstat_result) if lstat_result is not None else None,
|
||||
'workspace_sort_rank': workspace_sort_rank,
|
||||
})
|
||||
|
||||
if _DIR_FD_OK:
|
||||
@@ -1803,10 +1432,6 @@ def read_file_content(workspace: Path, rel: str) -> dict:
|
||||
if st.st_size > MAX_FILE_BYTES:
|
||||
raise ValueError(f"File too large ({st.st_size} bytes, max {MAX_FILE_BYTES})")
|
||||
raw = fh.read(MAX_FILE_BYTES + 1)
|
||||
if Path(str(rel)).suffix.lower() in {".docx", ".xlsx", ".pptx"}:
|
||||
from api.office_documents import preview_office_document
|
||||
|
||||
return preview_office_document(rel, raw)
|
||||
content = raw.decode('utf-8', errors='replace')
|
||||
return {'path': rel, 'content': content, 'size': len(raw), 'lines': content.count('\n') + 1}
|
||||
|
||||
@@ -2015,7 +1640,6 @@ def _run_git(args, cwd, timeout=3):
|
||||
r = subprocess.run(
|
||||
['git'] + args, cwd=str(cwd), capture_output=True,
|
||||
text=True, timeout=timeout,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
return r.stdout.strip() if r.returncode == 0 else None
|
||||
except (subprocess.TimeoutExpired, FileNotFoundError, OSError):
|
||||
|
||||
@@ -20,11 +20,10 @@ from dataclasses import dataclass
|
||||
from pathlib import Path
|
||||
from typing import Iterable
|
||||
|
||||
from api.subprocess_utils import windows_hide_flags
|
||||
from api.workspace import rmtree_anchored, safe_resolve_ws, unlink_anchored
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
from api.workspace import rmtree_anchored, safe_resolve_ws, unlink_anchored
|
||||
|
||||
|
||||
GIT_TIMEOUT = 5
|
||||
GIT_REMOTE_TIMEOUT = 60
|
||||
@@ -249,7 +248,6 @@ def _run_git(
|
||||
text=True,
|
||||
timeout=timeout,
|
||||
env=run_env,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
except subprocess.TimeoutExpired as exc:
|
||||
raise GitWorkspaceError("Git command timed out", "timeout") from exc
|
||||
@@ -292,7 +290,6 @@ def _config_names_for_scope(
|
||||
capture_output=True,
|
||||
timeout=GIT_TIMEOUT,
|
||||
env=env,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
if result.returncode not in {0, 1}:
|
||||
if ignore_unsupported:
|
||||
|
||||
+2
-37
@@ -10,8 +10,6 @@ from pathlib import Path
|
||||
|
||||
import logging
|
||||
|
||||
from api.subprocess_utils import windows_hide_flags
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@@ -23,7 +21,6 @@ def _run_git(args: list[str], cwd: str | Path, timeout: float = 2) -> subprocess
|
||||
capture_output=True,
|
||||
timeout=timeout,
|
||||
check=False,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
|
||||
|
||||
@@ -268,18 +265,6 @@ def remove_worktree_for_session(session, *, force: bool = False) -> dict:
|
||||
repo_root = getattr(session, "worktree_repo_root", None)
|
||||
if not repo_root:
|
||||
raise ValueError("Session missing worktree_repo_root")
|
||||
|
||||
# Unlock the creation-time bookkeeping lock before removing (fail-soft).
|
||||
# The agent locks every worktree it creates (cli._setup_worktree), and git
|
||||
# refuses to remove a locked worktree with a single --force, so without
|
||||
# this every WebUI-created worktree is unremovable. The dirty/untracked/
|
||||
# unpushed/stream/terminal guards above are the real safety layer, not
|
||||
# git's lock — mirrors the agent's own _cleanup_worktree ordering.
|
||||
try:
|
||||
_run_git(["worktree", "unlock", str(worktree_path)], str(repo_root), timeout=5)
|
||||
except (OSError, subprocess.TimeoutExpired):
|
||||
pass # already unlocked / never locked — non-fatal
|
||||
|
||||
try:
|
||||
remove_args = ["worktree", "remove"]
|
||||
if force:
|
||||
@@ -329,7 +314,6 @@ def find_git_repo_root(workspace: str | Path) -> Path:
|
||||
capture_output=True,
|
||||
timeout=5,
|
||||
check=False,
|
||||
creationflags=windows_hide_flags(),
|
||||
)
|
||||
except (OSError, subprocess.TimeoutExpired) as exc:
|
||||
raise ValueError("Workspace is not inside a git repository") from exc
|
||||
@@ -343,27 +327,8 @@ def find_git_repo_root(workspace: str | Path) -> Path:
|
||||
|
||||
def _setup_agent_worktree(repo_root: str) -> dict:
|
||||
try:
|
||||
import importlib.util
|
||||
from pathlib import Path
|
||||
|
||||
from api.config import _AGENT_DIR # Hermes Agent source root
|
||||
|
||||
# Use importlib to load cli.py from its absolute path instead of a bare
|
||||
# ``from cli import _setup_worktree``. The bare import is vulnerable to
|
||||
# namespace-package shadowing — any third-party package (e.g. the ``cli``
|
||||
# namespace shipped by stringzilla) that creates a cli/ directory in
|
||||
# site-packages will preempt the real hermes-agent/cli.py module.
|
||||
cli_path = str(Path(_AGENT_DIR) / "cli.py")
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"hermes_cli_worktree", cli_path,
|
||||
)
|
||||
if spec is None or spec.loader is None:
|
||||
raise RuntimeError(
|
||||
f"Could not locate hermes-agent worktree helper at {cli_path}"
|
||||
)
|
||||
cli_mod = importlib.util.module_from_spec(spec)
|
||||
spec.loader.exec_module(cli_mod)
|
||||
_setup_worktree = cli_mod._setup_worktree
|
||||
import api.config # noqa: F401 # ensure Hermes Agent dir is on sys.path
|
||||
from cli import _setup_worktree
|
||||
except Exception as exc:
|
||||
raise RuntimeError("Hermes Agent worktree helper is unavailable") from exc
|
||||
output = StringIO()
|
||||
|
||||
+1
-36
@@ -182,31 +182,6 @@ def _agent_dir_from_hermes_cli() -> Path | None:
|
||||
return None
|
||||
|
||||
|
||||
def _agent_dir_from_python(python_exe: str) -> Path | None:
|
||||
script = (
|
||||
'import importlib.util\n'
|
||||
'spec = importlib.util.find_spec("run_agent")\n'
|
||||
'print(spec.origin if spec else "")\n'
|
||||
)
|
||||
try:
|
||||
check = subprocess.run(
|
||||
[python_exe, "-c", script],
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
except OSError:
|
||||
return None
|
||||
if check.returncode != 0:
|
||||
return None
|
||||
lines = check.stdout.splitlines()
|
||||
if not lines:
|
||||
return None
|
||||
origin = Path(lines[0].strip())
|
||||
if not origin.is_absolute() or origin.name != "run_agent.py" or not origin.is_file():
|
||||
return None
|
||||
return origin.parent.resolve()
|
||||
|
||||
|
||||
def discover_agent_dir() -> Path | None:
|
||||
home = Path(os.getenv("HERMES_HOME", str(Path.home() / ".hermes"))).expanduser()
|
||||
candidates = [
|
||||
@@ -227,10 +202,7 @@ def discover_agent_dir() -> Path | None:
|
||||
candidate = Path(raw).expanduser().resolve()
|
||||
if candidate.exists() and (candidate / "run_agent.py").exists():
|
||||
return candidate
|
||||
agent_dir = _agent_dir_from_hermes_cli()
|
||||
if agent_dir:
|
||||
return agent_dir
|
||||
return _agent_dir_from_python(discover_launcher_python(None))
|
||||
return _agent_dir_from_hermes_cli()
|
||||
|
||||
|
||||
def discover_launcher_python(agent_dir: Path | None) -> str:
|
||||
@@ -298,13 +270,6 @@ def ensure_python_has_webui_deps(python_exe: str, agent_dir: Path | None = None)
|
||||
if _python_can_run_webui_and_agent(str(candidate), agent_dir):
|
||||
return str(candidate)
|
||||
|
||||
if (os.getenv("HERMES_WEBUI_DISABLE_LOCAL_VENV") or "").strip().lower() in {"1", "true", "yes", "on"}:
|
||||
raise RuntimeError(
|
||||
"Python environment cannot import both WebUI dependencies and Hermes Agent, "
|
||||
"and local .venv creation is disabled for this packaged launch. Set "
|
||||
"HERMES_WEBUI_PYTHON to an interpreter with Hermes Agent dependencies."
|
||||
)
|
||||
|
||||
venv_dir = REPO_ROOT / ".venv"
|
||||
venv_python = venv_dir / (
|
||||
"Scripts/python.exe" if platform.system() == "Windows" else "bin/python"
|
||||
|
||||
@@ -29,60 +29,6 @@ ensure_home() {
|
||||
mkdir -p "${HERMES_HOME}" "${DEFAULT_STATE_DIR}"
|
||||
}
|
||||
|
||||
_apply_env_file_safely() {
|
||||
local env_file="$1"
|
||||
local line key value
|
||||
while IFS= read -r line || [[ -n "${line}" ]]; do
|
||||
line="${line#${line%%[![:space:]]*}}"
|
||||
[[ -z "${line}" || "${line}" == \#* ]] && continue
|
||||
if [[ "${line}" =~ ^export[[:space:]]+(.+)$ ]]; then
|
||||
line="${BASH_REMATCH[1]}"
|
||||
line="${line#${line%%[![:space:]]*}}"
|
||||
fi
|
||||
[[ "${line}" == *=* ]] || continue
|
||||
|
||||
key="${line%%=*}"
|
||||
value="${line#*=}"
|
||||
key="${key//[[:space:]]/}"
|
||||
[[ "${key}" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
|
||||
case "${key}" in
|
||||
UID | GID | EUID | EGID | PPID) continue ;;
|
||||
esac
|
||||
|
||||
value="${value#${value%%[![:space:]]*}}"
|
||||
if [[ "${value}" =~ ^\"(([^\"\\]|\\.)*)\"([[:space:]]*\#.*)?[[:space:]]*$ ]]; then
|
||||
value="${BASH_REMATCH[1]}"
|
||||
value="$(printf '%s' "$value" | awk '{
|
||||
i = 1
|
||||
len = length($0)
|
||||
while (i <= len) {
|
||||
c = substr($0, i, 1)
|
||||
if (c == "\\" && i < len) {
|
||||
nc = substr($0, i+1, 1)
|
||||
if (nc == "n") printf "\n"
|
||||
else if (nc == "r") printf "\r"
|
||||
else if (nc == "t") printf "\t"
|
||||
else if (nc == "\"") printf "\""
|
||||
else if (nc == "\\") printf "\\"
|
||||
else { printf "\\%s", nc }
|
||||
i += 2
|
||||
} else {
|
||||
printf "%s", c
|
||||
i++
|
||||
}
|
||||
}
|
||||
}')"
|
||||
elif [[ "${value}" =~ ^\'([^\']*)\'([[:space:]]*\#.*)?[[:space:]]*$ ]]; then
|
||||
value="${BASH_REMATCH[1]}"
|
||||
else
|
||||
value="${value%%[[:space:]]\#*}"
|
||||
value="${value%${value##*[![:space:]]}}"
|
||||
fi
|
||||
|
||||
export "${key}=${value}"
|
||||
done < "${env_file}"
|
||||
}
|
||||
|
||||
_load_repo_dotenv_preserving_env() {
|
||||
[[ "${HERMES_WEBUI_NO_DOTENV:-0}" == "1" ]] && return 0
|
||||
local env_file="${REPO_ROOT}/.env"
|
||||
@@ -94,9 +40,7 @@ _load_repo_dotenv_preserving_env() {
|
||||
line="${line#${line%%[![:space:]]*}}"
|
||||
[[ -z "${line}" || "${line}" == \#* || "${line}" != *=* ]] && continue
|
||||
key="${line%%=*}"
|
||||
if [[ "${key}" =~ ^export[[:space:]]+(.+)$ ]]; then
|
||||
key="${BASH_REMATCH[1]}"
|
||||
fi
|
||||
key="${key#export }"
|
||||
key="${key//[[:space:]]/}"
|
||||
[[ "${key}" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
|
||||
# Skip shell-readonly names (UID/GID/EUID/EGID/PPID); re-exporting them
|
||||
@@ -110,50 +54,21 @@ _load_repo_dotenv_preserving_env() {
|
||||
fi
|
||||
done < "${env_file}"
|
||||
|
||||
_apply_env_file_safely "${env_file}"
|
||||
|
||||
local assignment
|
||||
if [[ ${#preserved[@]} -gt 0 ]]; then
|
||||
for assignment in "${preserved[@]}"; do
|
||||
export "${assignment}"
|
||||
done
|
||||
fi
|
||||
}
|
||||
|
||||
_load_hermes_dotenv() {
|
||||
# Also load ~/.hermes/.env so that ${VAR} references in config.yaml can
|
||||
# resolve against provider credentials defined in the Hermes env file.
|
||||
# Repo .env takes precedence (loaded above); variables already exported
|
||||
# into the shell environment (including those just set by repo .env) are
|
||||
# captured in preserved[] before _apply_env_file_safely runs and are
|
||||
# restored afterwards, so this acts as a fallback source for vars the
|
||||
# repo .env did not define.
|
||||
[[ "${HERMES_WEBUI_NO_DOTENV:-0}" == "1" ]] && return 0
|
||||
local hermes_home="${HERMES_HOME:-${HOME}/.hermes}"
|
||||
local hermes_env="${hermes_home}/.env"
|
||||
[[ -f "${hermes_env}" ]] || return 0
|
||||
|
||||
local -a preserved=()
|
||||
local line key value
|
||||
while IFS= read -r line || [[ -n "${line}" ]]; do
|
||||
line="${line#${line%%[![:space:]]*}}"
|
||||
[[ -z "${line}" || "${line}" == \#* || "${line}" != *=* ]] && continue
|
||||
key="${line%%=*}"
|
||||
if [[ "${key}" =~ ^export[[:space:]]+(.+)$ ]]; then
|
||||
key="${BASH_REMATCH[1]}"
|
||||
fi
|
||||
key="${key//[[:space:]]/}"
|
||||
[[ "${key}" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue
|
||||
case "${key}" in
|
||||
UID | GID | EUID | EGID | PPID) continue ;;
|
||||
esac
|
||||
if [[ -n "${!key+x}" ]]; then
|
||||
value="${!key}"
|
||||
preserved+=("${key}=${value}")
|
||||
fi
|
||||
done < "${hermes_env}"
|
||||
|
||||
_apply_env_file_safely "${hermes_env}"
|
||||
set -a
|
||||
# Filter out shell-readonly vars (UID, GID, EUID, EGID, PPID) before
|
||||
# sourcing. docker-compose.yml's macOS instructions document
|
||||
# `echo "UID=$(id -u)" >> .env`; under `set -euo pipefail` a raw
|
||||
# `source .env` then aborts with "UID: readonly variable" before
|
||||
# `ctl.sh status` can print anything. Mirrors the guard start.sh uses.
|
||||
local _ctl_env_filtered
|
||||
_ctl_env_filtered="$(mktemp "${TMPDIR:-/tmp}/hermes-webui-ctl-env.XXXXXX")"
|
||||
# shellcheck disable=SC2064
|
||||
trap "rm -f '${_ctl_env_filtered}'" RETURN
|
||||
grep -vE '^[[:space:]]*(export[[:space:]]+)?(UID|GID|EUID|EGID|PPID)=' "${env_file}" > "${_ctl_env_filtered}" || true
|
||||
# shellcheck source=/dev/null
|
||||
source "${_ctl_env_filtered}"
|
||||
rm -f "${_ctl_env_filtered}"
|
||||
set +a
|
||||
|
||||
local assignment
|
||||
if [[ ${#preserved[@]} -gt 0 ]]; then
|
||||
@@ -385,25 +300,11 @@ _pid_listens_on_port() {
|
||||
local pid="$1" port="$2"
|
||||
[[ "${pid}" =~ ^[0-9]+$ && "${port}" =~ ^[0-9]+$ ]] || return 2
|
||||
if command -v lsof >/dev/null 2>&1; then
|
||||
if lsof -nP -a -p "${pid}" -iTCP:"${port}" -sTCP:LISTEN >/dev/null 2>&1; then
|
||||
if lsof -nP -p "${pid}" -iTCP:"${port}" -sTCP:LISTEN >/dev/null 2>&1; then
|
||||
return 0 # PID is listening on that port → real conflict
|
||||
fi
|
||||
return 1 # PID is alive but NOT listening on that port → no conflict
|
||||
fi
|
||||
# Linux hosts (where the systemd guard runs) often lack lsof but ship ss.
|
||||
if command -v ss >/dev/null 2>&1; then
|
||||
local ss_out rows
|
||||
if ! ss_out="$(ss -tlnp 2>/dev/null)"; then
|
||||
return 2
|
||||
fi
|
||||
rows="$(printf '%s\n' "${ss_out}" | awk -v p="${port}" '$4 ~ (":" p "$")')"
|
||||
[[ -z "${rows}" ]] && return 1 # nothing listens on that port at all
|
||||
printf '%s\n' "${rows}" | grep -q "pid=${pid}," && return 0
|
||||
# Listener rows exist; when they carry pid= attribution and none is ours,
|
||||
# some OTHER process owns the port — not a conflict for this pid.
|
||||
printf '%s\n' "${rows}" | grep -q "pid=" && return 1
|
||||
return 2 # listener present but unattributable
|
||||
fi
|
||||
return 2 # can't determine
|
||||
}
|
||||
|
||||
@@ -438,156 +339,9 @@ _launchd_webui_pid() {
|
||||
esac
|
||||
}
|
||||
|
||||
_probe_target_host() {
|
||||
# A bind-all host (0.0.0.0 / ::) is not a connectable probe target; probe
|
||||
# loopback instead (mirrors server.py _abort_if_already_serving). IPv6
|
||||
# literals must be bracketed for URL interpolation ("http://[::1]:8787"),
|
||||
# or curl/wget reject the URL and an existing instance is missed.
|
||||
local host="${1:-}"
|
||||
case "${host}" in
|
||||
0.0.0.0 | '' | ::) printf '127.0.0.1' ;;
|
||||
\[*\]) printf '%s' "${host}" ;;
|
||||
*:*) printf '[%s]' "${host}" ;;
|
||||
*) printf '%s' "${host}" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
_port_answers_http() {
|
||||
# True when ANYTHING answers an HTTP(S) request on host:port — mirrors
|
||||
# server.py's _abort_if_already_serving, which treats any response bytes
|
||||
# (including an error status from a foreign app squatting the port) as a
|
||||
# conflict. Deliberately broader than hermes_webui_probe_health, which
|
||||
# requires a 200 from /health.
|
||||
local host="$1" port="$2" url rc scheme
|
||||
for scheme in http https; do
|
||||
url="${scheme}://${host}:${port}/health"
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
# No -f: an HTTP error status is still a responder. -k: a self-signed
|
||||
# cert is still a responder. --noproxy: this is a LOCAL ownership
|
||||
# check — routed through an http(s)_proxy it would report the proxy,
|
||||
# not the port (false block or, with a dead proxy, a missed conflict).
|
||||
curl -sS -o /dev/null -k --noproxy '*' --max-time 2 "${url}" 2>/dev/null
|
||||
rc=$?
|
||||
[[ ${rc} -eq 0 ]] && return 0
|
||||
elif command -v wget >/dev/null 2>&1; then
|
||||
wget -qO /dev/null --no-check-certificate --no-proxy "--timeout=2" --tries=1 "${url}" 2>/dev/null
|
||||
rc=$?
|
||||
# 0 = OK; 8 = server answered with an error status — both are responders.
|
||||
[[ ${rc} -eq 0 || ${rc} -eq 8 ]] && return 0
|
||||
else
|
||||
return 1
|
||||
fi
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
_port_listener_diag() {
|
||||
# Best-effort one-liner describing what listens on TCP port $1. Used in
|
||||
# error/status messages only — never fails the caller.
|
||||
local port="$1" line=""
|
||||
# The || true keeps the assignments non-failing when errexit is inherited
|
||||
# into command substitutions (shopt inherit_errexit, or a BASHOPTS env from
|
||||
# the invoking shell) — without it a no-match awk/ss aborts status/stop.
|
||||
if command -v ss >/dev/null 2>&1; then
|
||||
line="$(ss -tlnp 2>/dev/null | awk -v p="${port}" '$4 ~ (":" p "$") {print; exit}')" || true
|
||||
fi
|
||||
if [[ -z "${line}" ]] && command -v lsof >/dev/null 2>&1; then
|
||||
line="$(lsof -nP -iTCP:"${port}" -sTCP:LISTEN 2>/dev/null | awk 'NR==2 {print; exit}')" || true
|
||||
fi
|
||||
if [[ -n "${line}" ]]; then
|
||||
printf '%s' "${line}"
|
||||
fi
|
||||
return 0
|
||||
}
|
||||
|
||||
_systemd_unit_effective_port() {
|
||||
# Best-effort resolution of the port a systemd unit is configured to bind:
|
||||
# HERMES_WEBUI_PORT in its Environment=, then an explicit --port on its
|
||||
# ExecStart=. Prints nothing when the binding cannot be determined so the
|
||||
# caller can fall back to the conservative default-port guard.
|
||||
local scope="$1" unit="$2" env_block="" exec_block="" port=""
|
||||
env_block="$(systemctl "${scope}" show -p Environment --value "${unit}" 2>/dev/null)" || true
|
||||
if [[ "${env_block}" =~ HERMES_WEBUI_PORT=([0-9]+) ]]; then
|
||||
port="${BASH_REMATCH[1]}"
|
||||
fi
|
||||
if [[ -z "${port}" ]]; then
|
||||
exec_block="$(systemctl "${scope}" show -p ExecStart --value "${unit}" 2>/dev/null)" || true
|
||||
if [[ "${exec_block}" =~ --port[=[:space:]]([0-9]+) ]]; then
|
||||
port="${BASH_REMATCH[1]}"
|
||||
fi
|
||||
fi
|
||||
printf '%s' "${port}"
|
||||
}
|
||||
|
||||
_systemd_webui_conflict() {
|
||||
# Linux analog of _launchd_webui_pid: echo a short conflict descriptor and
|
||||
# return 0 when a systemd unit (default hermes-webui.service, override via
|
||||
# HERMES_WEBUI_SYSTEMD_UNIT) effectively owns the instance we are about to
|
||||
# start. Two conflict shapes:
|
||||
# - ActiveState=active and the unit's MainPID listens on our port.
|
||||
# - ActiveState=activating/reloading (Restart= backoff between attempts):
|
||||
# the port is briefly silent, but the unit WILL respawn and re-bind; a
|
||||
# ctl.sh-started server would trap it in a permanent crash loop
|
||||
# (each systemd attempt aborts on the "already responding" check, 5s
|
||||
# later the next one dies the same way — observed in the field).
|
||||
# Port scoping mirrors the launchd #3291 fix: when the unit's port cannot
|
||||
# be determined, only the default port is guarded, so alternate-port test
|
||||
# instances are never wrongly blocked.
|
||||
[[ "${HERMES_WEBUI_CTL_ALLOW_SYSTEMD_CONFLICT:-0}" == "1" ]] && return 1
|
||||
command -v systemctl >/dev/null 2>&1 || return 1
|
||||
local unit="${HERMES_WEBUI_SYSTEMD_UNIT:-hermes-webui.service}"
|
||||
[[ -n "${unit}" ]] || return 1
|
||||
local want_port="${CTL_PORT:-${HERMES_WEBUI_PORT:-8787}}"
|
||||
local scope state main_pid unit_port
|
||||
for scope in --system --user; do
|
||||
state="$(systemctl "${scope}" show -p ActiveState --value "${unit}" 2>/dev/null)" || continue
|
||||
case "${state}" in
|
||||
active)
|
||||
main_pid="$(systemctl "${scope}" show -p MainPID --value "${unit}" 2>/dev/null)"
|
||||
[[ "${main_pid}" =~ ^[0-9]+$ ]] || main_pid=0
|
||||
if (( main_pid > 0 )); then
|
||||
_pid_listens_on_port "${main_pid}" "${want_port}"
|
||||
case "$?" in
|
||||
0) printf 'unit %s is active (MainPID %s listens on port %s)' "${unit}" "${main_pid}" "${want_port}"; return 0 ;;
|
||||
1) continue ;; # active on a different port → no conflict
|
||||
esac
|
||||
fi
|
||||
# Port ownership unknown via PID: resolve the unit's CONFIGURED
|
||||
# binding and refuse only on actual overlap. Only when the binding
|
||||
# cannot be determined fall back to guarding the default port
|
||||
# (see #3291) so alternate-port instances are never wrongly blocked.
|
||||
unit_port="$(_systemd_unit_effective_port "${scope}" "${unit}")" || true
|
||||
if [[ -n "${unit_port}" ]]; then
|
||||
if [[ "${unit_port}" == "${want_port}" ]]; then
|
||||
printf 'unit %s is active (configured for port %s)' "${unit}" "${unit_port}"
|
||||
return 0
|
||||
fi
|
||||
elif [[ "${want_port}" == "8787" ]]; then
|
||||
printf 'unit %s is active' "${unit}"
|
||||
return 0
|
||||
fi
|
||||
;;
|
||||
activating | reloading)
|
||||
unit_port="$(_systemd_unit_effective_port "${scope}" "${unit}")" || true
|
||||
if [[ -n "${unit_port}" ]]; then
|
||||
if [[ "${unit_port}" == "${want_port}" ]]; then
|
||||
printf 'unit %s is %s (auto-restart pending on port %s)' "${unit}" "${state}" "${unit_port}"
|
||||
return 0
|
||||
fi
|
||||
elif [[ "${want_port}" == "8787" ]]; then
|
||||
printf 'unit %s is %s (auto-restart pending)' "${unit}" "${state}"
|
||||
return 0
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
done
|
||||
return 1
|
||||
}
|
||||
|
||||
start_cmd() {
|
||||
ensure_home
|
||||
_load_repo_dotenv_preserving_env
|
||||
_load_hermes_dotenv
|
||||
export HERMES_WEBUI_STATE_DIR="${HERMES_WEBUI_STATE_DIR:-${DEFAULT_STATE_DIR}}"
|
||||
mkdir -p "${HERMES_WEBUI_STATE_DIR}"
|
||||
_parse_launch_binding "$@"
|
||||
@@ -606,28 +360,6 @@ start_cmd() {
|
||||
echo "[ctl] Use launchctl kickstart -k gui/$(id -u)/${HERMES_WEBUI_LAUNCHD_LABEL:-${DEFAULT_LAUNCHD_LABEL}} or disable the launchd job before using ctl.sh start." >&2
|
||||
return 2
|
||||
fi
|
||||
local systemd_conflict
|
||||
if systemd_conflict="$(_systemd_webui_conflict 2>/dev/null)"; then
|
||||
echo "[ctl] Refusing to start a second Hermes WebUI: systemd ${systemd_conflict}." >&2
|
||||
echo "[ctl] Manage that instance with systemctl instead (e.g. sudo systemctl restart ${HERMES_WEBUI_SYSTEMD_UNIT:-hermes-webui.service}), or disable the unit before using ctl.sh start. Set HERMES_WEBUI_CTL_ALLOW_SYSTEMD_CONFLICT=1 to override." >&2
|
||||
return 2
|
||||
fi
|
||||
# Generic duplicate guard: whatever supervises it, if a live server already
|
||||
# answers on the target host:port, a second one is doomed — server.py aborts
|
||||
# on its "already responding" startup check a few seconds in, AFTER ctl.sh's
|
||||
# old 0.15s aliveness check had reported success and recorded a PID file
|
||||
# that goes stale on exit. Refuse up front instead.
|
||||
local probe_host
|
||||
probe_host="$(_probe_target_host "${CTL_HOST}")"
|
||||
if [[ "${HERMES_WEBUI_CTL_ALLOW_PORT_CONFLICT:-0}" != "1" ]] \
|
||||
&& _port_answers_http "${probe_host}" "${CTL_PORT}"; then
|
||||
echo "[ctl] Refusing to start: a live server is already responding on ${probe_host}:${CTL_PORT}." >&2
|
||||
local listener_diag
|
||||
listener_diag="$(_port_listener_diag "${CTL_PORT}")"
|
||||
[[ -n "${listener_diag}" ]] && echo "[ctl] listener: ${listener_diag}" >&2
|
||||
echo "[ctl] Stop that instance first (its own supervisor may restart it — check systemctl/launchctl)." >&2
|
||||
return 2
|
||||
fi
|
||||
_clear_stale_pid >/dev/null 2>&1 || true
|
||||
|
||||
local python_exe pid
|
||||
@@ -643,33 +375,7 @@ start_cmd() {
|
||||
|
||||
printf '%s\n' "${pid}" > "${PID_FILE}"
|
||||
_write_state "${pid}" "${CTL_HOST}" "${CTL_PORT}" "${python_exe}"
|
||||
# Watch the child through its startup window instead of a single 0.15s
|
||||
# aliveness check. A server that dies during startup (bad venv, import
|
||||
# error, port stolen between guard and bind) exits after ~1-3s — after the
|
||||
# old check had already printed success and left a stale PID file behind.
|
||||
# Break early as soon as /health answers; report failure the moment the
|
||||
# process dies. HERMES_WEBUI_START_GRACE (integer seconds, default 3)
|
||||
# bounds the wait for servers that need longer before /health responds —
|
||||
# on timeout we keep the optimistic legacy behavior and say so.
|
||||
local grace="${HERMES_WEBUI_START_GRACE:-3}"
|
||||
[[ "${grace}" =~ ^[0-9]+$ ]] || grace=3
|
||||
# 0 would skip startup monitoring entirely and restore the stale-PID
|
||||
# behavior this window exists to prevent; treat it like any invalid value.
|
||||
(( grace > 0 )) || grace=3
|
||||
local grace_steps=$(( grace * 4 )) step=0 healthy=0
|
||||
while (( step < grace_steps )); do
|
||||
if ! _is_alive "${pid}"; then
|
||||
echo "[ctl] Hermes WebUI failed to stay running. Log: ${LOG_FILE}" >&2
|
||||
rm -f "${PID_FILE}" "${STATE_FILE}"
|
||||
return 1
|
||||
fi
|
||||
if hermes_webui_probe_health "${probe_host}" "${CTL_PORT}" "/health" 1 direct >/dev/null 2>&1; then
|
||||
healthy=1
|
||||
break
|
||||
fi
|
||||
sleep 0.25
|
||||
step=$(( step + 1 ))
|
||||
done
|
||||
sleep 0.15
|
||||
if ! _is_alive "${pid}"; then
|
||||
echo "[ctl] Hermes WebUI failed to stay running. Log: ${LOG_FILE}" >&2
|
||||
rm -f "${PID_FILE}" "${STATE_FILE}"
|
||||
@@ -678,28 +384,6 @@ start_cmd() {
|
||||
echo "[ctl] Started Hermes WebUI (PID ${pid})"
|
||||
echo "[ctl] Bound: ${CTL_HOST}:${CTL_PORT}"
|
||||
echo "[ctl] Log: ${LOG_FILE}"
|
||||
if (( ! healthy )); then
|
||||
echo "[ctl] Note: /health did not respond within ${grace}s; check '$(basename "$0") status' shortly."
|
||||
fi
|
||||
}
|
||||
|
||||
_warn_if_unmanaged_instance_serving() {
|
||||
# After stop concluded "nothing to do", check whether a server is STILL
|
||||
# answering on the configured port. Silently reporting "stopped" while a
|
||||
# foreign (e.g. systemd-supervised) instance keeps serving is how operators
|
||||
# end up starting doomed duplicates.
|
||||
_load_state_if_present
|
||||
local host="${HOST:-${HERMES_WEBUI_HOST:-127.0.0.1}}"
|
||||
local port="${PORT:-${HERMES_WEBUI_PORT:-8787}}"
|
||||
local probe_host
|
||||
probe_host="$(_probe_target_host "${host}")"
|
||||
if _port_answers_http "${probe_host}" "${port}"; then
|
||||
echo "[ctl] Warning: an instance NOT managed by ctl.sh is still serving ${probe_host}:${port} — not touching it." >&2
|
||||
local listener_diag
|
||||
listener_diag="$(_port_listener_diag "${port}")"
|
||||
[[ -n "${listener_diag}" ]] && echo "[ctl] listener: ${listener_diag}" >&2
|
||||
echo "[ctl] If it is supervised (systemd/launchd), stop or restart it there instead." >&2
|
||||
fi
|
||||
}
|
||||
|
||||
stop_cmd() {
|
||||
@@ -707,15 +391,11 @@ stop_cmd() {
|
||||
local pid
|
||||
if ! pid="$(_pid_from_file 2>/dev/null)"; then
|
||||
echo "[ctl] Hermes WebUI is stopped"
|
||||
# Warn BEFORE deleting the state file: it carries the saved host/port
|
||||
# binding the probe needs when the instance was started off-default.
|
||||
_warn_if_unmanaged_instance_serving
|
||||
rm -f "${PID_FILE}" "${STATE_FILE}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if ! _is_alive "${pid}" || ! _is_owned_webui_pid "${pid}"; then
|
||||
_warn_if_unmanaged_instance_serving
|
||||
_clear_stale_pid
|
||||
return 0
|
||||
fi
|
||||
@@ -767,7 +447,6 @@ except Exception:
|
||||
status_cmd() {
|
||||
ensure_home
|
||||
_load_repo_dotenv_preserving_env
|
||||
_load_hermes_dotenv
|
||||
_load_state_if_present
|
||||
local host="${HOST:-${HERMES_WEBUI_HOST:-127.0.0.1}}"
|
||||
local port="${PORT:-${HERMES_WEBUI_PORT:-8787}}"
|
||||
@@ -785,29 +464,11 @@ status_cmd() {
|
||||
echo " Health: ${health}"
|
||||
else
|
||||
[[ -f "${PID_FILE}" ]] && _clear_stale_pid >/dev/null 2>&1 || true
|
||||
local probe_host
|
||||
probe_host="$(_probe_target_host "${host}")"
|
||||
if _port_answers_http "${probe_host}" "${port}"; then
|
||||
# Something serves the port but ctl.sh does not own it (systemd unit,
|
||||
# launchd job, manual run). Saying "stopped" here is what leads
|
||||
# operators to start a doomed duplicate.
|
||||
health="$(_health_line "${probe_host}" "${port}")"
|
||||
echo "● hermes-webui — running (not managed by ctl.sh)"
|
||||
local listener_diag
|
||||
listener_diag="$(_port_listener_diag "${port}")"
|
||||
echo " PID: -"
|
||||
[[ -n "${listener_diag}" ]] && echo " Listener: ${listener_diag}"
|
||||
echo " Bound: ${host}:${port}"
|
||||
echo " Log: ${log_path}"
|
||||
echo " Health: ${health}"
|
||||
echo " Note: manage it via its own supervisor (systemctl/launchctl) or the process directly."
|
||||
else
|
||||
echo "● hermes-webui — stopped"
|
||||
echo " PID: -"
|
||||
echo " Bound: ${host}:${port}"
|
||||
echo " Log: ${log_path}"
|
||||
echo " Health: not checked"
|
||||
fi
|
||||
echo "● hermes-webui — stopped"
|
||||
echo " PID: -"
|
||||
echo " Bound: ${host}:${port}"
|
||||
echo " Log: ${log_path}"
|
||||
echo " Health: not checked"
|
||||
fi
|
||||
}
|
||||
|
||||
|
||||
@@ -3,10 +3,6 @@
|
||||
# QUICK START:
|
||||
# docker compose -f docker-compose.three-container.yml up -d
|
||||
# Open http://localhost:8787 (chat) and http://localhost:9119 (dashboard)
|
||||
# Remote access (issue #6876): echo "HERMES_DASHBOARD_BIND=0.0.0.0" >> .env
|
||||
# and echo "API_SERVER_KEY=<16+ chars>" >> .env, then recreate the stack.
|
||||
# The WebUI reports "Hermes agent is not responding" when the gateway
|
||||
# listener is down — API_SERVER_KEY must be set for port 8642 to open.
|
||||
#
|
||||
# This extends the two-container setup with the Hermes Dashboard for
|
||||
# monitoring agent activity, sessions, and resource usage.
|
||||
@@ -43,10 +39,7 @@ services:
|
||||
container_name: hermes-agent
|
||||
command: gateway run
|
||||
ports:
|
||||
# Bind to loopback by default. For remote access (issue #6876), set
|
||||
# HERMES_AGENT_BIND=0.0.0.0 in .env — the gateway requires
|
||||
# API_SERVER_KEY (>=16 chars) to open the listener at all.
|
||||
- "${HERMES_AGENT_BIND:-127.0.0.1}:8642:8642"
|
||||
- "127.0.0.1:8642:8642"
|
||||
volumes:
|
||||
# Persist config, state, sessions, skills, memory across restarts
|
||||
- hermes-home:/home/hermes/.hermes
|
||||
@@ -58,15 +51,6 @@ services:
|
||||
# Defaults to 1000 to match WANTED_UID/WANTED_GID in the webui service.
|
||||
- HERMES_UID=${UID:-1000}
|
||||
- HERMES_GID=${GID:-1000}
|
||||
# Gateway API server (port 8642). The WebUI's Tasks/cron health probe
|
||||
# reaches this listener via HERMES_API_URL below. The agent only starts
|
||||
# the listener when API_SERVER_KEY is a usable value (>=16 chars); set
|
||||
# it in .env to enable the gateway API. Without a key the gateway daemon
|
||||
# still runs (messaging + cron) but port 8642 stays unbound and the
|
||||
# WebUI reports "Gateway endpoint not reachable".
|
||||
- API_SERVER_ENABLED=true
|
||||
- API_SERVER_HOST=0.0.0.0
|
||||
- API_SERVER_KEY=${API_SERVER_KEY:-}
|
||||
# Bind-mount permission handling for the agent — narrow set of overrides.
|
||||
# NOTE: The agent's HERMES_HOME_MODE applies to the HERMES_HOME *directory*
|
||||
# mode (default 0700) — NOT to credential files like the WebUI's variant.
|
||||
@@ -90,11 +74,7 @@ services:
|
||||
container_name: hermes-dashboard
|
||||
command: dashboard --host 0.0.0.0 --insecure
|
||||
ports:
|
||||
# Loopback-only by default (the dashboard runs --insecure). To reach the
|
||||
# dashboard from another machine (issue #6876), set
|
||||
# HERMES_DASHBOARD_BIND=0.0.0.0 in .env and put it behind auth / a
|
||||
# reverse proxy.
|
||||
- "${HERMES_DASHBOARD_BIND:-127.0.0.1}:9119:9119"
|
||||
- "127.0.0.1:9119:9119"
|
||||
volumes:
|
||||
- hermes-home:/home/hermes/.hermes
|
||||
environment:
|
||||
@@ -152,18 +132,14 @@ services:
|
||||
# echo "UID=$(id -u)" >> .env && echo "GID=$(id -g)" >> .env
|
||||
- WANTED_UID=${UID:-1000}
|
||||
- WANTED_GID=${GID:-1000}
|
||||
# Forward the API server key so the WebUI's gateway health probe can
|
||||
# authenticate (must match the agent's API_SERVER_KEY above).
|
||||
- HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
|
||||
# Optional: enable password auth. Set HERMES_WEBUI_PASSWORD in .env
|
||||
# (required if you expose the port beyond 127.0.0.1).
|
||||
- HERMES_WEBUI_PASSWORD=${HERMES_WEBUI_PASSWORD:-}
|
||||
# NOTE: When using bind-mount volumes shared across containers, ALL containers
|
||||
# that write to the same host directory must run as the same UID/GID.
|
||||
# If hermes-agent initialises the state dir as root (UID 0), hermes-webui
|
||||
# will get a PermissionError accessing those paths — including a crash on every
|
||||
# HTTP request if the auth signing-key file is unreadable. Either set WANTED_UID
|
||||
# to match the agent container's UID, or use a named Docker volume (preferred).
|
||||
# Optional: set a password for remote access
|
||||
# - HERMES_WEBUI_PASSWORD=your-secret-password
|
||||
# Bind-mount permission handling for the WebUI (fixes #1389, #1399).
|
||||
# NOTE: WebUI's HERMES_HOME_MODE is a credential-file threshold (allow
|
||||
# group bits on .env/.signing_key/etc.), DIFFERENT from the agent's
|
||||
|
||||
@@ -67,15 +67,6 @@ services:
|
||||
# The agent image's entrypoint already supports usermod remapping.
|
||||
- HERMES_UID=${UID:-1000}
|
||||
- HERMES_GID=${GID:-1000}
|
||||
# Gateway API server (port 8642). The WebUI's Tasks/cron health probe
|
||||
# reaches this listener via HERMES_API_URL below. The agent only starts
|
||||
# the listener when API_SERVER_KEY is a usable value (>=16 chars); set
|
||||
# it in .env to enable the gateway API. Without a key the gateway daemon
|
||||
# still runs (messaging + cron) but port 8642 stays unbound and the
|
||||
# WebUI reports "Gateway endpoint not reachable".
|
||||
- API_SERVER_ENABLED=true
|
||||
- API_SERVER_HOST=0.0.0.0
|
||||
- API_SERVER_KEY=${API_SERVER_KEY:-}
|
||||
# Bind-mount permission handling for the agent — narrow set of overrides.
|
||||
# NOTE: The agent's HERMES_HOME_MODE applies to the HERMES_HOME *directory*
|
||||
# mode (default 0700) — NOT to credential files like the WebUI's variant.
|
||||
@@ -126,12 +117,8 @@ services:
|
||||
# echo "UID=$(id -u)" >> .env && echo "GID=$(id -g)" >> .env
|
||||
- WANTED_UID=${UID:-1000}
|
||||
- WANTED_GID=${GID:-1000}
|
||||
# Forward the API server key so the WebUI's gateway health probe can
|
||||
# authenticate (must match the agent's API_SERVER_KEY above).
|
||||
- HERMES_WEBUI_GATEWAY_API_KEY=${API_SERVER_KEY:-}
|
||||
# Optional: enable password auth. Set HERMES_WEBUI_PASSWORD in .env
|
||||
# (required if you expose the port beyond 127.0.0.1).
|
||||
- HERMES_WEBUI_PASSWORD=${HERMES_WEBUI_PASSWORD:-}
|
||||
# Optional: set a password for remote access
|
||||
# - HERMES_WEBUI_PASSWORD=your-secret-password
|
||||
# Bind-mount permission handling for the WebUI (fixes #1389, #1399).
|
||||
# NOTE: WebUI's HERMES_HOME_MODE is a credential-file threshold (allow
|
||||
# group bits on .env/.signing_key/etc.), DIFFERENT from the agent's
|
||||
|
||||
+2
-3
@@ -42,9 +42,8 @@ services:
|
||||
- HERMES_WEBUI_STATE_DIR=/home/hermeswebui/.hermes/webui
|
||||
# Default workspace directory shown on first launch
|
||||
# - HERMES_WEBUI_DEFAULT_WORKSPACE=/workspace
|
||||
# Optional: enable password auth. Set HERMES_WEBUI_PASSWORD in .env
|
||||
# (required if you expose the port beyond 127.0.0.1).
|
||||
- HERMES_WEBUI_PASSWORD=${HERMES_WEBUI_PASSWORD:-}
|
||||
# Optional: set a password for remote access
|
||||
# - HERMES_WEBUI_PASSWORD=your-secret-password
|
||||
#
|
||||
# Bind-mount permission handling (fixes #1389, #1399):
|
||||
# When you mount an EXISTING ~/.hermes directory (the common case),
|
||||
|
||||
+26
-72
@@ -57,31 +57,17 @@ if [ ! -d "$itdir" ]; then error_exit "Failed to create $itdir"; fi
|
||||
# logic: if not set and file exists, use file value, else use default. Create file for persistence when the container is re-run
|
||||
# reasoning: needed when using docker compose as the file will exist in the stopped container, and changing the value from environment variables or configuration file must be propagated from the root init phase to the hermeswebui runtime phase
|
||||
it=$itdir/hermeswebui_user_uid
|
||||
# Remember where the value came from. Auto-detection must never overwrite an
|
||||
# identity the operator chose explicitly — including the legitimate value 1024,
|
||||
# which is also the built-in default and was therefore indistinguishable from
|
||||
# "unset" (#7027). The source is persisted next to the value because `su` drops
|
||||
# the environment when this script re-enters as the runtime user, so an explicit
|
||||
# choice would otherwise look like a detected one on the second pass.
|
||||
it_source=$itdir/hermeswebui_user_uid_source
|
||||
_wanted_uid_source=""
|
||||
if [ -n "${WANTED_UID+x}" ]; then
|
||||
_wanted_uid_source=explicit
|
||||
elif [ -f $it ]; then
|
||||
WANTED_UID=$(cat $it)
|
||||
if [ -f $it_source ]; then _wanted_uid_source=$(cat $it_source); fi
|
||||
if [ -z "${WANTED_UID+x}" ]; then
|
||||
if [ -f $it ]; then WANTED_UID=$(cat $it); fi
|
||||
fi
|
||||
# Auto-detect from mounted volumes if still unset (#569, #668, #7027).
|
||||
# Auto-detect from mounted volumes if still unset (#569, #668).
|
||||
# On macOS, host UIDs start at 501. Using the wrong UID means the container
|
||||
# user cannot read the bind-mounted files, making the workspace appear empty.
|
||||
# In two-container setups (hermes-agent + hermes-webui), the shared hermes-home
|
||||
# volume may be owned by the agent container's UID — detect from there first.
|
||||
if [ "$_wanted_uid_source" != "explicit" ] && { [ -z "${WANTED_UID+x}" ] || [ "${WANTED_UID}" = "1024" ]; }; then
|
||||
# Priority 1: the configured state dir first (#7027) — in a single-container
|
||||
# deploy it is a bind mount by definition, so its owner is the host identity
|
||||
# the container has to match — then the hermes-home shared volume, which
|
||||
# covers two-container Zeabur/Compose setups (#668)
|
||||
for _probe_dir in "${HERMES_WEBUI_STATE_DIR:-/app/data}" "/home/hermeswebui/.hermes" "$HERMES_HOME" "/opt/data"; do
|
||||
if [ -z "${WANTED_UID+x}" ] || [ "${WANTED_UID}" = "1024" ]; then
|
||||
# Priority 1: hermes-home shared volume — covers two-container Zeabur/Compose setups (#668)
|
||||
for _probe_dir in "/home/hermeswebui/.hermes" "$HERMES_HOME" "/opt/data"; do
|
||||
if [ -d "$_probe_dir" ]; then
|
||||
_detected_uid=$(stat -c '%u' "$_probe_dir" 2>/dev/null || echo "")
|
||||
if [ -n "$_detected_uid" ] && [ "$_detected_uid" != "0" ]; then
|
||||
@@ -92,10 +78,8 @@ if [ "$_wanted_uid_source" != "explicit" ] && { [ -z "${WANTED_UID+x}" ] || [ "$
|
||||
fi
|
||||
done
|
||||
fi
|
||||
if [ "$_wanted_uid_source" != "explicit" ] && { [ -z "${WANTED_UID+x}" ] || [ "${WANTED_UID}" = "1024" ]; }; then
|
||||
# Priority 2: /workspace bind-mount — a useful signal when it is actually
|
||||
# bind-mounted, but a stock image ships /workspace owned by the build-time
|
||||
# 1024, which says nothing about the host, so it stays below the state dir (#7027)
|
||||
if [ -z "${WANTED_UID+x}" ] || [ "${WANTED_UID}" = "1024" ]; then
|
||||
# Priority 2: /workspace bind-mount — the standard single-container mount point
|
||||
if [ -d "/workspace" ]; then
|
||||
_detected_uid=$(stat -c '%u' "/workspace" 2>/dev/null || echo "")
|
||||
if [ -n "$_detected_uid" ] && [ "$_detected_uid" != "0" ]; then
|
||||
@@ -106,22 +90,16 @@ if [ "$_wanted_uid_source" != "explicit" ] && { [ -z "${WANTED_UID+x}" ] || [ "$
|
||||
fi
|
||||
WANTED_UID=${WANTED_UID:-1024}
|
||||
write_privtmpfile $it "$WANTED_UID"
|
||||
write_privtmpfile $it_source "${_wanted_uid_source:-detected}"
|
||||
echo "-- WANTED_UID: \"${WANTED_UID}\""
|
||||
|
||||
it=$itdir/hermeswebui_user_gid
|
||||
it_source=$itdir/hermeswebui_user_gid_source
|
||||
_wanted_gid_source=""
|
||||
if [ -n "${WANTED_GID+x}" ]; then
|
||||
_wanted_gid_source=explicit
|
||||
elif [ -f $it ]; then
|
||||
WANTED_GID=$(cat $it)
|
||||
if [ -f $it_source ]; then _wanted_gid_source=$(cat $it_source); fi
|
||||
if [ -z "${WANTED_GID+x}" ]; then
|
||||
if [ -f $it ]; then WANTED_GID=$(cat $it); fi
|
||||
fi
|
||||
# Auto-detect GID from mounted volumes to match (#569, #668, #7027)
|
||||
if [ "$_wanted_gid_source" != "explicit" ] && { [ -z "${WANTED_GID+x}" ] || [ "${WANTED_GID}" = "1024" ]; }; then
|
||||
# Priority 1: configured state dir first, then the hermes-home shared volume
|
||||
for _probe_dir in "${HERMES_WEBUI_STATE_DIR:-/app/data}" "/home/hermeswebui/.hermes" "$HERMES_HOME" "/opt/data"; do
|
||||
# Auto-detect GID from mounted volumes to match (#569, #668)
|
||||
if [ -z "${WANTED_GID+x}" ] || [ "${WANTED_GID}" = "1024" ]; then
|
||||
# Priority 1: hermes-home shared volume
|
||||
for _probe_dir in "/home/hermeswebui/.hermes" "$HERMES_HOME" "/opt/data"; do
|
||||
if [ -d "$_probe_dir" ]; then
|
||||
_detected_gid=$(stat -c '%g' "$_probe_dir" 2>/dev/null || echo "")
|
||||
if [ -n "$_detected_gid" ] && [ "$_detected_gid" != "0" ]; then
|
||||
@@ -132,8 +110,8 @@ if [ "$_wanted_gid_source" != "explicit" ] && { [ -z "${WANTED_GID+x}" ] || [ "$
|
||||
fi
|
||||
done
|
||||
fi
|
||||
if [ "$_wanted_gid_source" != "explicit" ] && { [ -z "${WANTED_GID+x}" ] || [ "${WANTED_GID}" = "1024" ]; }; then
|
||||
# Priority 2: /workspace bind-mount — lower priority for the same reason as UID (#7027)
|
||||
if [ -z "${WANTED_GID+x}" ] || [ "${WANTED_GID}" = "1024" ]; then
|
||||
# Priority 2: /workspace bind-mount
|
||||
if [ -d "/workspace" ]; then
|
||||
_detected_gid=$(stat -c '%g' "/workspace" 2>/dev/null || echo "")
|
||||
if [ -n "$_detected_gid" ] && [ "$_detected_gid" != "0" ]; then
|
||||
@@ -144,7 +122,6 @@ if [ "$_wanted_gid_source" != "explicit" ] && { [ -z "${WANTED_GID+x}" ] || [ "$
|
||||
fi
|
||||
WANTED_GID=${WANTED_GID:-1024}
|
||||
write_privtmpfile $it "$WANTED_GID"
|
||||
write_privtmpfile $it_source "${_wanted_gid_source:-detected}"
|
||||
echo "-- WANTED_GID: \"${WANTED_GID}\""
|
||||
|
||||
echo "== Most Environment variables set"
|
||||
@@ -169,26 +146,7 @@ load_env() {
|
||||
obfuscate_part="${ENV_OBFUSCATE_PART}"
|
||||
if [ -f "$tocheck" ]; then
|
||||
echo "-- Loading environment variables from $tocheck (overwrite existing: $overwrite_if_different) (ignorelist: $ignore_list) (obfuscate: $obfuscate_part)"
|
||||
# Read whole lines, then split at the FIRST '=' only. Splitting on every
|
||||
# '=' (IFS='=' read -r key value) makes read drop a single trailing '='
|
||||
# from values, corrupting secrets such as openssl rand -base64 32 output
|
||||
# (44 chars ending in '=').
|
||||
while IFS= read -r key; do
|
||||
# Skip empty lines and comment lines.
|
||||
case "$key" in
|
||||
''|\#*) continue ;;
|
||||
esac
|
||||
if [[ "$key" == *=* ]]; then
|
||||
# Split at the FIRST '=' only. Derive value before shortening key,
|
||||
# since ${key%%=*} mutates key. Reusing the existing 'key' read
|
||||
# target (rather than a new 'line' var) avoids silently clobbering a
|
||||
# user env var named 'line' on the post-su reload.
|
||||
value=${key#*=}
|
||||
key=${key%%=*}
|
||||
else
|
||||
# Line without '=': whole line is the key with an empty value.
|
||||
value=
|
||||
fi
|
||||
while IFS='=' read -r key value; do
|
||||
doit=false
|
||||
# checking if the key is in the ignorelist
|
||||
for i in $ignore_list; do
|
||||
@@ -472,17 +430,14 @@ else
|
||||
# `egg_info` build step, however, touches `hermes_agent.egg-info/` inside the
|
||||
# source tree even under PEP 517 build isolation, which `EROFS`-fails on a
|
||||
# `:ro` mount and (under `set -e`) kills startup of every multi-container
|
||||
# deploy. Stage the source into a writable directory so the editable install
|
||||
# can write its metadata without touching the underlying mount.
|
||||
#
|
||||
# The staged copy lives under /app/ (persistent across container restarts)
|
||||
# because the editable install records this path in a .pth file — Python
|
||||
# imports resolve through it at runtime. Placing it alongside the venv gives
|
||||
# it the same lifecycle as .deps_installed: both survive restarts and both
|
||||
# are lost on container recreation (fresh boot re-stages automatically).
|
||||
# deploy. Stage the source into a writable tmpfs copy so the build can write
|
||||
# its metadata side-by-side without touching the underlying mount.
|
||||
#
|
||||
# The copy excludes any pre-baked `*.egg-info` / `build` / `dist` artifacts
|
||||
# to avoid the timestamp-update path setuptools takes when one is present.
|
||||
# to avoid the timestamp-update path setuptools takes when one is present,
|
||||
# and `--reflink=auto` makes the copy near-free on overlay2/btrfs where
|
||||
# supported. We rebuild on every container start (the agent source can
|
||||
# change across volume re-init); cost is one rsync of ~10MB of Python source.
|
||||
#
|
||||
# NB: `rsync -a` / `cp -a` preserve the source tree's mode bits, so a `:ro`
|
||||
# source mounted mode 555 leaves the staged copy also mode 555. setuptools
|
||||
@@ -490,14 +445,13 @@ else
|
||||
# with "Permission denied" even though `_stage_src` itself was created
|
||||
# writable by hermeswebui. Re-add owner-write on the staged tree after the
|
||||
# copy so the build dir is genuinely writable, not just owned by us.
|
||||
_stage_src="/app/hermes-agent-src"
|
||||
_stage_src="/tmp/hermes-agent-build"
|
||||
rm -rf "$_stage_src"
|
||||
mkdir -p "$_stage_src"
|
||||
if command -v rsync >/dev/null 2>&1; then
|
||||
rsync -a \
|
||||
--exclude='*.egg-info' --exclude='build' --exclude='dist' \
|
||||
--exclude='__pycache__' --exclude='.git' \
|
||||
--exclude='.playwright' \
|
||||
"$_agent_src"/ "$_stage_src"/ \
|
||||
|| error_exit "Failed to stage hermes-agent source to writable build dir"
|
||||
else
|
||||
@@ -506,13 +460,13 @@ else
|
||||
cp -a "$_agent_src"/. "$_stage_src"/ \
|
||||
|| error_exit "Failed to copy hermes-agent source to writable build dir"
|
||||
rm -rf "$_stage_src"/*.egg-info "$_stage_src"/build "$_stage_src"/dist 2>/dev/null || true
|
||||
rm -rf "$_stage_src"/.playwright 2>/dev/null || true
|
||||
find "$_stage_src" -type d -name __pycache__ -exec rm -rf {} + 2>/dev/null || true
|
||||
fi
|
||||
chmod -R u+w "$_stage_src" \
|
||||
|| error_exit "Failed to make staged hermes-agent source writable (rsync/cp preserved :ro mount perms)"
|
||||
uv pip install -e "$_stage_src[all]" --trusted-host pypi.org --trusted-host files.pythonhosted.org \
|
||||
uv pip install "$_stage_src[all]" --trusted-host pypi.org --trusted-host files.pythonhosted.org \
|
||||
|| error_exit "Failed to install hermes-agent's requirements"
|
||||
rm -rf "$_stage_src"
|
||||
else
|
||||
echo ""
|
||||
echo "!! WARNING: hermes-agent source not found."
|
||||
|
||||
+4
-26
@@ -23,29 +23,19 @@ contributor guidance; it does not change runtime behavior or CI gates.
|
||||
|
||||
## Runtime, durability, and state contracts
|
||||
|
||||
- [`docs/remote-workspaces.md`](remote-workspaces.md):
|
||||
architecture contract for remote terminal workspaces (SSH/Docker), target-side
|
||||
POSIX path preservation against macOS synthetic firmlink expansion, and
|
||||
per-profile isolation boundaries.
|
||||
- [`docs/rfcs/webui-run-state-consistency-contract.md`](rfcs/webui-run-state-consistency-contract.md):
|
||||
proposed consistency rules for current WebUI streaming, recovery, replay,
|
||||
model-context reconstruction, compression, UI scene/cache, and sidebar metadata
|
||||
repairs. Start here for narrow fixes that keep the existing WebUI execution
|
||||
path.
|
||||
- [`docs/rfcs/live-to-final-assistant-replies.md`](rfcs/live-to-final-assistant-replies.md):
|
||||
accepted product model for long-running assistant replies, live process text,
|
||||
proposed product model for long-running assistant replies, live process text,
|
||||
tool activity, recovery, terminal outcomes, and final-answer boundaries. Start
|
||||
here for UI/UX changes to running-session assistant reply rendering.
|
||||
- [`docs/rfcs/stable-assistant-turn-anchors.md`](rfcs/stable-assistant-turn-anchors.md):
|
||||
implemented presentation/reconciliation model that attaches live, settled,
|
||||
replayed, and recovered activity to one assistant-turn owner and projects one
|
||||
`activity_scene_v1` into Compact Worklog, Transparent Stream, or Final answer
|
||||
only. Remaining hardening stays tracked under #3400.
|
||||
- [`docs/architecture/stable-assistant-turn-anchor-phase0.md`](architecture/stable-assistant-turn-anchor-phase0.md):
|
||||
cumulative implementation inventory for the Stable Assistant Turn Anchors
|
||||
work under #3926. Use it to distinguish shipped wiring from historical slice
|
||||
boundaries before changing live SSE, replay, settlement, `INFLIGHT`, or
|
||||
`renderMessages()` paths.
|
||||
current Phase 0 inventory for the Stable Assistant Turn Anchors work under
|
||||
#3926. Use this before wiring anchor helpers into live SSE, replay,
|
||||
settlement, `INFLIGHT`, or `renderMessages()` paths.
|
||||
- [`docs/rfcs/canonical-session-resolution.md`](rfcs/canonical-session-resolution.md):
|
||||
proposed contract for resolving URL routes, query parameters, localStorage,
|
||||
sidebar rows, and compression-lineage IDs to one canonical visible session
|
||||
@@ -70,18 +60,6 @@ contributor guidance; it does not change runtime behavior or CI gates.
|
||||
queued messages, interrupt replacement, steer visibility, or leftover-steer
|
||||
recovery changes.
|
||||
- [`docs/rfcs/README.md`](rfcs/README.md): RFC conventions and current RFC index.
|
||||
- [`docs/rfcs/session-sse-contract-v1.md`](rfcs/session-sse-contract-v1.md):
|
||||
proposed contract vocabulary, cursor/resume semantics, replay identity, snapshot
|
||||
fallback, event taxonomy, and implementation gates for the per-session SSE
|
||||
stream `GET /api/sessions/{session_id}/events` (#4812). Distinct from the
|
||||
existing global session-list stream `GET /api/sessions/events`. Start here for
|
||||
any work that touches per-session SSE, `Last-Event-ID` replay, or session
|
||||
lifecycle event delivery. The Phase 1 **server route and journal relay** for
|
||||
`GET /api/sessions/{session_id}/events` are implemented; broader client,
|
||||
platform, and semantic-taxonomy claims in the RFC remain behind the recorded
|
||||
proof gates. Prefer the RFC's **Authoritative emitted events** table (live
|
||||
`/api/chat/stream` wire names) over the aspirational semantic taxonomy when
|
||||
writing clients against current source.
|
||||
|
||||
When a change touches streaming, recovery, replay, compression, context
|
||||
reconstruction, cancellation, approval/clarify, session metadata, or run state,
|
||||
|
||||
+22
-264
@@ -17,15 +17,6 @@ This is intentionally not a plugin marketplace or dependency system. It is a
|
||||
safe escape hatch for local dashboards, internal tooling, and workflow-specific
|
||||
panels that should not live in core Hermes WebUI.
|
||||
|
||||
> **The vetted extension library.** The curated, one-click-installable extensions
|
||||
> that appear in the gallery live in a separate public repo:
|
||||
> **[hermes-webui/hermes-webui-extensions](https://github.com/hermes-webui/hermes-webui-extensions)**.
|
||||
> "In the registry == vetted." That repo holds the entries, the authoring
|
||||
> conventions ([`docs/extension-entry.md`](https://github.com/hermes-webui/hermes-webui-extensions/blob/main/docs/extension-entry.md)),
|
||||
> the JSON schema, and the CI safety gates. This document covers the WebUI-side
|
||||
> *infrastructure* (loader, manifest contract, capabilities, install client);
|
||||
> see the library repo to browse existing extensions or contribute a new one.
|
||||
|
||||
## What extensions can do
|
||||
|
||||
Extensions can:
|
||||
@@ -44,7 +35,6 @@ Extensions cannot, by themselves:
|
||||
- serve files outside the configured extension directory
|
||||
- load third-party scripts/styles through the built-in injection config
|
||||
- register new WebUI backend routes or proxy arbitrary sidecar/backend traffic
|
||||
outside the fixed consented extension sidecar path described below
|
||||
- change Hermes Agent permissions, models, memory, or tools unless they call
|
||||
existing authenticated APIs that already allow those changes
|
||||
|
||||
@@ -151,8 +141,7 @@ manifest such as `extensions.json` keeps the existing
|
||||
`desktop-companion/manifest.json` resolves relative assets under
|
||||
`/extensions/desktop-companion/`.
|
||||
|
||||
Extension entries may also declare a loopback sidecar for diagnostics and the
|
||||
opt-in proxy:
|
||||
Extension entries may also declare a read-only loopback sidecar for diagnostics:
|
||||
|
||||
```json
|
||||
{
|
||||
@@ -165,230 +154,16 @@ opt-in proxy:
|
||||
"sidecar": {
|
||||
"type": "loopback",
|
||||
"origin": "http://127.0.0.1:17787",
|
||||
"health_path": "/health",
|
||||
"proxy_auth": "token-v1"
|
||||
"health_path": "/health"
|
||||
}
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Loopback sidecars do **not** change asset injection behavior. They are reported
|
||||
by diagnostics so an operator can see that a local companion service was
|
||||
declared and optionally check its health from the browser. If the operator later
|
||||
approves the proxy in **Settings → Extensions**, WebUI may proxy requests only
|
||||
through the fixed per-extension sidecar path for that extension.
|
||||
|
||||
### Sidecar proxy authentication (`proxy_auth`)
|
||||
|
||||
The loopback port a sidecar binds is reachable by **any local process**, and the
|
||||
proxy strips every inbound credential (cookies, `Authorization`, CSRF, `x-hermes-*`)
|
||||
before forwarding — so a sidecar cannot, on its own, tell a proxied request from a
|
||||
direct one. The `proxy_auth` field closes that gap:
|
||||
|
||||
- **`token-v1`** (recommended for any sidecar that mutates state) — WebUI mints a
|
||||
per-extension secret at `STATE_DIR/sidecar-auth/<id>.token` (mode `0600`) and
|
||||
injects it as the `X-Hermes-Sidecar-Token` header on every proxied request. The
|
||||
sidecar resolves the token file in this order — `HERMES_EXT_SIDECAR_TOKEN_FILE`
|
||||
→ `$HERMES_WEBUI_STATE_DIR/sidecar-auth/<id>.token` →
|
||||
`$HERMES_HOME/webui/sidecar-auth/<id>.token` → platform default
|
||||
(`~/.hermes/webui/…`, `%LOCALAPPDATA%\hermes\webui\…` on Windows) — and must
|
||||
validate the header on every route except `/health`, returning **`401` on a
|
||||
missing/mismatched token** and **`503` when the token file is absent/unreadable**.
|
||||
The canonical scaffold in the extensions repository (`examples/`, see
|
||||
`docs/SIDECAR_CONTRACT.md`) does all of this for you — do not hand-roll it.
|
||||
- **absent (or the explicit literal `"legacy"`)** — **legacy** mode (no token;
|
||||
unchanged behavior). Only appropriate for read-only, non-sensitive sidecars.
|
||||
- Any **other** value fails closed (the sidecar declaration is rejected).
|
||||
|
||||
**Auth-off posture:** WebUI authentication is optional and off by default. Because
|
||||
the consent endpoint and proxy route are unauthenticated in that mode, `token-v1`
|
||||
fails closed regardless of whether the sidecar origin is loopback: consent and proxy
|
||||
resolution return `403` until WebUI authentication is configured. Otherwise, any
|
||||
caller that can reach WebUI could ask core to inject the token and use it as a
|
||||
forwarding oracle. The extensions panel exposes the `local_unprotected` posture so
|
||||
the operator is told to enable authentication before granting consent. The token
|
||||
protects against other-UID and sandboxed local callers; it does **not** defend against
|
||||
arbitrary same-UID code (which can read the token file, WebUI's own signing key, or
|
||||
run the sidecar's tool directly).
|
||||
|
||||
Extension entries may declare browser-local settings when they also request
|
||||
extension-owned storage:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "desktop-companion",
|
||||
"permissions": {
|
||||
"storage": {
|
||||
"owned": true
|
||||
}
|
||||
},
|
||||
"settings_schema": [
|
||||
{
|
||||
"key": "show_badge",
|
||||
"type": "boolean",
|
||||
"label": "Show badge",
|
||||
"default": true
|
||||
},
|
||||
{
|
||||
"key": "mode",
|
||||
"type": "enum",
|
||||
"label": "Mode",
|
||||
"options": [
|
||||
{"value": "compact", "label": "Compact"},
|
||||
{"value": "full", "label": "Full"}
|
||||
],
|
||||
"default": "compact"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Settings are a first-pass browser feature. WebUI sanitizes the manifest schema, injects the accepted schema before extension scripts, and leaves persistence to the browser. The backend does not store extension settings or expose a generic settings write route, and it does not treat these values as secrets.
|
||||
|
||||
The sanitizer accepts only `boolean`, `string`, `number`, `integer`, and `enum`
|
||||
fields. It drops `sensitive: true` fields, unsupported types, malformed enum
|
||||
options, duplicate keys after the first valid field, and defaults that do not
|
||||
match the declared type. `settings_schema` is honored only when
|
||||
`permissions.storage.owned` is exactly `true`.
|
||||
|
||||
Extension scripts can use the sanctioned browser accessors:
|
||||
|
||||
```js
|
||||
const settings = window.HermesExtensionSettings.settingsForExtension("desktop-companion");
|
||||
const value = settings.get("show_badge");
|
||||
settings.set("show_badge", false);
|
||||
|
||||
const storage = window.HermesExtensionSettings.storageForExtension("desktop-companion");
|
||||
storage.set("lastPanel", "settings");
|
||||
|
||||
const sameSettings = window.hermesExt.settings.forExtension("desktop-companion");
|
||||
const sameStorage = window.hermesExt.storage.forExtension("desktop-companion");
|
||||
```
|
||||
|
||||
Settings persist only non-default overrides. Resetting settings removes those
|
||||
overrides and returns schema defaults. Extension-owned storage uses a separate
|
||||
browser-local namespace, and clearing storage removes that namespace without
|
||||
changing settings.
|
||||
|
||||
### Cooperative extension identity
|
||||
|
||||
An injected extension can claim a stable browser-page identity together with its
|
||||
existing settings and storage accessors:
|
||||
|
||||
```js
|
||||
const ext = window.hermesExt.register("desktop-companion");
|
||||
if (ext) {
|
||||
ext.settings.set("show_badge", false);
|
||||
ext.storage.set("lastPanel", "settings");
|
||||
}
|
||||
```
|
||||
|
||||
`register(id)` trims the ID and succeeds only for an ID in the effective-enabled,
|
||||
Core-sanitized manifest inventory that was present at the initial page boot. It
|
||||
returns `null` for empty, malformed, unknown, or untrusted IDs without creating
|
||||
settings or storage state. Re-registering the same ID in one page returns the
|
||||
same handle object; different IDs receive different handles. The handle exposes
|
||||
only the canonical `id`, `settings`, `storage`, and `events` fields. Settings and
|
||||
storage are backed by the same factories used by the legacy accessors.
|
||||
|
||||
Manifest enable/disable changes still take effect after a WebUI reload. A handle
|
||||
already created in the current page may remain cached across a status refresh;
|
||||
this is expected and does not implement runtime unload. This identity is a
|
||||
cooperative attribution convenience for extensions sharing the page—not a
|
||||
sandbox, isolation mechanism, capability or permission system, or security
|
||||
boundary. Extensions still execute with the full WebUI session authority
|
||||
described above.
|
||||
|
||||
### Custom Configure editors
|
||||
|
||||
Extensions with structured configuration that does not fit scalar
|
||||
`settings_schema` fields can register one custom editor entry point on their
|
||||
scoped E0 settings handle:
|
||||
|
||||
```js
|
||||
const ext = window.hermesExt?.register?.("dictionary-manager");
|
||||
const unregister = ext?.settings?.registerConfigure?.(({ opener, restoreFocus }) => {
|
||||
openDictionaryManager({ opener, restoreFocus });
|
||||
});
|
||||
```
|
||||
|
||||
`registerConfigure(handler)` is available only through a valid boot-trusted E0
|
||||
handle. It does not require `settings_schema` or extension-owned storage. The
|
||||
first handler registered for an extension wins; duplicates return `null` and do
|
||||
not replace it. A successful registration returns an idempotent unregister
|
||||
function.
|
||||
|
||||
Core shows one **Configure** button only for the extension's current
|
||||
effective-enabled row under **Settings → Extensions → Installed**. Diagnostics
|
||||
never shows the button. Late registration updates an already-mounted Installed
|
||||
row. Disable or uninstall status removes the entry point immediately; uninstall
|
||||
followed by a same-ID reinstall cannot revive the old page-local handler until a
|
||||
full WebUI reload injects and registers the new extension script.
|
||||
|
||||
Core enters a pending state before invoking the handler and passes the activated
|
||||
button as `opener` plus a one-shot `restoreFocus()` callback. The extension owns
|
||||
its dialog, validation, persistence, and close lifecycle. It must either call
|
||||
`restoreFocus()` when its UI closes or return a thenable whose settlement means
|
||||
the Configure UI is closed. Both paths end pending and converge on Core's
|
||||
one-shot focus restoration. A synchronous non-thenable handler remains disabled
|
||||
while unsettled. If it never calls `restoreFocus()`, it remains disabled until reload.
|
||||
Core does not guess dialog lifetime with a timeout.
|
||||
|
||||
Synchronous throws, thenable-access failures, and asynchronous rejections are
|
||||
isolated, logged with the extension ID, and surfaced as a generic failure. Focus
|
||||
returns to the connected opener when possible, otherwise to the current visible
|
||||
Configure button or the Installed tab without forcing navigation. The hook does
|
||||
not let an extension return DOM for Core to render, add a backend settings route,
|
||||
or change the trusted same-origin extension model.
|
||||
|
||||
### Turn lifecycle events
|
||||
|
||||
A registered extension can react when a session turn observed by the current page
|
||||
starts or reaches a terminal state:
|
||||
|
||||
```js
|
||||
const ext = window.hermesExt.register("desktop-companion");
|
||||
const stop = ext && ext.events.on("turn:complete", event => {
|
||||
console.log(event.sessionId, event.streamId, event.status);
|
||||
});
|
||||
|
||||
// Remove the listener when the extension no longer needs it.
|
||||
if (stop) stop();
|
||||
```
|
||||
|
||||
The supported event types are `turn:start`, `turn:complete`, `turn:error`, and
|
||||
`turn:cancel`. The frozen event object always contains `type`, `sessionId`,
|
||||
`streamId`, and `timestamp` (Unix seconds). Start events also contain
|
||||
`startedAt`; terminal events contain `endedAt` and may contain `status`.
|
||||
`events.on(type, handler)` returns an idempotent unsubscribe function, or `null`
|
||||
for an unsupported type or non-function handler. An exception in one extension
|
||||
listener is logged and does not prevent other listeners or the chat UI from
|
||||
continuing.
|
||||
|
||||
The version-1 mapping is deliberately small: attaching a confirmed live session
|
||||
stream emits `turn:start`; SSE `done` emits `turn:complete`; SSE `cancel` and
|
||||
application errors classified as `cancelled` or `interrupted` emit
|
||||
`turn:cancel`; other application errors and an unrecoverable live-stream
|
||||
connection failure emit `turn:error`. Transport-only `stream_end` does not emit
|
||||
a second terminal event.
|
||||
|
||||
Core suppresses reconnect duplicates so a recently observed
|
||||
`sessionId`/`streamId` pair produces at most one start event and one terminal
|
||||
event. `sessionId` is the owner of the original stream; server-side compression
|
||||
or session rotation at completion does not change that lifecycle identity.
|
||||
Terminal callbacks run after Core has cleared the original stream ownership,
|
||||
applied the current session's immediate terminal transcript projection, and
|
||||
returned the owned pane to its idle state. They do not wait for unrelated
|
||||
follow-up work such as notifications, queued turns, or optional recovery
|
||||
refreshes.
|
||||
|
||||
This is a page-local, live-stream notification surface. It does not replay
|
||||
history, report turns that finish without this page observing their stream, or
|
||||
provide token, tool, approval, or metrics events. It follows the same cooperative
|
||||
trust model as `register(id)` and adds no sandbox or permission boundary. A
|
||||
durable or global agent event stream is a separate Core contract. Ephemeral
|
||||
`/btw` answer streams are not included in this version.
|
||||
Loopback sidecars do **not** change asset injection behavior. They are only
|
||||
reported by diagnostics so an operator can see that a local companion service was
|
||||
declared and optionally check its health from the browser.
|
||||
|
||||
## URL rules
|
||||
|
||||
@@ -427,10 +202,9 @@ URLs are ignored rather than injected.
|
||||
|
||||
Manifest-bundled extensions may integrate with a trusted local sidecar process,
|
||||
such as a desktop companion listening on `http://127.0.0.1:17787`. The injected
|
||||
extension JavaScript can talk to that sidecar directly from the browser, and
|
||||
WebUI diagnostics still use that direct browser path. WebUI may also proxy the
|
||||
same sidecar through a fixed per-extension sidecar path after explicit persisted
|
||||
user consent. WebUI does not create arbitrary extension-owned backend routes.
|
||||
extension JavaScript talks to that sidecar directly from the browser; Hermes
|
||||
WebUI does not proxy those requests and does not create extension-owned backend
|
||||
routes.
|
||||
|
||||
Loopback sidecar origins are already included in WebUI's enforced CSP
|
||||
`connect-src` directive:
|
||||
@@ -554,7 +328,6 @@ window.registerHermesSkin({
|
||||
name: 'E-Ink', // display name (also the picker label)
|
||||
value: 'e-ink', // optional stable key; slugified from name if omitted
|
||||
label: 'E-Ink', // optional explicit picker label
|
||||
scheme: 'light', // optional: force a light or dark base while selected
|
||||
colors: ['#000000', '#ffffff', '#555555'], // up to 3 preview swatches
|
||||
tokens: { // CSS design-token overrides for this skin
|
||||
'--bg': '#ffffff',
|
||||
@@ -573,13 +346,6 @@ shows up in the picker, can be selected, and persists across reloads exactly
|
||||
like a built-in skin. Registering the same key again updates it in place
|
||||
(idempotent), which is what a live theme editor relies on while the user edits.
|
||||
|
||||
Use `scheme` when a skin is light-only or dark-only. Accepted values are
|
||||
`"light"` and `"dark"`; any other value is ignored. This does not rewrite the
|
||||
user's saved Theme setting (`Light`, `Dark`, or `System Default`). It only
|
||||
controls the effective base theme class while that extension skin is selected,
|
||||
so a dark editor skin is not mixed with light-mode code/table tokens, and a
|
||||
light E-Ink skin is not mixed with dark-mode tokens.
|
||||
|
||||
**Core does the security-sensitive work for you.** Because token values are
|
||||
written into CSS, every value is sanitized in core, once, so every theme
|
||||
extension inherits the guard:
|
||||
@@ -600,8 +366,6 @@ extension inherits the guard:
|
||||
- **Reserved keys are protected** — an extension cannot overwrite a built-in
|
||||
skin key (e.g. `default`, `ares`, `graphite`).
|
||||
- A descriptor with no valid tokens after sanitization is rejected entirely.
|
||||
- **Skin scheme is constrained** — only `light` and `dark` are accepted. Invalid
|
||||
scheme values are ignored rather than rendered into CSS.
|
||||
|
||||
This is the supported, forward-looking way for theme-pack and theme-creator
|
||||
extensions to integrate with the built-in appearance system.
|
||||
@@ -700,16 +464,6 @@ If host CSS overrides `[hidden]`, add an extension-scoped rule such as:
|
||||
}
|
||||
```
|
||||
|
||||
### Contributing to the extension library
|
||||
|
||||
To publish an extension in the vetted gallery, open a PR against
|
||||
**[hermes-webui/hermes-webui-extensions](https://github.com/hermes-webui/hermes-webui-extensions)**
|
||||
following [`docs/extension-entry.md`](https://github.com/hermes-webui/hermes-webui-extensions/blob/main/docs/extension-entry.md)
|
||||
(entry layout, `extension.json`/`manifest.json` shape, and the capability +
|
||||
best-practice conventions). Every entry PR runs the repo's CI validators and
|
||||
safety scan before it can merge, and merged entries are published to the registry
|
||||
that powers Settings → Extensions.
|
||||
|
||||
## Minimal example
|
||||
|
||||
Create a local extension directory:
|
||||
@@ -764,8 +518,8 @@ for operators who prefer to inspect extension state from the browser. Installed
|
||||
manifest entries can be enabled or disabled from that panel through the
|
||||
authenticated `POST /api/extensions/toggle` endpoint. The toggle writes only a
|
||||
WebUI-managed override in the WebUI state directory; it does not edit extension
|
||||
manifests, fetch new extension assets, uninstall files, or add extension-owned
|
||||
backend routes. Manifest entries with `"enabled": false` remain
|
||||
manifests, fetch new extension assets, uninstall files, proxy sidecars, or add
|
||||
extension-owned backend routes. Manifest entries with `"enabled": false` remain
|
||||
manifest-disabled and cannot be re-enabled from WebUI.
|
||||
|
||||
The diagnostics return the same public asset URLs that can already be injected
|
||||
@@ -786,11 +540,15 @@ return `HERMES_WEBUI_EXTENSION_DIR`, resolved manifest paths, raw environment
|
||||
values, rejected URL strings, rejected sidecar origins, rejected health paths, or
|
||||
the override state-file path.
|
||||
|
||||
When sanitized loopback sidecars are present, **Settings → Extensions** renders a sidecar monitor card. The browser checks each declared `health_url` directly with `fetch(..., { credentials: 'omit', cache: 'no-store' })` and a short timeout. A successful HTTP response is shown as healthy, a non-OK HTTP response as unhealthy, and CORS/network/timeouts as unreachable or blocked; raw health response bodies are never rendered. If a healthy response includes an optional top-level `runtime` object, the panel may parse it and render only allowlisted scalar fields such as `sidecar`, `native_host`, `bridge`, `last_seen_at`, and `webui_origin`. This keeps sidecar-specific diagnostics machine-readable without making WebUI depend on any one extension's private payload shape.
|
||||
|
||||
The same card also exposes proxy consent through `POST /api/extensions/sidecar-proxy-consent` and reports the fixed per-extension sidecar path `/api/extensions/<extension-id>/sidecar/<relative-path>`. WebUI strips `Cookie`, `Authorization`, and CSRF headers before contacting the sidecar, and sidecar `Set-Cookie` headers are stripped before the browser sees the response. WebUI does not create arbitrary extension-owned backend routes; the proxy surface stays on that fixed per-extension sidecar path.
|
||||
|
||||
When sanitized settings are present, Settings -> Extensions renders
|
||||
browser-local controls for installed manifest entries. Save, reset, and clear
|
||||
storage actions call `window.HermesExtensionSettings`; they do not call backend
|
||||
storage routes and do not write WebUI settings.
|
||||
When sanitized loopback sidecars are present, **Settings → Extensions** renders a
|
||||
read-only sidecar monitor card. The browser checks each declared `health_url`
|
||||
directly with `fetch(..., { credentials: 'omit', cache: 'no-store' })` and a
|
||||
short timeout. WebUI does **not** proxy sidecar requests and does not send WebUI
|
||||
cookies to sidecars. A successful HTTP response is shown as healthy, a non-OK
|
||||
HTTP response as unhealthy, and CORS/network/timeouts as unreachable or blocked;
|
||||
raw health response bodies are never rendered. If a healthy response includes an
|
||||
optional top-level `runtime` object, the panel may parse it and render only
|
||||
allowlisted scalar fields such as `sidecar`, `native_host`, `bridge`,
|
||||
`last_seen_at`, and `webui_origin`. This keeps sidecar-specific diagnostics
|
||||
machine-readable without making WebUI depend on any one extension's private
|
||||
payload shape.
|
||||
|
||||
@@ -1,159 +0,0 @@
|
||||
# Change guidelines — writing a PR that lands cleanly
|
||||
|
||||
Hermes WebUI is deliberately simple: Python on the server, vanilla JS in the browser, no build
|
||||
step. That simplicity only survives if every change is scoped, verified, and complete. This
|
||||
document is the distilled set of principles that make the difference between a PR that merges on the
|
||||
first review and one that needs several rounds of back-and-forth. It applies to human and
|
||||
AI-assisted contributions alike — but it is written especially for AI coding agents, because the
|
||||
mistakes below are the ones they make most often and most confidently.
|
||||
|
||||
If you read nothing else, read the four root causes. The ten rules are just their enforcement.
|
||||
|
||||
---
|
||||
|
||||
## The four root causes of a change that has to be redone
|
||||
|
||||
Almost every change that fails review fails for one of these reasons. Learn to catch them in your
|
||||
own work before you open the PR.
|
||||
|
||||
1. **Fixing the instance instead of the class.** *The single most common one.* A bug usually has
|
||||
more than one home — the same mistake lives at several call sites, across more than one backend
|
||||
or code path, in a companion endpoint, in a second layout, on more than one lifecycle exit. It's
|
||||
easy to fix the exact line the report points at and leave the siblings broken. The missed one is
|
||||
usually a *different dimension* than the one you were looking at (a second backend, an unusual
|
||||
layout, the cancel path, the multi-item case).
|
||||
|
||||
2. **Mistaking a proxy for proof.** "The test passes," "the code contains the string," "that branch
|
||||
ran," "I mocked it and it returned the right thing" — none of these prove the *user-visible
|
||||
behavior* is correct. A test that would still pass if you reverted the fix is testing nothing.
|
||||
|
||||
3. **Reasoning about a value's content but not its lifetime.** You get the value right but miss when
|
||||
it goes stale, who is responsible for cleaning it up, and what happens to it on every exit path
|
||||
(success, error, cancel, replace, shrink, concurrent access). Leaked resources, orphaned caches,
|
||||
stale reads, and "the decision used one value but the action used another" all live here.
|
||||
|
||||
4. **Letting the diff grow past the task.** Extra changes you weren't asked for, or re-implementing
|
||||
something the codebase already centralizes (a fallback, a shared helper, an extension point).
|
||||
Every extra line is a new risk surface and makes the change harder to review and to trust.
|
||||
|
||||
A fifth applies only to *visible* changes: **placing a control where the code is, not where the
|
||||
user's attention should go.**
|
||||
|
||||
---
|
||||
|
||||
## The ten rules
|
||||
|
||||
### 1. Fix the class, not the instance.
|
||||
Before writing the fix, search for every sibling — other call sites, producers and consumers,
|
||||
alternate backends, companion endpoints, other layouts, every lifecycle exit — and fix at the
|
||||
**shared chokepoint**. If you deliberately leave one out of scope, name it in the PR and say why.
|
||||
One guard in the shared function is a smaller and more correct change than one guard per caller.
|
||||
The chokepoint is the *smallest* boundary that contains the fault, not the widest one you can
|
||||
reach: fixing the class means guarding every sibling of the bug, not disabling a whole stage or
|
||||
pipeline to suppress one bad output. If your fix blocks more than the fault occupies, it has grown
|
||||
past the task (rule 9).
|
||||
|
||||
### 2. Trace one authoritative value end-to-end.
|
||||
Follow the value from `input → normalize → decision → action → persist → cleanup`, and use the
|
||||
**same** resolved value at every stage. Don't let the code that *decides* see one value while the
|
||||
code that *acts* uses the stale or un-normalized one — that split is a silent correctness bug.
|
||||
Before you key a decision on a value, confirm it's *authoritative*: written at the point of intent
|
||||
(a canonical kind field, a provenance marker stamped by the action itself, a recorded origin) — not
|
||||
inferred from an id prefix, content shape, emptiness, or DOM state. Search for an existing canonical
|
||||
field first; inference that happens to work on the case in front of you is the most common way a
|
||||
guard misfires on a sibling. If no authoritative field exists, add a test with an adjacent case that
|
||||
matches your heuristic and must *not* trigger.
|
||||
|
||||
### 3. When you can't confirm something, fail closed and say so.
|
||||
For anything touching authority, capability, identity, or containment: if you cannot *confirm* it's
|
||||
safe, deny it — don't take the permissive branch on uncertainty, and never report a failure as a
|
||||
success. Treat "unknown", "absent", and "unverifiable" as distinct from "allowed." Stating "I could
|
||||
not verify X" in the PR is good engineering; hiding it is how a security or reliability bug ships.
|
||||
|
||||
### 4. Enumerate the state-space before you edit.
|
||||
Write down which dimensions the change touches — entry point, backend, item count (0 / 1 / many /
|
||||
duplicate), lifecycle exit (success / error / cancel / replace / shrink / teardown), auth on vs off,
|
||||
concurrency (two profiles or workers at once), input shape (empty / hostile / aliased) — and cover
|
||||
each, or mark it out of scope on purpose. Most redo rounds are one un-considered dimension. Those
|
||||
axes are generic; the ones that actually bite are subsystem-specific and are not obvious to a
|
||||
first-time contributor. Before editing an unfamiliar subsystem, find its real variants: read the
|
||||
closed PRs and review threads that touched the same code, where the discriminating dimensions
|
||||
(every session-lineage kind, every replay/recovery source, each stream response shape) were already
|
||||
enumerated. Reuse that list rather than inventing your own from the one case you were handed.
|
||||
|
||||
### 5. Assume inputs and check-then-use gaps are adversarial.
|
||||
Input crossing a trust boundary can be crafted to break a naive check (odd delimiters, casing, YAML
|
||||
aliases, path traversal). Filesystem and process state can change between the moment you check it
|
||||
and the moment you use it, so validate at the **point of use** (hold a file descriptor/handle rather
|
||||
than re-resolving a path). Scope caches by their *complete* identity or they leak across
|
||||
profiles/sessions under concurrency. Validation is worthless if the thing you finally use isn't the
|
||||
thing you validated.
|
||||
|
||||
### 6. A test must fail before your fix and pass after it.
|
||||
Run your new test against the current code *first* and confirm it fails for the right reason; only
|
||||
then apply the fix. Assert the **observable behavior** (the row updates, the secret is absent from
|
||||
the log, the request carries the right value) — not a source string, and not through a mock that
|
||||
stands in for the very thing under test. If the bug is about picking the right one of several items,
|
||||
the test must include several items, or it can't catch picking the wrong one. When the issue ships a
|
||||
reproduction — a session capture, a script, exact steps, an attachment — your test loads **that**,
|
||||
not a fixture you rebuilt from your reading of it. A fix and a test written from the same wrong model
|
||||
of the bug will base-fail and head-pass on the fixture forever while the real symptom sits untouched;
|
||||
loading the reporter's actual scene is what breaks that agreement. A reproduction is whatever *pins
|
||||
the bug's shape* — a downloadable capture, but a fenced JSON structure, the field-level trigger
|
||||
conditions, or exact steps bind it just as tightly. Bind your fixture to that shape: satisfy every
|
||||
condition it names and let the assertion fail when any one is violated, instead of granting the
|
||||
fixture a property the report never had so a guard will fire. Reconstruct the shape yourself only
|
||||
when the issue genuinely leaves it unpinned — and then say so, and say what you assumed.
|
||||
|
||||
### 7. Name the owner of every piece of state and prove it's released.
|
||||
For each resource or mutation you introduce (a cache entry, a lock, a temporary env change, a
|
||||
loading flag, a DOM node, a pending-map entry), show it is cleaned up or invalidated on **every**
|
||||
exit — success, error, cancellation, replacement, shrink/clear, and teardown — not just the happy
|
||||
path. Happy-path testing structurally cannot see these leaks.
|
||||
|
||||
### 8. Fallbacks and defaults are contracts — extend the mechanism, don't copy it.
|
||||
If your change means editing several parallel blocks identically, stop: the codebase almost
|
||||
certainly has a mechanism for this (a fallback, a shared helper, an extension point) and you've
|
||||
missed it. Add to the one canonical place. For example, new user-facing copy goes in the `en` locale
|
||||
and rides the existing fallback — don't paste English into every locale block.
|
||||
|
||||
### 9. The diff is the task and nothing else.
|
||||
Change exactly what the task requires. Anything else you noticed goes in the PR description as a
|
||||
note, not into the diff. Before opening the PR, run the affected tests **and** a broader sweep of
|
||||
neighboring tests (not only the ones you added), and remove every unrelated change.
|
||||
|
||||
### 10. A visible control costs attention on every future visit — place it accordingly.
|
||||
Where a control lives should be decided by how often it's used and by where mainstream chat apps put
|
||||
the equivalent — not by where your diff already is. Rare per-item actions belong in an overflow/`⋮`
|
||||
menu; global or data actions belong in Settings; only genuinely daily-use controls earn a spot on a
|
||||
hot surface like the composer. Then verify it *visually*: capture desktop and narrow-viewport,
|
||||
before and after, and confirm nothing is clipped, no overflow-collapse is tripped, and no hover-only
|
||||
affordance is stranded on touch.
|
||||
|
||||
---
|
||||
|
||||
## Show your work in the PR description
|
||||
|
||||
The fastest review is one where the reviewer can *see* that the above is handled instead of having
|
||||
to discover the gaps. Make your PR description carry the evidence:
|
||||
|
||||
- **The siblings you found** for rule 1/4 — and any you deliberately left out of scope.
|
||||
- **Proof the test bites** — that your new test failed before the fix.
|
||||
- **Verification run** — the affected + neighboring tests, not just your new ones.
|
||||
- **Before/after images** for any visible change (desktop + narrow).
|
||||
- **What you could not verify** — an explicit list. An admitted gap is fine; a hidden one is a bug
|
||||
waiting to be found in review.
|
||||
- **Who owns the truth** for any claim the repo doesn't — browser behavior, a provider's API, a
|
||||
registry, an OS convention. Name the owner and check your proof is one they'd accept: in-page
|
||||
automation proves page behavior, not that the browser didn't swallow the event first; a mock
|
||||
proves your intent, not the provider's contract; a synthetic fixture proves nothing about the live
|
||||
system. The failure mode here isn't knowing you couldn't verify — it's thinking you did.
|
||||
|
||||
Following this imperfectly still helps enormously, because the reviewer can see exactly where your
|
||||
coverage stops instead of finding it the hard way.
|
||||
|
||||
---
|
||||
|
||||
*Companion to [`AGENTS.md`](../AGENTS.md), [`CONTRIBUTING.md`](../CONTRIBUTING.md), and
|
||||
[`docs/CONTRACTS.md`](CONTRACTS.md). For UI/UX specifics see [`docs/UIUX-GUIDE.md`](UIUX-GUIDE.md)
|
||||
and [`DESIGN.md`](../DESIGN.md).*
|
||||
+9
-18
@@ -96,25 +96,16 @@ unless the code and tests prove it is implemented.
|
||||
|
||||
## Typography and content
|
||||
|
||||
Use three explicit font tokens:
|
||||
Use split typography intentionally:
|
||||
|
||||
- `--font-ui`: shell chrome, controls, composer, labels, and ordinary UI text
|
||||
- `--font-conversation`: user/assistant message prose; by default this is
|
||||
`var(--font-ui)` in `static/style.css`
|
||||
- `--font-mono`: code, file paths, command lines, tool payloads, technical logs,
|
||||
and terminal output
|
||||
|
||||
Use semantic tokens for typography. Keep prose on `--font-conversation` by
|
||||
default so it tracks `--font-ui` whenever a skin intentionally retunes UI type.
|
||||
Override `--font-conversation` only for a skin that intentionally wants a
|
||||
distinct prose face.
|
||||
Conversation prose must remain the system sans by default. Do not introduce a
|
||||
global conversation serif through the `--font-conversation` token or
|
||||
selector-level overrides without explicit design approval plus code and test
|
||||
evidence; keep distinct editorial prose typography explicitly opt-in or
|
||||
skin-scoped.
|
||||
Avoid hard-coding selector-level font stacks when a token already carries the
|
||||
intent.
|
||||
- assistant prose uses the same system sans stack as the rest of the UI by
|
||||
default (`--font-ui` in `static/style.css`),
|
||||
- editorial serif assistant prose is historical/proposal or skin-scoped only;
|
||||
do not reintroduce a global assistant serif without explicit design approval
|
||||
plus code and test evidence,
|
||||
- user bubbles and functional UI also use the system sans stack unless a scoped
|
||||
skin intentionally overrides typography,
|
||||
- monospace is for code, file paths, commands, tool names, and compact metadata.
|
||||
|
||||
Keep scale tight. Avoid introducing near-duplicate one-off font sizes, colors,
|
||||
radius values, or spacing values when an existing token works.
|
||||
|
||||
@@ -49,50 +49,6 @@ every new browser turn. The browser only receives a compact status event
|
||||
(`source`, `label`, message count, compaction metadata, and redacted errors),
|
||||
never the prefill message bodies.
|
||||
|
||||
## Session title generation
|
||||
|
||||
Hermes WebUI derives a provisional session title from the first user message
|
||||
and, after the first response, may call an LLM to generate a better title
|
||||
(and periodically refresh it for long sessions).
|
||||
|
||||
For structured messages containing text and native images, title generation
|
||||
uses the user text without flattening or modifying the stored message. Title
|
||||
comparison and title-model inputs remove the internal `[Workspace::v1: ...]`
|
||||
prefix and one terminal `[Attached files: ...]` or
|
||||
`[Attached files for this steer: ...]` suffix separated by a blank line.
|
||||
For structured content, this cleanup applies to the first text part that
|
||||
provides title content. Literal legacy `[Workspace: ...]` text and later text
|
||||
parts remain unchanged. Initial generation, explicit regeneration, and adaptive
|
||||
refresh use this title-specific cleanup.
|
||||
|
||||
Background generation requires user text and a substantive assistant response.
|
||||
It recognizes the sanitized provisional title as well as the existing raw
|
||||
placeholder, so internal metadata does not make an image-containing turn look
|
||||
manually titled. Image-only or metadata-only content does not provide title
|
||||
text. Existing manual-title protection and the title-generation setting still
|
||||
apply; this cleanup does not rewrite the transcript or native image parts.
|
||||
|
||||
Automatic title-generation LLM calls honor the active Hermes profile's
|
||||
`auxiliary.title_generation.enabled` setting (default: `true`):
|
||||
|
||||
```yaml
|
||||
auxiliary:
|
||||
title_generation:
|
||||
enabled: false
|
||||
```
|
||||
|
||||
When disabled:
|
||||
|
||||
- the provisional first-message title stays in place and is never replaced
|
||||
or overwritten by an automatic LLM call or local fallback;
|
||||
- the periodic adaptive refresh is skipped;
|
||||
- the explicit "regenerate title" action returns a
|
||||
`title_generation_disabled` response instead of calling a title model.
|
||||
|
||||
The WebUI's `auto_title_refresh_every` setting remains a separate control for
|
||||
periodic refreshes of already-generated titles; it does not re-enable
|
||||
automatic generation when the auxiliary flag is off.
|
||||
|
||||
## Gateway-backed browser chat
|
||||
|
||||
By default, browser chat runs through WebUI's in-process legacy runtime. Advanced
|
||||
@@ -107,33 +63,6 @@ HERMES_WEBUI_GATEWAY_API_KEY=... \
|
||||
./ctl.sh restart
|
||||
```
|
||||
|
||||
Gateway-backed approval prompts need one more explicit opt-in because they use the Gateway runs API path:
|
||||
|
||||
```bash
|
||||
HERMES_WEBUI_CHAT_BACKEND=gateway \
|
||||
HERMES_WEBUI_GATEWAY_BASE_URL=http://127.0.0.1:8642 \
|
||||
HERMES_WEBUI_GATEWAY_API_KEY=... \
|
||||
HERMES_WEBUI_GATEWAY_USE_RUNS_API=true \
|
||||
./ctl.sh restart
|
||||
```
|
||||
|
||||
Use this when the connected gateway advertises approval support and you want tool approval cards to appear in WebUI. Without `HERMES_WEBUI_GATEWAY_USE_RUNS_API=true`, gateway chat stays on the legacy chat-completions transport and approval-capable commands can remain pending in the agent without a WebUI approval card.
|
||||
|
||||
When YOLO is enabled for a gateway-backed browser session, WebUI approves every
|
||||
approval already parked for that session: Runs API prompts are relayed by their
|
||||
exact `run_id` and mirror token, and local/no-run waiters are all released. It
|
||||
then automatically answers later Runs API approval requests while the WebUI
|
||||
session flag remains active. The flag is committed only after every currently
|
||||
parked remote relay succeeds; a later prompt that races that unconfirmed drain
|
||||
remains visible instead of being speculatively auto-approved. The handoff is
|
||||
also shared with local approval admission: a local waiter arriving after
|
||||
the current drain snapshot waits for the same session handoff and is released
|
||||
immediately if YOLO has committed, rather than being parked behind an enabled
|
||||
session. This is client-managed compatibility behavior: the current Runs API has
|
||||
no session-YOLO toggle, so a request briefly reaches the approval boundary before WebUI answers
|
||||
it, and Agent-owned policy such as unrestricted computer-use mode is unchanged.
|
||||
Native API session YOLO is tracked in [Hermes Agent PR #61946](https://github.com/NousResearch/hermes-agent/pull/61946).
|
||||
|
||||
`HERMES_WEBUI_CHAT_BACKEND` is intentionally strict: only `gateway`,
|
||||
`api_server`, or `api-server` enable the bridge. Generic truthy values such as
|
||||
`1` or `true` are ignored so existing deployments do not change execution
|
||||
|
||||
@@ -41,109 +41,10 @@ helpers are genuinely shared code.
|
||||
| `docker_agent_source_volume` | Compose files and Docker docs expose `hermes-agent-src` and `/opt/hermes` to make the agent checkout visible to WebUI. | Remove the WebUI source mount only after startup install and runtime imports have migrated. This needs Docker/compose follow-up work, not a runtime behavior change in this audit PR. |
|
||||
| `startup_dependency_install` | `api/startup.py` discovers `HERMES_WEBUI_AGENT_DIR` or `$HERMES_HOME/hermes-agent`; `server.py` calls `auto_install_agent_deps()` after import verification fails; `docker_init.bash` installs from the staged agent source. | Replace source-tree pip installs with a packaged hermes-agent WebUI client plus an agent health/version capability contract. Keep `HERMES_WEBUI_AGENT_DIR` during migration as an override/debug path, but it should stop being required in normal multi-container startup. |
|
||||
| `runtime_auxiliary_model_metadata` | `api/streaming.py`, `api/routes.py`, `api/config.py`, and `api/providers.py` import `agent.auxiliary_client`, `agent.model_metadata`, `agent.models_dev`, `hermes_cli.models`, and `agent.account_usage`. | Existing provider/model WebUI endpoints can keep serving UI data where they already wrap agent helpers. Missing surfaces need hermes-agent endpoints or a client package for auxiliary task config, text auxiliary calls, context length, token estimate, provider catalog, and account usage. |
|
||||
| `runtime_session_state` | `api/streaming.py`, `api/goals.py`, and `api/state_sync.py` import `hermes_state.SessionDB` directly. `api/models.py` also reads the `messages` table directly (see [state.db message content encoding](#statedb-message-content-encoding) for the storage-format coupling that creates) and opens the active profile's canonical `state.db` for scoped session deletion because the current canonical helper does not preserve branch/compression evidence ahead of inherited delegate metadata or expose retryable artifact-cleanup semantics. | Move cross-container state reads and writes, including destructive session deletion, behind hermes-agent session/state endpoints once the agent API provides equivalent lineage precedence, transaction, and retry-manifest guarantees. WebUI-only presentation state can remain local, but agent session storage should not be opened from the WebUI container. |
|
||||
| `runtime_session_state` | `api/streaming.py`, `api/goals.py`, and `api/state_sync.py` import `hermes_state.SessionDB` directly. | Move cross-container state reads and writes behind hermes-agent session/state endpoints. WebUI-only presentation state can remain local, but agent session storage should not be opened from the WebUI container. |
|
||||
| `runtime_gateway_provider` | `api/streaming.py` and `api/routes.py` import `hermes_cli.runtime_provider`; adapter helpers such as `agent.anthropic_adapter` are also imported for gateway normalization. | Provider resolution, runtime routing, and gateway invocation should be hermes-agent API calls. WebUI can keep request validation and display formatting, but it should not import runtime provider internals from the agent checkout. |
|
||||
| `webui_local_or_client_package` | WebUI imports `hermes_cli.auth`, `hermes_cli.config`, `hermes_cli.plugins`, `hermes_cli.profiles`, `hermes_cli.goals`, `agent.skill_utils`, `agent.credential_pool`, and `hermes_constants`. | Pure schemas, constants, and parsing helpers can move into a small versioned client/shared package. Privileged data such as credential pools, auth status, profile mutation, plugin discovery, and goal persistence need hermes-agent endpoints. UI-only formatting can remain in WebUI. |
|
||||
|
||||
## state.db message content encoding
|
||||
|
||||
`api/models.py` reads the agent's `messages` table with its own SQL, so it also
|
||||
depends on how hermes-agent *encodes* that table, not only on its schema. This
|
||||
is a storage-format coupling and belongs with the `runtime_session_state`
|
||||
dependency class above.
|
||||
|
||||
`hermes_state` stores list/dict message content (multimodal parts) as a
|
||||
sentinel-prefixed JSON string, because sqlite3 binds only scalars:
|
||||
|
||||
```
|
||||
_CONTENT_JSON_PREFIX = "\x00json:" # hermes_state.py
|
||||
```
|
||||
|
||||
It provides `_decode_content()` to reverse this. Any WebUI read path that
|
||||
projects that column must apply an equivalent decode; a raw read hands the
|
||||
frontend an encoded string that no reader recognises, and an image part's
|
||||
base64 data URI then renders as literal transcript text.
|
||||
|
||||
### WebUI decoding contract
|
||||
|
||||
`_decode_state_db_content()` in `api/models.py` is the single decode point. It
|
||||
is deliberately narrower than the agent's own decoder, because the WebUI can
|
||||
only accept shapes the rest of its pipeline already renders:
|
||||
|
||||
| Input | Result | Why |
|
||||
| --- | --- | --- |
|
||||
| Sentinel + list with non-whitespace text and only valid image parts | decoded `list` | `msgContent()` joins the text parts, so the row renders its text |
|
||||
| Sentinel + image-only list, or text that is empty/whitespace | unchanged string | `msgContent()` discards image parts, so `_messageIsRenderable()` would hide the row with no error |
|
||||
| Sentinel + list containing a malformed image part | unchanged string | a part must carry a valid per-type payload, not just a matching `type` |
|
||||
| Sentinel + dict or scalar root | unchanged string | a dict reaches `_getCachedRender()`, and `_renderCacheKey()` calls `text.slice()` on it, blanking the turn |
|
||||
| Sentinel + `NaN`/`Infinity`/overflowed float | unchanged string | Python emits them, browser `JSON.parse()` rejects the whole `/api/session` payload |
|
||||
| Sentinel + unsupported part shapes | unchanged string | `input_text`, `output_text`, scalar and unknown parts are dropped by the JS readers, so decoding them would silently lose content that is visible today |
|
||||
| Anything without the sentinel | unchanged | non-sentinel content is not this contract's concern |
|
||||
|
||||
Supported parts are `{"type": "text", "text": <str>}` plus image parts whose
|
||||
payload validates for their type: `image_url` with a non-empty URL (string or
|
||||
`{"url": ...}`), `input_image` with a URL or `file_id`, and `image` with a
|
||||
`base64` source carrying `data` and `media_type` or a `url` source. At least one
|
||||
text part must contain non-whitespace text.
|
||||
|
||||
**Image parts do not render from this projection.** The shared JS readers drop
|
||||
them, and the state.db projection supplies no `attachments`. Decoding a
|
||||
text-and-image row shows its text and keeps the base64 payload out of the DOM;
|
||||
it does not display the image. Rendering images from state.db rows would need a
|
||||
shared inline-image projection first, at which point image-only lists could be
|
||||
accepted too.
|
||||
|
||||
Widening the accepted schema requires teaching every shared content reader
|
||||
through one extractor first; until then unsupported shapes must keep falling
|
||||
back to the raw string.
|
||||
|
||||
### Consequences for identity and bounded reads
|
||||
|
||||
Decoding changes the runtime type of `content`, so every consumer that derives
|
||||
an identity from it must agree on one representation:
|
||||
|
||||
- Every key -- merge, dedup, content, visible and the fuzzy fallback -- derives
|
||||
content identity through `_content_identity_for_key()`. Non-list values key
|
||||
exactly as on master, `str(content or "")`. Non-empty lists get an
|
||||
**out-of-band** tuple identity, so no message body can compare equal to one:
|
||||
an in-band string marker would be forgeable by a scalar that contains it.
|
||||
Two rich turns sharing visible text and timestamp stay distinct when their
|
||||
images differ.
|
||||
- Fuzzy duplicate matching is text-only. Structured identities match by exact
|
||||
identity or not at all, so a rich row can never fuzzy-match a scalar.
|
||||
- The merge key cache never writes a key component back into message content.
|
||||
To avoid re-serialising large payloads for every key it instead memoises the
|
||||
canonical serialisation per content object, scoped to one
|
||||
`merge_session_messages_append_only()` call.
|
||||
- The multimodal mirror bridge pairs one rich image-bearing row with one scalar
|
||||
mirror only. `require_image_parts` and `require_scalar_mirror` are mutually
|
||||
exclusive so rich-to-rich pairing cannot occur.
|
||||
- Every read path that projects the `content` column applies the decoder, so
|
||||
keys derived on one path cannot disagree with keys derived on another. There
|
||||
are exactly three such call sites:
|
||||
|
||||
| Call site | Role |
|
||||
| --- | --- |
|
||||
| `_project_state_db_message()` | canonical row projection, shared by the transcript read and the regeneration tail |
|
||||
| `get_state_db_session_message_keys_before_timestamp()` | bounded prefix keys |
|
||||
| `get_state_db_regeneration_tail_snapshot()` | regeneration prefix keys |
|
||||
|
||||
If prefix keys stayed encoded while the projected tail was decoded, the
|
||||
prefix/tail collision proof could miss a genuine repeated recovered turn and
|
||||
`_bounded_tail_snapshot_if_safe` would reject the bounded path, reading the
|
||||
entire transcript during regeneration.
|
||||
|
||||
Two nearby paths deliberately need no decode.
|
||||
`get_state_db_session_message_prefix_summary()` projects only timestamp
|
||||
counts and never selects `content`. State-db sidecar reconstruction
|
||||
(`_sync_sidecar_from_state_db_if_newer()`) sources its rows through
|
||||
`get_state_db_session_messages()`, so it inherits the canonical decoded
|
||||
projection rather than reading the column itself.
|
||||
|
||||
When session state moves behind hermes-agent endpoints, this decode should move
|
||||
with it: the agent should return structured content over the API and the WebUI
|
||||
should stop depending on the sentinel format at all.
|
||||
|
||||
## Replacement contract
|
||||
|
||||
### Existing endpoint candidates
|
||||
@@ -166,10 +67,7 @@ These surfaces require an agent-owned endpoint because they read agent state,
|
||||
perform provider/runtime decisions, or expose privileged data:
|
||||
|
||||
- SessionDB/session state read and write operations now using
|
||||
`hermes_state.SessionDB`, plus the scoped direct-SQL session deletion in
|
||||
`api/models.py`. The replacement endpoint must preserve branch/compression
|
||||
precedence and expose retryable post-commit artifact cleanup before that
|
||||
compatibility path can be removed.
|
||||
`hermes_state.SessionDB`.
|
||||
- Runtime provider resolution and gateway normalization now using
|
||||
`hermes_cli.runtime_provider` and `agent.anthropic_adapter`.
|
||||
- Auxiliary task execution and configuration now using `agent.auxiliary_client`.
|
||||
|
||||
@@ -1,16 +1,12 @@
|
||||
# Stable Assistant Turn Anchors Phase 0 Inventory
|
||||
|
||||
This inventory began as the first non-visual slice of
|
||||
[`stable-assistant-turn-anchors.md`](../rfcs/stable-assistant-turn-anchors.md) and
|
||||
now records the shipped phase progression. Current `master` wires Anchor-owned
|
||||
`activity_scene_v1` data into live and settled Compact Worklog, Transparent
|
||||
Stream, journal hydration, session re-entry, and transcript-backed persistence.
|
||||
This inventory implements the first non-visual slice of
|
||||
[`stable-assistant-turn-anchors.md`](../rfcs/stable-assistant-turn-anchors.md).
|
||||
It documents the current per-turn state layers and the event-shape contract that
|
||||
future anchor phases must consume. It does not claim that anchors are wired into
|
||||
streaming or rendering yet.
|
||||
|
||||
The slice-by-slice sections below are retained as implementation history.
|
||||
Present-tense statements such as “unwired” or “future phase” describe the point
|
||||
when that slice landed; they do not override the current coverage summary above.
|
||||
|
||||
## Shipped RFC Phase Progress
|
||||
## RFC Phase Progress
|
||||
|
||||
- The #3962 Phase 0 scaffold shipped through #3977 / v0.51.359: inventory the
|
||||
current state layers, encode the owner seed, and pin the source classification
|
||||
@@ -54,13 +50,6 @@ when that slice landed; they do not override the current coverage summary above.
|
||||
`_anchor_activity_scene` snapshot so the folded activity summary can appear
|
||||
above the final answer. This handoff does not claim Transparent Stream wiring
|
||||
or durable hard-reload scene persistence.
|
||||
- The first RFC Phase 6 ownership slice routes the production `state_saved` SSE
|
||||
into the active Assistant Turn Anchor and carries `artifacts` and
|
||||
`side_effects` through `activity_scene_v1` settlement persistence and reload
|
||||
hydration. These owned outcomes remain off the readable Worklog by default;
|
||||
memory/skill saves still surface through their existing toast and natural UI.
|
||||
Production artifact-reference derivation and run-journal reconstruction of
|
||||
side effects remain follow-up work.
|
||||
|
||||
## State Layers
|
||||
|
||||
@@ -306,28 +295,6 @@ activity order for Compact Worklog and Transparent Stream renderers. The
|
||||
Transparent Stream raw `content[]` helper remains a fallback for settled
|
||||
messages that do not yet carry `_anchor_activity_scene`.
|
||||
|
||||
### Settled Fallback Ownership Matrix
|
||||
|
||||
After the mixed `content[]` bridge, `_anchor_activity_scene` is the semantic
|
||||
owner for settled assistant activity whenever it is present. Legacy raw-message
|
||||
paths stay as compatibility fallbacks for older or non-anchor transcripts; they
|
||||
must not compete with an anchor-owned turn.
|
||||
|
||||
| Surface | Anchor-present owner | Compatibility fallback | Fallback exit |
|
||||
| --- | --- | --- | --- |
|
||||
| Settled Compact Worklog activity | `_renderSettledAnchorSceneForMessage()` renders `activity_scene_v1` rows before the final answer. | Legacy `S.toolCalls`, `tool_calls`, `_partial_tool_calls`, and raw `content[].tool_use` rebuilds. | `anchorOwnedAssistantRawIdxs` excludes the anchor-owned turn from message metadata scans, fallback source collection, tool buckets, thinking buckets, and worklog-source mirroring. |
|
||||
| Settled Transparent Stream activity | `_renderSettledAnchorSceneTransparentForMessage()` renders the same scene rows as transparent event rows or inline prose before the final answer segment. | `_transparentStreamOrderedParts()` rebuilds raw `content[]` order for historical messages without an anchor scene. | `_transparentStreamOrderedParts()` returns `null` when `message._anchor_activity_scene` exists, leaving the dedicated anchor renderer in charge. |
|
||||
| Historical / non-anchor transcripts | Complete, user-bounded transcripts with unique top-level `assistant.tool_calls[].id` declarations, exactly one matching role=`tool` `tool_call_id` result per declaration, and a later final assistant answer hydrate an in-memory `activity_scene_v1` through the Anchor module adapter before settled rendering. | Raw `content[]`, persisted `session.tool_calls`, role=`tool` rows, and partial tool-call snapshots continue to recover ambiguous, incomplete, or richer historical shapes. | The ID-linked shape exits fallback ownership only after strict turn-local validation; missing or duplicate IDs/results, cross-turn matches, process prose/reasoning, and other tool schemas fail closed to the legacy renderer. |
|
||||
| Live reattach / session switch | `renderLiveAnchorActivityScene()` consumes the projected live scene, and session switch first attempts runtime-journal anchor scene restore. | Saved live DOM snapshots and `INFLIGHT` tool replay. | Snapshot fallback only runs when no usable live anchor scene can be rendered. |
|
||||
|
||||
This matrix is the current settled-render contract, backed by runtime-journal
|
||||
hydration parity and behavior-level renderer ownership coverage. Fallback paths
|
||||
are compatibility-only rebuilds for historical or genuinely non-anchor
|
||||
transcripts, and the renderer must gate them by explicit raw transcript indexes
|
||||
so object identity or duplicate message references cannot let fallback rows
|
||||
compete with an anchor-owned turn. Removing a compatibility path still requires
|
||||
new evidence that the supported transcript shape hydrates through anchors.
|
||||
|
||||
## Source Event Classification
|
||||
|
||||
Phase 0 classifies current sources before changing render behavior:
|
||||
|
||||
@@ -11,27 +11,6 @@ This is the comprehensive Docker reference. For a 5-minute quickstart, see the [
|
||||
| **Three-container** | Two-container PLUS the dashboard for monitoring. | `docker-compose.three-container.yml` |
|
||||
| **All-in-one image** (community fork — third-party, not maintained by us) | Podman 3.4 / multi-arch / supervisord-style preference. | [sunnysktsang/hermes-suite](https://github.com/sunnysktsang/hermes-suite) — see [#1399](https://github.com/nesquena/hermes-webui/issues/1399) for the original discussion |
|
||||
|
||||
### Available Docker tags
|
||||
|
||||
The WebUI Docker image is published to `ghcr.io/nesquena/hermes-webui` with these tags:
|
||||
|
||||
| Tag | Channel | Description |
|
||||
|---|---|---|
|
||||
| `:latest` | stable | The most recent stable release (from `v*` tags). Suitable for production. |
|
||||
| `:experimental` | experimental | The most recent experimental release (from `exp-v*` tags). For early testing; may include breaking changes or unfinished features. Do not run in production. |
|
||||
| `:X.Y` / `:X.Y.Z` | stable | Pinned stable releases (e.g., `:1.5`, `:1.5.0`). |
|
||||
| `:X.Y` / `:X.Y.Z` | experimental | Pinned experimental releases — same version numbers but pushed from `exp-v*` tags. The `:experimental` floating tag always points at the latest of these. |
|
||||
|
||||
To track experimental builds in Docker Compose, use the `:experimental` tag:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
hermes-webui:
|
||||
image: ghcr.io/nesquena/hermes-webui:experimental
|
||||
```
|
||||
|
||||
> **Note:** updating between `:experimental` builds requires `docker compose pull` followed by `docker compose up -d` — the floating tag is updated only when a new `exp-v*` release is pushed. Experimental builds are not pushed on every commit to the default branch.
|
||||
|
||||
> **Note (v0.14+):** If you use `docker-compose.three-container.yml`, both
|
||||
> `hermes-agent` and `hermes-dashboard` initialise from the same image and write
|
||||
> to the same `hermes-home` volume simultaneously. This can cause overlapping lock
|
||||
@@ -193,22 +172,6 @@ cp .env.docker.example .env
|
||||
docker compose -f docker-compose.two-container.yml up -d
|
||||
```
|
||||
|
||||
The compose files forward `API_SERVER_KEY` from `.env` into the `hermes-agent`
|
||||
container. The agent only starts the gateway API listener (port 8642) when
|
||||
`API_SERVER_KEY` is a usable value (>=16 chars) — `API_SERVER_ENABLED` alone
|
||||
does nothing. Without a key, the gateway daemon still runs but port 8642 stays
|
||||
unbound and the WebUI keeps showing **"Gateway endpoint not reachable"**. To
|
||||
enable scheduled ticking and the green gateway pill, set a long random string
|
||||
in `.env`:
|
||||
|
||||
```bash
|
||||
echo "API_SERVER_KEY=$(openssl rand -hex 24)" >> .env
|
||||
docker compose -f docker-compose.two-container.yml up -d --force-recreate
|
||||
```
|
||||
|
||||
The compose file forwards the same value to the WebUI as
|
||||
`HERMES_WEBUI_GATEWAY_API_KEY`, so the health probe authenticates automatically.
|
||||
|
||||
The three-container layout adds the dashboard but is otherwise the same shape. If you must stay single-container, you can run `hermes gateway` inside the container as a long-lived background process, but the compose split is sturdier.
|
||||
|
||||
If you maintain a custom compose file, make sure the **WebUI service** points at
|
||||
@@ -222,20 +185,6 @@ services:
|
||||
# HERMES_WEBUI_GATEWAY_BASE_URL=http://hermes-agent:8642 also works.
|
||||
```
|
||||
|
||||
If WebUI browser chat is routed through that gateway and you expect approval prompts for guarded tools, set the gateway chat backend and opt into the runs API path in the **WebUI service**:
|
||||
|
||||
```yaml
|
||||
services:
|
||||
hermes-webui:
|
||||
environment:
|
||||
- HERMES_WEBUI_CHAT_BACKEND=gateway
|
||||
- HERMES_WEBUI_GATEWAY_BASE_URL=http://hermes-agent:8642
|
||||
- HERMES_WEBUI_GATEWAY_USE_RUNS_API=true
|
||||
# HERMES_WEBUI_GATEWAY_API_KEY=... when the gateway requires API auth.
|
||||
```
|
||||
|
||||
`HERMES_WEBUI_GATEWAY_USE_RUNS_API=true` is required for gateway approval cards because approval-capable gateway runs emit approval requests on the runs API transport. Leaving it unset keeps browser chat on the legacy chat-completions path.
|
||||
|
||||
Do not copy only `API_SERVER_ENABLED=true` / `API_SERVER_HOST=0.0.0.0` into the
|
||||
agent service as a standalone fix. If you intentionally enable the agent API
|
||||
server, the agent also requires a real `API_SERVER_KEY` (at least 8 characters),
|
||||
@@ -468,44 +417,6 @@ services:
|
||||
|
||||
Then configure the URL as `http://host.docker.internal:<port>`. Also ensure the host service binds to an address reachable from containers (not only a loopback interface the Docker bridge cannot reach) and that your host firewall allows the connection.
|
||||
|
||||
### 9. "Failed to verify state directory" / restart loop on a bind-mounted state dir (#7027)
|
||||
|
||||
**Symptom**: Single-container deploy with the state directory bind-mounted from a
|
||||
host directory, no `WANTED_UID` set. The container exits 1 and restart-loops:
|
||||
|
||||
```
|
||||
-- Auto-detected workspace UID: 1024 (from /workspace)
|
||||
touch: cannot touch '/app/data/.testfile': Permission denied
|
||||
!! ERROR: Failed to verify state directory at /app/data
|
||||
```
|
||||
|
||||
**Cause**: UID auto-detection used to read `/workspace` before the configured
|
||||
state directory. In a stock image `/workspace` exists and is owned by the
|
||||
image's own build-time `1024:1024`, so detection returned a value that carries
|
||||
no information about the host — and because `1024` is also the fallback default,
|
||||
the log read as if detection had found nothing.
|
||||
|
||||
**Fix**: Fixed in the init script — the configured `HERMES_WEBUI_STATE_DIR` is
|
||||
now probed first. The full order is:
|
||||
|
||||
1. `$HERMES_WEBUI_STATE_DIR` (default `/app/data`) — a bind mount by definition
|
||||
in a single-container deploy, so its owner is the host identity to match
|
||||
2. `/home/hermeswebui/.hermes`, `$HERMES_HOME`, `/opt/data` — the hermes-home
|
||||
shared volume in two-container setups (#668)
|
||||
3. `/workspace` — used only when nothing above resolves
|
||||
4. `1024` — fallback default
|
||||
|
||||
Root-owned candidates (UID 0, e.g. a freshly created named volume) are skipped
|
||||
at every step. An explicitly supplied `WANTED_UID`/`WANTED_GID` always wins and
|
||||
is never overwritten by detection — including the value `1024`, which earlier
|
||||
versions treated as "unset".
|
||||
|
||||
If you are on an older image, the workaround is to set the IDs explicitly:
|
||||
|
||||
```bash
|
||||
docker run -e WANTED_UID=$(id -u) -e WANTED_GID=$(id -g) ...
|
||||
```
|
||||
|
||||
## Multi-container architecture
|
||||
|
||||
The two- and three-container setups use **named Docker volumes** (not bind mounts) by default for a reason: named volumes solve the UID/GID problem by construction. Docker creates the volume's root directory with the correct ownership, all containers reading/writing to it see the same files, no host-side permission setup required.
|
||||
@@ -626,7 +537,6 @@ volumes:
|
||||
- #681 — tools running in WebUI container, not agent container (architectural)
|
||||
- #668 — auto-detect UID/GID from mounted volume
|
||||
- #569 — UID/GID detection priority order
|
||||
- #7027 — state dir probed before `/workspace` in UID/GID detection (see [#9 above](#9-failed-to-verify-state-directory--restart-loop-on-a-bind-mounted-state-dir-7027))
|
||||
|
||||
If you hit a new failure mode not covered here, please [open an issue](https://github.com/nesquena/hermes-webui/issues/new) with:
|
||||
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 51 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 47 KiB |
@@ -168,8 +168,6 @@ Docker UID/GID and mount guidance in [Docker setup guide](docker.md).
|
||||
Password protection is optional for localhost-only installs. Enable it if you
|
||||
expose WebUI outside `127.0.0.1`, behind a reverse proxy, or on a LAN.
|
||||
|
||||
For installed PWAs, prefer WebUI's built-in password over proxy basic auth. Reverse proxies are supported, but HTTP basic-auth challenges in front of the WebUI origin can interrupt the service-worker and shell-asset fetches the installed app relies on during updates. If you keep proxy auth, scope it so same-origin `sw.js`, manifest, and shell update requests can complete.
|
||||
|
||||
The password is stored through the normal WebUI settings path and hashed
|
||||
server-side. You can change it later from Settings.
|
||||
|
||||
|
||||
+5
-39
@@ -34,53 +34,19 @@ The Hermes Web UI is fully responsive with a mobile-optimized layout
|
||||
(hamburger sidebar, sidebar top tabs in the drawer, touch-friendly controls),
|
||||
so it works well as a daily-driver agent interface from your phone.
|
||||
|
||||
**Preferred setup: Tailscale Serve**
|
||||
**Setup:**
|
||||
|
||||
1. Install [Tailscale](https://tailscale.com/download) on your server and
|
||||
your iPhone/Android.
|
||||
2. Keep the WebUI bound to localhost and enable password auth:
|
||||
|
||||
```bash
|
||||
HERMES_WEBUI_PASSWORD=your-secret ./start.sh
|
||||
```
|
||||
|
||||
3. Publish the local WebUI port through Tailscale Serve:
|
||||
|
||||
```bash
|
||||
tailscale serve --bg 8787
|
||||
```
|
||||
|
||||
4. Open the HTTPS MagicDNS URL that Tailscale prints in your phone's browser.
|
||||
|
||||
Tailscale Serve keeps WebUI on loopback while giving your tailnet an HTTPS
|
||||
MagicDNS hostname. On Linux, changing Serve configuration may require elevated
|
||||
permissions. If `tailscale serve --bg 8787` reports `Access denied: serve
|
||||
config denied`, either run it with sudo:
|
||||
|
||||
```bash
|
||||
sudo -S -p '' tailscale serve --bg 8787
|
||||
```
|
||||
|
||||
Or allow the supervised non-root WebUI/Hermes user to manage Tailscale:
|
||||
|
||||
```bash
|
||||
sudo -S -p '' tailscale set --operator=$USER
|
||||
tailscale serve --bg 8787
|
||||
```
|
||||
|
||||
**Fallback: direct tailnet IP**
|
||||
|
||||
Use direct tailnet access when Tailscale Serve is unavailable, disabled, or not
|
||||
permitted. Because this binds WebUI beyond loopback, always enable password
|
||||
auth:
|
||||
2. Start the WebUI listening on all interfaces with password auth enabled:
|
||||
|
||||
```bash
|
||||
HERMES_WEBUI_HOST=0.0.0.0 HERMES_WEBUI_PASSWORD=your-secret ./start.sh
|
||||
```
|
||||
|
||||
Then open `http://<server-tailscale-ip>:8787` in your phone's browser (find
|
||||
your server's Tailscale IP in the Tailscale app or with `tailscale ip -4` on
|
||||
the server).
|
||||
3. Open `http://<server-tailscale-ip>:8787` in your phone's browser
|
||||
(find your server's Tailscale IP in the Tailscale app or with
|
||||
`tailscale ip -4` on the server).
|
||||
|
||||
That's it. Traffic is encrypted end-to-end by WireGuard, and password auth
|
||||
protects the UI at the application level. You can add it to your home screen
|
||||
|
||||
@@ -1,29 +0,0 @@
|
||||
# Remote Terminal Workspaces
|
||||
|
||||
Architecture contract and path-resolution semantics for remote terminal profiles (SSH, Docker) in Hermes WebUI.
|
||||
|
||||
---
|
||||
|
||||
## 1. Overview
|
||||
|
||||
When a Hermes profile is configured with a remote terminal backend (e.g. `terminal.backend: "ssh"` or `"docker"`), its working directory (`terminal.cwd`) lives on the remote target host rather than the local WebUI host filesystem.
|
||||
|
||||
On hosts such as macOS, local path resolution via `Path.resolve()` or `os.path.realpath()` expands synthetic firmlinks (e.g. rewriting `/home/<user>` to `/System/Volumes/Data/home/<user>`). Because the target-side path does not exist on the local macOS server, unconstrained local resolution causes runtime validation failures and session corruption.
|
||||
|
||||
---
|
||||
|
||||
## 2. Path Resolution Contract
|
||||
|
||||
1. **Target-side POSIX Path Preservation:**
|
||||
- Any valid POSIX path at or beneath a remote profile's configured `terminal.cwd` is preserved verbatim as a `Path` object without calling host-local `Path.resolve()`.
|
||||
- Traversal escapes (`..`), null bytes (`\0`), and blocked system roots (`/etc`, `/usr`, `/var`, `/sys`, `/proc`, `/dev`) continue to be strictly rejected.
|
||||
|
||||
2. **Profile Isolation & Boundary Enforcement:**
|
||||
- Remote path recognition is **strictly scoped to the target/active profile**.
|
||||
- If an active profile has a local backend, target-side remote paths belonging to inactive named profiles are **not** treated as remote for the active local profile.
|
||||
- For local profiles, workspace addition (`/api/workspaces/add`) enforces host-local directory existence and permissions.
|
||||
- For remote profiles, workspace addition validates against the profile's remote `terminal.cwd` and skips host-local directory creation (`mkdir`).
|
||||
|
||||
3. **Session & Streaming Lifecycle:**
|
||||
- `Session.__init__` and `Session.load` normalize session workspace paths through `_resolve_path(..., profile=profile)`, preserving target-side paths for remote profiles and preventing corruption of `session.workspace` or `session.created_workspace`.
|
||||
- Streaming execution (`_run_agent_streaming`) and multimodal asset root resolution preserve the remote session workspace when updating runtime run state.
|
||||
+8
-12
@@ -47,17 +47,16 @@ First-time contributor RFCs should be discussed in an issue before opening a PR.
|
||||
replay, compression, and session metadata coherent during active and recovered
|
||||
WebUI runs.
|
||||
- [`live-to-final-assistant-replies.md`](live-to-final-assistant-replies.md)
|
||||
— #3400 accepted product model for long-running assistant replies, live
|
||||
process prose, tool activity, recovery, terminal outcomes, display
|
||||
projections, and the final-answer boundary.
|
||||
— #3400 product model for long-running assistant replies, live process prose,
|
||||
tool activity, recovery, terminal outcomes, and the final-answer boundary.
|
||||
- [`transparent-stream-activity-mode.md`](transparent-stream-activity-mode.md)
|
||||
— #3820 implemented opt-in display mode for power users who need a
|
||||
transparent, chronological Thinking / progress / tool-call stream alongside
|
||||
the default Compact Worklog and opt-in Final answer only projections.
|
||||
— #3820 opt-in display mode for power users who need a transparent,
|
||||
chronological Thinking / progress / tool-call stream alongside the default
|
||||
Compact Worklog model.
|
||||
- [`stable-assistant-turn-anchors.md`](stable-assistant-turn-anchors.md) — #3926
|
||||
implemented frontend presentation/reconciliation model for anchoring live
|
||||
assistant activity, settled final answers, replay, and all activity display
|
||||
modes to one assistant turn; remaining hardening is tracked under #3400.
|
||||
proposed frontend presentation/reconciliation model for anchoring live
|
||||
assistant activity, settled final answers, replay, and Compact/Transparent
|
||||
render modes to one assistant turn.
|
||||
- [`canonical-session-resolution.md`](canonical-session-resolution.md) — #2361
|
||||
focused contract for resolving URL, query parameter, localStorage, sidebar,
|
||||
and compression-lineage session IDs to one canonical visible chat target.
|
||||
@@ -67,6 +66,3 @@ First-time contributor RFCs should be discussed in an issue before opening a PR.
|
||||
#3058 control-surface companion to #3400 for Queue, Steer, Stop-and-send,
|
||||
Interrupt, and leftover-steer inputs submitted while a long-running agent
|
||||
session is active.
|
||||
- [`session-sse-contract-v1.md`](session-sse-contract-v1.md) — #4812 Proposed
|
||||
contract vocabulary, replay identity, event taxonomy, cursor/resume semantics,
|
||||
and implementation gates for `GET /api/sessions/{session_id}/events`.
|
||||
|
||||
@@ -3,7 +3,6 @@
|
||||
- **Status:** Accepted (parent contract; implementation tracked in [#3400](https://github.com/nesquena/hermes-webui/issues/3400))
|
||||
- **Author:** @franksong2702
|
||||
- **Created:** 2026-06-03
|
||||
- **Updated:** 2026-07-16
|
||||
- **Tracking issue:** [#3400](https://github.com/nesquena/hermes-webui/issues/3400)
|
||||
|
||||
## Background: Long-Running Sessions Are The Anchor
|
||||
@@ -205,43 +204,6 @@ Requirements:
|
||||
- Very long final answers remain complete and readable. They should not be
|
||||
hidden inside the activity summary or replaced by a progress/status artifact.
|
||||
|
||||
### Activity display and disclosure addenda
|
||||
|
||||
The accepted lifecycle now has three presentation strategies over the same
|
||||
assistant-turn activity data:
|
||||
|
||||
- **Compact Worklog** remains the default. Its top-level Worklog is expanded
|
||||
while live so progress remains visible, and the user may collapse or reopen it.
|
||||
Nested tool/reasoning detail remains progressively disclosed.
|
||||
- **Transparent Stream** is opt-in and renders the same ordered activity as
|
||||
chronological rows. It does not create a second live or settled owner.
|
||||
- **Final answer only** is opt-in (`hide_all_activity` in persisted settings).
|
||||
It suppresses activity rows without deleting the persisted Anchor scene or
|
||||
changing the final-answer owner.
|
||||
|
||||
For normal completed turns, the settled Compact Worklog remains collapsed by
|
||||
default. When a terminal error-family turn has actual Worklog content, the
|
||||
Worklog defaults open so readable partial work is not hidden behind the error
|
||||
outcome. The current disclosure error family is `error`, `no_response`,
|
||||
`degraded`, `connection_lost`, `tool_limit_reached`, and
|
||||
`compression_exhausted`; cancelled turns and the parent `interrupted` terminal
|
||||
outcome keep their separate semantics. An explicit user disclosure choice wins
|
||||
over these defaults and may be restored across a render rebuild.
|
||||
|
||||
`degraded` and `connection_lost` are Anchor-level reconstruction/transport
|
||||
outcomes defined in
|
||||
[`stable-assistant-turn-anchors.md`](stable-assistant-turn-anchors.md), not
|
||||
additional canonical product states in the table below. In this parent
|
||||
contract, `connection_lost` is the transport-specific Anchor form of an
|
||||
interruption, while `degraded` is the explicit recovery-state counterpart to the
|
||||
restoring/degraded path. They use error-family disclosure only because they can
|
||||
leave readable partial Worklog content without a normal final answer.
|
||||
|
||||
These are presentation and disclosure rules only. They do not change reply
|
||||
ownership, terminal classification, durable transcript truth, or the requirement
|
||||
that all three strategies converge across live, settle, reload, session switch,
|
||||
and reconnect.
|
||||
|
||||
### Recovery and replay
|
||||
|
||||
Refresh, reconnect, session switching, and replay should preserve the same
|
||||
@@ -511,9 +473,7 @@ for current open/merged/superseded status.
|
||||
| Track | Scope | Current vehicle |
|
||||
| --- | --- | --- |
|
||||
| Parent product RFC | Define the long-running live-to-final assistant reply lifecycle and review checklist. | This RFC; tracking issue [#3400](https://github.com/nesquena/hermes-webui/issues/3400). |
|
||||
| First reply lifecycle implementation | Live process prose, quiet tool activity, settled activity summary above final answer, replay/reattach consistency, live-only compression status, supporting stream ownership fixes. | [#3401](https://github.com/nesquena/hermes-webui/pull/3401), absorbed through [#3741](https://github.com/nesquena/hermes-webui/pull/3741). |
|
||||
| Activity display projections | Compact Worklog by default, opt-in Transparent Stream, and opt-in Final answer only over the same assistant-turn data. | [#3820](https://github.com/nesquena/hermes-webui/issues/3820), [`transparent-stream-activity-mode.md`](transparent-stream-activity-mode.md), tracking issue [#3400](https://github.com/nesquena/hermes-webui/issues/3400). |
|
||||
| Assistant-turn presentation ownership | Normalize live, settled, replayed, and recovered activity into one Anchor / `activity_scene_v1`, with legacy rendering limited to historical or non-anchor compatibility. | [#3926](https://github.com/nesquena/hermes-webui/issues/3926), [`stable-assistant-turn-anchors.md`](stable-assistant-turn-anchors.md); core implementation shipped through the #4411/#4564 and #5242/#5243 capstones. |
|
||||
| First reply lifecycle implementation | Live process prose, quiet tool activity, settled activity summary above final answer, replay/reattach consistency, live-only compression status, supporting stream ownership fixes. | [#3401](https://github.com/nesquena/hermes-webui/pull/3401). |
|
||||
| Terminal/no-final stabilization | Compression exhausted, tool-tail/no-final transcript shape, context-compaction marker suppression, terminal error routing. | [#3315](https://github.com/nesquena/hermes-webui/issues/3315), [#3316](https://github.com/nesquena/hermes-webui/pull/3316). |
|
||||
| Cancel ownership hardening | Frontend cancel should close its own SSE source and clear only its own busy state. | [#3344](https://github.com/nesquena/hermes-webui/issues/3344), [#3345](https://github.com/nesquena/hermes-webui/pull/3345). |
|
||||
| Early-cancel startup race | Backend cancel should still interrupt the worker when the SSE registry detached before startup fully settled. | [#3475](https://github.com/nesquena/hermes-webui/issues/3475), [#3476](https://github.com/nesquena/hermes-webui/pull/3476). |
|
||||
|
||||
@@ -1,303 +0,0 @@
|
||||
# Session SSE Contract v1
|
||||
|
||||
- **Status:** Proposed
|
||||
- **Author:** @rodboev
|
||||
- **Created:** 2026-07-04
|
||||
- **Tracking:** #4812
|
||||
|
||||
Refs #4812
|
||||
|
||||
---
|
||||
|
||||
## Problem
|
||||
|
||||
hermes-webui has no stable, cross-client contract for observing the lifecycle
|
||||
of an individual session over SSE. Five or more future consumers — WebUI
|
||||
reconnect/multi-tab, Android wrapper, iOS/PWA wrapper, desktop/TWA wrapper, and
|
||||
test/CLI observers — each need a resumable, dedupe-safe event stream. Without a
|
||||
shared contract, every client invents its own cursor, heartbeat, and event-type
|
||||
semantics, multiplying coordination cost as new producers are added.
|
||||
|
||||
The maintainer asked for a docs-only RFC first, holding implementation until
|
||||
sequence and replay semantics are settled (comment 2026-06-24T17:14:05Z,
|
||||
2026-06-25T04:50:06Z on #4812). This document settles the contract vocabulary
|
||||
against current source before any route is added.
|
||||
|
||||
## Goals
|
||||
|
||||
- Define the SSE envelope and event-type vocabulary for a proposed per-session
|
||||
stream `GET /api/sessions/{session_id}/events`.
|
||||
- Specify replay identity using the existing run-journal cursor model.
|
||||
- Specify the snapshot fallback for stale or evicted cursors.
|
||||
- Document the distinction from the existing global session-list stream.
|
||||
- Record open implementation gates that must be resolved before the endpoint
|
||||
ships.
|
||||
|
||||
## Non-goals
|
||||
|
||||
- This RFC does **not** implement `GET /api/sessions/{session_id}/events`. No
|
||||
route, handler, or related code is added in this PR.
|
||||
- This RFC does **not** modify `GET /api/sessions/events` (the existing global
|
||||
session-list invalidation stream routed in `api/routes.py` and
|
||||
implemented by `_handle_session_events_stream()` in `api/routes.py`).
|
||||
- This RFC does **not** replace or modify existing streams: `/api/chat/stream`,
|
||||
`/api/approval/stream`, or `/api/clarify/stream`.
|
||||
- This RFC does **not** introduce Android, iOS, or PWA client code.
|
||||
- This RFC does **not** claim Android/iOS background reconnect behavior or
|
||||
production proxy delivery; those require owner-reaching proof in a later
|
||||
implementation PR.
|
||||
- This RFC does **not** promise a new session-global sequence counter in Phase 1.
|
||||
|
||||
## Current source inventory
|
||||
|
||||
### Existing global session-list stream
|
||||
|
||||
`GET /api/sessions/events` is a **different endpoint** from the one this RFC
|
||||
proposes. It is routed in `api/routes.py` and implemented by
|
||||
`_handle_session_events_stream()` in `api/routes.py`. It emits bare
|
||||
`sessions_changed` events and keepalives for any change to the session list. It
|
||||
is a global invalidation signal, not a per-session lifecycle stream. The proposed
|
||||
`GET /api/sessions/{session_id}/events` is per-session and path-distinct.
|
||||
|
||||
### Heartbeat
|
||||
|
||||
`_SSE_HEARTBEAT_INTERVAL_SECONDS = 5` (defined in `api/routes.py`) is the current
|
||||
heartbeat interval for SSE streams. Phase 1 reuses this constant rather than
|
||||
adding a separate configurable knob.
|
||||
|
||||
### Run-journal cursor and replay
|
||||
|
||||
Current replay identity is run/stream-scoped:
|
||||
|
||||
Symbols in this inventory were verified against WebUI `master` when this RFC
|
||||
was written. Function, constant, and endpoint **names** are the stable anchors:
|
||||
this RFC deliberately cites them by name (not by line number) so a source-layout
|
||||
shift in `api/routes.py` cannot invalidate the doc or its contract test.
|
||||
|
||||
- `_parse_run_journal_event_id()` and `_parse_run_journal_after_seq()` (both in
|
||||
`api/routes.py`) parse the replay cursor from the `after_event_id` /
|
||||
`after_seq` **query params** (not the
|
||||
`Last-Event-ID` header — that header is the *proposed* new-endpoint contract
|
||||
below, §Reconnect).
|
||||
- `_runner_event_id()` (in `api/routes.py`) constructs the event `id`
|
||||
field as `stream_id:seq`.
|
||||
- SSE frames carry their `id:` via the `_sse_with_id()` helper, emitted on the
|
||||
live `/api/chat/stream` path, on the runner-observe path, and during journal
|
||||
replay — all in `api/routes.py`.
|
||||
- `_replay_run_journal()` (in `api/routes.py`) reads events by
|
||||
`(session_id, stream_id)`.
|
||||
- `api/streaming.py` writes current live agent streams to
|
||||
`STREAMS[stream_id]`.
|
||||
- `api/streaming.py` appends SSE events to the run journal and carries
|
||||
per-item `event_id` into the live queue.
|
||||
|
||||
The existing run journal represents `session_id`, `stream_id`, `seq`, and
|
||||
`event_id`, but **not** a session-global monotonic sequence. Phase 1 must not
|
||||
promise a session-global counter because current source does not provide one.
|
||||
|
||||
## Proposed endpoint
|
||||
|
||||
```
|
||||
GET /api/sessions/{session_id}/events
|
||||
```
|
||||
|
||||
This endpoint is **path-distinct** from `GET /api/sessions/events`. The
|
||||
`{session_id}` path segment is required; the global endpoint has no such segment.
|
||||
|
||||
Response: `Content-Type: text/event-stream`. Authentication and session
|
||||
visibility checks reuse existing mechanisms.
|
||||
|
||||
## Envelope
|
||||
|
||||
Each SSE event carries a JSON payload with this structure:
|
||||
|
||||
```json
|
||||
{
|
||||
"schema_version": 1,
|
||||
"session_id": "<session_id>",
|
||||
"event_type": "<string>",
|
||||
"event_id": "<opaque cursor>",
|
||||
"stream_id": "<stream_id>",
|
||||
"seq": <integer>,
|
||||
"emitted_at": "<ISO-8601 UTC>",
|
||||
"payload": { ... },
|
||||
"meta": { ... }
|
||||
}
|
||||
```
|
||||
|
||||
- `schema_version`: integer, always `1` for Phase 1 events.
|
||||
- `session_id`: the session this event belongs to.
|
||||
- `event_type`: one of the event types listed in the taxonomy below.
|
||||
- `event_id`: opaque client cursor (see Cursor and resume semantics).
|
||||
- `stream_id`: the run journal stream this event came from, if applicable.
|
||||
- `seq`: monotonic within a stream/run (see Cursor and resume semantics).
|
||||
- `emitted_at`: server-side emission timestamp in ISO-8601 UTC.
|
||||
- `payload`: event-type-specific data.
|
||||
- `meta`: optional; reserved for tracing and debug metadata.
|
||||
|
||||
Server-generated events that do not originate in the run journal, currently
|
||||
`heartbeat` and `session_snapshot`, need an explicit `event_id` / `stream_id` /
|
||||
`seq` rule before implementation. This RFC records that as an implementation
|
||||
gate rather than inventing values without source support.
|
||||
|
||||
## Event taxonomy (Phase 1 draft)
|
||||
|
||||
> **Semantic names below are aspirational for the per-session endpoint.** Live
|
||||
> `/api/chat/stream` wire names are listed in **Authoritative emitted events**
|
||||
> immediately after this table — use those when writing clients against current
|
||||
> source.
|
||||
|
||||
| event_type | Source | Description |
|
||||
|---|---|---|
|
||||
| `chat_delta` | run journal / live stream | Token or chunk from an assistant reply. |
|
||||
| `tool_call` | run journal / live stream | Tool invocation record. |
|
||||
| `tool_result` | run journal / live stream | Tool result record. |
|
||||
| `approval_request` | run journal | Approval prompt sent to the user. |
|
||||
| `clarify_request` | run journal | Clarification prompt sent to the user. |
|
||||
| `run_started` | run journal | Run entered active state. |
|
||||
| `run_finished` | run journal | Run reached a terminal state (complete, cancelled, error). |
|
||||
| `session_snapshot` | server fallback | Current session projection; emitted when replay is unavailable. |
|
||||
| `heartbeat` | server | Keepalive emitted on the `_SSE_HEARTBEAT_INTERVAL_SECONDS` cadence. |
|
||||
|
||||
## Authoritative emitted events (`/api/chat/stream`)
|
||||
|
||||
These are the **real wire `event:` names** emitted by `api/streaming.py` today
|
||||
(23 names). Clients and docs must use this table — not the semantic draft above —
|
||||
when integrating with the live chat SSE relay.
|
||||
|
||||
| Wire name | Role |
|
||||
|---|---|
|
||||
| `token` | Assistant text delta |
|
||||
| `reasoning` | Model reasoning / thinking delta |
|
||||
| `tool` | Tool call started |
|
||||
| `tool_complete` | Tool call finished (result or error) |
|
||||
| `interim_assistant` | Mid-turn assistant prose (pre-final) |
|
||||
| `approval` | Destructive-command approval prompt |
|
||||
| `clarify` | Structured clarification prompt |
|
||||
| `compressing` | Context compression started |
|
||||
| `compressed` | Context compression finished |
|
||||
| `title` | Session title update (often after `done`) |
|
||||
| `title_status` | Title generation status / skip reason |
|
||||
| `warning` | Non-fatal provider/fallback warning |
|
||||
| `apperror` | Terminal application error (no trailing `stream_end`) |
|
||||
| `cancel` | Run cancelled |
|
||||
| `done` | Turn finalized (session payload); title/`stream_end` may follow |
|
||||
| `stream_end` | SSE fence — close the client EventSource |
|
||||
| `metering` | Token/cost metering snapshot |
|
||||
| `context_status` | Context window / usage status |
|
||||
| `goal` | Goal / plan card update |
|
||||
| `goal_continue` | Goal continuation signal |
|
||||
| `pending_steer_leftover` | Leftover steer text after interrupt |
|
||||
| `state_saved` | Durable state write acknowledgment |
|
||||
| `todo_state` | Todo / checklist panel update |
|
||||
|
||||
Relay close set (stop draining the live queue): `stream_end`, `cancel`,
|
||||
`apperror`, and legacy `error` — see `api.run_journal.SSE_RELAY_CLOSE_EVENTS`.
|
||||
`done` is **not** a relay-close event because `title` and `stream_end` follow it.
|
||||
|
||||
The semantic taxonomy table remains a draft for the proposed per-session
|
||||
endpoint vocabulary and must be confirmed during maintainer review before that
|
||||
endpoint claims parity.
|
||||
|
||||
## Cursor and resume semantics
|
||||
|
||||
`Last-Event-ID` is the standard SSE reconnect header. Clients send the last
|
||||
`event_id` value seen on reconnect; the server uses it to resume replay from that
|
||||
position.
|
||||
|
||||
**`event_id` is opaque to clients.** Its current source-compatible form is
|
||||
`stream_id:seq`, as constructed by `_runner_event_id()` in `api/routes.py`.
|
||||
Clients must treat it as an opaque string and must
|
||||
not parse or construct cursor values.
|
||||
|
||||
**`seq` is monotonic within a stream/run.** It is not a session-global counter
|
||||
and is not promised to increase monotonically across streams or runs. Phase 1
|
||||
does not claim a pre-existing session-global sequence because current source does
|
||||
not provide one.
|
||||
|
||||
**Clients dedupe by `event_id`.** If a reconnect causes overlap with already-seen
|
||||
events, clients use `event_id` to detect and skip duplicates.
|
||||
|
||||
## Replay source
|
||||
|
||||
Phase 1 uses the **durable run journal** as the replay source for replayable
|
||||
events. The live `STREAMS[stream_id]` queue (in `api/streaming.py`) is
|
||||
not a reliable replay source because it holds only recent in-memory state.
|
||||
|
||||
A future implementation must replay from the run journal via the existing
|
||||
`_replay_run_journal()` path (in `api/routes.py`) and fall back to the
|
||||
snapshot mechanism when journal entries are unavailable for a given cursor.
|
||||
|
||||
## Snapshot fallback
|
||||
|
||||
When the `Last-Event-ID` cursor is evicted, expired, unknown, or refers to a
|
||||
stream that is no longer replayable, the server must:
|
||||
|
||||
1. Emit a `session_snapshot` event containing the current session projection.
|
||||
2. Continue the live stream from the present without pretending that missed
|
||||
events were replayed.
|
||||
|
||||
`session_snapshot` is a recovery boundary, not proof of exact missed-event
|
||||
replay. Clients receiving a snapshot must treat prior cursor state as invalid and
|
||||
resync from the snapshot payload.
|
||||
|
||||
## Heartbeat
|
||||
|
||||
Phase 1 reuses `_SSE_HEARTBEAT_INTERVAL_SECONDS` (defined in `api/routes.py`) for
|
||||
heartbeat cadence. A new per-session configurable heartbeat knob is **not** added
|
||||
in Phase 1. The implementation PR must follow whatever value the constant holds
|
||||
at implementation time; it must not hard-code a separate interval.
|
||||
|
||||
## Security and privacy
|
||||
|
||||
- Reuse existing auth and session visibility checks. A client must not be able to
|
||||
subscribe to events for a session it does not own.
|
||||
- Payloads must not include credentials, raw provider API keys, or unsanitized
|
||||
internal error details.
|
||||
- `meta` fields are for tracing and debug metadata and must not carry
|
||||
security-sensitive values in production.
|
||||
|
||||
## Implementation gates (open questions)
|
||||
|
||||
The following decisions must be resolved before any implementation PR for this
|
||||
endpoint is accepted:
|
||||
|
||||
1. **Sequence semantics**: Maintainer must confirm that stream/run-scoped `seq`
|
||||
(not session-global) is acceptable for Phase 1 clients.
|
||||
2. **Retention policy**: How long are run journal entries retained for replay?
|
||||
What is the eviction boundary that triggers the snapshot fallback?
|
||||
3. **Event-type table**: The taxonomy above is a draft. The complete event-type
|
||||
list must be confirmed during review of this RFC.
|
||||
4. **Auth behavior on reconnect**: Does `Last-Event-ID` replay require the same
|
||||
auth token, or can it continue across token refresh?
|
||||
5. **Client proof**: At least one browser-based client (WebUI) and at least one
|
||||
non-browser client (Android wrapper or CLI) must provide owner-reaching
|
||||
reconnect proof before implementation closes #4812.
|
||||
6. **Proxy and keepalive**: The 5 s heartbeat choice must survive real proxy
|
||||
deployments. This requires manual-owner-proof or standards-doc evidence in the
|
||||
implementation PR.
|
||||
7. **Server-generated event identity**: Maintainer must confirm how
|
||||
`heartbeat` and `session_snapshot` populate `event_id`, `stream_id`, and
|
||||
`seq`, because those events do not originate in the run journal.
|
||||
|
||||
## Bypass risks
|
||||
|
||||
Future implementation work must not:
|
||||
|
||||
- Introduce an in-memory-only cursor that bypasses the run journal.
|
||||
- Conflate `GET /api/sessions/events` (global session-list invalidation) with
|
||||
`GET /api/sessions/{session_id}/events` (per-session lifecycle).
|
||||
- Promise a session-global monotonic sequence without defining a migration from
|
||||
the current stream/run-scoped model.
|
||||
|
||||
Tests in `tests/test_issue4812_session_sse_contract_rfc.py` assert these
|
||||
boundaries so review catches regressions against this contract.
|
||||
|
||||
## Rollout plan
|
||||
|
||||
1. This RFC is accepted by maintainer review on #4812.
|
||||
2. Retention and event-type decisions are confirmed.
|
||||
3. Client proof (browser + non-browser) is provided.
|
||||
4. An implementation PR adds `GET /api/sessions/{session_id}/events` following
|
||||
this contract vocabulary.
|
||||
5. Implementation PR closes #4812.
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user