Files
browser/.github/workflows/agent-regression.yml
Thibaud-Vdb 7e60cc4d5a ci: permissions on every workflow, no stored checkout token where nothing pushes, packaging image by digest
Six workflows had no permissions block and ran on whatever the
repository default grants. They get contents: read at the workflow
level. The three perf-fmt jobs pull ghcr.io/lightpanda-io/perf-fmt, a
private image, with GITHUB_TOKEN, so they keep packages: read on top of
it. No job in these workflows writes to the repository.

actions/checkout leaves the job token in .git/config by default, where
every later step and tool in the job can read it. Every checkout opts
out except the three whose job pushes a branch with that token: the
orderfile regen and the two nix dependency updates keep theirs.

The Debian package build ran in debian:stable-slim, a tag that moves
with every Debian point release. It is pinned to the current index
digest, with the tag kept next to it for readability.
2026-10-30 12:23:13 +01:00

103 lines
3.6 KiB
YAML

name: agent-regression
on:
schedule:
# Offset from wpt's 02:21 nightly.
- cron: "37 3 * * *"
workflow_dispatch:
inputs:
model:
description: "Gemini model id for the live layer (empty = suite default, see agent/run.sh in lightpanda-io/demo)"
required: false
permissions:
contents: read
jobs:
# Keep identical to e2e-test.yml's zig-build-release so the nightly suite
# exercises the same build variant that gates PRs.
zig-build-release:
name: zig build release
runs-on: ubuntu-latest
timeout-minutes: 20
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
fetch-depth: 0
- uses: ./.github/actions/install
- uses: ./.github/actions/v8-snapshot
- name: zig build release
run: zig build -Dsnapshot_path=../../snapshot.bin -Dprebuilt_v8_path=v8/libc_v8.a -Doptimize=fast -Dcpu=x86_64
- name: upload artifact
uses: actions/upload-artifact@043fb46d1a93c77aae656e7c1c64a875d1fc6a0a # v7.0.1
with:
name: lightpanda-build-release
path: |
zig-out/bin/lightpanda
retention-days: 1
agent-regression:
name: agent-regression
needs: zig-build-release
runs-on: ubuntu-latest
# The live HN save task drives the LLM through many turns (~7 min); give
# the whole suite generous headroom.
timeout-minutes: 25
steps:
- uses: actions/checkout@df4cb1c069e1874edd31b4311f1884172cec0e10 # v6.0.3
with:
persist-credentials: false
repository: 'lightpanda-io/demo'
fetch-depth: 0
- name: download lightpanda release
uses: actions/download-artifact@3e5f45b2cfb9172054b4087a40e8e0b5a5461e7c # v8.0.1
with:
name: lightpanda-build-release
path: bin
- run: chmod a+x bin/lightpanda
- name: run agent regression suite (deterministic + live)
# Explicit bash gets pipefail, so tee doesn't mask the suite's exit code.
shell: bash
env:
LPD_PATH: ${{ github.workspace }}/bin/lightpanda
LPD_ARGS: --load-resources iframe --load-resources worker
GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
LP_MODEL: ${{ github.event.inputs.model }}
# Optional: news.ycombinator.com often blocks datacenter IPs. Reuse
# the residential proxy secret if present; empty is fine locally.
LP_HTTP_PROXY: ${{ secrets.MASSIVE_PROXY_RESIDENTIAL_US }}
run: ./agent/run.sh all 2>&1 | tee result.log
# run.sh colors its output unconditionally; strip the ANSI codes for
# Slack (the Actions log above keeps them).
- name: prepare slack log
id: log
if: ${{ !cancelled() }}
run: |
sed 's/\x1b\[[0-9;]*m//g' result.log > slack.log 2>/dev/null \
|| echo "suite produced no output (earlier step failed)" > slack.log
echo "summary=$(grep -o 'SUMMARY: .*' slack.log | tail -1)" >> "$GITHUB_OUTPUT"
- name: Send result to slack
if: ${{ !cancelled() }}
uses: slackapi/slack-github-action@af78098f536edbc4de71162a307590698245be95 # v3.0.1
with:
errors: true
method: files.uploadV2
token: ${{ secrets.CI_SLACK_BOT_TOKEN }}
payload: |
channel_id: ${{ vars.AGENT_SLACK_CHANNEL_ID }}
initial_comment: "Agent regression — ${{ steps.log.outputs.summary || 'no summary (suite crashed?)' }}"
file: "./slack.log"
filename: "agent-regression-${{ github.sha }}.txt"