mirror of https://github.com/ankitects/anki.git synced 2026-10-21 06:39:46 +08:00

Files

T

Fernando Lins 587d6ec796 fix: run static SonarCloud analysis on fork PRs (#5596 )

## Linked issue

Follow-up to #5498, which was closed in favor of a smaller and more
maintainable implementation.

## Summary / motivation

Run CI-based SonarCloud analysis for internal and fork pull requests
while keeping the Quality Gate informational.

The analysis matrix is:

| Event | Static analysis | Coverage in CI | Coverage in SonarCloud |
| --- | --- | --- | --- |
| Internal pull request | Yes | Yes | Yes |
| Push to `main` or `release/**` | Yes | Yes | Yes |
| Fork pull request | Yes | Yes | No |

Fork pull requests continue to run the full test coverage and
coverage-regression checks in the unprivileged CI workflow. Their
contributor-produced coverage reports are not passed to the privileged
SonarCloud workflow.

The fork scan reports static bugs, vulnerabilities, security hotspots,
and code smells. It:

- validates the PR state, repositories, branches, and tested SHA against
the GitHub API;
- checks out the exact revision tested by CI;
- replaces `sonar-project.properties` with the trusted default-branch
version;
- provides `SONAR_TOKEN` only to the scanner step;
- never builds, installs, or executes contributor code;
- keeps scanner-side Clippy and SCA disabled;
- clears coverage report paths and excludes coverage calculation for
forks.

Automatic Analysis remains disabled because it cannot be combined with
CI-based analysis and does not provide the required Rust and coverage
support.

This also configures Vitest to produce repository-relative LCOV paths
such as `SF:ts/lib/...`, allowing SonarCloud to resolve TypeScript files
when reports are downloaded into a separate checkout.

## How to test

### Checklist

- [x] I ran `just check` or an equivalent relevant check locally.
- [ ] I added or updated tests when the change is non-trivial or
behavior changed. The remaining behavior is specific to GitHub's
`workflow_run` environment and can only be exercised end-to-end after
the workflow is available on the default branch.

### Details

The following checks passed locally:

- `just test-ts --coverage`
- `just test-ts --coverage --html`
- `just fmt`
- `just check`
- YAML and JSON parsing
- `git diff --check`

Both TypeScript coverage runs passed all 67 tests. The generated LCOV
report was also checked to confirm that source paths are
repository-relative and begin with `SF:ts/`.

After this lands, end-to-end validation should cover:

1. A push to `main`, confirming that Python, TypeScript, and Rust
coverage reports are imported.
2. An internal PR, confirming static analysis and coverage.
3. A fork PR, confirming static analysis while the coverage download is
skipped and no coverage condition is evaluated.

## Before / after behavior

Before:

- fork PR analysis fails at the protected checkout;
- the `actions/checkout` version comments are outdated;
- TypeScript LCOV paths are relative to `ts/` and may not resolve in the
SonarCloud checkout.

After:

- internal PRs and trusted branch pushes receive static analysis and
coverage;
- fork PRs receive static analysis without importing
contributor-produced coverage;
- fork tests and coverage-regression checks continue to run in regular
CI;
- TypeScript LCOV paths resolve from the repository root;
- technical scanner failures remain visible without making the Sonar
Quality Gate a required merge check.

## Risk / compatibility / migration

The fork analysis runs in a privileged `workflow_run` and necessarily
asks the SonarScanner to parse untrusted source while it has access to a
project-scoped token.

The workflow limits this exposure by using GitHub-hosted runners,
read-only permissions, trusted scanner configuration, an exact tested
SHA, API-validated PR metadata, and no builds, dependency installation,
local Actions, caches, or fork-produced coverage artifacts.

This is intentionally a smaller trust boundary than the data-only
snapshot and custom coverage parser proposed in #5498. It prevents
direct execution of contributor code but does not eliminate potential
vulnerabilities in GitHub Actions or Sonar analyzers.

Automatic Analysis must remain disabled, `SONAR_TOKEN` should remain
restricted to this project, and the SonarCloud check must remain
non-required.

2026-09-14 15:30:42 -03:00

4.3 KiB

Raw Permalink Blame History

Testing and Coverage

CI runs Rust, Python, and TypeScript tests plus lint/type checks. Coverage is orchestrated in justfile with direct CLI calls, using ninja only to prepare generated build artifacts that the test commands need.

Quick reference

just test                        # run all tests (no coverage)
just test --coverage             # run all tests + enforce coverage thresholds
just test --coverage --html      # same + generate HTML reports under out/coverage/

just test-rust                   # Rust only
just test-rust --coverage
just test-rust --coverage --html

just test-py                     # Python (pylib + qt) only
just test-py --coverage
just test-py --coverage --html

just test-ts                     # TypeScript/Svelte Vitest only
just test-ts --coverage
just test-ts --coverage --html

HTML reports are written under out/coverage/ (gitignored).

Coverage tools and thresholds

Stack	Test runner	Coverage tool	Minimum
Rust workspace	`cargo nextest` via `cargo-llvm-cov`	`cargo-llvm-cov`	60%
Python `pylib/anki`	`pytest pylib/tests`	`coverage.py`	63%
Python `qt/aqt`	`pytest qt/tests`	`coverage.py`	20%
TypeScript/Svelte	`vitest run`	Vitest V8	5%

Linux pull requests run just test --coverage in CI. macOS and Windows jobs run just test (no coverage enforcement) for now.

Notes

Rust — cargo-llvm-cov is installed on demand into out/bin/ to avoid polluting the global cargo install. Coverage runs rebuild the workspace with instrumentation, so they are slower than plain just test-rust. Windows ARM64 (aarch64-pc-windows-msvc) is not supported: the Rust compiler produces malformed .profraw files on that target (rust-lang/rust#150123, cargo-llvm-cov#436). just test-rust --coverage exits with a clear message on ARM64 Windows; use just test-rust (no coverage) or rely on CI (Linux) for enforcement.
Python — coverage is split across two suites (pylib and qt) because they have different PYTHONPATH setups and test folders.
TypeScript — coverage is measured only over code reachable through Vitest's module graph. Svelte component rendering behavior is not covered.

SonarCloud

SonarCloud runs automatically after successful CI pushes and pull requests. It runs static analysis across Python, TypeScript, and Rust. For internal pull requests and trusted branch pushes, it also aggregates the coverage reports generated by the three stacks.

Dashboard: https://sonarcloud.io/project/overview?id=ankitects_anki
Internal coverage: uses the same LCOV/XML reports produced by just test --coverage
Fork coverage: remains enforced in CI but is not imported into SonarCloud
Analysis results: SonarCloud reports quality gate status on each analyzed PR

The quality gate is configured as non-blocking — a failing gate does not prevent merging, but the results are visible on the PR.

See SonarCloud analysis for the analysis matrix and the fork trust boundary.

Gaps and future improvements

Raise thresholds gradually as the test suite grows and CI timings stabilise.
Exclude generated files from coverage denominators where appropriate.
Publish out/coverage/ as a CI artifact so reviewers can browse HTML reports directly from a PR.
Consider diff/changed-file coverage once baselines are stable — it is a better enforcement mechanism for incremental improvement than whole-repo thresholds.
Add component or browser tests for Svelte UI surfaces if Svelte coverage is intended to cover rendered component behaviour.

4.3 KiB Raw Permalink Blame History