Files
dependabot[bot]andMichael D'Angelo 664fac181c chore(deps): bump the codex group across 3 directories with 2 updates (#2770)
* chore(deps): bump the codex group across 3 directories with 2 updates

Bumps the codex group with 1 update in the /evals/triage-finding directory: [@openai/codex-sdk](https://github.com/openai/codex/tree/HEAD/sdk/typescript).
Bumps the codex group with 1 update in the /plugins/codex-security/mcp-app directory: [@openai/codex-sdk](https://github.com/openai/codex/tree/HEAD/sdk/typescript).
Bumps the codex group with 2 updates in the /sdk/typescript directory: [@openai/codex-sdk](https://github.com/openai/codex/tree/HEAD/sdk/typescript) and [@openai/codex](https://github.com/openai/codex/tree/HEAD/codex-cli).


Updates `@openai/codex-sdk` from 0.170.0 to 0.171.0

Updates `@openai/codex-sdk` from 0.170.0 to 0.171.0

Updates `@openai/codex-sdk` from 0.170.0 to 0.171.0

Updates `@openai/codex` from 0.170.0 to 0.171.0

---
updated-dependencies:
- dependency-name: "@openai/codex-sdk"
  dependency-version: 0.171.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codex
- dependency-name: "@openai/codex-sdk"
  dependency-version: 0.171.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codex
- dependency-name: "@openai/codex-sdk"
  dependency-version: 0.171.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codex
- dependency-name: "@openai/codex"
  dependency-version: 0.171.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: codex
...

Signed-off-by: dependabot[bot] <support@github.com>

* fix(container): record Codex 0.171.0 native source

---------

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Michael D'Angelo <mdangelo@openai.com>
2026-11-20 07:39:28 -08:00
..

Evaluations

Start with the deterministic triage checks or the harness tests linked below. These run locally without model calls. Model evaluations are opt-in and consume usage on the account you configure.

  • Dependency reachability grades local CVE and package investigations against controlled application builds and measured stack evidence.
  • Triage finding: Promptfoo input contracts, OSS calibration, and SastBench. This suite owns its private package and lockfile.
  • Secret discovery: checks whether the production core audit finds synthetic credentials in source and keeps them in its final findings, with deterministic grading and harness checks.
  • Standard time budget: paired comparison of production Standard scans with and without an advisory time allowance on synthetic vulnerabilities and safe controls, with source-evidence grading.
  • Workflow routing: checks whether repository-installed production skills route natural-language requests to the appropriate workflow, stopping before workflow execution.
  • The skill review fixtures cover pre-install decisions for individual skill files and bundles, including cross-file behavior.

CI runs the deterministic triage, secret-discovery, time-budget, and workflow-routing checks. The MCP suite covers Deep Scan iterations, recovery, and publication.

Build the evaluation tools

For the secret-discovery, Standard time-budget and workflow-routing suites, install the SDK and MCP dependencies, then run these commands from the repository root. The eval type-check includes the triage sources, so it needs the triage package’s dependencies too.

pnpm --dir evals/triage-finding install --frozen-lockfile
pnpm --dir sdk/typescript run build:evals

For Promptfoo triage and SastBench runs, use the triage setup, which also builds the native policy helper for your platform.