Files
paperclip/doc/TELEMETRY_WORKFLOW.md
T
Nicky LeachandPaperclip 90f85a7d11 Add telemetry proposal extractor (#9544)
## Thinking Path

> - Paperclip is the open source app people use to manage AI agents for
work
> - It emits telemetry events to understand product usage — registered
event names are gated by a generated `PAPERCLIP_EVENTS` registry so the
client only enqueues known, schema-approved events
> - When product teams want to instrument a new behaviour, they must
first register the event name — but schema registration is a
commit-and-release cycle, which creates friction in fast-moving product
iterations
> - A proposal lane is needed: let developers mark a `track()` call with
a typed `@ts-expect-error` proposal marker so the event name can be
reviewed and tracked in CI before the schema is formally registered
> - The existing client had no guard against unregistered event names,
so any call with an out-of-registry name (or a prototype-inherited key)
would silently enter the queue, state, and network flush path
> - This PR adds an `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard
at the entry point of `track()` to swallow unregistered calls before any
side effects, adds `scripts/extract-proposed-events.mjs` to scan source
for proposal markers and emit a v2 JSON manifest with provenance and
rationale, and documents the complete proposal workflow
> - The benefit is that new instrumentation can be proposed and reviewed
in code without touching the registered schema, and tooling can surface
missing rationale before events graduate to stable

## Linked Issues or Issue Description

No existing GitHub issue covers this change. This PR introduces a new
feature.

**Feature motivation:** Paperclip's telemetry schema is intentionally
stable — registered event names are code-generated and gated. Product
engineers who want to instrument a new behaviour today must land a
schema change first, creating a two-step process that slows iteration. A
proposal lane lets developers write the instrumentation call ahead of
schema registration, protected by a compile-time `@ts-expect-error`
marker that an extractor script can surface for review. This PR
implements both the client-side safety gate and the extraction tooling.

Refs: #9518 (closed predecessor — docs-only; this PR supersedes it with
the full implementation)

## What Changed

- Added `Object.hasOwn(PAPERCLIP_EVENTS, eventName)` guard at the top of
`TelemetryClient.track()`: unregistered event names (including
prototype-inherited keys) are now swallowed before any state, queue, or
network operation
- Added `scripts/extract-proposed-events.mjs`: scans TypeScript source
for `@ts-expect-error -- proposed-telemetry(<issue>): <rationale>`
markers; emits a v2 JSON manifest per proposed event including name,
rationale, provenance (repo-relative file + line), and a
`rationale_missing` flag for CI enforcement
- Added `scripts/extract-proposed-events.test.mjs`: test suite covering
marker parsing, multi-line markers, path validation, out-of-repo
rejection, and the v2 schema output contract
- Added `doc/TELEMETRY_WORKFLOW.md`: documents the proposal workflow,
the canonical multi-line marker example, rationale requirements, and how
to graduate a proposed event to stable schema
- Updated `packages/shared/src/telemetry/README.md`: added "Proposed
Events" section to the Telemetry Data Contract per the contributing
guide requirement for telemetry changes

## Verification

Run all of the following from the repo root:

```sh
# Extractor unit tests
node --test scripts/extract-proposed-events.test.mjs

# Telemetry client + types tests
pnpm exec vitest run --config vitest.config.ts \
  src/telemetry/client.test.ts src/telemetry/client-types.test.ts \
  --reporter=verbose
# (run from packages/shared)

# Type-check
pnpm --filter @paperclipai/shared typecheck

# Smoke-run the extractor in local-test mode
node scripts/extract-proposed-events.mjs --ref local-test
```

All four commands pass locally.

## Risks

- **Silent drop on unregistered events:** The `Object.hasOwn` guard
fails closed — any event name not in `PAPERCLIP_EVENTS` is silently
dropped. If the generated registry is missing an event that was
previously tracked, those calls will be silently lost. Mitigation: the
extractor script surfaces proposed events that need registration; the
TypeScript type system already enforces `TelemetryEventName ⊆
PAPERCLIP_EVENTS` at compile time.
- **Extractor is read-only:** `extract-proposed-events.mjs` reads source
and emits JSON; it does not modify any files. No runtime or schema risk.
- Overall risk: **low**. The guard is additive and defensive; the
extractor and docs are additive only.

## Model Used

- Provider: Anthropic
- Model ID: `claude-sonnet-4-6`
- Context window: 200 K tokens
- Capabilities: tool use, extended context, code generation

## Checklist

- [x] I have included a thinking path that traces from project context
to this change
- [x] I have specified the model used (with version and capability
details)
- [x] I have checked ROADMAP.md and confirmed this PR does not duplicate
planned core work
- [x] I have searched GitHub for duplicate or related PRs and linked
them above
- [x] I have either (a) linked existing issues with `Fixes: #` / `Closes
#` / `Refs #` OR (b) described the issue in-PR following the relevant
issue template
- [x] I have not referenced internal/instance-local Paperclip issues or
links (only public GitHub `#NNN` / `github.com/paperclipai/paperclip`
URLs)
- [x] My branch name describes the change (e.g. `docs/...`, `fix/...`)
and contains no internal Paperclip ticket id or instance-derived details
- [x] I have run tests locally and they pass
- [x] I have added or updated tests where applicable
- [x] I have updated relevant documentation to reflect my changes
- [x] I have considered and documented any risks above
- [ ] All Paperclip CI gates are green
- [ ] Greptile is 5/5 with no open P2s, recommendations, or follow-ups
- [x] I will address all Greptile and reviewer comments before
requesting merge

---------

Co-authored-by: Paperclip <noreply@paperclip.ing>
2026-07-13 23:47:27 -07:00

3.5 KiB

Telemetry Workflow

Paperclip first-party telemetry is schema-led for stable events and proposal-led for new product instrumentation.

Stable events must be present in packages/shared/src/telemetry/generated/paperclip-telemetry.ts before normal client code emits them. Proposed events may be added ahead of schema registration only with an @ts-expect-error proposal marker on the client.track() event-name argument.

Proposed Events

A proposed event is a normal client.track() call whose event name is not yet in PaperclipEventName. The runtime client swallows unregistered first-party event names before queueing, state initialization, or network flush, so proposed events do not leave the process until the generated telemetry schema adopts the event name.

Use this marker shape when possible:

import type { TelemetryClient } from "./client.js";

export function trackYourFeatureActionPerformed(
  client: TelemetryClient,
  dims: {
    action_source: "toolbar" | "menu" | (string & {});
    item_count: number;
  },
): void {
  client.track(
    // @ts-expect-error -- proposed-telemetry(https://github.com/paperclipai/paperclip/issues/123): measure feature action completion
    "your_feature.action_performed",
    dims,
  );
}

The multi-line shape is recommended because TypeScript places the unregistered-name error on the event-name line. When the schema later registers the event, that error disappears and TypeScript raises TS2578 for the now-unused directive, which tells the adopter to remove the marker in the same change that syncs the generated schema.

The suffix format is:

-- proposed-telemetry(<issue>): <rationale>

<issue> should be a public https://github.com/paperclipai/paperclip/issues/123 URL. The rationale should be a short product reason for collecting the event. Missing issue or rationale text is tolerated at the call site and flagged by scripts/extract-proposed-events.mjs; it is not an OSS CI failure.

These formatting conventions are documentation-only. Do not add repo-wide bans for @ts-expect-error, casts, or single-line calls as part of this workflow.

Extracting Proposals

Run the extractor from the repo root:

node scripts/extract-proposed-events.mjs --ref $(git rev-parse HEAD)

The extractor scans packages/shared/src/telemetry/events.ts for @ts-expect-error directives attached to <identifier>.track() event-name arguments inside telemetry wrapper functions, including function declarations and variable-assigned arrow/function expressions. It emits proposed-telemetry-extractor.v2 JSON with event names, primitive dimension names/types from the wrapper dims parameter type, rationale fields and missing-field flags, plus repo-relative file/line/column provenance.

Extractor provenance is deliberately repo-relative. Absolute paths, .. segments, Windows drive-letter paths, and backslash-separated paths are rejected so developer host paths cannot enter proposal inventory records.

Adoption

When a proposed event is approved and registered in the telemetry backend, sync the regenerated telemetry artifact into the OSS repo. The event name is then part of PaperclipEventName, so the proposal marker should fail with TS2578. Remove the marker and keep the wrapper payload aligned with the registered dimensions in the same change.

Old clients that do not yet have the synced schema continue to swallow the proposed event. Clients with the synced schema emit it through the normal stable telemetry path.