trending/treg
1
0
Fork 0
mirror of https://github.com/superdesigndev/treg.git synced 2026-10-02 03:24:35 +08:00
Code Issues Packages Projects Releases Wiki Activity

Merge origin/main into dev/hub: the tool hub, behind TREG_HUB_ENABLED

Browse Source 
Brings main's 86 commits (dashboard boot and loading, legacy dashboard removal, test pruning,
overflow and routing fixes) together with the tool hub branch.

Conflicts, both sides kept unless noted:
- the legacy dashboard stays deleted, as on main;
- App.vue and the dashboard state: main's search page and loading states plus the hub pages;
- ci.yml: main's Postgres job, with the hub tests added to its list;
- dev-local.sh: main's server environment plus the hub flag passthrough;
- test_call_application_contract.py, test_marketplace_call.py: main's pruned files plus the
  hub branch's sync `settle: usage` test.

Not conflicts: main and the hub branch fixed the same CompanyEnrich empty-page billing; main's
rule runs first, so the hub branch's copy and its test are dropped. The Listing-tab test reads
the Vue source instead of the deleted legacy page.
This commit is contained in:
UncleCode
2026-09-26 07:32:53 +08:00
parent 19d5fa7d4b a5513dac17
commit 3ee1948c64
377 changed files with 8678 additions and 24330 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.agents/skills/tools-registry-context/MAP.md
+32 -13
View File
@@ -33,6 +33,8 @@ Regenerate via `scripts/build-map.py`.
| `frontend/src/App.vue` | interface/dashboard.md, interface/seo.md |
| `frontend/src/api.ts` | interface/dashboard.md |
| `frontend/src/components/DashboardNavigation.vue` | interface/dashboard.md |
| `frontend/src/components/FindAnswer.vue` | interface/dashboard.md |
| `frontend/src/components/LandingNavigation.vue` | interface/dashboard.md |
| `frontend/src/components/PublicNavigation.vue` | interface/dashboard.md, interface/seo.md |
| `frontend/src/components/SignInDialog.vue` | interface/dashboard.md |
| `frontend/src/components/SignedOutPage.vue` | interface/dashboard.md, interface/landing-sandbox.md |
@@ -66,6 +68,7 @@ Regenerate via `scripts/build-map.py`.
| `frontend/src/pages/PlatformPage.vue` | interface/dashboard.md |
| `frontend/src/pages/ProviderPage.vue` | architecture/instagram-oauth.md, interface/dashboard.md |
| `frontend/src/pages/ReferralsPage.vue` | interface/dashboard.md |
| `frontend/src/pages/SearchPage.vue` | interface/dashboard.md |
| `frontend/src/pages/SecretsPage.vue` | interface/dashboard.md |
| `frontend/src/pages/TeamPage.vue` | architecture/auth-secrets.md, interface/dashboard.md |
| `frontend/src/pages/TeamResourcesPage.vue` | interface/dashboard.md |
@@ -87,6 +90,8 @@ Regenerate via `scripts/build-map.py`.
| `frontend/src/state/data.js` | interface/dashboard.md |
| `frontend/src/state/details.js` | interface/dashboard.md |
| `frontend/src/state/detailsComputed.js` | interface/dashboard.md |
| `frontend/src/state/find.js` | interface/dashboard.md |
| `frontend/src/state/findComputed.js` | interface/dashboard.md |
| `frontend/src/state/format.js` | interface/dashboard.md |
| `frontend/src/state/governance.js` | interface/dashboard.md |
| `frontend/src/state/help.js` | interface/dashboard.md |
@@ -96,6 +101,7 @@ Regenerate via `scripts/build-map.py`.
| `frontend/src/state/navigation.js` | interface/dashboard.md |
| `frontend/src/state/onboarding.js` | interface/dashboard.md, interface/onboarding.md |
| `frontend/src/state/onboardingComputed.js` | interface/dashboard.md |
| `frontend/src/state/pile.ts` | interface/dashboard.md |
| `frontend/src/state/projects.js` | interface/dashboard.md |
| `frontend/src/state/referrals.js` | interface/dashboard.md |
| `frontend/src/state/resources.js` | interface/dashboard.md |
@@ -194,6 +200,7 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/application/billing.py` | architecture/money.md |
| `src/treg/application/call/__init__.py` | architecture/import-boundaries.md |
| `src/treg/application/call/access.py` | architecture/import-boundaries.md, architecture/instagram-oauth.md, interface/api.md |
| `src/treg/application/call/async_bridge.py` | architecture/catalog.md, architecture/money.md |
| `src/treg/application/call/authorize.py` | architecture/import-boundaries.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/evidence.py` | architecture/import-boundaries.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/idempotency.py` | architecture/import-boundaries.md, architecture/money.md, architecture/proxy-model.md, interface/api.md |
@@ -202,12 +209,14 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/application/call/overflow.py` | architecture/import-boundaries.md, ops/capacity.md |
| `src/treg/application/call/reserve.py` | architecture/import-boundaries.md, architecture/money.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/resolve.py` | architecture/import-boundaries.md, architecture/instagram-oauth.md, architecture/money.md, architecture/multi-tenancy.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/route.py` | architecture/catalog.md, architecture/import-boundaries.md |
| `src/treg/application/call/route.py` | architecture/catalog.md, architecture/import-boundaries.md, architecture/money.md |
| `src/treg/application/call/service.py` | architecture/archive.md, architecture/hub.md, architecture/import-boundaries.md, architecture/instagram-oauth.md, architecture/money.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/settle.py` | architecture/archive.md, architecture/import-boundaries.md, architecture/money.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/call/types.py` | architecture/import-boundaries.md, architecture/proxy-model.md, interface/api.md |
| `src/treg/application/catalog_find.py` | architecture/search-experiment.md |
| `src/treg/application/catalog_stats.py` | architecture/catalog.md |
| `src/treg/application/connect.py` | architecture/auth-secrets.md, architecture/composition.md, guides/expanding-a-category.md, interface/api.md |
| `src/treg/application/evidence_retention.py` | architecture/super-admin.md |
| `src/treg/application/feedback.py` | architecture/feedback.md |
| `src/treg/application/hub/__init__.py` | architecture/hub.md |
| `src/treg/application/hub/health.py` | architecture/hub.md |
@@ -258,6 +267,13 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/catalog/examples/adyntel.meta-ads.library.advertiser.json` | architecture/catalog.md |
| `src/treg/catalog/examples/adyntel.meta-ads.library.search.json` | architecture/catalog.md |
| `src/treg/catalog/examples/adyntel.tiktok-ads.library.search.company.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.company.profile.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.post.comments.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.post.engagement.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.post.reactions.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.user.posts.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.user.profile.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fetchinio.linkedin.user.reactions.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fishaudio.tts.s2-1-pro.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fishaudio.voices.create.json` | architecture/catalog.md |
| `src/treg/catalog/examples/fishaudio.voices.discover.json` | architecture/catalog.md |
@@ -268,6 +284,11 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/catalog/examples/tinyfish.web.search.json` | architecture/catalog.md |
| `src/treg/catalog/examples/tinyfish.web.search.news.json` | architecture/catalog.md |
| `src/treg/catalog/examples/tinyfish.web.search.publications.json` | architecture/catalog.md |
| `src/treg/catalog/examples/wiza.people.email.find.json` | architecture/catalog.md |
| `src/treg/catalog/examples/wiza.people.email.find.terminal.json` | architecture/catalog.md |
| `src/treg/catalog/examples/wiza.people.phone.find.json` | architecture/catalog.md |
| `src/treg/catalog/examples/wiza.people.phone.find.terminal.json` | architecture/catalog.md |
| `src/treg/catalog/fetchinio.yaml` | architecture/catalog.md |
| `src/treg/catalog/financialdatasets.yaml` | architecture/catalog.md |
| `src/treg/catalog/fishaudio.yaml` | architecture/catalog.md |
| `src/treg/catalog/fx.yaml` | architecture/catalog.md |
@@ -305,6 +326,7 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/catalog/tomba.yaml` | architecture/catalog.md, architecture/money.md |
| `src/treg/catalog/trestleiq.yaml` | architecture/catalog.md |
| `src/treg/catalog/trykitt.yaml` | architecture/catalog.md |
| `src/treg/catalog/wiza.yaml` | architecture/catalog.md |
| `src/treg/cli.py` | architecture/hub.md, architecture/instagram-oauth.md, interface/cli.md, interface/onboarding.md, interface/shell.md |
| `src/treg/cli_analytics.py` | interface/cli.md |
| `src/treg/client_identity.py` | architecture/import-boundaries.md, architecture/proxy-model.md, interface/api.md |
@@ -437,7 +459,6 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/web/catalog.css` | interface/seo.md |
| `src/treg/web/claude-connector.html` | architecture/mcp-oauth.md |
| `src/treg/web/connect-demo.html` | architecture/mcp-oauth.md |
| `src/treg/web/dashboard-legacy/README.md` | interface/dashboard.md |
| `src/treg/web/enrich-arena.html` | interface/enrich-arena.md |
| `src/treg/web/enrich-arena/arena.css` | interface/enrich-arena.md |
| `src/treg/web/enrich-arena/arena.js` | interface/enrich-arena.md |
@@ -455,6 +476,7 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/web/logos/apollo.svg` | interface/enrich-arena.md |
| `src/treg/web/logos/branddev.svg` | interface/enrich-arena.md |
| `src/treg/web/logos/companyenrich.svg` | interface/enrich-arena.md |
| `src/treg/web/logos/fetchinio.svg` | architecture/catalog.md |
| `src/treg/web/logos/findymail.svg` | interface/enrich-arena.md |
| `src/treg/web/logos/hunter.svg` | interface/enrich-arena.md |
| `src/treg/web/logos/icypeas.svg` | interface/enrich-arena.md |
@@ -509,9 +531,6 @@ Regenerate via `scripts/build-map.py`.
| `src/treg/worker.py` | architecture/hub.md, ops/capacity.md, ops/deploy.md |
| `tests/callmatrix/test_hub_run.py` | architecture/hub.md |
| `tests/fake_object_store.py` | architecture/archive.md |
| `tests/js/arena-bench.test.cjs` | interface/enrich-arena.md |
| `tests/js/arena-template.test.cjs` | interface/enrich-arena.md |
| `tests/js/enrich-arena.test.cjs` | interface/enrich-arena.md |
| `tests/test_aigc_pr_b.py` | architecture/catalog.md |
| `tests/test_alembic_expand_safety.py` | architecture/data-model.md |
| `tests/test_api_keys.py` | architecture/auth-secrets.md, architecture/data-model.md, interface/cli.md, interface/dashboard.md |
@@ -534,10 +553,9 @@ Regenerate via `scripts/build-map.py`.
| `tests/test_capacity_protect.py` | ops/capacity.md |
| `tests/test_capacity_smoothing.py` | ops/capacity.md |
| `tests/test_catalog_api.py` | architecture/catalog.md |
| `tests/test_catalog_find.py` | architecture/search-experiment.md |
| `tests/test_catalog_validate.py` | architecture/catalog.md |
| `tests/test_cli_key_compatibility.py` | interface/cli.md |
| `tests/test_dashboard_rollout.py` | interface/dashboard.md |
| `tests/test_dataforseo_constraints.py` | architecture/catalog.md |
| `tests/test_enrich_arena.py` | interface/enrich-arena.md |
| `tests/test_error_capture.py` | architecture/proxy-model.md |
| `tests/test_feedback.py` | architecture/feedback.md |
@@ -574,6 +592,7 @@ Regenerate via `scripts/build-map.py`.
| `tests/test_team_limit.py` | architecture/multi-tenancy.md |
| `tests/test_tinyfish.py` | architecture/catalog.md, architecture/money.md, ops/capacity.md |
| `tests/test_token_revocation.py` | architecture/multi-tenancy.md |
| `tests/test_wiza.py` | architecture/catalog.md |
## Fragment → sources
@@ -582,7 +601,7 @@ Regenerate via `scripts/build-map.py`.
| `architecture/ads-conversions.md` | `adsconv.py`, `signup.py`, `adtrack.js`, `gtag.js` |
| `architecture/archive.md` | `archive.py`, `hunter.yaml`, `results.py`, `0031_archive_result_admission.py`, `test_cache_result_admission.py`, `archive_bodies.py`, `config.py`, `object_store.py`, `0032_archive_body_storage.py`, `test_archive_r2.py`, `fake_object_store.py`, `smoke_archive_r2.py`, `0002_archive_tables.py`, `0003_callrecord_cached.py`, `0004_archivekey_request_shape.py`, `0011_callrecord_archive_link.py`, `service.py`, `settle.py`, `0039_archive_own_key_and_repeat_pricing.py`, `backfill_call_archive_links.py`, `api.py`, `bootstrap.py`, `admin.py`, `asynctasks.py` |
| `architecture/auth-secrets.md` | `injectors.py`, `ssrf.py`, `crypto.py`, `oauth.py`, `__init__.py`, `authorization.py`, `oauth_flow.py`, `refresh.py`, `oauth_exchange.py`, `oauth_refresh.py`, `oauth_providers.py`, `session.js`, `keys.js`, `TeamPage.vue`, `health.py`, `connect.py`, `connections.py`, `resources.py`, `__init__.py`, `bindings.py`, `bundles.py`, `api_keys.py`, `access.py`, `api_keys.py`, `test_api_keys.py`, `test_oauth_refresh.py`, `test_financialdatasets.py`, `test_key_providers.py`, `config.py` |
| `architecture/catalog.md` | `fishaudio.yaml`, `fishaudio.tts.s2-1-pro.json`, `fishaudio.voices.create.json`, `fishaudio.voices.discover.json`, `provider_resources.py`, `provider_resources.py`, `provider_resources.py`, `tavily.yaml`, `keenable.yaml`, `olostep.yaml`, `tinyfish.yaml`, `tinyfish.web.search.json`, `tinyfish.web.search.news.json`, `tinyfish.web.search.publications.json`, `tinyfish.web.fetch.json`, `tinyfish.web.agent.run.json`, `tinyfish.web.agent.run.get.json`, `tinyfish.web.agent.run.cancel.json`, `test_tinyfish.py`, `exa.yaml`, `anyapi.extended.yaml`, `adyntel.yaml`, `adyntel.meta-ads.library.advertiser.json`, `adyntel.meta-ads.library.search.json`, `adyntel.linkedin.search.ads.company.json`, `adyntel.linkedin.search.ads.keyword.json`, `adyntel.google.ads.transparency.json`, `adyntel.tiktok-ads.library.search.company.json`, `adyntel.google.domain.keywords.overview.json`, `adyntel.svg`, `trestleiq.yaml`, `financialdatasets.yaml`, `test_financialdatasets.py`, `quickenrich.yaml`, `influencersclub.yaml`, `quickenrich.extended.yaml`, `trykitt.yaml`, `contracts.yaml`, `millionverifier.yaml`, `adapters.yaml`, `prospeo.yaml`, `test_route_cost_ceiling.py`, `tomba.yaml`, `__init__.py`, `contracts.py`, `paths.py`, `plan.py`, `synthetic.py`, `route.py`, `test_routing.py`, `catalog-drift.yml`, `catalog_drift.py`, `catalog_ingest.py`, `catalog_validate.py`, `aliases.yaml`, `fx.yaml`, `cloro.yaml`, `aviato.yaml`, `crustdata.yaml`, `google-search-console.yaml`, `google-search-console.extended.yaml`, `google-tag-manager.yaml`, `google-tag-manager.extended.yaml`, `instagram.yaml`, `instagram.extended.yaml`, `justoneapi.extended.yaml`, `minimax.yaml`, `apify.yaml`, `brightdata.yaml`, `companyenrich.yaml`, `oceanio.yaml`, `akta.extended.yaml`, `dataforseo.yaml`, `dataforseo.extended.yaml`, `test_dataforseo_constraints.py`, `scrapecreators.yaml`, `scrapecreators.extended.yaml`, `serpapi.yaml`, `serpapi.extended.yaml`, `diffbot.yaml`, `diffbot.extended.yaml`, `tikhub.extended.yaml`, `lusha.extended.yaml`, `openrouter.yaml`, `openrouter.extended.yaml`, `replicate.yaml`, `replicate.extended.yaml`, `reapi.yaml`, `piapi.yaml`, `__init__.py`, `store.py`, `hunter.yaml`, `mcp.py`, `settlement.py`, `stats.py`, `catalog_observations.py`, `catalog_stats.py`, `0038_endpoint_day_stats.py`, `catalog.py`, `test_aigc_pr_b.py`, `test_catalog_api.py`, `test_catalog_validate.py` |
| `architecture/catalog.md` | `fetchinio.yaml`, `fetchinio.svg`, `fetchinio.linkedin.user.profile.json`, `fetchinio.linkedin.company.profile.json`, `fetchinio.linkedin.user.posts.json`, `fetchinio.linkedin.user.reactions.json`, `fetchinio.linkedin.post.comments.json`, `fetchinio.linkedin.post.reactions.json`, `fetchinio.linkedin.post.engagement.json`, `fishaudio.yaml`, `fishaudio.tts.s2-1-pro.json`, `fishaudio.voices.create.json`, `fishaudio.voices.discover.json`, `provider_resources.py`, `provider_resources.py`, `provider_resources.py`, `tavily.yaml`, `keenable.yaml`, `olostep.yaml`, `tinyfish.yaml`, `tinyfish.web.search.json`, `tinyfish.web.search.news.json`, `tinyfish.web.search.publications.json`, `tinyfish.web.fetch.json`, `tinyfish.web.agent.run.json`, `tinyfish.web.agent.run.get.json`, `tinyfish.web.agent.run.cancel.json`, `test_tinyfish.py`, `exa.yaml`, `anyapi.extended.yaml`, `adyntel.yaml`, `adyntel.meta-ads.library.advertiser.json`, `adyntel.meta-ads.library.search.json`, `adyntel.linkedin.search.ads.company.json`, `adyntel.linkedin.search.ads.keyword.json`, `adyntel.google.ads.transparency.json`, `adyntel.tiktok-ads.library.search.company.json`, `adyntel.google.domain.keywords.overview.json`, `adyntel.svg`, `trestleiq.yaml`, `financialdatasets.yaml`, `test_financialdatasets.py`, `quickenrich.yaml`, `influencersclub.yaml`, `quickenrich.extended.yaml`, `trykitt.yaml`, `contracts.yaml`, `millionverifier.yaml`, `adapters.yaml`, `prospeo.yaml`, `test_route_cost_ceiling.py`, `tomba.yaml`, `__init__.py`, `contracts.py`, `paths.py`, `plan.py`, `synthetic.py`, `async_bridge.py`, `route.py`, `wiza.yaml`, `wiza.people.email.find.json`, `wiza.people.email.find.terminal.json`, `wiza.people.phone.find.json`, `wiza.people.phone.find.terminal.json`, `test_routing.py`, `test_wiza.py`, `catalog-drift.yml`, `catalog_drift.py`, `catalog_ingest.py`, `catalog_validate.py`, `aliases.yaml`, `fx.yaml`, `cloro.yaml`, `aviato.yaml`, `crustdata.yaml`, `google-search-console.yaml`, `google-search-console.extended.yaml`, `google-tag-manager.yaml`, `google-tag-manager.extended.yaml`, `instagram.yaml`, `instagram.extended.yaml`, `justoneapi.extended.yaml`, `minimax.yaml`, `apify.yaml`, `brightdata.yaml`, `companyenrich.yaml`, `oceanio.yaml`, `akta.extended.yaml`, `dataforseo.yaml`, `dataforseo.extended.yaml`, `scrapecreators.yaml`, `scrapecreators.extended.yaml`, `serpapi.yaml`, `serpapi.extended.yaml`, `diffbot.yaml`, `diffbot.extended.yaml`, `tikhub.extended.yaml`, `lusha.extended.yaml`, `openrouter.yaml`, `openrouter.extended.yaml`, `replicate.yaml`, `replicate.extended.yaml`, `reapi.yaml`, `piapi.yaml`, `__init__.py`, `store.py`, `hunter.yaml`, `mcp.py`, `settlement.py`, `stats.py`, `catalog_observations.py`, `catalog_stats.py`, `0038_endpoint_day_stats.py`, `catalog.py`, `test_aigc_pr_b.py`, `test_catalog_api.py`, `test_catalog_validate.py` |
| `architecture/composition.md` | `bootstrap.py`, `bootstrap_handlers.py`, `bootstrap_http.py`, `call_surface.py`, `connect.py`, `mcp_oauth.py`, `session.py`, `admin.py`, `auth.py`, `billing.py`, `call.py`, `connections.py`, `onboard.py`, `orgs.py`, `resources.py`, `referrals.py`, `web.py`, `dump_surface.py`, `test_app_roles.py` |
| `architecture/data-model.md` | `0042_pinned_read_scope.py`, `alembic.ini`, `env.py`, `0001_baseline_current_schema.py`, `0002_archive_tables.py`, `0003_callrecord_cached.py`, `0004_archivekey_request_shape.py`, `0005_capacity_policy_snapshot.py`, `0006_overflow_route.py`, `0007_overflow_spend.py`, `0008_org_platform_overflow_disabled.py`, `0009_callrecord_hit.py`, `0017_async_task_record.py`, `0018_async_resource_ownership.py`, `0019_async_poll_failures.py`, `0020_callrecord_created_at_indexes.py`, `0021_ledgerentry_org_created_at_index.py`, `0022_org_spent_today_counter.py`, `0023_callrecord_org_user_created_at_index.py`, `0024_membership_calls_today_counter.py`, `0027_enrich_arena.py`, `0028_arena_insights.py`, `0029_arena_verification_snapshot.py`, `0011_callrecord_archive_link.py`, `0015_idempotentcall_membership_cascade.py`, `0034_managed_api_keys.py`, `0035_default_key_generation.py`, `0036_activity_key_indexes.py`, `0038_endpoint_day_stats.py`, `maintenance.py`, `sitetrack.js`, `models.py`, `0031_archive_result_admission.py`, `0032_archive_body_storage.py`, `0039_archive_own_key_and_repeat_pricing.py`, `0043_provider_resources.py`, `provider_resources.py`, `provider_resources.py`, `0033_signup_promo_eligibility.py`, `0041_searchlog.py`, `timeutil.py`, `db.py`, `referrals.py`, `audit.py`, `analytics.py`, `bootstrap_handlers.py`, `ratestore.py`, `auth.py`, `test_postgres_reset.py`, `test_alembic_expand_safety.py`, `test_api_keys.py` |
| `architecture/feedback.md` | `feedback_contract.py`, `__init__.py`, `reports.py`, `reviews.py`, `hints.py`, `config.py`, `call.py`, `invite.py`, `kv.py`, `feedback.py`, `feedback.py`, `0025_feedback.py`, `0026_callreview.py`, `0030_feedback_handling.py`, `test_feedback_handling_schema.py`, `feedback.md`, `test_feedback.py`, `test_reviews.py`, `test_hints.py`, `test_kv.py` |
@@ -593,18 +612,18 @@ Regenerate via `scripts/build-map.py`.
| `architecture/local-run.md` | `localrun.py`, `egress.py`, `fsjail.py` |
| `architecture/mcp-oauth.md` | `auth.py`, `mcp.py`, `health.py`, `mcp_oauth.py`, `session.py`, `access.py`, `api_keys.py`, `api_keys.py`, `auth.py`, `claude-connector.html`, `connect-demo.html`, `CLAUDE-CONNECTOR-SUBMISSION.md`, `test_mcp.py`, `test_mcp_oauth.py`, `test_mcp_directory.py`, `test_marketplace_call.py` |
| `architecture/media.md` | `media.py`, `media.py`, `models.py`, `0037_media_hosting.py`, `test_media.py` |
| `architecture/money.md` | `tavily.yaml`, `tinyfish.yaml`, `test_tinyfish.py`, `__init__.py`, `settlement.py`, `__init__.py`, `models.py`, `billing.py`, `idempotency.py`, `intake.py`, `resolve.py`, `service.py`, `reserve.py`, `settle.py`, `tomba.yaml`, `asynctasks.py`, `0017_async_task_record.py`, `0018_async_resource_ownership.py`, `0019_async_poll_failures.py`, `referrals.py`, `budgets.py`, `__init__.py`, `stripe.py`, `reconcile.py`, `referrals.py`, `api.py`, `signup.py`, `promotions.py`, `0033_signup_promo_eligibility.py`, `admin.py`, `billing.py`, `call.py`, `orgs.py`, `referrals.py`, `test_call_architecture.py`, `test_marketplace_call.py`, `test_asynctasks.py` |
| `architecture/money.md` | `tavily.yaml`, `tinyfish.yaml`, `test_tinyfish.py`, `__init__.py`, `settlement.py`, `__init__.py`, `models.py`, `billing.py`, `idempotency.py`, `intake.py`, `resolve.py`, `service.py`, `async_bridge.py`, `route.py`, `reserve.py`, `settle.py`, `tomba.yaml`, `asynctasks.py`, `0017_async_task_record.py`, `0018_async_resource_ownership.py`, `0019_async_poll_failures.py`, `referrals.py`, `budgets.py`, `__init__.py`, `stripe.py`, `reconcile.py`, `referrals.py`, `api.py`, `signup.py`, `promotions.py`, `0033_signup_promo_eligibility.py`, `admin.py`, `billing.py`, `call.py`, `orgs.py`, `referrals.py`, `test_call_architecture.py`, `test_marketplace_call.py`, `test_asynctasks.py` |
| `architecture/multi-tenancy.md` | `access.py`, `0042_pinned_read_scope.py`, `test_pinned_read_scope.py`, `models.py`, `api.py`, `caller_metadata.py`, `auth.py`, `asynctasks.py`, `resolve.py`, `provider_resources.py`, `provider_resources.py`, `provider_resources.py`, `0043_provider_resources.py`, `signup.py`, `access.py`, `budgets.py`, `publicdemo.py`, `teams.py`, `usage.py`, `access.py`, `api_keys.py`, `session.py`, `promotions.py`, `test_team_limit.py`, `test_auth.py`, `test_token_revocation.py`, `auth.py`, `orgs.py`, `resources.py`, `bundles.py`, `db.py`, `0017_async_task_record.py`, `0018_async_resource_ownership.py`, `test_router_dependencies.py`, `test_asynctasks.py` |
| `architecture/proxy-model.md` | `relay.py`, `ssrf.py`, `api.py`, `authorize.py`, `idempotency.py`, `intake.py`, `resolve.py`, `reserve.py`, `settle.py`, `evidence.py`, `service.py`, `types.py`, `asynctasks.py`, `client_identity.py`, `call_surface.py`, `sandbox_identity.py`, `access.py`, `publicdemo.py`, `usage.py`, `call.py`, `test_ssrf_public_addresses.py`, `test_call_application_contract.py`, `test_call_cancellation.py`, `test_call_response_limits.py`, `test_error_capture.py`, `test_marketplace_call.py`, `test_oauth_billed.py`, `test_passthrough.py`, `test_tag_billing.py`, `test_tag_billing_adversarial.py`, `test_call_architecture.py`, `test_asynctasks.py`, `test_relay_content_length.py` |
| `architecture/search-experiment.md` | `search_experiment.py`, `interleave.py`, `judge.py`, `0041_searchlog.py`, `search_experiment_report.sql`, `test_search_experiment.py` |
| `architecture/super-admin.md` | `api.py`, `admin.py`, `access.py`, `config.py` |
| `architecture/search-experiment.md` | `search_experiment.py`, `interleave.py`, `judge.py`, `0041_searchlog.py`, `search_experiment_report.sql`, `test_search_experiment.py`, `catalog_find.py`, `test_catalog_find.py` |
| `architecture/super-admin.md` | `api.py`, `admin.py`, `evidence_retention.py`, `access.py`, `config.py` |
| `foundation/charter.md` | `2026-06-30-jason-tools-registry.md`, `README.md` |
| `guides/expanding-a-category.md` | `oauth_providers.py`, `authorization.py`, `oauth_flow.py`, `oauth_exchange.py`, `connect.py`, `connections.py`, `config.py` |
| `interface/api.md` | `media.py`, `sitetrack.js`, `api.py`, `bootstrap_handlers.py`, `bootstrap_http.py`, `call_surface.py`, `caller_metadata.py`, `client_identity.py`, `auth.py`, `provider_resources.py`, `access.py`, `authorize.py`, `idempotency.py`, `intake.py`, `resolve.py`, `reserve.py`, `settle.py`, `evidence.py`, `service.py`, `types.py`, `relay.py`, `connect.py`, `onboard.py`, `referrals.py`, `signup.py`, `__init__.py`, `admin.py`, `auth.py`, `auth_helpers.py`, `billing.py`, `call.py`, `catalog.py`, `connections.py`, `onboard.py`, `orgs.py`, `provider_resources.py`, `api_keys.py`, `resources.py`, `referrals.py`, `signup_cookies.py`, `web.py`, `access.py`, `api_keys.py`, `teams.py`, `access.py`, `budgets.py`, `publicdemo.py`, `usage.py`, `mcp_oauth.py`, `session.py`, `timeutil.py`, `store.py`, `email.py`, `runner.py`, `ratestore.py` |
| `interface/catalog-review-proposal.md` | `store.py`, `capabilities.yaml` |
| `interface/cli.md` | `cli.py`, `test_released_cli_compat.py`, `test_cli_key_compatibility.py`, `auth_helpers.py`, `cli_analytics.py`, `convert.py`, `agents.py`, `api_keys.py`, `test_api_keys.py` |
| `interface/dashboard.md` | `sitetrack.js`, `index.html`, `package.json`, `vite.config.ts`, `TeamResourcesPage.vue`, `FishVoiceDialog.vue`, `resources.js`, `resourcesComputed.js`, `App.vue`, `api.ts`, `DashboardNavigation.vue`, `PublicNavigation.vue`, `SignInDialog.vue`, `SignedOutPage.vue`, `AcceptInvitesDialog.vue`, `AgentGuideDialog.vue`, `CallDetailsDialog.vue`, `ConnectTokenDialog.vue`, `ConnectionMethodDialog.vue`, `CopyToolDialog.vue`, `EditToolDialog.vue`, `ExtraCredentialDialog.vue`, `ImportSkillDialog.vue`, `RecipeDialog.vue`, `RequestToolDialog.vue`, `ResourcePickerDialog.vue`, `RunToolDialog.vue`, `ShareDialog.vue`, `TopUpDialog.vue`, `TryEndpointDialog.vue`, `WelcomeDialog.vue`, `main.ts`, `ActivityPage.vue`, `AdminPage.vue`, `CatalogPage.vue`, `DetailPage.vue`, `GettingStartedPage.vue`, `HelpPage.vue`, `PlatformPage.vue`, `ProviderPage.vue`, `ReferralsPage.vue`, `SecretsPage.vue`, `TeamPage.vue`, `ToolsPage.vue`, `activity.js`, `admin.js`, `agents.js`, `agentsComputed.js`, `analytics.js`, `billing.js`, `billingComputed.js`, `boot.js`, `catalog.js`, `catalogComputed.js`, `connections.js`, `constants.js`, `context.ts`, `controller.js`, `data.js`, `details.js`, `detailsComputed.js`, `format.js`, `governance.js`, `help.js`, `keys.js`, `lifecycle.js`, `navigation.js`, `onboarding.js`, `onboardingComputed.js`, `projects.js`, `referrals.js`, `secrets.js`, `session.js`, `sessionComputed.js`, `sharing.js`, `skills.js`, `snippets.js`, `team.js`, `tools.js`, `tryTool.js`, `base.css`, `agent-setup.js`, `dashboard.css`, `SOURCES.md`, `README.md`, `copy-runtime.mjs`, `tutorial.js`, `tutorial.html`, `tour.js`, `index.html`, `api.py`, `test_dashboard_rollout.py`, `README.md`, `web.py`, `session.py`, `api_keys.py`, `test_api_keys.py` |
| `interface/enrich-arena.md` | `arena.py`, `arena.py`, `arena.py`, `models.py`, `0027_enrich_arena.py`, `teams.py`, `auth.py`, `bootstrap.py`, `enrich-arena.html`, `arena.js`, `bench.js`, `arena-bench.test.cjs`, `arena-template.test.cjs`, `arena.css`, `agent-setup.js`, `arena_verification_insights.py`, `0029_arena_verification_snapshot.py`, `import_arena_verification.py`, `test_arena_verification_insights.py`, `arena_insights.py`, `arena_insights.py`, `0028_arena_insights.py`, `test_arena_insights.py`, `apollo.svg`, `branddev.svg`, `companyenrich.svg`, `findymail.svg`, `hunter.svg`, `icypeas.svg`, `leadmagic.svg`, `leadsforge.svg`, `lusha.svg`, `pdl.svg`, `predictleads.svg`, `thecompaniesapi.svg`, `tomba.svg`, `sitetrack.js`, `test_enrich_arena.py`, `enrich-arena.test.cjs` |
| `interface/dashboard.md` | `sitetrack.js`, `index.html`, `package.json`, `vite.config.ts`, `TeamResourcesPage.vue`, `FishVoiceDialog.vue`, `resources.js`, `resourcesComputed.js`, `App.vue`, `api.ts`, `DashboardNavigation.vue`, `PublicNavigation.vue`, `SignInDialog.vue`, `SignedOutPage.vue`, `AcceptInvitesDialog.vue`, `AgentGuideDialog.vue`, `CallDetailsDialog.vue`, `ConnectTokenDialog.vue`, `ConnectionMethodDialog.vue`, `CopyToolDialog.vue`, `EditToolDialog.vue`, `ExtraCredentialDialog.vue`, `ImportSkillDialog.vue`, `RecipeDialog.vue`, `RequestToolDialog.vue`, `ResourcePickerDialog.vue`, `RunToolDialog.vue`, `ShareDialog.vue`, `TopUpDialog.vue`, `TryEndpointDialog.vue`, `WelcomeDialog.vue`, `main.ts`, `ActivityPage.vue`, `AdminPage.vue`, `CatalogPage.vue`, `DetailPage.vue`, `GettingStartedPage.vue`, `HelpPage.vue`, `PlatformPage.vue`, `ProviderPage.vue`, `ReferralsPage.vue`, `SecretsPage.vue`, `TeamPage.vue`, `ToolsPage.vue`, `activity.js`, `admin.js`, `agents.js`, `agentsComputed.js`, `analytics.js`, `billing.js`, `billingComputed.js`, `boot.js`, `catalog.js`, `catalogComputed.js`, `find.js`, `findComputed.js`, `pile.ts`, `FindAnswer.vue`, `SearchPage.vue`, `LandingNavigation.vue`, `connections.js`, `constants.js`, `context.ts`, `controller.js`, `data.js`, `details.js`, `detailsComputed.js`, `format.js`, `governance.js`, `help.js`, `keys.js`, `lifecycle.js`, `navigation.js`, `onboarding.js`, `onboardingComputed.js`, `projects.js`, `referrals.js`, `secrets.js`, `session.js`, `sessionComputed.js`, `sharing.js`, `skills.js`, `snippets.js`, `team.js`, `tools.js`, `tryTool.js`, `base.css`, `agent-setup.js`, `dashboard.css`, `SOURCES.md`, `README.md`, `copy-runtime.mjs`, `tutorial.js`, `tutorial.html`, `tour.js`, `index.html`, `api.py`, `web.py`, `session.py`, `api_keys.py`, `test_api_keys.py` |
| `interface/enrich-arena.md` | `arena.py`, `arena.py`, `arena.py`, `models.py`, `0027_enrich_arena.py`, `teams.py`, `auth.py`, `bootstrap.py`, `enrich-arena.html`, `arena.js`, `bench.js`, `arena.css`, `agent-setup.js`, `arena_verification_insights.py`, `0029_arena_verification_snapshot.py`, `import_arena_verification.py`, `test_arena_verification_insights.py`, `arena_insights.py`, `arena_insights.py`, `0028_arena_insights.py`, `test_arena_insights.py`, `apollo.svg`, `branddev.svg`, `companyenrich.svg`, `findymail.svg`, `hunter.svg`, `icypeas.svg`, `leadmagic.svg`, `leadsforge.svg`, `lusha.svg`, `pdl.svg`, `predictleads.svg`, `thecompaniesapi.svg`, `tomba.svg`, `sitetrack.js`, `test_enrich_arena.py` |
| `interface/env-import.md` | `providers.py`, `skills.py` |
| `interface/landing-sandbox.md` | `sandbox.py`, `sandbox_identity.py`, `pubfeed.py`, `sandbox.py`, `__init__.py`, `sandbox.py`, `api.py`, `onboard.py`, `web.py`, `boot.js`, `SignedOutPage.vue`, `install.sh` |
| `interface/onboarding.md` | `auth.py`, `__init__.py`, `demo.py`, `cli.py`, `auth.py`, `onboard.py`, `onboarding.js`, `WelcomeDialog.vue`, `GettingStartedPage.vue` |
.claude-plugin/plugin.json
+1 -1
View File
@@ -2,7 +2,7 @@
"$schema": "https://json.schemastore.org/claude-code-plugin-manifest.json",
"name": "treg",
"displayName": "treg",
"version": "0.21.2",
"version": "0.21.3",
"description": "OpenRouter for tools - 2,896 agent-friendly tools, pay for the usage, not subscription",
"author": {
"name": "Superdesign dev, Inc.",
.cursor-plugin/marketplace.json
+1 -1
View File
@@ -6,7 +6,7 @@
},
"metadata": {
"description": "OpenRouter for tools - 2,896 agent-friendly tools, pay for the usage, not subscription",
"version": "0.21.2"
"version": "0.21.3"
},
"plugins": [
{
.github/workflows/ci.yml
+14 -19
View File
@@ -65,8 +65,7 @@ jobs:
if: steps.docs.outputs.only != 'true'
# -n auto: the suite is 2,700+ small independent tests — serial execution is what let a
# throttled runner turn minutes into tens of minutes. xdist arrives via --with rather than
# the lockfile on purpose (local daily default uses the same --with). The Postgres job
# below must stay serial: workers would share one database while reset_db() drops tables.
# the lockfile on purpose (local daily default uses the same --with).
run: uv run --with pytest-xdist pytest -n auto -v -o faulthandler_timeout=300
test-postgres:
@@ -103,14 +102,14 @@ jobs:
run: bash scripts/build-dashboard.sh
- name: Install dependencies
run: uv sync --locked
- name: Run Postgres tests serially
# Never add xdist here. Every worker would share this one database while reset_db() drops
# and recreates its tables, reproducing the 1,022-error cross-worker failure shape.
- name: Run Postgres tests
# Each xdist worker gets its own database (treg_test_gw0, ...; see tests/conftest.py), so
# reset_db() in one worker never empties another's tables.
# -v + faulthandler: this job has twice been killed at its time limit with zero output on
# this runner while the identical list passes locally in 2 minutes. Per-test lines locate
# the stall; the fault handler dumps every thread's stack after 300s of one test.
# this runner while the identical list passes locally. Per-test lines locate the stall;
# the fault handler dumps every thread's stack after 300s of one test.
run: >-
uv run --frozen python -m pytest -v -o faulthandler_timeout=300
uv run --frozen --with pytest-xdist python -m pytest -n auto -v -o faulthandler_timeout=300
tests/callmatrix
tests/test_team_limit.py
tests/test_ledger.py
@@ -137,6 +136,11 @@ jobs:
tests/test_health.py
tests/test_localrun.py
tests/test_orgs.py
tests/test_orgs_isolation.py
tests/test_orgs_mgmt.py
tests/test_hub.py
tests/test_hub_sandbox.py
tests/test_mcp.py
- name: Dump Postgres activity on failure
# Three CI-only hangs at archive drains were undiagnosable from Python stacks alone; this
# shows the DATABASE's view: every session, its state (idle in transaction = a leaked or
@@ -146,13 +150,8 @@ jobs:
env:
PGPASSWORD: treg
run: |
psql -h 127.0.0.1 -U treg -d treg_test -x -c "select pid, state, wait_event_type, wait_event, xact_start, state_change, left(query,140) as query from pg_stat_activity where datname='treg_test' order by xact_start nulls last;" || true
psql -h 127.0.0.1 -U treg -d treg_test -x -c "select datname, pid, state, wait_event_type, wait_event, xact_start, state_change, left(query,140) as query from pg_stat_activity where datname like 'treg_test%' order by xact_start nulls last;" || true
psql -h 127.0.0.1 -U treg -d treg_test -c "select l.pid, l.locktype, l.mode, l.granted, a.state, left(a.query,90) as query from pg_locks l join pg_stat_activity a using(pid) where not l.granted;" || true
tests/test_orgs_isolation.py
tests/test_orgs_mgmt.py
tests/test_hub.py
tests/test_hub_sandbox.py
tests/test_mcp.py
# RESTORED 2026-08-24. This job is a REQUIRED check on main, and the CI rewrite in 528383d
# dropped it while keeping the requirement — so for three weeks every pull request waited on a
@@ -215,11 +214,7 @@ jobs:
assert assets, 'Dashboard entry must reference compiled assets'
for asset in assets:
assert wheel.read('treg/web/dashboard/' + asset)
legacy = wheel.read('treg/web/dashboard-legacy/index.html').decode()
legacy_assets = re.findall(r'/app/legacy/(assets/[^"\s]+)', legacy)
assert legacy_assets
for asset in legacy_assets:
assert wheel.read('treg/web/dashboard-legacy/' + asset)
assert not [name for name in wheel.namelist() if 'dashboard-legacy' in name]
arena = wheel.read('treg/web/enrich-arena.html').decode()
for asset in re.findall(r'src="(/vendor/vue-[^"\s]+)"', arena):
assert wheel.read('treg/web' + asset)
.gitignore
+1 -2
View File
@@ -94,7 +94,7 @@ tests/screenshots/
# ArcTerm writes this local backup beside CLAUDE.md; it is not project content.
CLAUDE.md.arcterm-bak
# Frontend dependencies, build output and browser test artifacts.
node_modules/
node_modules
/src/treg/web/dashboard/
/frontend/test-results/
/frontend/playwright-report/
@@ -102,4 +102,3 @@ node_modules/
# Global Vue runtime copied from the npm lockfile during the frontend build.
/src/treg/web/vendor/*.js
/src/treg/web/vendor/LICENSE
/src/treg/web/dashboard-legacy/assets/*/vendor/
AGENTS.md
+5 -9
View File
@@ -105,7 +105,8 @@ agents then built against a constitution that was wrong.
- **Table ownership.** One writer module per table; cross-domain reads are fine. Three recorded
exceptions: only money writes `org.balance_micro`, the daily-spend counter (`spent_today_*`) and
the auto-top-up fields; the call runtime may persist an OAuth token refresh into `secret`; audit
writes `callrecord`, domains only read it.
writes `callrecord`, domains only read it (`application/evidence_retention.py` also updates it,
blanking the two evidence columns past retention).
- **Feedback handling.** This repo owns `FeedbackHandling` and `FeedbackHandlingEvent` models and
migrations; the private admin service is their only runtime writer. Original reports remain
owned by the feedback domain. See `docs/context/architecture/feedback.md`.
@@ -157,9 +158,9 @@ uv run lint-imports # the import-linter contracts (CI
scripts/dev-local.sh up # live dev stack on :18790 with its own sqlite DB
```
xdist is pulled via `--with`, not the lockfile — same as CI. The Postgres CI job
(`test-postgres`) must stay serial: every worker would share one database while
`reset_db()` drops tables.
xdist is pulled via `--with`, not the lockfile — same as CI. Every test process gets its own
database (a sqlite file per pid; under `TREG_TEST_DB_URL`, a Postgres database per xdist worker),
so parallel runs and side-by-side runs never share one.
- **Dependencies change through `uv add` or `uv lock`, never by hand.** `pyproject.toml` pins
`required-version` so an old uv refuses to run instead of rewriting `uv.lock`; CI uses `--locked`.
@@ -167,11 +168,6 @@ xdist is pulled via `--with`, not the lockfile — same as CI. The Postgres CI j
`[server]` extra, the certificate authority is `[proxy]`. Never import a heavy dependency at the
top of a CLI-path module; the "Lightweight CLI modules" import-linter contract lists them and
fails the build.
- **Frontend rollout.** `frontend/README.md` documents account assignment and rollback.
`src/treg/web/dashboard-legacy/` is **deprecated**, retained only for temporary rollout and
rollback. Never hand-edit it or mirror new features/fixes into it; `frontend/` is the only
maintained Dashboard source. Follow the retirement checklist in `frontend/README.md` to remove
it after rollout, including anonymous entries that still use legacy at 100%.
- **The dashboard** lives in `frontend/` (Vue components, TypeScript entry/transport, Vite).
Build with `bash scripts/build-dashboard.sh`; generated assets in `src/treg/web/dashboard/`
ship with Python. Run `npm --prefix frontend test` and `npm --prefix frontend run test:e2e`.
CONTRIBUTING.md
+1 -1
View File
@@ -44,7 +44,7 @@ require a frontend build, so CLI and background-worker development remains indep
Commit messages follow Conventional Commits (`feat(scope): …`, `fix: …`, `docs: …`).
3. Add or update tests; run `uv run --with pytest-xdist pytest -n auto -q` (all green).
Serial `uv run --frozen python -m pytest -q` is for debugging one test or order.
The Postgres CI job must stay serial (`reset_db()` drops tables on a shared database).
Against Postgres, set `TREG_TEST_DB_URL`; each xdist worker creates its own database from it.
4. If you changed a subsystem, update its fragment in `docs/context/` in the same PR.
5. Open a PR. CI runs the tests + a secret scan; a maintainer reviews.
README.md
+2 -1
View File
@@ -150,7 +150,8 @@ Out of balance is an HTTP **402** carrying `balance_micro`, `estimated_cost_micr
so an agent can act on it without reading prose.
**Enrich Arena** lives at `/enrich-arena`, outside the dashboard. Compare enrichment answers with each vendor’s cost and speed,
vote for the best answer in one click, or watch a sequential waterfall. Browsing is
vote for the best answer in one click, or watch a sequential waterfall. Capability-compatible
async providers participate too; Arena handles submit and polling internally. Browsing is
public; submitting requires login, and billable attempts use your team's credits. See the
[Arena guide](docs/context/interface/enrich-arena.md).
USAGE.md
+6
View File
@@ -137,6 +137,7 @@ treg tool add google-ads --base-url https://googleads.googleapis.com \
| `treg catalog search` | `"what you want to do"` | find endpoints by capability |
| `treg catalog get` | `ENDPOINT_ID` | docs, parameters, **the price**, and how you would be served |
| `treg call ENDPOINT_ID` | `--query K=V`, `--data STR` | call it |
| `treg --json call ENDPOINT_ID` | same | for scripts: one JSON line `{"result": <body>, "_treg": {http_status, call_id, charged_micro}}` on stdout, nothing on stderr |
| `treg call ENDPOINT_ID --await` | `--timeout N` (default 900) | a generation call (video/image): submit, poll the provider, print the final response |
| `treg host FILE` | `--content-type T`, `--json` | host a reference image/audio/video at a public URL a vendor can fetch (30 MB, 7-day TTL, free); prints the URL for `image_urls` / `audio_urls` |
| `treg catalog request` | `"what's missing"` | searched, not there? file it — requests steer what gets added next |
@@ -257,6 +258,11 @@ on by default), cheapest first, within `X-Treg-Route-Max-Cost` (default $1 per c
attempt settles at its real price and misses on per-success providers are free. `X-Treg-Route-Waterfall: 0`
stops at the first miss. `X-Treg-Route-Prefer` / `X-Treg-Route-Exclude` name providers. A filter the serving provider could not apply is named in `X-Treg-Ignored-Filters` (and `_treg.ignored_filters`); `X-Treg-Route-Strict-Filters: 1` refuses such a call with a 422 (unbilled) instead. Vendor endpoints are still relayed verbatim; only `treg.*` rows model an API.
When a routed child is asynchronous, treg submits and polls it internally for up to 60 seconds. If
it is still processing, the route returns HTTP 202 with `_treg.outcome: "pending"`, the child call
reference and poll descriptor, `reserved_micro`, and `charged_micro: null`. That attempt stops the
waterfall; it is not safe to start another billable provider while the first may still complete.
## Calling
| Command | Options | What it does |
design.md
+8 -8
View File
@@ -85,20 +85,20 @@ Use fluid layout instead of copying Figma's absolute coordinates. Set `min-width
| --- | --- | --- | --- |
| Page title | Geist Pixel | 22 / 27.5px | 400 |
| Short introduction, such as the Try it out intro | Geist Pixel | 16 / 22px | 400 |
| Module title | Google Sans Flex | 16 / 24px | 500 |
| Body copy and form instructions | Google Sans Flex | 14 / 21px | 400; key field labels 600 |
| Navigation and selectors | Google Sans Flex | 13 / 19.5px | 400-600, according to state |
| Supporting copy | Google Sans Flex | 12.5 / 19.375px | 400 |
| Example card label | Inter | 12.5 / 19.375px | 400 |
| Example task copy | Inter | 13 / 20.15px | 600 |
| Module title | System UI font | 16 / 24px | 500 |
| Body copy and form instructions | System UI font | 14 / 21px | 400; key field labels 600 |
| Navigation and selectors | System UI font | 13 / 19.5px | 400-600, according to state |
| Supporting copy | System UI font | 12.5 / 19.375px | 400 |
| Example card label | System UI font | 12.5 / 19.375px | 400 |
| Example task copy | System UI font | 13 / 20.15px | 600 |
| Commands, tokens, and code | DM Mono | 12 / 18.6px | 400 |
| Balance values and step numbers | DM Mono | 12 / 18px | 500 |
Body copy falls back to Inter and system sans-serif; code falls back to system monospace. If Geist Pixel is unavailable, fall back to mono without blocking the page. Chinese content needs a system font fallback with complete glyph coverage; do not force it into a pixel font with missing glyphs.
Interface text uses the operating system's UI font (`--sans`: SF Pro on Apple platforms, Segoe UI on Windows, Roboto on Android), as ChatGPT's product does. It downloads nothing, so text is final from the first paint and never reflows when a web font arrives; the brand is carried by Geist Pixel titles and DM Mono figures, not by body text. Those two faces are the only web fonts: pinned `@fontsource` npm packages (OFL-1.1), bundled same-origin with the Dashboard. Do not add a body web font without a brand reason strong enough to pay for its download and swap. Code falls back to system monospace. If Geist Pixel is unavailable, fall back to mono without blocking the page. Chinese content needs a system font fallback with complete glyph coverage; do not force it into a pixel font with missing glyphs.
Preserve DM Mono for balances as a treg detail instead of adopting Vercel's rule of using Sans for every financial figure. Use tabular numerals for comparisons, with consistent units and precision.
The wordmark target is Figma's Google Sans Flex at 15px / 500. The current prototype's `.brand` still inherits an earlier mono declaration. This is a known discrepancy, not a second brand typography rule.
The wordmark target is the system UI font at 15px / 500 (Figma used Google Sans Flex). The current prototype's `.brand` still inherits an earlier mono declaration. This is a known discrepancy, not a second brand typography rule.
## 6. Color and themes
docs/context/README.md
+2 -2
View File
@@ -19,7 +19,7 @@ covers (frontmatter `sources:`). Regenerate this index with
| [Google Ads conversion tracking — capture, outbox, upload](architecture/ads-conversions.md) | shipped | adsconv.py, signup.py, adtrack.js, gtag.js |
| [Archive - versioned history and cache admission](architecture/archive.md) | building | archive.py, hunter.yaml, results.py, 0031_archive_result_admission.py, … |
| [Auth & secrets — injectors, encryption, OAuth freshness, health](architecture/auth-secrets.md) | shipped | injectors.py, ssrf.py, crypto.py, oauth.py, … |
| [Endpoint catalog — what you can DO with a connected key, and which provider should do it](architecture/catalog.md) | shipped | fishaudio.yaml, fishaudio.tts.s2-1-pro.json, fishaudio.voices.create.json, fishaudio.voices.discover.json, … |
| [Endpoint catalog — what you can DO with a connected key, and which provider should do it](architecture/catalog.md) | shipped | fetchinio.yaml, fetchinio.svg, fetchinio.linkedin.user.profile.json, fetchinio.linkedin.company.profile.json, … |
| [Application composition and deployment roles](architecture/composition.md) | shipped | bootstrap.py, bootstrap_handlers.py, bootstrap_http.py, call_surface.py, … |
| [Data model — the registry tables, async DB, audit writer](architecture/data-model.md) | shipped | 0042_pinned_read_scope.py, alembic.ini, env.py, 0001_baseline_current_schema.py, … |
| [Feedback - private intake for problems and suggestions](architecture/feedback.md) | shipped | feedback_contract.py, __init__.py, reports.py, reviews.py, … |
@@ -34,7 +34,7 @@ covers (frontmatter `sources:`). Regenerate this index with
| [Multi-tenancy — orgs, memberships, invites, per-org scoping](architecture/multi-tenancy.md) | shipped | access.py, 0042_pinned_read_scope.py, test_pinned_read_scope.py, models.py, … |
| [The proxy — faithful credential-injecting relay + tool resolution](architecture/proxy-model.md) | shipped | relay.py, ssrf.py, api.py, authorize.py, … |
| [Discovery experiment — a relevance judge behind catalog search, measured on what the caller does next](architecture/search-experiment.md) | building | search_experiment.py, interleave.py, judge.py, 0041_searchlog.py, … |
| [Super-admin — cross-tenant read + control](architecture/super-admin.md) | shipped | api.py, admin.py, access.py, config.py |
| [Super-admin — cross-tenant read + control](architecture/super-admin.md) | shipped | api.py, admin.py, evidence_retention.py, access.py, … |
## Interfaces (API · CLI · skill)
docs/context/architecture/ads-conversions.md
+5 -12
View File
@@ -192,19 +192,12 @@ moved to v25 on 2026-08-17 across every place a version is hard-coded (`oauth_pr
for the full four-places-at-once list and the two-failure-modes note (a dead version returns a typed
`UNSUPPORTED_VERSION`, not the HTML 404 a never-existent version returns).
## Testing hazard: the shared test database
## Testing: one database per test process
The suite's default SQLite files live under `$TMPDIR/treg-tests/` (per-xdist-worker, out of the repo
tree so file watchers stay quiet), and `reset_db()` (test-only) drops and recreates
every table. Two pytest runs against the same file concurrently corrupt each other — one run's
`reset_db()` mid-flight drops a table the other run is about to query — and the failure surfaces as a
misleading `no such table` error that looks like a flake, not a concurrency bug. This cost this
feature's development several hours and one conversation-round wrongly dismissing a real bug as a
flake before the cause was found. Isolate a run with:
```bash
TREG_TEST_DB_URL="sqlite+aiosqlite:///./some-other.db" uv run --frozen python -m pytest -q
```
The suite's default SQLite files live under `$TMPDIR/treg-tests/`, one per test process (named by
pid, removed at exit), out of the repo tree so file watchers stay quiet. Concurrent runs in one
checkout therefore never share a file. Before this, two runs against one file wiped each other's
tables mid-test and surfaced as a misleading `no such table` error that looked like a flake.
## Not built
docs/context/architecture/archive.md
+38 -15
View File
@@ -199,15 +199,21 @@ error because a worker completion has no pending caller event to annotate.
Readers use `archive_bodies.pointer` to collect R2 metadata inside
a session (`defer(ArchiveSnapshot.body)` for R2-first), then close it before `archive_bodies.read`. When DB bytes are needed, including after an R2 failure, a new short DB
session for fallback bytes. Terminal batches use at most eight simultaneous reads. This applies to lookup, call-result reads,
and terminal-result reads, including the Activity routes' outer auth/query sessions. `r2-first`
only tries R2 for a published `both`/`r2` location; missing objects, timeouts, errors and checksum
mismatches fall back to DB. Lookup selects `result_snapshot_id` under the existing result-state
session for fallback bytes. Terminal and Arena batches use at most eight simultaneous reads. This
applies to lookup, call-result, terminal, admin and Arena reads, including outer request sessions.
`r2-first` tries the selected snapshot's content hash independently of `body_storage`: historical
backfills do not rewrite the original storage marker, and a pruned DB copy may still exist in R2.
Missing objects, timeouts, errors and checksum mismatches fall back to the exact DB snapshot/carrier.
A hash-only row does not promise an R2 object; an unsuccessful read returns unavailable, never a
different version. Authorization, key scope and result selection precede body resolution.
Lookup selects `result_snapshot_id` under the existing result-state
and observed-version guards, then classifies the resolved body after closing the session. Unknown
results retain the decisive snapshot; empty results invalidate serving without deleting history.
Pruning protects the decisive snapshot and DB carriers of surviving versions. Eligible `both`
rows lose DB bytes and become `r2`; their objects remain untouched. `db` does not contact R2, including for R2-only rows. The admin body
viewer remains a DB-only diagnostic in this first delivery. Read switches should be enabled
rows lose DB bytes and become `r2`; their objects remain untouched. `db` does not contact R2,
including for R2-only rows. Admin and Arena follow the result read switch; comparison and lazy
initialization follow lookup. Admin fallback uses the admin pool; observation/initialization use
the background pool; Arena uses its worker session factory. Read switches should be enabled
before any future R2-only write rollout. No serving allowlist, cohort or production setting is
changed here.
@@ -220,10 +226,21 @@ All paths log bounded reasons without exception text, keys, bodies or credential
`hash_mismatch` are ERROR. Oversized objects are also ERROR (`too_large`); other transport errors
and an unavailable client are WARNING (`store_error`, `store_unavailable`). HTTP 429 is
`rate_limited`, HTTP 5xx is `upstream_error`, both WARNING on read fallback. These same reason
names appear on failed uploads. Logs include the exception class, never the exception text. Result and terminal
reads use these logs because they have no `tool_called`. Existing per-path process counters remain;
names appear on failed uploads. Logs include the exception class, never the exception text.
A missing object with a NULL legacy storage marker logs at INFO: this also includes intentionally
hash-only history, so absence is not proof of loss. Existing per-path process counters remain;
additional bounded per-path/reason counters distinguish the failure classes.
Every common-reader invocation emits one best-effort `archive_body_read` completion event from
`finally`, including cancellation and DB fallback errors. It reports `path`, original `storage`
(`legacy` for NULL), `read_mode`, final `source`, `outcome` (`r2`, `db`, `db_fallback`, `unavailable`,
`db_error`, `cancelled`), `fallback_reason`, `r2_attempts`, `r2_retry_reason`, `r2_retry_recovered`,
`r2_read_ms`, `db_read_ms`, `total_ms` and returned `bytes`. Timings cover the common reader, not
the preceding metadata query; in DB mode the pointer may already hold bytes. The event contains
no body, hash, URL, call or team identity and receives the normal build/config fingerprints.
`read_<path>_<outcome>` process counters include successful reads as a denominator; fallback logs
alone do not prove DB rescued a read. Analytics remains best effort, not evidence of full coverage.
DB fallback requires a snapshot that still has DB bytes or a DB carrier, normally written during
`db`, `both`, or a failed R2-only upload. Successful `r2` writes have no DB copy: an R2 read failure
becomes a cache miss and calls upstream for lookup; history returns `stored=false` with no response
@@ -713,14 +730,19 @@ carries only a bounded reason and the underlying exception class. The real-wheel
Pruning still strips eligible DB bytes during double writing. A stripped `both` row becomes
`r2`; its content hash and object remain intact, and logical retained-body statistics do not
decrease. Existing deduplicated DB carriers and result baselines retain their protections.
The admin DB body viewer identifies object-stored bodies without fetching them. Retired
The admin body viewer uses the common reader and retains dedup carrier-version metadata. Retired
`volatile_paths` remains in the schema but is no longer displayed.
Known result-admission upgrade limit: when a historical R2-only row has no current
`result_state`/observed-version metadata, the write path does not GET its old body to classify
it. The baseline becomes unknown; the next decisive result establishes a new baseline without
a stability comparison. Subsequent observations learn normally. This conservative loss of one
learning interval avoids object I/O inside a write session or an extra speculative GET per write.
For R2-first recording, `_ignored_matches` also preclassifies the latest snapshot when result
state needs lazy initialization. Raw-identical bytes reuse the current response; otherwise this
shares the precomparison read and deadline, with path `initialization`. `_store_locked` accepts
that classification only for the actual newest snapshot under the key lock. If a race or the
precomparison deadline leaves no classification, the writer retains the original DB-only lazy
initialization for that exact newest snapshot/carrier. This preserves an existing decisive result
when R2 is slow; no available body still means an unknown baseline. The `change_outcomes` counters
`initialization_db_recovered` and `initialization_db_unavailable` report this last fallback outside
the common reader. The writer selects body presence rather than loading blobs for deduplication,
and never performs object I/O under its transaction.
`WritePlan` is the single body-retention decision passed into the DB writer. DB retention is
inferred from its storage location; failed eligible R2-only uploads become `db` plans. Each started
@@ -809,7 +831,8 @@ pool. It cannot enable R2 independently of the existing startup checks or rollba
(default true). Declared ignore-path TTL comparison remains a separate decision mechanism.
Both ignore comparison and change reporting share the recorder/touch semaphore budget of two,
including fallback sessions and CPU work. No DB connection is held during object I/O. New bodies
not retained by policy/size/storage and known hash-only previous snapshots skip observation.
not retained by policy/size/storage skip observation. DB-mode reads also skip known hash-only
previous snapshots; R2-first can recover a previous body's migrated object after DB pruning.
Process-local `change_outcomes` counts `observed`, `body_unavailable`, `observation_failed`,
`ignore_body_unavailable` and `ignore_comparison_failed`. `/admin/archive` exposes this mapping and
`body_outcomes` (archive_bodies.outcomes), including on cached report responses; counters reset on
docs/context/architecture/auth-secrets.md
+17
View File
@@ -39,12 +39,29 @@ related:
# Auth & secrets
Fetchin uses a pasted `X-API-Key` at `https://api.fetchin.io`. Its free internal
`GET /api/v1/subscription` probe rejects invalid credentials and accepts a valid account even when
its credit balance is zero. `TREG_PLATFORM_KEY_FETCHINIO` supplies the optional shared binding;
the own-key-first ladder keeps a team's credential unmetered. The same free route supplies capacity
data and is not exposed as a catalog tool.
Tavily uses a pasted Bearer key at `https://api.tavily.com`. Its free internal `GET /usage` probe
rejects invalid credentials and validates both team-owned and optional platform credentials without
exposing usage as a catalog tool. `TREG_PLATFORM_KEY_TAVILY` supplies the server-held fallback; the
existing own-key-first ladder means a team's key always wins and remains unmetered. The public
surface is limited to Search, Extract, Map, and Crawl.
ScrapeGraphAI uses a pasted raw `SGAI-APIKEY` header at `https://v2-api.scrapegraphai.com`. Its free
internal `GET /api/credits` probe rejects invalid credentials and validates team-owned and optional
platform credentials while also supplying capacity data. `TREG_PLATFORM_KEY_SCRAPEGRAPHAI` supplies
the server-held fallback; the existing own-key-first ladder keeps a team's credential unmetered.
Serper uses a pasted raw `X-API-KEY` header at `https://google.serper.dev`. Its free internal
`GET /account` probe validates team-owned and optional platform credentials while also supplying
balance and rate-limit evidence. `CatalogTarget` approves `https://scrape.serper.dev` for the same
credential without broadening the primary host. `TREG_PLATFORM_KEY_SERPER` supplies the server-held
fallback; the existing own-key-first ladder keeps a team's credential unmetered.
`ADYNTEL` is the first pasted-key provider whose two credentials ride in the JSON request body.
The primary `api_key` and second `email` are ordinary declarative bindings with `location: json`;
the relay contains no Adyntel branch. Tier 4 reads `TREG_PLATFORM_KEY_ADYNTEL` and
docs/context/architecture/catalog.md
+67 -6
View File
@@ -2,6 +2,15 @@
title: Endpoint catalog — what you can DO with a connected key, and which provider should do it
status: shipped
sources:
- src/treg/catalog/fetchinio.yaml
- src/treg/web/logos/fetchinio.svg
- src/treg/catalog/examples/fetchinio.linkedin.user.profile.json
- src/treg/catalog/examples/fetchinio.linkedin.company.profile.json
- src/treg/catalog/examples/fetchinio.linkedin.user.posts.json
- src/treg/catalog/examples/fetchinio.linkedin.user.reactions.json
- src/treg/catalog/examples/fetchinio.linkedin.post.comments.json
- src/treg/catalog/examples/fetchinio.linkedin.post.reactions.json
- src/treg/catalog/examples/fetchinio.linkedin.post.engagement.json
- src/treg/catalog/fishaudio.yaml
- src/treg/catalog/examples/fishaudio.tts.s2-1-pro.json
- src/treg/catalog/examples/fishaudio.voices.create.json
@@ -50,8 +59,15 @@ sources:
- src/treg/domain/catalog/routing/paths.py
- src/treg/domain/catalog/routing/plan.py
- src/treg/domain/catalog/routing/synthetic.py
- src/treg/application/call/async_bridge.py
- src/treg/application/call/route.py
- src/treg/catalog/wiza.yaml
- src/treg/catalog/examples/wiza.people.email.find.json
- src/treg/catalog/examples/wiza.people.email.find.terminal.json
- src/treg/catalog/examples/wiza.people.phone.find.json
- src/treg/catalog/examples/wiza.people.phone.find.terminal.json
- tests/test_routing.py
- tests/test_wiza.py
- .github/workflows/catalog-drift.yml
- scripts/catalog_drift.py
- scripts/catalog_ingest.py
@@ -76,7 +92,6 @@ sources:
- src/treg/catalog/akta.extended.yaml
- src/treg/catalog/dataforseo.yaml
- src/treg/catalog/dataforseo.extended.yaml
- tests/test_dataforseo_constraints.py
- src/treg/catalog/scrapecreators.yaml
- src/treg/catalog/scrapecreators.extended.yaml
- src/treg/catalog/serpapi.yaml
@@ -112,6 +127,31 @@ related:
# Endpoint catalog — platform-grouped operations per provider
## Fetchin
`fetchinio.yaml` curates Fetchin's seven public LinkedIn data routes: profile, company, member
posts and reactions, post comments and reactions, and the combined engagement read. All seven are
strict-query GET tools and are available through BYOK or the platform key; the normal own-key-first
ladder keeps a team's credential unmetered. `GET /api/v1/subscription` is deliberately internal:
it is the free connection probe and capacity collector rather than an account-kind catalog tool.
Verified adapters add profile, company, member posts, post comments and post reactions to their
provider-neutral LinkedIn routes. The member-posts contract compares Fetchin with Aviato and
HarvestAPI; member reactions and combined engagement remain direct Fetchin tools because no shared
contracts describe those provider-native operations.
Live balance deltas on 2026-09-24 confirmed one credit for every ordinary successful route and two
for combined engagement. The shared account's acquired PAYG replacement rate is $1.50 per 1,000
credits, so the rows are $0.0015 and $0.003 per call. Fetchin also bills one credit for a 404 while
every other failure is free. Its response contains no per-call charge evidence, and treg's generic
settlement rule never charges a rejected response on an estimate; the shared tier therefore absorbs
that upstream 404 cost. `fullProfile=true` can cost either one or two credits without reporting
which happened, so the strict curated profile row excludes it. A team's raw BYOK tool remains a
faithful relay and can still request it.
The examples were captured from public-figure and company fixtures. Comment and reaction actors,
their text, cursors and profile identifiers are replaced with reserved synthetic values before
commit; only the public test post identifier remains.
## Fish Audio v1
Fish Audio contributes synchronous S2.1 Pro TTS, public-voice discovery, and private voice
@@ -472,6 +512,11 @@ against v1), so a field-wise merge only produced descriptors nobody had written
serves the effective descriptor on the normalized endpoint. An explicit endpoint `async: false` opts
a utility or synchronous endpoint out of the provider default; absence means inherit.
An async endpoint may also declare `terminal_example_response`. Its ordinary `example_response`
remains the submission response shown by the catalog, while adapter verification uses the terminal
fixture returned by the poll endpoint. This lets an async tool join a routed capability without
pretending that its kickoff body is the final enrichment result.
**Poll mode in practice.** Every listed provider polls a static catalog id (`poll.endpoint`), which
the CLI reaches through `/call/<id>` on any credential tier. Replicate offers both `urls.get` and the
stable `GET /v1/predictions/{id}`; the static form is listed (`replicate.predictions.get`) because a
@@ -900,8 +945,9 @@ and state the break-even volume, and `fee_usd_month` must be present as data (th
and edited by hand. The full ladder: docs/SHARED-PLAN-PRICING-PLAN.md; the billing side (429 never
billable, the recovery report): architecture/money.md.
For synchronous providers that disclose the exact USD charge in the response, a paid cost may
declare `reported_charge: {path: ..., unit: usd}`. The catalog estimate still reserves a safe
For synchronous providers that disclose the exact charge in the response, a paid cost may declare
`reported_charge: {path: ..., unit: usd}` or use `unit: credit` when the provider has an `fx.yaml`
credit rate. The catalog estimate still reserves a safe
ceiling. A finite nonnegative response value settles the call at that amount; missing, invalid, or
non-finite evidence falls back to the normal estimate/miss rules. `reported_charge` is generic
catalog metadata, not a provider-specific billing branch, and cannot be combined with `cost.settle`.
@@ -1662,6 +1708,11 @@ to choose (`docs/CAPABILITY-ROUTING-PLAN.md`). Everything else in the catalog st
admission-only contract: its adapters verify like any other (which is what the archive's
`has_result_rules` reads), but no `treg.<capability>` row is ever generated from it. For a
capability whose "children" are one provider's price tiers, not a choice treg should make.
`scoping` names identity keys that scope the answer rather than describe it (`people.search`:
`company_domain`). A candidate whose adapter never sends one the caller supplied is dropped from
the plan with the reason, not ranked down like an ignored filter: a title-only search asked for
one company's CEO returns title-matched strangers for any company and bills them as a hit. The
rule is per candidate, so `{q, company_domain}` also drops the `q`-only providers.
- **Adapters** — `adapters.yaml`, one per endpoint: `accepts` (identity variants), `in` (contract
field → `queryParams.x` / `body.x`), `const` (fixed provider params), `out` (core field →
expression over the body), `miss`. The expression language (`domain/catalog/routing/paths.py`)
@@ -1702,9 +1753,11 @@ to choose (`docs/CAPABILITY-ROUTING-PLAN.md`). Everything else in the catalog st
is the measured hit rate when ≥ 20 decided samples exist, else `ok_rate`, else 1.0 (flagged
`unmeasured`). `build_plan` reads that evidence through bootstrap's shared process cache; cold or
unavailable observations degrade to unmeasured ranking while the cache refreshes off the request
path. `X-Treg-Route-Prefer` / `-Exclude` override; exhausted providers (capacity view)
and providers with no key on the deployment are dropped and named in `dropped` (`needs {…}`
says which identity variant a dropped child wanted).
path. `X-Treg-Route-Prefer` / `-Exclude` override. An exhausted platform provider with an enabled
overflow route remains a candidate at the overflow route's price, so the ordinary child ladder can
skip the known-dry direct account and use the aggregator; without an enabled route it is dropped.
Providers with no key on the deployment are also dropped and named in `dropped` (`needs {…}` says
which identity variant a dropped child wanted).
- **Execution** — `application/call/route.py`, entered from `service._execute_call` when the
resolved catalog row is `kind: routed`. Each attempt is a **full child `execute_call`** on a
`CallContext` whose `call_ref` is `{parent}:r{n}` — its hold id, ladder (tiers 1/2/4/overflow),
@@ -1760,6 +1813,14 @@ to choose (`docs/CAPABILITY-ROUTING-PLAN.md`). Everything else in the catalog st
the idempotency label (a success replays without touching a provider; a failure now costs
nothing, so it is not stored and a retry with the same key tries again) and writes one audit row
(`credential_tier: routed`) beside the children's.
An async child uses the shared async bridge to submit once and poll through ordinary authenticated
child calls. The final poll response, not the kickoff response, is passed to the adapter. Routed
execution waits for up to 60 seconds. If the task is still processing, or a foreground poll
cannot prove a declared terminal state, it returns HTTP 202 with `_treg.outcome: pending`, the
provider and endpoint, child call reference, poll descriptor, `reserved_micro`, and
`charged_micro: null`. A pending attempt stops that waterfall because the child may still complete
and charge; the existing async worker owns eventual settlement. Only declared terminal misses and
failures may continue under the normal bounded fallback rules.
- **Hit rate** — `CallRecord.hit` (nullable, alembic `0009`, last column) is the adapter's verdict
written at settle; `stats.observed` publishes `hit_rate`/`hit_samples` (floor 20) and, for
per-success endpoints, reads historical rows too (a 2xx with `cost_observed_micro == 0` is a miss).
docs/context/architecture/composition.md
+2
View File
@@ -53,6 +53,8 @@ verification, then owns the asynchronous client until archive and analytics drai
conditional resource setup does no object I/O at startup and adds no worker. Tests can supply
`create_app(..., archive_object_store=...)`; `configure_archive_object_store` is the shared
in-memory injection seam. See [archive](archive.md) for switches and queue behavior.
The same `archive_object_store` context owns the client for the Arena insights worker; that command
does not start a web lifespan or its background tasks.
For every role, the factory wires the Catalog observation port to one process-local
`CachedEndpointObservationReader` backed by short `background_session_maker` reads — the cache never
docs/context/architecture/data-model.md
+11 -7
View File
@@ -282,8 +282,9 @@ uses this metadata, never the encrypted token's shape.
`error_request` / `error_response` hold redacted, truncated failure evidence across platform,
own-key and own-tool calls. Successes leave them empty. Captured provider headers use an
allowlist covering retry/auth/rate-limit and request/trace identifiers. `/calls` neither fetches
nor exposes these wide fields; `GET /admin/errors` owns access and the 14-day retention purge
(replacing expired evidence with `<expired>`).
nor exposes these wide fields; `GET /admin/errors` owns read access (read-only) and the
`treg-worker admin purge-evidence` cron (`application/evidence_retention.py`) the 14-day
retention purge, replacing expired evidence with `<expired>`.
Redaction in `application.call.evidence` is security-sensitive:
@@ -297,7 +298,8 @@ uses this metadata, never the encrypted token's shape.
Unmetered uploads are buffered for evidence only with declared `Content-Length <= 64 KiB`.
Failed streaming responses retain at most the first 8 KiB, replaying all bytes to the caller.
Purging stays on the admin path; request-session dependencies do not commit a lazy purge marker.
Purging never runs on a request: an admin page reading errors once blanked evidence
platform-wide on every load.
`archive_key_hash` / `archive_content_hash` link eligible metered platform responses to
`ArchiveKey` / `ArchiveSnapshot` for `GET /calls/{id}/result`. They are nullable, unindexed
@@ -411,9 +413,11 @@ session is committed before the relay so none of them ever waits on it, see
later server loop cannot inherit connections bound to the closed loop. `verify_db()` is the read-only
lifespan and worker guard: it keeps the missing-Fernet-key refusal, requires a stamp at head, refuses a
known older revision, and warns but serves on an unknown-newer revision for additive-era rollback.
`reset_db()` is test-only: it disposes every loop-bound pool, recreates the SQLite schema or truncates
application tables on Postgres, then writes the Alembic head stamp. Avoiding per-test Alembic runs and
Postgres DDL keeps the suite fast without weakening the autogenerate drift guard. `get_session()` and
`reset_db()` is test-only: it disposes every loop-bound pool, deletes every application row (and
rewinds Postgres sequences), then writes the Alembic head stamp. It rebuilds the schema only when its
column-and-index fingerprint differs from the one it last built: a fresh database, or a test that
altered the shared schema. Avoiding per-test Alembic runs and DDL keeps the suite fast without weakening the
autogenerate drift guard. `get_session()` and
`get_admin_session()` are the FastAPI dependencies. SQLite locally (`aiosqlite`), Postgres on Render, same code. **Timestamps are
naive UTC:** `_now()` (the `created_at` default) drops tzinfo because the columns are `TIMESTAMP WITHOUT
TIME ZONE` and asyncpg rejects tz-aware values on Postgres; the app compares naive UTC throughout.
@@ -483,7 +487,7 @@ module is off** (self-hosters and the test suite send nothing). `$groups: {team:
browser's `posthog.group('team', slug)`. Every event also carries `build` (`TREG_BUILD`, else the
commit variable the host exports, else the installed package version; `build_id`) and
`archive_config` (a 12-hex digest of the archive settings that change what a call does:
mode, serving allowlist and percentage, repeat price, age ceilings, body storage, change
mode, serving allowlist and percentage, repeat price, age ceilings, body write and all three read modes, change
observation; `archive_config_id`), and the lifespan emits one `service_started` per process with
the role and those archive settings. They exist so an analysis can be bounded to one code version
or one cache configuration instead of a remembered deploy time: a property that an older build
docs/context/architecture/import-boundaries.md
+6 -3
View File
@@ -62,8 +62,9 @@ archive body PUT chain because it persists the paid response outside any DB tran
post-relay provider-resource register/rename/tombstone calls because shared-key resource ownership
must be established or advanced by the call that receives the provider's successful response.
The separate `test-postgres` job runs its database-sensitive subset serially against Postgres 16;
it uses unbuffered Python output and a 15-minute job budget so a slow test remains diagnosable. The
The separate `test-postgres` job runs its database-sensitive subset against Postgres 16, one
database per xdist worker; it uses unbuffered Python output and a 15-minute job budget so a slow
test remains diagnosable. The
subset includes agent attribution, managed API-key lifecycle and concurrency, credential health,
local-run reporting and ads-conversion coverage so naive-UTC assumptions are exercised by asyncpg
rather than hidden by SQLite's permissive adapter.
@@ -146,7 +147,9 @@ application imports the capacity domain inward (`resolve` → `view`, `settle`
the domain never imports back; `application.call.overflow` composes the capacity domain, the
aggregator envelopes and the money primitives, and the aggregator adapters stay pure envelope code;
`routers.catalog` reads the capacity domain's `routes_view` for the overflow price disclosure (a read
of the worker-owned table through the same in-process copy the call path uses, never a write); `application.call.route` composes the pure
of the worker-owned table through the same in-process copy the call path uses, never a write);
`application.call.route` reads both capacity views before planning so an exhausted provider with an
enabled overflow route reaches the ordinary child ladder, and composes the pure
`domain.catalog.routing` package (contracts, adapters, ranking) with the call use case itself. The
aggregator envelopes live under `treg.infra.upstream.aggregators` and inherit the upstream contract
(no HTTP adapters, no routers); the capacity domain's `verify` module may import them because they are
docs/context/architecture/money.md
+11 -1
View File
@@ -14,6 +14,8 @@ sources:
- src/treg/application/call/intake.py
- src/treg/application/call/resolve.py
- src/treg/application/call/service.py
- src/treg/application/call/async_bridge.py
- src/treg/application/call/route.py
- src/treg/application/call/reserve.py
- src/treg/application/call/settle.py
- src/treg/catalog/tomba.yaml
@@ -250,6 +252,14 @@ An async status declared as `billed_failure` is still presented as failure by th
worker settles its usage evidence and records the terminal outcome; this covers cancellation after
billable work without manufacturing a successful result.
Routed tools and Enrich Arena may wait for an async child through the shared async bridge. Every
poll still uses the ordinary call path, so BYOK remains unmetered and platform polls enforce task
ownership. Foreground polling and the worker may observe the same terminal response, but the task
row lock lets only one close the original hold. A foreground timeout or inconclusive poll response
returns a pending result with the reservation still open; the worker later settles or releases it.
Terminal UI and routed results read the task's settled amount, while pending results expose only the
maximum reservation.
The worker selects due candidates, acquires provider/global concurrency slots, then atomically
claims each still-due row. `attempts` fences stale workers from changing a newer claim's state.
The 60-second lease exceeds the 30-second processing deadline; queued rows are not leased.
@@ -508,7 +518,7 @@ Provider-specific calculation stays outside the faithful relay.
| Evidence | Settlement behavior |
|---|---|
| Generic catalog-reported charge | A paid synchronous cost may name `reported_charge.path` with unit `usd`. A finite nonnegative response value, including zero, settles exactly; invalid or absent evidence falls through to the normal estimate/miss behavior |
| Generic catalog-reported charge | A paid synchronous cost may name `reported_charge.path` with unit `usd` or provider `credit`. A finite nonnegative response value, including zero, settles exactly; credit conversion is frozen from `fx.yaml` when the call resolves, while invalid or absent evidence falls through to the normal estimate/miss behavior |
| Tavily Search | Reserve one credit for Basic, Fast and Ultra-fast or two for Advanced and an auto-selected depth; an explicit Basic depth overrides automatic selection. Platform Search requires caller-supplied `include_usage: true` and settles finite nonnegative per-request `usage.credits`. Empty results remain a paid routing miss. Missing or malformed usage keeps the frozen reserve. BYOK is unmetered and need not request usage. The endpoint-specific rate table must be complete, positive and finite; catalog validation rejects bad declarations and runtime refuses the call before reserve or relay instead of pricing it at zero |
| Tavily Extract | Reserve the requested URL count (bounded by the documented 20-URL maximum) at 0.2 credit per Basic or 0.4 per Advanced extraction. Settle that fractional allocation for each valid entry in `results`; `failed_results` and grouped `usage.credits` do not charge the caller. A documented empty results list is free; malformed evidence keeps the frozen reserve |
| Tavily Map | Platform calls require an explicit integer `limit` from 1 to 20. Reserve that many pages at 0.1 credit each, or 0.2 when the caller supplied nonempty `instructions`; settle valid URL strings in `results` at the frozen per-page unit. Empty results are free, malformed evidence keeps the reserve, and grouped `usage.credits` is ignored |
docs/context/architecture/search-experiment.md
+49 -4
View File
@@ -8,6 +8,8 @@ sources:
- src/treg/alembic/versions/0041_searchlog.py
- scripts/search_experiment_report.sql
- tests/test_search_experiment.py
- src/treg/application/catalog_find.py
- tests/test_catalog_find.py
---
# Discovery experiment
@@ -33,7 +35,10 @@ Three layers, imports pointing inward:
- **`infra/judge.py`** — TypeSafe's System One API (Jev). One request carries the query and every
candidate as `state`, and one Noul question per candidate; the answer is a probability per row.
It never raises: timeout, non-200, malformed body all return `probs=None` with a reason, and the
caller serves the baseline. Answers are cached in-process by (model, query, candidate ids).
caller serves the baseline. Answers are cached in-process by (model, query, candidate ids, and
any criteria or extra questions). A caller may attach Noul `criteria` to every candidate question
and add `extra` questions about the same state; the experiment passes neither, so its question
is unchanged while it runs.
- **`application/search_experiment.py`** — the use case. Judges the candidates, builds the judged
page with the SAME finishing steps the baseline had (evidence rerank, routed grouping, cut to the
page — the MCP layer passes that function in), deals the caller an arm, decides what is shown, and
@@ -99,6 +104,45 @@ team + email within ten minutes of the search, on endpoints that were on the ser
4. re-query rate — a second search within two minutes and no call in between is a page that did
not do its job.
## Served to people: find tools for a job
`GET /catalog/find?q=` (`application/catalog_find.py`) is the same mechanism with a person on the
other end: `store.candidates` recall, one `infra.judge` request, the same `search_judge_keep` /
`search_judge_high` cuts. It backs the dashboard's Catalog search box (Enter on a described job) and
the public `/search` page (see `interface/dashboard.md`). It differs from the experiment where the
audience differs:
- **Wider recall, looser timeout.** `find_candidates` (60) and `find_timeout_s` (6 s). The judge
scores a request's candidates in parallel, so 60 measured the same wall time as 30, and it lets
rows the lexical order ranks low reach the judge ("why is my blog losing google traffic" found
the Search Console performance report only at 60).
- **Streamed.** Two NDJSON events: `candidates` as soon as the recall is computed, `judged` when the
judge answers. The pages animate the gap on the first event.
- **A stricter question, and a name question.** Each candidate question carries `FIT_CRITERIA`,
whose `false` side includes "the task only names a product, company or platform": without it a
bare "google" scored 0.6+ against every Google endpoint and read as a weak answer. The same
request asks one extra Noul, whether `task` is only a name. Measured on hand-labelled queries,
the criteria left real fits level or slightly higher, and the name question put bare names at
0.9+ and short jobs ("backlinks", "tiktok ads") under 0.6.
- **A verdict, not a page.** `strong` (a row at or over `high`), `closest` (kept rows, none strong),
`none` (nothing kept), `keyword` when the judge abstained and the rows are the lexical page,
unjudged, or `name`: no strong fit, and the query is a name (the judge's name probability at or
over `find_name_min`, or exactly a platform's name or slug). Its rows are what the name offers:
the platforms whose name contains it, the one it starts first, each cut to its first 40
endpoints; else a provider of that name's endpoints; unjudged. The event's `named` says which
(`platform` or `provider`), and /search groups the answer by it. A name the catalog does not carry
falls through to the judged verdict. Kept rows are best fit first (no `interleave.bucketed` lexical order inside a bucket),
each with its fit and the catalog's own price shape; the event carries `high` so the pages draw
the strong cut from the server's setting. The probability is shown to people; agents still never
see it.
- **Open and rate limited.** No identity is needed, so `admit` bounds use per IP and per deployment
(`find_max_per_ip_hour`, `find_max_per_hour`) through `ratestore`, in a session that is committed
and closed before the judge is called.
- **Logged in the same tables.** One `SearchLog` row with `mode=find`, `source=web-find` and no
identity (so no outcome join yet), and a `SearchMiss` when nothing fit.
Agents are unaffected: `/catalog/search` and MCP `catalog_search` answer exactly as before.
## Guardrails and what is deliberately not here
- The judge can add at most `typesafe_timeout_s` (2.5 s) to a search and can never fail one. Live
@@ -108,7 +152,8 @@ team + email within ten minutes of the search, on endpoints that were on the ser
- The agent-facing response does not carry the judge's probability. Exposing it would change how
agents pick and turn the experiment into a different one.
- Page length is the same in every arm, so "more options" cannot masquerade as "better options".
- Nothing here touches `/call/`, money, or the HTTP search route. The routed-discovery switch
- Nothing here touches `/call/`, money, or the HTTP search route (`/catalog/find` is its own
route). The routed-discovery switch
(`routed_discovery`) applies to the judged page through the shared finishing function.
- Not yet built: a read path for `SearchMiss` other than the report scripts, and any use of the
judge outside search.
- Not yet built: a read path for `SearchMiss` other than the report scripts, any use of the judge
outside catalog discovery, and crediting a `/catalog/find` answer with what the person did next.
docs/context/architecture/super-admin.md
+5 -4
View File
@@ -4,6 +4,7 @@ status: shipped
sources:
- src/treg/api.py
- src/treg/routers/admin.py
- src/treg/application/evidence_retention.py
- src/treg/domain/identity/access.py
- src/treg/config.py
related:
@@ -60,10 +61,10 @@ endpoints are unaffected (they use `require_superadmin`).
[data-model](data-model.md)). `tier` filters an exact marketplace tier; an empty value selects plain
own tools. Superadmin and not org-admin
because the rows hold customers' request content; `GET /calls` deliberately does **not** expose
these columns, and it defers them so they are not even fetched. This route also performs the
14-day retention pass (`_purge_expired_error_evidence`, blanking to `'<expired>'` on its own
committed session) — ageing lives here because there is no scheduler and the request path cannot
hold a lazy marker, `get_admin_session` never committing one.
these columns, and it defers them so they are not even fetched. The route is read-only: the
14-day retention purge (blanking both columns to `'<expired>'`) is the `treg-worker admin
purge-evidence` cron (`application/evidence_retention.py`), in bounded batches. A row past the
window is listed as `expired` with no evidence even before the cron reaches it.
- **Reconciliation (Phase 5):** `admin_reconcile_drift|spend|repeats` (`?since_days=30`) — cross-org
aggregates over platform-tier spend, so super-admin and not org-admin: price drift per endpoint,
settled spend per provider (the invoice comparison), and the repeat-query rate. Query-time reports
docs/context/interface/api.md
+6 -4
View File
@@ -327,8 +327,8 @@ validated before resolving the shared HTTP client. `/auth/logout` remains an HTT
itself ran that journal count - 2.8 s per call for a member with 110k rows that day.
- **Super-admin (cross-tenant, `require_superadmin`):** `/admin/stats|orgs|orgs/{id}|users|tools|calls|
errors|health` (reads - `errors` is failed calls across every credential tier with captured,
admin-only request/response evidence, supports a `tier` filter, and runs the 14-day retention pass;
see [super-admin](../architecture/super-admin.md))
admin-only request/response evidence, supports a `tier` filter, and withholds evidence past the
14-day retention window, which the `treg-worker admin purge-evidence` cron blanks; see [super-admin](../architecture/super-admin.md))
+ `/admin/users/{id}/superadmin|suspend`, `DELETE /admin/users/{id}`,
`/admin/orgs/{id}/suspend`, `DELETE /admin/orgs/{id}` (Phase-2). See
[super-admin](../architecture/super-admin.md).
@@ -389,7 +389,8 @@ validated before resolving the shared HTTP client. `/auth/logout` remains an HTT
virtual-memory cap crashes Go CLIs (gh/stripe/doctl) and `RLIMIT_NPROC` is per-uid, shared with the
server. Full **filesystem/network** isolation needs a container deploy and is a planned follow-up.
- **Meta:** `meta` (`GET /meta`, open) → `{public_url, github, google, app_version, treg_version,
posthog_key/posthog_host, intercom_app_id}` for the dashboard. The last three are the opt-in
posthog_key/posthog_host, intercom_app_id, referral}` for the dashboard. `referral` carries the
two configured reward amounts so the top-bar entry can name them without `GET /referrals`. The last three are the opt-in
third-party keys (analytics, support chat): empty on a deployment that didn't set them, so
self-hosted pages load neither PostHog nor the Intercom Messenger. `intercom_app_id` is paired
server-side with `intercom_secret`, which never leaves the server: `_intercom_user_hash` (HMAC-SHA256
@@ -403,9 +404,10 @@ validated before resolving the shared HTTP client. `/auth/logout` remains an HTT
| Route | Contract |
|---|---|
| `GET /catalog/platforms` | Non-empty platforms with capability/endpoint counts and providers, ordered by endpoint count |
| `GET /catalog/platforms` | Non-empty platforms with capability/endpoint counts and providers, ordered by endpoint count; `providers` names every browsable vendor |
| `GET /catalog/platforms/{slug}` | Capabilities, extended endpoints, dashboard domain rows and provider metadata; unknown slug is 404 |
| `GET /catalog/search?q=&limit=` | Ranked endpoint views, count/total and hints; default 25, maximum 100 |
| `GET /catalog/find?q=` | Find tools for a described job: NDJSON stream of `candidates` then `judged` (verdict + kept rows with probabilities; a bare platform or provider name gets verdict `name` and its endpoints, unjudged); rate limited per IP, 503 without a judge key |
| `GET /catalog/endpoints/{id}` | Endpoint, provider, capability siblings, call template, inline example and next-step hints; `overflow_price_usd` / `overflow_price_unit` / `overflow_via` on the endpoint when the deployment can relay it |
| `GET /catalog/examples/{id}` | Captured JSON, resolved through the catalog before constructing a file path |
| `POST /tool-requests` | Open, rate-limited demand report with capped fields and optional caller attribution |
docs/context/interface/cli.md
+6 -3
View File
@@ -75,7 +75,11 @@ epilog (a `mk()` helper + `_ex()` + `RawDescriptionHelpFormatter`), so `treg <cm
`treg --version` / `treg version` print `cli_version()` (package metadata); `treg update` (`cmd_update`)
re-runs the server's `install.sh` to upgrade the CLI in place. A global **`--json`** flag (stripped in
`main` like `--org`) makes the human-table commands (`org ls`, `agents ls`, `catalog` in all its forms)
emit raw JSON instead — one stable contract for agents; commands that already print JSON are unaffected.
emit raw JSON instead — one stable contract for agents. On `call` (not `--await`) it prints one
compact envelope, `{"result": <body>, "_treg": {http_status, call_id, charged_micro | reserved_micro,
replay?, async?, hint?}}` (`_call_envelope`; text as a string, binary as base64), and suppresses the
charge, hint and failure-diagnostic stderr lines: a script that merged the streams once discarded
every result it had paid for. Exit status is unchanged (1 on HTTP >= 400).
**`TREG_CONFIG`** points the CLI at an alternate config file (CI/agents/tests; default
`~/.treg/config.json`). `org use` validates the slug against `/orgs`, then gets that membership's
active Default key before it saves either value. If that exchange fails, the previous team and token
@@ -157,8 +161,7 @@ CONFIG config · login · logout · on
The order **is** the pitch: what you can do with no setup comes before what you have to register
yourself, and `balance`/`topup` sit next to the thing that spends them rather than under team
management. `test_help_is_grouped_and_hides_aliases` pins both the order and that `catalog` precedes
`tool`, so a drift back to vault-first fails the suite.
management.
**Old → new.** Every one of these still parses and routes exactly as before — hidden, not removed:
docs/context/interface/dashboard.md
+103 -34
View File
@@ -56,6 +56,12 @@ sources:
- frontend/src/state/boot.js
- frontend/src/state/catalog.js
- frontend/src/state/catalogComputed.js
- frontend/src/state/find.js
- frontend/src/state/findComputed.js
- frontend/src/state/pile.ts
- frontend/src/components/FindAnswer.vue
- frontend/src/pages/SearchPage.vue
- frontend/src/components/LandingNavigation.vue
- frontend/src/state/connections.js
- frontend/src/state/constants.js
- frontend/src/state/context.ts
@@ -93,8 +99,6 @@ sources:
- src/treg/web/tour/tour.js
- src/treg/web/tour/index.html
- src/treg/api.py
- tests/test_dashboard_rollout.py
- src/treg/web/dashboard-legacy/README.md
- src/treg/routers/web.py
- src/treg/domain/identity/session.py
- src/treg/routers/api_keys.py
@@ -196,32 +200,29 @@ per-application state available to extracted components during this incremental
not a singleton, and this boundary is not yet a fully typed domain store. The TypeScript entry,
JSON transport and development configuration are checked with `vue-tsc` before every build.
Initialization renders a neutral loading state until session and route resolution finish, with a
retry on unexpected failure. Signed-out arrivals get a focused sign-in entry or shared-link gate;
retry on unexpected failure. `index.html` paints the same `.boot-status` markup before any script
runs, so mounting swaps the screen for itself. A fast boot shows only the page ground: the
indicator fades in after a delay, on the page's own clock (`bootStartedAt`), so the node Vue swaps
in does not restart it. `index.html` also starts
`/meta` and `/auth/me` alongside the bundle download (`window.__tregBoot`, taken over by boot) and
applies the saved theme before first paint. `loadAll` waits on one round trip per dependency step:
`/orgs` with `/invites/mine`, then the bearer with the team's tools, health and skills.
Catalog data does not wait for the session: boot starts the shelves (and a shelf's endpoints,
through `prefetchPlatform`, which `loadPlatform` takes over) alongside `/meta` and `/auth/me`.
**A view renders nothing it cannot yet know.** Empty states, zero figures and fallback views wait
for their data to answer (`plats.settled`, `callsLoaded`, `orgMembersLoaded`, `ref.loaded`); text
whose values are still loading keeps its space invisibly rather than showing zeros. Signed-out arrivals get a focused sign-in entry or shared-link gate;
the obsolete embedded marketing page is removed. The public landing page remains at `/`.
History navigation retains existing hashes, catalog URLs and shared links in `state/navigation.js`,
`state/catalog.js`, `state/details.js` and `state/boot.js`.
Mainline Team resources and Fish Audio upload, voice-management and audio-preview flows live
in `TeamResourcesPage.vue`, `FishVoiceDialog.vue`, `TryEndpointDialog.vue` and their state modules.
`_new_dashboard` selects the compiled entry by verified session user ID: the master rollout switch
must be on, then an ID allowlist or a stable SHA-256 bucket below the configured percentage selects
new. Defaults are off and zero percent. Anonymous and token-only browser entries retain the frozen
`dashboard-legacy/index.html`, whose Vue/onboarding/tutorial JavaScript has revision-qualified legacy asset
URLs. No query parameter, team selection or analytics service controls assignment. All dashboard,
shared-link and catalog entries use this decision and `private, no-store` plus `Vary: Cookie`.
Environment changes require restarting Web processes. Existing tabs switch on reload; the version
stamp also incorporates rollout settings to offer a refresh when assignment policy changes.
Every signed-in selection emits `dashboard_served` (variant, assignment, bucket, percentage) and
sets the `dashboard_variant` and `dashboard_bucket` person properties. Analytics only observes the
decision: PostHog persons carry no user ID to recompute the bucket from, and the bucket alone cannot
date an account's switch when the percentage moves.
The legacy snapshot is deprecated and scheduled for removal after rollout, not a second maintained
Dashboard. New features and routine fixes belong only in `frontend/`; normal main-branch syncs must
not refresh the frozen artifact. `frontend/README.md` owns the retirement checklist: migrate
anonymous and token-only entries as well as signed-in accounts, then remove the snapshot, legacy
asset route, selection settings and obsolete rollout plumbing. A 100% account rollout alone does
not retire legacy.
`_dashboard_index` returns the one compiled entry for every Dashboard, shared-link and catalog
request, signed in or not, so those pages no longer look up the session to choose a frontend. They
are served `private, no-store` with `Vary: Cookie`. The frozen legacy snapshot and its percentage
rollout were retired once every visitor was on this app; rollback is a deploy of the previous build.
The version stamp in `/meta` is the bundle hash, so an open tab offers a refresh after a deploy.
`GET /app` serves the selected document same-origin from the Python package, preserving local
sign-in and parked OAuth authorization. Catalog and shared-link handlers modify that same document's
@@ -238,7 +239,7 @@ and Vite, using a local-only development entry for hot updates. See `CONTRIBUTIN
Vue is pinned in the npm lockfile and bundled from the same origin, so a blocked CDN cannot
prevent startup. The shared onboarding widgets in `/agent-setup.js` still serve both Dashboard and
Arena; their templates use Vue's bundled compiler. The global Vue runtime for standalone pages and the legacy snapshot is
Arena; their templates use Vue's bundled compiler. The global Vue runtime for the standalone Arena page is
copied from the npm package at build time, with its license; generated copies are not committed. Agent icons and Google Fonts remain optional external presentation assets.
The unmounted entry displays a loading message and a reload link rather than hiding a raw template.
The authenticated redesign follows the root `design.md`.
@@ -283,8 +284,9 @@ user. On narrow screens navigation scrolls in a second row; team switching and o
remain available. The public catalog and logged-out landing retain their separate shells.
The authenticated wrapper's `.redesign` class scopes `media/redesign/dashboard.css`, served through
the existing `/media` mount. It uses Google Sans Flex for interface text, Geist Pixel for page titles,
and DM Mono for commands and balances, with light and dark semantic colors. Getting started uses
the existing `/media` mount. It uses the system UI font for interface text, Geist Pixel for page titles,
and DM Mono for commands and balances (the only two web fonts, bundled from pinned `@fontsource`
packages; see `design.md`), with light and dark semantic colors. Getting started uses
an approximately 1080px centered column, a split agent-preview/setup card, image-backed prompt cards,
and the existing optional Build on treg and manual setup flows. On mobile the setup card and prompt
grid stack. Images are copied from the pinned designer repository; provenance is in
@@ -634,7 +636,9 @@ selected account stamps a runnable containers-list path into the provisioned too
platform logo assets both carry the Google Tag Manager mark, so the catalog tile, platform header,
provider page, and expanded endpoint rows resolve to the same identity.
The tab bar itself is `v-if`'d on `plats.list.length` and `mkTabActive` collapses to `'platform'` when
the catalog is absent, so a build that predates `/catalog` renders exactly the old marketplace.
the catalog is absent, so a build that predates `/catalog` renders exactly the old marketplace. It
collapses only once `plats.settled` (the request answered, even with a failure): falling back while
the shelves loaded flashed the integration list on every visit.
The catalog page's header carries a **Request a tool** button (`reqAsk` modal): a short form —
what's missing, an optional note, a contact field only when signed out (`!me`) — POSTed to
@@ -860,7 +864,7 @@ stays in the price column, and the provider/endpoint counts live in the cell's t
a second line of their own.
**Merged rows expand in TWO levels.** Clicking one opens its providers as collapsed `.lsub` sub-rows —
one line each: logo, name, `costShort`, ✓/·, the connected chip, and a truncated `METHOD path`.
one line each: logo, name, `costShort`, a ✓ when verified, the connected chip, and a truncated `METHOD path`.
Clicking a sub-row (`toggleEp` → `epOpen[e.id]`) opens **that** provider's instruction. Dropping six
full parameter tables on one click buried the comparison the merge exists to make. A single row has
nothing to compare, so it skips the middle level and renders its detail straight away — the SAME
@@ -869,12 +873,19 @@ the instruction differently. Inside a merged sub-row the detail drops the provid
sub-row above already shows, and leads with the chips.
**The filter bar is sticky** under the top bar, and the section headings stick under *it* (`--lbar-top` /
`--lsec-top`); the domain chips **scroll** rather than wrap, because a bar that grew a second row as you
filtered would push the headings out from under it. Text, `verified only` and the domain chips narrow the
`--lsec-top`). The domain chips **wrap**: a scrolling strip with a hidden scrollbar cut its last chip in
half and gave a mouse no way to reach the rest. So the bar's height varies with the platform and the
filters, and `stickLedgerBar` (PlatformPage.vue) measures it and writes `--lsec-top` on the bar's parent;
the redesign shell redeclares the variable on its own element, so a value on the document root never
reached the headings. At phone width nothing sticks (a wrapped bar would cover half the screen) and each
row stacks: title, then route, price and ✓ on one line, with the separator drawn on the row. Unverified
rows show nothing in the Verified column. Text, `verified only` and the domain chips narrow the
same row list (`platRowsPreDomain` → `platLedger`); a section with no surviving rows disappears rather
than showing an empty heading, chip counts are taken after the other two filters so a chip never promises
rows they have already removed, and a live `N rows · M endpoints` line counts both — a merged row stands
for several endpoints. Both the wrapper and the table drop their `overflow` clip (an `overflow:hidden`
rows they have already removed, and a live `N rows · M endpoints` line counts both when they differ — a
merged row stands for several endpoints. Only the wrapper draws the rounded frame: a collapsed table cannot
round its own border, so a second one showed as a square frame inside it, and the last row's cells round
their own corners because nothing clips a hover fill. Both the wrapper and the table drop their `overflow` clip (an `overflow:hidden`
ancestor is a scroll container, and a sticky heading inside one never escapes it) and the table is
`table-layout:fixed`, so a nowrap path or `treg call` line scrolls **inside** its cell instead of widening
the table past the page.
@@ -999,6 +1010,63 @@ CSS classes or template source spelling. `tests/test_catalog_api.py` locks the s
merged/single split, the domain resolution ladder, and a delivery-mode path segment never becoming a
subject.
## Find tools for a job (Catalog search box, `/search`)
Both surfaces read `GET /catalog/find` (see `architecture/search-experiment.md`) through
`state/find.js`, which parses the NDJSON stream, aborts a superseded request, and keeps one `find`
state (`idle | recall | reading | done | error`). `state/findComputed.js` groups the judged rows by
capability: the job is the card, its providers are the lines in the server's order, and the card's
fit is its best provider's. The page never re-ranks providers.
- **Catalog search** (`CatalogPage.vue`, `view==='connections'`, signed in or on the public
catalog). A large box under the page title, not the corner search the other views use. A short
query is a name and keeps the instant platform filter; four words or a question mark makes it a
job and shows **Find tools ↵** (`isJobQuery`). The finder runs by itself once typing pauses
(`findSchedule`, `FIND_DEBOUNCE_MS`), because people did not discover Enter; Enter runs it at once.
Typing again drops the previous answer so a name filters the shelves meanwhile, and "No platform
is called that" waits while a find is scheduled (`findSoon`). The answer renders `FindAnswer.vue` between the box and
the tabs: one list, a row per job (platform, providers, lowest price, a fit bar), strong fits
first and weaker ones after them in a lighter tone, with no bucket labels; **Copy** appears on
hover, the row opens the platform. `closest` adds one line saying nothing fits closely; `none` is
a single sentence with Request a tool pre-filled. `name` (Enter on a bare platform or provider
name) lists what that name offers in the server's order, with no fit bars. The shelves stay: platforms the answer landed on
sort first with a match count, the rest dim. Clearing the box (× or Esc) returns to browsing. A
name that matches no platform says so and points at Enter, instead of the old "no catalogued
platforms on this server" message; tab counts follow the name filter. The page title's catalog
size is computed from `/catalog/platforms` (`toolCountText`), never hard-coded.
- **`/search`** (`SearchPage.vue`, a public view like `/catalog`, public for members too). It
looks like the landing page's first screen, not the dashboard: the landing top bar
(`LandingNavigation.vue`; "Open dashboard" for a member), the landing tokens, and the landing
hero's glyph field (`/media/landing/hero-particles.js`, mounted through `window.tregMountField`
and ticked by this page). One viewport tall, never scrolls; a long answer scrolls inside its
panel. Every platform and every vendor is a tile in a Matter.js pile (`state/pile.ts`) on the
floor of the page, keyed `p:`/`v:` since a slug can be both; the vendors come from
`/catalog/platforms`' `providers`. A platform tile opens its platform, a vendor tile its busiest
platform. Tiles can be picked up and thrown, the recall's platforms and vendors hop while the
judge reads, the fitting ones leave the physics world and fly to their answer cards, and the next
search drops them back in; × or Esc clears the answer the same way. A described job is answered
**by vendor** (so is a vendor's name, `named: provider`): one card per provider, the vendor's tile landing in the logo place and, on the
first card naming it, the platform's tile beside the platform name (later cards show a still
copy), with the vendor's jobs and prices. A bare name (`name`, titled "Tools for …") is answered
by platform: each platform's tile lands in its card's logo place, the cards list jobs with
provider counts, and the vendors on those platforms stay lit in the pile. An empty box submits its placeholder. Reduced motion
settles the pile unseen and skips the flights. `?q=` runs a search on load and is what **Share**
copies. Any result (a card, a job line, a tile) opens that platform in the dashboard: directly
for a member; otherwise sign-in first, the destination kept in localStorage for ten minutes and
resumed by boot (`findResume`) however sign-in returns, and first-run onboarding leaves a
visitor on that platform rather than on Getting started. The server serves this page to every
visitor.
**Analytics for finds** (PostHog through `track`, anonymous until sign-in, when the visitor's
earlier events join the identified person): `search_opened` (`ref`: the landing's Tools link sends
`?ref=landing-nav` or `landing-footer`, else the referring host), `catalog_find` (a find ran:
`surface` search or catalog, `words`, `auto`), `search_answered` (`verdict`, `results`,
`providers`, `top_fit`), `search_result_clicked` (`from` card, job or tile; `platform`,
`provider`, `rank`, `signed_in`) and `search_copied` (`scope` all, job or share). The landing's
Tools links also send `nav_clicked`. A cohort of people who performed `catalog_find`, followed
through `signup_completed`, `tool_called` and `topup_completed`, is the search-to-conversion
funnel.
## Code surfaces (every page)
Snippet blocks (`.lc-codewrap` on Getting started, the in-app CLI tutorial's `.term` panes, the
standalone `/tutorial`, the connect/setup instruction panes, the ledger's `treg call` line and captured
@@ -1161,9 +1229,10 @@ them.
## The Referrals view
`ReferralsPage.vue` renders the referrals view. The maintained Dashboard exposes a fixed
`Refer a friend` link at the bottom left, leaving the bottom right for the support messenger.
`dashboard.css` keeps this placement on desktop and mobile.
`ReferralsPage.vue` renders the referrals view. The maintained Dashboard's entry is a pill in the
top bar that names the offer ("Give $5, get $5") from `/meta.referral`, falling back to
`Refer a friend` when either amount is zero or `/meta` has not loaded. Narrow screens show only
its gift icon.
**`'referrals'` must appear in BOTH view whitelists** — `viewFromHash()` and the `popstate` handler.
`go('referrals')` works on click regardless of them; those two lists are what make the view survive
docs/context/interface/enrich-arena.md
+23 -17
View File
@@ -13,8 +13,6 @@ sources:
- src/treg/web/enrich-arena.html
- src/treg/web/enrich-arena/arena.js
- src/treg/web/enrich-arena/bench.js
- tests/js/arena-bench.test.cjs
- tests/js/arena-template.test.cjs
- src/treg/web/enrich-arena/arena.css
- src/treg/web/agent-setup.js
- src/treg/application/arena_verification_insights.py
@@ -40,7 +38,6 @@ sources:
- src/treg/web/logos/tomba.svg
- src/treg/web/sitetrack.js
- tests/test_enrich_arena.py
- tests/js/enrich-arena.test.cjs
related:
- architecture/catalog.md
- architecture/money.md
@@ -196,8 +193,10 @@ configured platform margin. Account reads and the excluded batch/file surface do
Email inputs require a nonempty mailbox and dotted domain. Malformed domain/LinkedIn URLs,
invalid ports, embedded credentials and non-web schemes return validation errors before pricing
or charging, including malformed bracketed hosts that URL parsing would otherwise reject with an exception.
Each provider contributes one eligible synchronous endpoint. Bulk jobs,
asynchronous submissions and personal-email finders are excluded from the work-email task.
Each provider contributes one eligible endpoint whose adapter verifies the task contract. Verified
asynchronous submissions participate through the same planner and quote surfaces as synchronous
providers; Arena handles submit, poll and terminal normalization internally. Bulk jobs and
personal-email finders remain excluded from the work-email task.
`?capability=people.email.find&mode=waterfall` opens a task/mode directly.
## Historical vendor insights
@@ -264,8 +263,15 @@ database the money path depends on. In the worker process it uses the API pool,
there. The rewind that
revisits ten minutes of evidence is constrained to the Arena's endpoints so it rides
`ix_callrecord_endpoint_id_created_at` instead of walking the table.
It reads 100 audit records per transaction, follows their exact archive key/content and optional body
carrier, reclassifies stored responses with current Arena required-field rules, and upserts anonymous
It reads a bounded batch of 100 audit records and their exact archive key/content pointers, closes
its metadata session, then resolves bodies through `archive_bodies.read` with path `arena` and the
result read switch. Object reads have concurrency eight and retain the existing decode-size cap.
Invalid compressed DB bodies remain unresolved per record rather than stopping collection of
other evidence, including when decompression occurs in the common reader's DB fallback.
The worker uses `bootstrap.archive_object_store` for the client lifecycle and drains read telemetry
before exiting. It reacquires the cursor row lock and validates cursor, cutoff and completion state
before publishing; evidence from a superseded batch is discarded. Thus no cursor lock or DB
connection is held across R2 I/O. It reclassifies responses with current required-field rules and upserts anonymous
`ArenaObservation` facts. It never calls vendors or trusts `CallRecord.hit`. No money writes or proxy
changes are involved. Evidence lookup deduplicates key/content pairs and finds each pair's newest
matching snapshot through the existing `(key_id, version)` index. This avoids repeatedly scanning
@@ -466,7 +472,9 @@ Arena never writes balances or holds. Own keys remain unmetered by treg. Aggrega
disabled for these comparisons, and archive lookup is bypassed so runs measure fresh calls.
Database sessions are short and closed before upstream requests. Attempt state and call references
persist before dispatch. Each leg has a 90-second deadline and the run has a 240-second deadline.
persist before dispatch. Synchronous legs have a 90-second deadline; an async leg may poll within
the remaining 240-second per-entry run deadline. No database session remains open during polling
waits or upstream I/O.
Cancellation is polled between writes and interrupts in-flight tasks through the normal call
cleanup. Shutdown drains Arena owners before closing the shared HTTP client. A process-lost run
becomes interrupted after its persisted deadline; it is never automatically retried. Unknown
@@ -486,8 +494,13 @@ original intermittent failure. Audit/archive drains alone cannot release this re
Waterfall uses ascending quoted prices, retaining planner order for ties, and the bounded error fallback policy. It stops
at the first structural hit: the adapter supplies the contract's required fields. Found work email
does not mean verified deliverability; phone found does not mean a live line. A negative mailbox
verification verdict is a successful answer. Each step shows queued/running, found/no match,
error/timeout, skipped/not attempted, timing, charge, and the reason for stopping or skipping.
verification verdict is a successful answer. An async attempt that remains in progress, or whose
foreground poll cannot prove a declared terminal state, is saved and shown as pending with its
reservation and call reference. Pending stops only that entry's Waterfall; Battle may finish other
explicitly selected providers concurrently. Only a declared terminal provider status permits the
entry to advance. Terminal attempts show the actual settled charge, not the maximum reservation.
Each step shows queued/running, pending, found/no match, error/timeout, skipped/not attempted,
timing, charge, and the reason for stopping or skipping.
## Additional vendor calls and issue reports
@@ -583,9 +596,6 @@ Alembic revision `0027` creates `arenarun` and `arenaevaluation`. Run `python -m
before serving the new release. Tests cover auth/private access, aggregate admission, direct billing,
own keys, cancellation, duplicate start/vote, attributed progress/results and pre-charge name validation, waterfall progression and OAuth return.
Frontend billing-flow checks: `node --test tests/js/enrich-arena.test.cjs` exercises inline pricing,
price invalidation, login gating, duplicate clicks, quote expiry and the correct-team top-up link.
### Conversion tracking
`TregTracking` in `sitetrack.js` connects anonymous pageviews to the authenticated email and
@@ -719,10 +729,6 @@ the page discloses that difference and does not manufacture an overall score or
numbers as individual vendor hit rates or email-verification accuracy. The new charts update when
the existing landing source changes. No paid benchmark execution is triggered by viewing them.
`tests/js/arena-template.test.cjs` compiles the shared page and component templates using the
bundled Vue runtime. This catches malformed template expressions that method-only tests miss,
including the nested footer interpolation that previously prevented all Arena views from mounting.
## Published verification pilot
`application.arena_verification_insights` validates and publishes aggregate-only pilot data in
docs/context/interface/env-import.md
+2 -2
View File
@@ -23,7 +23,7 @@ Bare `treg upload` does **both** sides of the dir; `treg upload env` / `treg upl
each as a tool (+ recipe) or a recipe-only bundle — see "Skill directories" below.
## The provider catalog (`CATALOG`)
~80 curated providers (`CATALOG_VERSION`, now **15** — LimaData (`x-api-key`) joined on 2026-09-17; MoltSets, Financial Datasets, Exa, Crustdata and Aviato are included, and
~80 curated providers (`CATALOG_VERSION`, now **21** — Fetchin (`X-API-Key`) joined on 2026-09-25; ScrapeGraphAI, LimaData, MoltSets, Financial Datasets, Exa, Crustdata and Aviato are included, and
`required_headers` can describe a fixed protocol header such as Crustdata's API-version pin; `probe`
remains the cheap authenticated GET path per provider so an imported tool self-validates, followed by
a wave of `cli` local-run blocks plus the CLI-only
@@ -57,7 +57,7 @@ real machine test (docs lie — Vercel ships an env var it ignores, so it inject
verified); `beta` marks an unverified entry. Several entries now carry **`deny`** patterns for subcommands
that would print the injected key or run member code as the isolated runner (`gh extension`/`alias`/`auth
token`/`--show-token`, `flyctl|turso auth token`, `doppler|infisical run`, …) — enforced by `check_deny` at
grant (see [local-run](../architecture/local-run.md)). `CATALOG_VERSION` is now **15**. An `unsupported:true` block is first-class: it tells the analyzer
grant (see [local-run](../architecture/local-run.md)). `CATALOG_VERSION` is now **21**. An `unsupported:true` block is first-class: it tells the analyzer
WHY and what to do instead (e.g. **Azure** — device-login only → register a service principal as an HTTP tool).
The catalog can never ENABLE a local run — only the owner's `tool.cli.enabled` does.
docs/context/interface/seo.md
+8 -2
View File
@@ -247,8 +247,14 @@ in `frontend/e2e/dashboard.spec.ts` checks public catalog navigation and reachab
### The no-JS fallback
Vue compiles `#app`'s own innerHTML as its template, so prerendered markup **cannot go inside it**.
`#prerender` is a sibling, removed by the app on boot.
Vue replaces `#app`'s content on mount, so prerendered markup **cannot go inside it**. `#prerender`
is a sibling, removed by the app on boot, and **visually hidden** (`_PRERENDER_HIDDEN`): shown to
people, the plain list flashed past as an older second page before the app replaced it. It stays in
the document for readers that run no script, which include most AI crawlers and agent fetchers.
`/search` keeps its own rule: a full-viewport ground in the page colour, so it opens with no
loading step at all. The same response embeds the `/catalog/platforms` body as
`<script id="catalog-platforms" type="application/json">`, which `loadPlatforms` reads instead of
fetching, so the app's first render already has the shelves.
It is deliberately plainer than the Vue view. The ledger's row-merging is a chain of client-side
computeds (`platRowsAll` → `platRowsPreDomain` → `platLedger`), and reproducing that server-side
docs/context/ops/capacity.md
+40 -2
View File
@@ -45,6 +45,16 @@ related:
# Provider capacity
Fetchin capacity is `credits / manual / api`. `collectors._fetchinio` calls the free internal
`GET /api/v1/subscription` route with the platform `X-API-Key`, accepts only a finite nonnegative
`creditsRemaining`, and retains plan status, PAYG remainder, renewal date and the account's reported
RPS limit as informational notes. The route can be polled after quota exhaustion and remains
internal capacity evidence rather than a catalog tool. The funded account reported 5 requests per
second. Because combined post engagement consumes two rate-limit units and smoothing is not
endpoint-weighted, the shared-key policy conservatively uses two calls per second; BYOK bypasses it.
The account was not deliberately exhausted, so the documented generic HTTP 402 is acknowledged
without a provider-specific empty-balance body or overflow route.
TinyFish capacity is `cash / manual / api`. `collectors._tinyfish` calls the free internal
`GET /v1/wallet` route with the platform `X-API-Key`, accepts only a finite nonnegative
`available_balance`, and retains the response currency plus whether vendor auto-reload is enabled.
@@ -83,6 +93,22 @@ top-up. A controlled `/usage` burst did not reproduce its documented 10-per-10-m
rate policy remains documentation-derived. The funded account was not deliberately exhausted;
432/433 signatures are documentation-derived rather than live-observed.
ScrapeGraphAI's internal collector calls the free `GET /api/credits` route with the platform
`SGAI-APIKEY`. It accepts only a finite nonnegative `remaining` credit balance and retains the plan,
used-credit count, and crawl/monitor job quotas as informational notes. The policy is
`credits / subscription / api`: the API balance is exact and the shared account uses subscription
funding. Shared-key smoothing uses the configured 500 requests per minute;
live responses supplied no usable rate-limit headers. The credits route remains internal capacity
evidence rather than a catalog tool, and no funding automation or exhaustion signature is inferred.
Serper's internal collector calls `GET /account` with the platform `X-API-KEY`. It accepts only a
finite nonnegative `balance` and records Serper's numeric `rateLimit` as an informational note. The
policy is `credits / auto_recharge / api`: the API balance is exact, and vendor auto recharge was
manually enabled and verified in the dashboard. The live shared account reports 50 requests per
second, so shared-key smoothing uses 50 requests per second. The account route remains internal
capacity evidence rather than a catalog tool. The funded account was not deliberately exhausted,
so no provider-specific empty-balance signature or overflow route is claimed.
TrestleIQ publishes no free balance or usage API. Capacity reports the wallet as Developer
Portal-only and does not spend a validation query to read it. The policy records cash with vendor
auto recharge, manually verified as enabled in the portal, and a documented 10 requests/second
@@ -339,7 +365,8 @@ pays the aggregator's real price, 0% markup, disclosed in-band when it ships (st
parameterized locations have no mapped fallback.
- **Mark scope on a failed child** (`overflow.py`): only `aggregator_auth` and `aggregator_balance`
mark `overflow:<aggregator>` for every provider. Everything else - the aggregator's account for
the vendor being dry, and a `malformed` answer (a 5xx, a transport timeout, a non-envelope) - marks
the vendor being dry, a vendor-specific authentication or authorization refusal, and a
`malformed` answer (a 5xx, a transport timeout, a non-envelope) - marks
`overflow:<aggregator>:<provider>`. On 2026-09-17 one Orthogonal Apollo relay answering
"timeout of 30000ms exceeded" marked the whole aggregator and refused every other provider's
fallback for 15 minutes, including 62 Influencers Club `similar` calls from one team. A dead
@@ -370,6 +397,8 @@ pays the aggregator's real price, 0% markup, disclosed in-band when it ships (st
- **`infra/upstream/aggregators/`** — the envelopes, and nothing else: `build()` wraps the
vendor request (Orthogonal `POST /run {api, path, query, body}`; Monid `POST /run {provider,
endpoint, input}`), `parse()` unwraps the vendor status + body + the real in-band charge, and
`build()` restores JSON scalar types that Monid validates and converts Akta enrichment's native
comma-separated `sections` query value to Monid's array-shaped envelope field, and
names who to blame when the aggregator itself refused (`AGGREGATOR_SIDE` = `aggregator_auth`,
`aggregator_balance`, `malformed` - the call path marks the aggregator unhealthy for everyone, the
verifier leaves the route alone; `contract` - the aggregator's own per-request refusal, including
@@ -379,6 +408,12 @@ pays the aggregator's real price, 0% markup, disclosed in-band when it ships (st
signature table - the one place a relayed body is read - is the aggregator's account for THIS
vendor (a relayed 402, Apollo's 422, a period 429): the call path marks
`overflow:<aggregator>:<provider>` only, so one vendor's cap never takes the others offline.
An otherwise unrecognized relayed vendor 401/403 becomes `VENDOR_REFUSAL` and uses that same
provider-scoped mark: repeated calls pause briefly without treating the aggregator as globally
unavailable.
A valid Monid run envelope takes precedence over its outer HTTP status: Monid mirrors relayed
vendor 401/402/403 statuses, so only a refusal with no run id proves the Monid key or balance
failed; a completed run unwraps `providerResponse.error` for the vendor signature table.
Deliberately not the direct path's strike ladder: the mark is immediate and a flat 15 min, because
a relayed body carries no headers to tell a burst from a cap and the caller has already paid the
aggregator's round trip;
@@ -388,7 +423,10 @@ pays the aggregator's real price, 0% markup, disclosed in-band when it ships (st
- **`verify.py`** + `treg-worker overflow verify` — the weekly re-verify: one cheap call per
route through the aggregator (and, when we hold the vendor key, directly), compare the shape
fingerprint (keys and list/leaf markers, values ignored), stamp `last_verified_at` or disable
with the reason. Two per-route price caps and one run budget: a route that is enabled or was
with the reason. A mismatch prints a bounded key-only structural diff, never response values;
identifier-shaped map keys are replaced before logging or persistence, so an operator can
distinguish omitted metadata from an incompatible body without exposing PII.
Two per-route price caps and one run budget: a route that is enabled or was
stamped before is a **renewal**, held to `--renew-max-usd` (default $1); a never-verified pair is
**discovery**, visited only under `--all` and held to `--max-usd` (default 2¢). Renewals go first,
oldest stamp first, so the route nearest its 7-day decay is reached before `--budget-usd`
docs/context/ops/deploy.md
+16 -14
View File
@@ -81,8 +81,8 @@ closed maintenance loop. Calling `maintenance.upgrade()` directly does not dispo
metadata. It never creates tables, stamps versions or runs release tasks. Worker commands use the
same check.
- **Schema changes are revision-only.** An autogenerate drift guard requires Alembic head and
`SQLModel.metadata` to match exactly. `reset_db()` uses `create_all` only for fast test isolation
and stamps that test schema directly at head.
`SQLModel.metadata` to match exactly. `reset_db()` uses `create_all` only to build a missing test
schema and stamps it directly at head.
- **A missing encryption key fails loudly on a real database.** If `TREG_SECRET_KEY` is empty and
`database_url` is not SQLite, `verify_db()` raises. On SQLite development it logs a warning.
@@ -233,16 +233,12 @@ database is local SQLite. Hosted deployments must still leave it false.
## Web service and generic Render example
The redesigned homepage ships to all homepage visitors independently of Dashboard rollout.
Its rollback requires a code rollback/revert; the Dashboard master switch does not change it.
The redesigned homepage ships to all homepage visitors; its rollback, like the Dashboard's, is a
code revert or a deploy of the previous build.
`GET /app` selects either the frozen legacy artifact or the Vite-built Vue application.
The rollout defaults to legacy. Set `TREG_DASHBOARD_ROLLOUT_ENABLED=true` with a JSON array in
`TREG_DASHBOARD_ROLLOUT_USER_IDS` for an account allowlist, then increase
`TREG_DASHBOARD_ROLLOUT_PERCENT` from zero. Disabling the master switch forces legacy, including
allowlisted accounts. Environment changes require restarting Web processes, not rebuilding assets.
Both frontends ship together; anonymous catalog/sign-in entries remain legacy even at 100%.
See `frontend/README.md` for the full rollout and retirement contract.
`GET /app`, the catalog pages and shared links all serve the Vite-built Vue application. There is
no frontend switch to configure; a Dashboard rollback is a deploy of the previous build. See
`frontend/README.md`.
The frontend is authored in `frontend/` within the same repository. `GET /` retains the existing
landing behavior. Dashboard assets, tutorials, agent files and installer assets ship with the wheel.
Hosted-page MP4 demos remain in Git checkout deployments but are excluded from published wheels and
@@ -340,14 +336,20 @@ without importing the heavy database stack into the light `treg` CLI.
- `treg-worker asynctasks settle` completes durable holds for asynchronous upstream operations.
- `treg-worker arena insights` folds new audit rows into the rolling Arena aggregate
(`--max-seconds`, default 110, bounds one pass; schedule it every two minutes).
It requires the archive object-store settings when R2 reads are enabled, opens the same client
lifecycle as the web service, and flushes read analytics before exiting.
- `treg-worker catalog stats` folds new audit rows into per-endpoint, per-day reliability buckets
(`--max-rows`, default 500,000, bounds one pass; schedule it every few minutes). The catalog keeps
computing observations live until this command has caught up once, so it can be scheduled after
the application deploys, and a self-hosted registry that never schedules it loses nothing.
- `treg-worker jev xboost` runs the `/jev` launch-radar demo once a day: it calls treg's own `/call/` API
with `TREG_JEV_TREG_TOKEN` (a member token of the demo team, so the spend is an ordinary bill) and jev
through the Vercel AI Gateway (`TREG_AI_GATEWAY_API_KEY`), and stores the run under Ephemeral for the page.
Both variables also belong on the web service, which needs them for the visitor judge endpoint.
(`--max-rows`, default 500,000, bounds one pass; schedule it every few minutes). The catalog keeps
computing observations live until this command has caught up once, so it can be scheduled after
the application deploys, and a self-hosted registry that never schedules it loses nothing.
- `treg-worker admin purge-evidence` blanks failed-call evidence past the 14-day retention window
(`--batch-size`, default 5000, rows per transaction; schedule it daily). `GET /admin/errors` is
read-only and already withholds evidence past the window, so an unscheduled purge keeps the old
bytes in the database but never shows them.
The two analytics commands exist so that no web process aggregates the audit table beside the
money path; `callrecord` is read only through the persisted cursors they own. Workers call
dsh/skills/treg/SKILL.md
+11 -2
View File
@@ -1,6 +1,6 @@
---
name: treg
description: Reach for this first for external or live data. 3,600+ endpoints across 94 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
description: Reach for this first for external or live data. 3,600+ endpoints across 97 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
---
## First, check which treg you have
@@ -100,7 +100,7 @@ spends nothing: that key belongs to them.
## Task — the catalog: what treg can do for you (start here)
3,600+ catalogued endpoints across 94 providers, grouped by what they DO: keyword & rank tracking,
3,600+ catalogued endpoints across 97 providers, grouped by what they DO: keyword & rank tracking,
backlinks & authority, AI visibility, trending & discovery, publishing to the team's own social
accounts, people & company enrichment, ads management & creative, measurement, video & image
generation.
@@ -127,6 +127,11 @@ Notes:
to the balance — they take priority automatically). A 402 with `error: route_max_cost` is
different: YOUR `X-Treg-Route-Max-Cost` header refused the call before anything was charged —
ask for fewer rows/targets or raise the ceiling.
- **Scripting many calls:** use `treg --json call …`. Stdout is one line,
`{"result": <provider body>, "_treg": {"http_status", "call_id", "charged_micro"}}`, and nothing
goes to stderr, so a script that merges the streams still parses every answer (`--await` output
is unchanged). Run a handful and check the parsed results before looping over the whole list: a
parse bug throws away answers that were already billed.
- The real charge is the response header `X-Treg-Cost-Micro` (micro-USD), with `X-Treg-Call-Id`
as the id to quote. On an asynchronous submission that header is the reserved ceiling; the CLI
labels it as a reservation, and the terminal task settles the real charge. The catalog `~$/call`
@@ -192,6 +197,10 @@ Notes:
still sent to the others, and the answer names it in `X-Treg-Ignored-Filters` / `_treg.ignored_filters`
— post-filter, or send `X-Treg-Route-Strict-Filters: 1` to get a 422 (unbilled) instead of a looser
answer. `catalog_get treg.people.email.find` shows the plan and prices.
An async child is submitted and polled internally for up to 60 seconds. If it is still running,
treg returns HTTP 202 with `_treg.outcome: pending`, its call reference and poll descriptor,
`reserved_micro`, and `charged_micro: null`; do not retry or start another provider, because the
existing task may still complete and charge.
- **A found contact is not a confirmed one.** An email or phone find returns the provider's best
match; only `output.verified: true` means it checked the mailbox. When it is not, the answer
carries `_treg.advice` naming the verify step (`treg.people.email.verify`, a fraction of a cent)
frontend/README.md
+8 -45
View File
@@ -14,10 +14,7 @@ This is an incremental extraction. The old use cases still share per-application
`state/context.ts`; their JavaScript and the shared onboarding widgets are not fully typed.
New isolated components should use typed props and events. Existing hash navigation and deep links
remain in the navigation/catalog/details modules; this change does not replace their URL contract.
The deprecated, frozen rollback artifact lives in `src/treg/web/dashboard-legacy/` and exists
only during rollout. `frontend/` is the only maintained Dashboard source. Do not backport features
or routine fixes, or refresh the snapshot when syncing main. The server selects the frontend by
authenticated user ID.
`frontend/` is the only Dashboard source: every entry, signed in or not, serves this compiled app.
## Develop
@@ -49,8 +46,8 @@ Browser tests start their own server on :18791 with a disposable database and no
They use full Chromium in headless mode so back/forward cache restoration is exercised.
`PLAYWRIGHT_CHANNEL=chrome` can use an installed Chrome for local checks.
Builds also copy the npm-installed Vue global runtime and license for standalone pages and the
legacy snapshot; these generated files are packaged but never committed. Page runtime versions
Builds also copy the npm-installed Vue global runtime and license for the standalone Arena page;
these generated files are packaged but never committed. Page runtime versions
must match the npm lockfile. Three.js and Lenis on the landing page use pinned CDN URLs.
Builds generate `src/treg/web/dashboard/`, which is ignored by Git and included in wheels/sdists.
@@ -58,43 +55,9 @@ Do not edit generated files. Distributable package builds fail if these assets a
Python installs and background workers do not require Node. The Web build script is
`scripts/build-web.sh`, which compiles the app and retains the locked Python installation.
## Gradual rollout
## Serving and rollback
The homepage (`/`) uses the new landing page for all visitors; it has no experiment or rollout
switch. The settings below apply only to the Dashboard, catalog and shared-link entries.
Disabling Dashboard rollout does not revert the homepage.
Production defaults to the frozen legacy Dashboard frontend. Configure the Web service:
- `TREG_DASHBOARD_ROLLOUT_ENABLED=true` enables rollout; `false` forces legacy for everyone.
- `TREG_DASHBOARD_ROLLOUT_USER_IDS='[123,456]'` is the JSON array of allowed numeric user IDs.
- `TREG_DASHBOARD_ROLLOUT_PERCENT=0` starts with only the allowlist. Increase toward 100 to
include stable account buckets; email changes, team switches and browser changes do not reshuffle them.
Anonymous visitors (including the public catalog and token-only browsers) stay on legacy.
After browser sign-in, the reload selects the account's frontend. All dashboard, catalog and
shared-link entries use the same selection and private, no-store HTML. Frontend selection grants
no API permissions. Legacy JavaScript is frozen under its own revision-qualified asset URLs.
PostHog is not involved. Environment changes require a process restart/rolling deployment;
rollback needs no frontend rebuild. Existing tabs switch on reload, and configuration changes
also change the app-version stamp so open tabs can offer a refresh.
The local dev script enables 100% for signed-in accounts by default. Override its rollout variables
to rehearse production settings.
## Retire the deprecated Dashboard
Legacy is temporary, not a permanently supported version. Remove it in a follow-up change once
the new Dashboard is validated at full account rollout and the release no longer needs the frozen
fallback. Setting the percentage to 100 is not retirement: anonymous and token-only visitors still
use legacy under the current policy.
- Route every Dashboard entry to the compiled app, including anonymous catalog, shared links,
token-only entries and sign-in. Verify those flows and authenticated account flows in the browser.
- Remove `src/treg/web/dashboard-legacy/`, `/app/legacy/assets/{path:path}`, the account-selection
branch, all three `TREG_DASHBOARD_ROLLOUT_*` settings and their app-version stamp inputs.
- Remove obsolete rollout tests, local defaults and packaging checks; retain coverage for the
surviving entry routes, sessions and compiled assets. Update build/deployment documentation and
remove the retired settings from the private deployment configuration in a paired change.
- Remove the deprecation instructions from `AGENTS.md` and context docs once removal ships.
Deployment rollback remains the recovery path after the in-process fallback is removed.
The server hands every Dashboard, catalog and shared-link entry the same compiled `index.html`,
as private, no-store HTML with `Vary: Cookie`. There is no in-process frontend switch: roll back a
Dashboard change by deploying the previous build. Open tabs compare the app-version stamp in `/meta`
and offer a refresh when a deploy changes the bundle.
frontend/e2e/dashboard.spec.ts
+4 -76
View File
@@ -32,46 +32,10 @@ test('sign in, create team, switch pages, refresh and navigate back', async ({ p
await page.locator('.rd-account-menu summary').click()
await page.locator('.rd-account-menu').getByRole('button', { name: 'Billing', exact: true }).click()
await expect(page).toHaveURL(/#orgs$/)
expect(errors).toEqual([])
})
test('signed-in users can visit the homepage and return to the dashboard', async ({ page }) => {
await page.emulateMedia({ reducedMotion: 'reduce' })
await signIn(page)
await page.getByRole('link', { name: 'treg home' }).click()
await expect(page).toHaveURL('http://127.0.0.1:18791/')
await expect(page.getByRole('heading', { level: 1 })).toContainText('OpenRouter for agent tools')
await expect(page.getByRole('link', { name: 'Sign in', exact: true })).toHaveCount(0)
await page.locator('.nav').getByRole('button', { name: 'Open dashboard' }).click()
await expect(page.getByRole('navigation', { name: 'Primary navigation' })).toBeVisible()
})
test('onboarding controls and images work on mobile and dark theme', async ({ page }, testInfo) => {
const errors: string[] = []
page.on('pageerror', error => errors.push(error.message))
await signIn(page)
await page.getByRole('button', { name: 'Getting started', exact: true }).click()
await page.setViewportSize({ width: 390, height: 844 })
await expect(page.locator('.rd-start')).toBeVisible()
const trigger = page.locator('[aria-controls="rd-agent-options"]')
await trigger.click()
await page.locator('#rd-agent-options').getByRole('button', { name: 'Codex', exact: true }).click()
await expect(trigger).toContainText('Codex')
await page.getByRole('button', { name: 'Show key', exact: true }).click()
await expect(page.getByRole('button', { name: 'Hide key', exact: true })).toBeVisible()
await page.getByRole('button', { name: 'Hide key', exact: true }).click()
await page.evaluate(() => Object.defineProperty(navigator, 'clipboard', {
configurable: true, value: { writeText: () => Promise.reject(new Error('denied')) },
}))
await page.locator('.rd-setup-panel').first().getByRole('button', { name: 'Copy', exact: true }).click()
await expect(page.getByRole('alert')).toContainText('Could not copy')
await page.getByRole('button', { name: 'Dismiss', exact: true }).click()
await page.locator('.rd-account-menu summary').click()
await page.getByRole('button', { name: 'Dark appearance' }).click()
await expect(page.locator('html')).toHaveAttribute('data-theme', 'dark')
await page.waitForFunction(() => [...document.querySelectorAll<HTMLImageElement>('.rd-try .try-ico')].every(img => img.complete && img.naturalWidth > 0))
expect(await page.evaluate(() => document.documentElement.scrollWidth <= innerWidth)).toBe(true)
await page.screenshot({ path: testInfo.outputPath('mobile-dark.png'), fullPage: true })
const referral = page.getByRole('link', { name: 'Refer a friend: Give $5, get $5', exact: true })
await expect(referral).toHaveText('Give $5, get $5')
await referral.click()
await expect(page).toHaveURL(/#referrals$/)
expect(errors).toEqual([])
})
@@ -92,39 +56,3 @@ test('public catalog and shared deep links remain available without a session',
await expect(page.getByRole('dialog', { name: 'Sign in' })).toBeVisible()
expect(errors).toEqual([])
})
test('session initialization never flashes the old signed-out landing page', async ({ page }) => {
await signIn(page)
let releaseSession!: () => void
const sessionGate = new Promise<void>(resolve => { releaseSession = resolve })
await page.route('**/auth/me', async route => { await sessionGate; await route.continue() })
await page.reload()
await expect(page.getByRole('status')).toHaveText('Loading treg…')
await expect(page.getByText('Sign in to treg', { exact: true })).toHaveCount(0)
releaseSession()
await expect(page.getByRole('navigation', { name: 'Primary navigation' })).toBeVisible()
})
test('mainline team resources survive navigation and open the voice tools', async ({ page }) => {
await signIn(page)
await page.route('**/provider-resources?source=platform', route => route.fulfill({
json: [{ id: 1, provider: 'fishaudio', kind: 'voice', upstream_id: 'test-private-voice', display_name: 'Test voice', status: 'active' }],
}))
await page.getByRole('navigation', { name: 'Primary navigation' }).getByRole('button', { name: 'Your own tools', exact: true }).click()
await page.getByRole('button', { name: 'Team resources', exact: true }).click()
await expect(page.getByRole('heading', { name: 'Team resources', exact: true })).toBeVisible()
await expect(page.getByText('Test voice', { exact: true })).toBeVisible()
await page.reload()
await expect(page.getByRole('heading', { name: 'Team resources', exact: true })).toBeVisible()
await page.getByRole('button', { name: 'Rename', exact: true }).click()
const dialog = page.getByRole('dialog', { name: 'Rename voice', exact: true })
await expect(dialog.getByRole('textbox')).toHaveValue('Test voice')
await expect(dialog.getByRole('textbox')).toBeFocused()
await dialog.getByRole('button', { name: 'Cancel', exact: true }).click()
await page.getByRole('button', { name: 'Use in TTS', exact: true }).click()
// The disposable server has no provider credentials; verify the prepared request
// through the API tab, which is available without enabling paid execution.
const drawer = page.getByRole('dialog').filter({ hasText: 'Try “fishaudio.tts.s2-1-pro”' })
await drawer.getByRole('button', { name: 'API', exact: true }).click()
await expect(drawer.locator('pre')).toContainText('"reference_id": "test-private-voice"')
})
frontend/e2e/landing.spec.ts
+5 -101
View File
@@ -1,92 +1,5 @@
import { expect, test } from '@playwright/test'
// Enable BFCache for history regression coverage.
test.use({ launchOptions: { ignoreDefaultArgs: ['--disable-back-forward-cache'] } })
test.describe('browser history', () => {
test('restores the 3D scene and catalog scrolling from the back/forward cache', async ({ page }) => {
// Software WebGL on CI renders the scene much more slowly than a desktop GPU.
test.setTimeout(120000)
// Allow BFCache on local HTTP; production cache headers stay unchanged.
await page.route('http://127.0.0.1:18791/', async route => {
const response = await route.fetch()
await route.fulfill({ response, headers: { ...response.headers(), 'cache-control': 'private, no-cache' } })
})
await page.goto('/')
await page.unrouteAll()
await expect(page.locator('.gateway-sculpture')).toHaveAttribute('data-model-state', 'ready', { timeout: 20000 })
await page.keyboard.press('Escape')
await page.evaluate(() => {
window.addEventListener('pageshow', event => {
document.documentElement.dataset.historyRestored = String(event.persisted)
})
})
for (let cycle = 0; cycle < 2; cycle++) {
await page.evaluate(() => window.scrollTo({ top: 0, behavior: 'instant' }))
await page.locator('.nav').getByRole('link', { name: 'Catalog', exact: true }).click()
await expect(page).toHaveURL(/\/catalog$/)
await page.evaluate(() => history.back())
await expect(page.locator('html')).toHaveAttribute('data-history-restored', 'true')
await expect(page.locator('.gateway-webgl')).toBeVisible()
await expect(page.locator('.gateway-webgl')).toHaveCount(1)
await expect(page.locator('.hero-particles')).toHaveCount(1)
await expect(page.locator('.command-beam')).toHaveCount(1)
await page.evaluate(() => window.scrollTo({ top: document.querySelector<HTMLElement>('#catalog')!.offsetTop + 200, behavior: 'instant' }))
const track = page.locator('#catalog .catwrap')
await expect.poll(() => track.evaluate(el => getComputedStyle(el).transform)).not.toBe('none')
const before = await track.evaluate(el => getComputedStyle(el).transform)
await page.mouse.wheel(0, 450)
await expect.poll(() => track.evaluate(el => getComputedStyle(el).transform)).not.toBe(before)
}
})
})
test('the hero does not flash a placeholder while the 3D module loads', async ({ page }) => {
let release!: () => void
const loading = new Promise<void>(resolve => { release = resolve })
await page.route('**/media/landing/gateway-3d.js', async route => {
await loading
await route.continue()
})
try {
await page.goto('/', { waitUntil: 'commit' })
await expect(page.locator('.gateway-sculpture')).toBeAttached()
await expect(page.locator('.hcore')).toBeHidden()
} finally {
release()
}
await expect(page.locator('.gateway-sculpture')).toHaveAttribute('data-model-state', 'ready', { timeout: 20000 })
})
test('landing renders its CDN-backed 3D scene and copies the serving-origin setup command', async ({ page, context }) => {
test.setTimeout(120000)
const errors: string[] = []
const failedAssets: string[] = []
page.on('pageerror', error => errors.push(error.message))
page.on('response', response => {
if ((response.url().includes('/media/landing/') || response.url().includes('cdn.jsdelivr.net/npm/')) && !response.ok()) failedAssets.push(response.url())
})
await context.grantPermissions(['clipboard-read', 'clipboard-write'])
await page.goto('/')
await expect(page.locator('.gateway-sculpture')).toHaveAttribute('data-model-state', 'ready', { timeout: 20000 })
await page.keyboard.press('Escape')
await expect(page.locator('html')).not.toHaveClass(/opening-stage/)
// Record the short-lived announcement in the page, so a busy software WebGL renderer
// cannot make the test runner miss it between protocol round trips.
await page.getByRole('status').evaluate(status => {
new MutationObserver(() => {
if (status.textContent === 'Copied') status.setAttribute('data-copy-announced', 'true')
}).observe(status, { childList: true, characterData: true, subtree: true })
})
await page.getByRole('button', { name: 'Copy agent command', exact: true }).click()
await expect(page.getByRole('status')).toHaveAttribute('data-copy-announced', 'true')
expect(await page.evaluate(() => navigator.clipboard.readText())).toBe('set up treg - http://127.0.0.1:18791/llms.txt')
await page.getByRole('button', { name: 'Next agent scenario' }).click()
await expect(page.locator('#sc-tools button')).toHaveCount(6)
expect(failedAssets).toEqual([])
expect(errors).toEqual([])
})
test('landing email sign-in reaches the dashboard', async ({ page }) => {
await page.emulateMedia({ reducedMotion: 'reduce' })
await page.goto('/')
@@ -101,7 +14,7 @@ test('landing email sign-in reaches the dashboard', async ({ page }) => {
await expect(page.getByPlaceholder('Team name, e.g. Superdesign')).toBeVisible()
})
test('mobile reduced-motion landing remains usable when WebGL is unavailable', async ({ page }, testInfo) => {
test('landing stays usable at phone width without WebGL or the library CDN', async ({ page }) => {
await page.setViewportSize({ width: 390, height: 844 })
await page.emulateMedia({ reducedMotion: 'reduce' })
await page.addInitScript(() => {
@@ -111,7 +24,8 @@ test('mobile reduced-motion landing remains usable when WebGL is unavailable', a
return getContext.apply(this, [type, ...args] as Parameters<typeof getContext>)
} as typeof getContext
})
await page.goto('/')
await page.route('https://cdn.jsdelivr.net/npm/**', route => route.abort('failed'))
await page.goto('/', { waitUntil: 'domcontentloaded' })
await expect(page.locator('.gateway-sculpture')).toHaveAttribute('data-model-state', 'fallback')
await expect(page.locator('.hcore')).toBeVisible()
await expect(page.locator('html')).not.toHaveClass(/opening-stage/)
@@ -121,20 +35,10 @@ test('mobile reduced-motion landing remains usable when WebGL is unavailable', a
}))
await page.getByRole('button', { name: 'Copy agent command', exact: true }).click()
await expect(page.getByRole('status')).toHaveText('Copy failed, please retry')
await page.locator('footer').scrollIntoViewIfNeeded()
await expect(page.getByRole('link', { name: 'Enrich Arena' })).toBeVisible()
await page.screenshot({ path: testInfo.outputPath('landing-mobile-fallback.png'), fullPage: true })
})
test('landing keeps native scrolling and sign-in when the library CDN is unavailable', async ({ page }) => {
await page.route('https://cdn.jsdelivr.net/npm/**', route => route.abort('failed'))
await page.goto('/', { waitUntil: 'domcontentloaded' })
await expect(page.locator('.gateway-sculpture')).toHaveAttribute('data-model-state', 'fallback')
await expect(page.locator('.hcore')).toBeVisible()
await expect(page.locator('html')).not.toHaveClass(/opening-stage/)
await page.mouse.wheel(0, 450)
await expect.poll(() => page.evaluate(() => scrollY)).toBeGreaterThan(0)
await page.locator('footer').scrollIntoViewIfNeeded()
await expect(page.getByRole('link', { name: 'Enrich Arena' })).toBeVisible()
await page.getByRole('link', { name: 'Sign in', exact: true }).click()
await expect(page.getByRole('dialog', { name: 'Sign in', exact: true })).toBeVisible()
})
frontend/index.html
+19 -4
View File
@@ -7,9 +7,20 @@
<!-- The dashboard is an authenticated app: every view needs a session and supports shared deep links. Nothing here is indexable, and a crawler that got in would index a sign-in prompt. -->
<meta name="robots" content="noindex, follow"/>
<link rel="icon" type="image/svg+xml" href="/favicon.svg"/>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Google+Sans+Flex:opsz,wght@6..144,400..700&family=Geist+Pixel&family=DM+Mono:ital,wght@0,400;0,500;1,400&family=Inter:wght@400;450;500;600&display=swap" rel="stylesheet">
<!-- Boot's first two requests start here, alongside the bundle download, instead of after it
(boot.js picks them up), and the saved theme applies before the first paint, not after mount. -->
<script>
(function(){
var h={'ngrok-skip-browser-warning':'1'};
window.__tregBoot={
meta:fetch('/meta',{headers:h}).then(function(r){return r.json()}),
me:fetch('/auth/me',{credentials:'include',headers:h}).then(function(r){return r.ok?r.json():null})
};
// boot.js attaches its own handlers; these only keep an unused promise from logging a rejection.
window.__tregBoot.meta.catch(function(){}); window.__tregBoot.me.catch(function(){});
try{ document.documentElement.dataset.theme=localStorage.getItem('treg-theme')||'light'; }catch(e){}
})();
</script>
@@ -18,7 +29,11 @@
<script src="/adtrack.js"></script>
</head>
<body>
<div id="app"><p role="status">Loading treg…</p><p>If this page does not load, <a href="">reload</a> to get the latest version.</p></div>
<!-- The same markup App.vue shows while it boots, so mounting swaps it for itself. Nothing is
visible for the first moments; a thin bar fades in at the top only if boot is slow
(base.css, .boot-status). The status line hides itself inline: in development the stylesheet
arrives with the script, after this markup has painted. -->
<div id="app"><main class="boot-status" aria-busy="true"><span class="boot-bar" aria-hidden="true"></span><p role="status" style="position:absolute;width:1px;height:1px;overflow:hidden;clip-path:inset(50%);white-space:nowrap">Loading treg…</p></main></div>
<script src="/sitetrack.js"></script>
<script src="/agent-setup.js"></script>
<script type="module" src="/src/main.ts"></script>
frontend/package-lock.json
Generated
+35
View File
@@ -6,10 +6,14 @@
"": {
"name": "treg-dashboard",
"dependencies": {
"@fontsource/dm-mono": "5.3.0",
"@fontsource/geist-pixel": "5.3.0",
"matter-js": "0.20.0",
"vue": "3.5.41"
},
"devDependencies": {
"@playwright/test": "1.63.0",
"@types/matter-js": "0.20.2",
"@types/node": "26.6.2",
"@vitejs/plugin-vue": "6.0.9",
"typescript": "5.9.3",
@@ -67,6 +71,24 @@
"node": ">=6.9.0"
}
},
"node_modules/@fontsource/dm-mono": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@fontsource/dm-mono/-/dm-mono-5.3.0.tgz",
"integrity": "sha512-OINjI8C1S/wpchhQxl7njZdMn4+hnDCpQ4YtvvOpKNARo+0J8O1x1IcrChxNjHOhfVv1by8C/FQoy3hXK+C1Ug==",
"license": "OFL-1.1",
"funding": {
"url": "https://github.com/sponsors/ayuhito"
}
},
"node_modules/@fontsource/geist-pixel": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/@fontsource/geist-pixel/-/geist-pixel-5.3.0.tgz",
"integrity": "sha512-lno6MaxiPlE8ex9T8GMXvU9ZzGBdt1GSQ7ub00RMrhd7SUs9Qy/L6f3sLwzs1F7YcwAZiIa8pxW5dIqZz3zgCg==",
"license": "OFL-1.1",
"funding": {
"url": "https://github.com/sponsors/ayuhito"
}
},
"node_modules/@jridgewell/resolve-uri": {
"version": "3.1.2",
"resolved": "https://registry.npmjs.org/@jridgewell/resolve-uri/-/resolve-uri-3.1.2.tgz",
@@ -407,6 +429,13 @@
"dev": true,
"license": "MIT"
},
"node_modules/@types/matter-js": {
"version": "0.20.2",
"resolved": "https://registry.npmjs.org/@types/matter-js/-/matter-js-0.20.2.tgz",
"integrity": "sha512-3PPKy3QxvZ89h9+wdBV2488I1JLVs7DEpIkPvgO8JC1mUdiVSO37ZIvVctOTD7hIq8OAL2gJ3ugGSuUip6DhCw==",
"dev": true,
"license": "MIT"
},
"node_modules/@types/node": {
"version": "26.6.2",
"resolved": "https://registry.npmjs.org/@types/node/-/node-26.6.2.tgz",
@@ -1016,6 +1045,12 @@
"@jridgewell/sourcemap-codec": "^1.5.5"
}
},
"node_modules/matter-js": {
"version": "0.20.0",
"resolved": "https://registry.npmjs.org/matter-js/-/matter-js-0.20.0.tgz",
"integrity": "sha512-iC9fYR7zVT3HppNnsFsp9XOoQdQN2tUyfaKg4CHLH8bN+j6GT4Gw7IH2rP0tflAebrHFw730RR3DkVSZRX8hwA==",
"license": "MIT"
},
"node_modules/muggle-string": {
"version": "0.4.1",
"resolved": "https://registry.npmjs.org/muggle-string/-/muggle-string-0.4.1.tgz",
frontend/package.json
+4
View File
@@ -11,10 +11,14 @@
"predev": "node scripts/copy-runtime.mjs"
},
"dependencies": {
"@fontsource/dm-mono": "5.3.0",
"@fontsource/geist-pixel": "5.3.0",
"matter-js": "0.20.0",
"vue": "3.5.41"
},
"devDependencies": {
"@playwright/test": "1.63.0",
"@types/matter-js": "0.20.2",
"@types/node": "26.6.2",
"@vitejs/plugin-vue": "6.0.9",
"typescript": "5.9.3",
frontend/scripts/copy-runtime.mjs
+8 -11
View File
@@ -1,4 +1,4 @@
// Standalone pages need Vue's global build; the maintained app uses Vite's npm import.
// The standalone Arena page needs Vue's global build; the maintained app uses Vite's npm import.
import { copyFileSync, mkdirSync, readFileSync } from 'node:fs'
import { fileURLToPath } from 'node:url'
import { dirname, resolve } from 'node:path'
@@ -6,13 +6,10 @@ import { dirname, resolve } from 'node:path'
const web = fileURLToPath(new URL('../../src/treg/web/', import.meta.url))
const vue = fileURLToPath(new URL('../node_modules/vue/', import.meta.url))
const { version } = JSON.parse(readFileSync(resolve(vue, 'package.json'), 'utf8'))
for (const page of ['enrich-arena.html', 'dashboard-legacy/index.html']) {
const html = readFileSync(resolve(web, page), 'utf8')
const url = html.match(/src="([^"]*\/vendor\/vue-([^/"]+)\.global\.prod\.js)"/)
if (!url || url[2] !== version) throw new Error(`${page} must use the installed Vue ${version}`)
const relative = url[1].replace(/^\/app\/legacy\//, 'dashboard-legacy/').replace(/^\//, '')
const target = resolve(web, relative)
mkdirSync(dirname(target), { recursive: true })
copyFileSync(resolve(vue, 'dist/vue.global.prod.js'), target)
copyFileSync(resolve(vue, 'LICENSE'), resolve(dirname(target), 'LICENSE'))
}
const html = readFileSync(resolve(web, 'enrich-arena.html'), 'utf8')
const url = html.match(/src="([^"]*\/vendor\/vue-([^/"]+)\.global\.prod\.js)"/)
if (!url || url[2] !== version) throw new Error(`enrich-arena.html must use the installed Vue ${version}`)
const target = resolve(web, url[1].replace(/^\//, ''))
mkdirSync(dirname(target), { recursive: true })
copyFileSync(resolve(vue, 'dist/vue.global.prod.js'), target)
copyFileSync(resolve(vue, 'LICENSE'), resolve(dirname(target), 'LICENSE'))
frontend/src/App.vue
+28 -21
View File
@@ -16,9 +16,11 @@ import ReferralsPage from './pages/ReferralsPage.vue'
import HelpPage from './pages/HelpPage.vue'
import HubPage from './pages/HubPage.vue'
import HubRunPage from './pages/HubRunPage.vue'
import SearchPage from './pages/SearchPage.vue'
import SignedOutPage from './components/SignedOutPage.vue'
import BrandMark from './components/BrandMark.vue'
import PublicNavigation from './components/PublicNavigation.vue'
import LandingNavigation from './components/LandingNavigation.vue'
import DashboardNavigation from './components/DashboardNavigation.vue'
import ConnectTokenDialog from './dialogs/ConnectTokenDialog.vue'
import TopUpDialog from './dialogs/TopUpDialog.vue'
@@ -38,18 +40,20 @@ import RunToolDialog from './dialogs/RunToolDialog.vue'
import CallDetailsDialog from './dialogs/CallDetailsDialog.vue'
import TryEndpointDialog from './dialogs/TryEndpointDialog.vue'
import SignInDialog from './components/SignInDialog.vue'
export default { ...controller, components: { ...controller.components, TeamResourcesPage, FishVoiceDialog, CatalogPage, ProviderPage, PlatformPage, ToolsPage, DetailPage, SecretsPage, TeamPage, ActivityPage, AdminPage, GettingStartedPage, ReferralsPage, HelpPage, HubPage, HubRunPage, SignedOutPage, BrandMark, PublicNavigation, DashboardNavigation, ConnectTokenDialog, TopUpDialog, AgentGuideDialog, ConnectionMethodDialog, ResourcePickerDialog, ExtraCredentialDialog, EditToolDialog, AcceptInvitesDialog, WelcomeDialog, CopyToolDialog, ImportSkillDialog, RequestToolDialog, ShareDialog, RecipeDialog, RunToolDialog, CallDetailsDialog, TryEndpointDialog, SignInDialog } }
export default { ...controller, components: { ...controller.components, TeamResourcesPage, FishVoiceDialog, CatalogPage, ProviderPage, PlatformPage, ToolsPage, DetailPage, SecretsPage, TeamPage, ActivityPage, AdminPage, GettingStartedPage, ReferralsPage, HelpPage, SearchPage, HubPage, HubRunPage, SignedOutPage, BrandMark, PublicNavigation, LandingNavigation, DashboardNavigation, ConnectTokenDialog, TopUpDialog, AgentGuideDialog, ConnectionMethodDialog, ResourcePickerDialog, ExtraCredentialDialog, EditToolDialog, AcceptInvitesDialog, WelcomeDialog, CopyToolDialog, ImportSkillDialog, RequestToolDialog, ShareDialog, RecipeDialog, RunToolDialog, CallDetailsDialog, TryEndpointDialog, SignInDialog } }
</script>
<template>
<div>
<main v-if="!bootReady || bootFailed" class="boot-status" aria-live="polite" :aria-busy="!bootReady">
<main v-if="bootFailed" class="boot-status" aria-live="polite">
<a href="/" class="brand"><BrandMark/>treg</a>
<template v-if="bootFailed">
<p role="alert">The dashboard couldn't load. Please try again.</p>
<button class="btn" @click="reloadApp()">Try again</button>
</template>
<p v-else role="status">Loading treg…</p>
<p role="alert">The dashboard couldn't load. Please try again.</p>
<button class="btn" @click="reloadApp()">Try again</button>
</main>
<!-- Continues index.html's loader on the page's own clock, so mounting does not restart it. -->
<main v-else-if="!bootReady" class="boot-status" aria-busy="true" :style="{'--boot-t': -Math.round(bootStartedAt)+'ms'}">
<span class="boot-bar" aria-hidden="true"></span>
<p role="status" style="position:absolute;width:1px;height:1px;overflow:hidden;clip-path:inset(50%);white-space:nowrap">Loading treg…</p>
</main>
<div v-else :class="{redesign:authed && !publicCatalog}">
<!-- Focused sign-in entry after session initialization. -->
@@ -61,14 +65,17 @@ export default { ...controller, components: { ...controller.components, TeamReso
(org switcher, global tool search, member nav) is furniture for a job they have not started.
They get the marketing site's nav instead, so /catalog reads as part of treg.to rather than
as a dashboard someone forgot to lock. -->
<PublicNavigation v-if="publicCatalog" />
<LandingNavigation v-if="view==='find'" />
<PublicNavigation v-else-if="publicCatalog" />
<DashboardNavigation v-else />
<img v-if="authed && !publicCatalog && view==='start'" class="rd-background" src="/media/redesign/ascii-background.jpg" alt="" aria-hidden="true">
<div v-if="startCopyError" class="rd-copy-error" role="alert">{{startCopyError}}<br><button class="btn sm" @click="startCopyError=''">Dismiss</button></div>
<span class="rd-sr-only" role="status">{{startCopied ? 'Copied to clipboard' : ''}}</span>
<div class="layout" :class="{solo:publicCatalog}">
<main id="maincontent" tabindex="-1">
<div v-if="!publicCatalog && (view==='tools'||view==='resources'||view==='connections')" class="rd-view-search search"><img src="/media/redesign/search.svg" alt=""><input :ref="el => setElement('search', el)" v-model="q" :placeholder="view==='connections'?'Search the catalog…':view==='resources'?'Search team resources…':'Search your own tools…'" aria-label="Search"></div>
<main id="maincontent" tabindex="-1" :class="{flush:view==='find'}">
<!-- The Catalog page has its own, larger search (CatalogPage.vue): there it also finds tools
for a described job. -->
<div v-if="!publicCatalog && (view==='tools'||view==='resources')" class="rd-view-search search"><img src="/media/redesign/search.svg" alt=""><input :ref="el => setElement('search', el)" v-model="q" :placeholder="view==='resources'?'Search team resources…':'Search your own tools…'" aria-label="Search"></div>
<div v-if="err" class="banner">{{err}}</div>
<div v-if="pendingInvites.length" class="banner" style="display:flex;align-items:center;gap:10px;flex-wrap:wrap">
<span>You've been invited:</span>
@@ -81,6 +88,9 @@ export default { ...controller, components: { ...controller.components, TeamReso
<!-- TOOLS -->
<CatalogPage v-if="view==='connections'" />
<!-- FIND: /search, a described job answered over the platform pile -->
<SearchPage v-if="view==='find'" />
<!-- MARKETPLACE: one integration -->
<ProviderPage v-if="view==='provider' && mkProvider" />
@@ -249,14 +259,15 @@ export default { ...controller, components: { ...controller.components, TeamReso
<CallDetailsDialog v-if="callView" />
<TryEndpointDialog v-if="epTry" />
<!-- access reminder toast: fired when a new tool is registered while some members have customized access -->
<div v-if="newVersion" class="tut-notice" style="position:fixed;bottom:18px;right:18px;max-width:360px;z-index:200;box-shadow:0 6px 20px rgba(0,0,0,.25);background:var(--card)">
A new version of the dashboard is available.
<div style="margin-top:8px;display:flex;gap:8px"><button class="btn sm" @click="reloadApp()">Refresh now</button><button class="btn sm" @click="newVersion=false">Later</button></div>
<!-- Toasts, bottom right: a newer dashboard build is live, and the access reminder (fired when
a new tool is registered while some members have customized access). -->
<div v-if="newVersion" class="app-toast" role="status">
<p>A new version of the dashboard is available.</p>
<div class="app-toast-a"><button class="btn sm" @click="newVersion=false">Later</button><button class="btn sm primary" @click="reloadApp()">Refresh now</button></div>
</div>
<div v-if="accessNote" class="tut-notice" style="position:fixed;bottom:18px;right:18px;max-width:360px;z-index:200;box-shadow:0 6px 20px rgba(0,0,0,.25);background:var(--card)">
{{accessNote}}
<div style="margin-top:8px;display:flex;gap:8px"><button class="btn sm" @click="go('org')">Open Team</button><button class="btn sm" @click="accessNote=''">Dismiss</button></div>
<div v-if="accessNote" class="app-toast" role="status">
<p>{{accessNote}}</p>
<div class="app-toast-a"><button class="btn sm" @click="accessNote=''">Dismiss</button><button class="btn sm primary" @click="go('org')">Open Team</button></div>
</div>
</template>
@@ -269,7 +280,3 @@ export default { ...controller, components: { ...controller.components, TeamReso
</div>
</template>
<style scoped>
.boot-status { min-height: 70vh; display: flex; flex-direction: column; align-items: center; justify-content: center; gap: 16px; color: var(--muted); }
.boot-status .brand { color: var(--text); text-decoration: none; }
</style>
frontend/src/components/DashboardNavigation.vue
+1 -1
View File
@@ -42,7 +42,7 @@ export default { components: { BrandMark }, setup: useDashboard }
<button v-if="authed && hubOn" class="rd-nav" :class="{active:view==='hub'||view==='run'}" :aria-current="(view==='hub'||view==='run')?'page':null" @click="go('hub')"><img src="/media/redesign/nav-hub.svg" alt="">Hub</button>
<button v-if="authed" class="rd-nav" :class="{active:view==='orgs'}" :aria-current="(view==='orgs')?'page':null" @click="go('orgs')"><img src="/media/redesign/nav-team.svg" alt="">Team</button></nav>
<div class="rd-account">
<a v-if="authed" class="rd-referral" href="#referrals" @click.prevent="go('referrals')" :aria-current="view==='referrals'?'page':null" aria-label="Refer a friend"><img src="/media/redesign/referral-gift.svg" alt=""><span>Refer a friend</span></a>
<a v-if="authed" class="rd-referral" href="#referrals" @click.prevent="go('referrals')" :aria-current="view==='referrals'?'page':null" :aria-label="refEntryLabel()==='Refer a friend' ? 'Refer a friend' : 'Refer a friend: '+refEntryLabel()"><img src="/media/redesign/referral-gift.svg" alt=""><span>{{refEntryLabel()}}</span></a>
<div class="rd-social"><a href="https://github.com/superdesigndev/treg" target="_blank" rel="noopener" aria-label="GitHub"><img src="/media/redesign/social-github.svg" alt=""></a><a href="https://discord.gg/6mQYYfFMAn" target="_blank" rel="noopener" aria-label="Discord"><img src="/media/redesign/social-discord.svg" alt=""></a></div>
<button v-if="billing" class="rd-balance" @click="orgTab='billing'; go('orgs')"><span>Balance</span><b>{{money(billing.balance_micro)}}</b></button>
<details class="rd-account-menu" :ref="el => setElement('accountMenu', el)">
frontend/src/components/FindAnswer.vue
+115
View File
@@ -0,0 +1,115 @@
<script>
import { useDashboard } from '../state/context'
const SHOWN = 8 // rows before "Show more"
// The Catalog page's answer to a described job (state/find.js): one quiet list, one row per job.
// Strong fits first at full weight, weaker ones after them in a lighter tone. No labels for the
// buckets: the order and the fit bar already say it. Clearing the search box is how you leave.
export default {
setup: useDashboard,
data(){ return { all:false } },
computed: {
shown(){ return this.all ? this.findGroups : this.findGroups.slice(0, SHOWN); },
nothing(){ return this.find.verdict==='none' || (this.find.verdict==='keyword' && !this.findGroups.length); },
},
watch: { 'find.q'(){ this.all=false; } },
}
</script>
<template>
<section class="fa" aria-live="polite" :aria-busy="findBusy">
<div v-if="findBusy" class="fa-list" aria-label="Finding tools">
<div v-for="i in 3" :key="i" class="fa-skel"><span></span><span></span><span></span></div>
</div>
<p v-else-if="find.phase==='error'" class="fa-note">{{find.error}}
<button class="fa-link" type="button" @click="findRun(find.q)">Try again</button></p>
<p v-else-if="find.phase==='done' && nothing" class="fa-note">
Nothing in the catalog does this yet.
<button class="fa-link" type="button" @click="findRequestTool()">Request it</button> and it steers what we add next.</p>
<template v-else-if="find.phase==='done'">
<div class="fa-head">
<span>{{findGroups.length}} {{find.verdict==='keyword' ? 'keyword match' : 'tool'}}{{findGroups.length===1 ? '' : (find.verdict==='keyword' ? 'es' : 's')}} for <b>{{find.q}}</b></span>
<button class="fa-link" type="button" @click="findCopyAll()">
{{findCopied==='all' ? 'Copied' : 'Copy for your agent'}}</button>
</div>
<p v-if="find.verdict==='closest'" class="fa-sub">Nothing fits closely. These come nearest.
<button class="fa-link" type="button" @click="findRequestTool()">Request a better tool</button></p>
<ul class="fa-list">
<li v-for="(g, i) in shown" :key="g.key" class="fa-row" :class="{weak:findWeak(g)}">
<button class="fa-main" type="button" @click="findOpen(g, i+1)">
<span class="fa-logo" :class="{gen:platLogoBad[g.platform]}"
:style="platLogoBad[g.platform] ? {background:platTileBg(g.platform)} : null">
<img v-if="!platLogoBad[g.platform]" :src="'/logos/platforms/'+g.platform+'.svg'" alt="" @error="platLogoBad[g.platform]=true">
<span v-else>{{platInitial({label:g.platform_label, slug:g.platform})}}</span>
</span>
<span class="fa-what"><b>{{g.label}}</b><small>{{platShort(g.platform_label)}}</small></span>
<span class="fa-provs" :title="g.rows.map(r=>r.provider_display||r.provider).join(', ')">
<span class="fa-stack"><img v-for="p in findProviders(g).slice(0,3)" :key="p" :src="'/logos/'+p+'.svg'" alt=""
@error="$event.target.style.visibility='hidden'"></span>
{{findProviders(g).length}} provider{{findProviders(g).length===1?'':'s'}}
</span>
<span class="fa-price">{{findPrice(g)}}</span>
<span v-if="g.p!=null" class="fa-fit" :title="'Fit for this job: '+Math.round(g.p*100)+'%'">
<i :style="{width:Math.round(g.p*100)+'%'}"></i></span>
</button>
<button class="fa-copy" type="button" @click="findCopy([g], g.key)">
{{findCopied===g.key ? 'Copied' : 'Copy for agent'}}</button>
</li>
</ul>
<button v-if="findGroups.length>shown.length" class="fa-link fa-more" type="button" @click="all=true">
Show {{findGroups.length-shown.length}} more</button>
</template>
</section>
</template>
<style scoped>
.fa{margin:0 0 30px;font-family:var(--sans)}
.fa-head{display:flex;align-items:baseline;justify-content:space-between;gap:16px;flex-wrap:wrap;margin:0 0 8px;font-size:13.5px;color:var(--muted)}
.fa-head b{color:var(--ink);font-weight:500}
.fa-sub,.fa-note{margin:0 0 8px;font-size:13.5px;color:var(--muted)}
.fa-note{padding:14px 0}
.fa-link{border:0;background:none;padding:0;font:inherit;font-size:13px;color:var(--ink);text-decoration:underline;text-underline-offset:3px;
text-decoration-color:var(--line2,var(--line));cursor:pointer}
.fa-link:hover{text-decoration-color:currentColor}
.fa-list{list-style:none;margin:0;padding:0;border-top:1px solid var(--line)}
.fa-row{position:relative;border-bottom:1px solid var(--line);animation:fa-in .3s both}
.fa-row:nth-child(2){animation-delay:30ms}.fa-row:nth-child(3){animation-delay:60ms}.fa-row:nth-child(4){animation-delay:90ms}
.fa-row:nth-child(5){animation-delay:120ms}.fa-row:nth-child(n+6){animation-delay:150ms}
@keyframes fa-in{from{opacity:0}}
.fa-main{width:100%;display:grid;grid-template-columns:36px minmax(0,1fr) 150px 120px 56px;align-items:center;gap:16px;
padding:12px 140px 12px 8px;border:0;background:none;text-align:left;color:var(--ink);cursor:pointer;border-radius:10px;font:inherit}
.fa-main:hover{background:var(--hover,rgba(0,0,0,.035))}
.fa-logo{width:36px;height:36px;border-radius:10px;background:#fff;border:1px solid var(--line);display:grid;place-items:center;color:#fff;font-weight:600;font-size:14px}
.fa-logo img{width:21px;height:21px;object-fit:contain}
.fa-what{display:flex;flex-direction:column;gap:1px;min-width:0}
.fa-what b{font-weight:500;font-size:14.5px;line-height:1.3;overflow:hidden;text-overflow:ellipsis;white-space:nowrap}
.fa-what small{font-size:12.5px;color:var(--muted)}
.fa-provs{display:flex;align-items:center;gap:8px;font-size:12.5px;color:var(--muted);white-space:nowrap}
.fa-stack{display:flex;padding-left:6px}
.fa-stack img{width:20px;height:20px;margin-left:-6px;border-radius:6px;background:#fff;border:1.5px solid var(--bg);object-fit:contain;padding:1px}
.fa-price{font-family:var(--mono);font-size:12px;color:var(--muted);text-align:right;white-space:nowrap;font-variant-numeric:tabular-nums}
.fa-fit{height:4px;border-radius:2px;background:var(--line);overflow:hidden}
.fa-fit i{display:block;height:100%;background:var(--ink);border-radius:2px}
.fa-row.weak .fa-what b,.fa-row.weak .fa-logo{opacity:.62}
.fa-row.weak .fa-fit i{background:var(--muted)}
.fa-copy{position:absolute;right:8px;top:50%;transform:translateY(-50%);border:1px solid var(--line2,var(--line));background:var(--surface,var(--panel));
color:var(--ink);border-radius:8px;padding:5px 11px;font:inherit;font-size:12.5px;cursor:pointer;opacity:0;transition:opacity .15s}
.fa-row:hover .fa-copy,.fa-copy:focus-visible{opacity:1}
.fa-more{margin-top:10px}
.fa-skel{display:grid;grid-template-columns:36px minmax(0,1fr) 150px;gap:16px;align-items:center;padding:12px 8px;border-bottom:1px solid var(--line)}
.fa-skel span{height:12px;border-radius:6px;background:linear-gradient(90deg,var(--line),var(--hover,var(--panel2)),var(--line));background-size:200% 100%;animation:fa-sh 1.2s linear infinite}
.fa-skel span:first-child{height:36px;border-radius:10px}
@keyframes fa-sh{to{background-position:-200% 0}}
@media (max-width:760px){
.fa-main{grid-template-columns:36px minmax(0,1fr) auto;padding-right:8px}
.fa-provs,.fa-fit{display:none}
.fa-copy{display:none}
}
@media (hover:none){.fa-copy{opacity:1}}
@media (prefers-reduced-motion:reduce){.fa-row,.fa-skel span{animation:none}}
</style>
frontend/src/components/LandingNavigation.vue
+57
View File
@@ -0,0 +1,57 @@
<script>
import { useDashboard } from '../state/context'
import BrandMark from './BrandMark.vue'
// The landing page's top bar (src/treg/web/landing.html `.navwrap`), for /search: the two first
// screens a visitor meets should look like one site. Same links, same pill; a signed-in visitor
// gets their way into the dashboard instead of sign-in.
export default { components: { BrandMark }, setup: useDashboard }
</script>
<template>
<div class="lnav-wrap">
<nav class="lnav" aria-label="treg">
<a class="lnav-brand" href="/"><BrandMark/>treg</a>
<div class="lnav-links">
<a class="hidem ico" href="https://github.com/superdesigndev/treg" target="_blank" rel="noopener"><svg viewBox="0 0 16 16" aria-hidden="true"><path fill="currentColor" d="M8 0C3.58 0 0 3.58 0 8c0 3.54 2.29 6.53 5.47 7.59.4.07.55-.17.55-.38 0-.19-.01-.82-.01-1.49-2.01.37-2.53-.49-2.69-.94-.09-.23-.48-.94-.82-1.13-.28-.15-.68-.52-.01-.53.63-.01 1.08.58 1.23.82.72 1.21 1.87.87 2.33.66.07-.52.28-.87.51-1.07-1.78-.2-3.64-.89-3.64-3.95 0-.87.31-1.59.82-2.15-.08-.2-.36-1.02.08-2.12 0 0 .67-.21 2.2.82.64-.18 1.32-.27 2-.27s1.36.09 2 .27c1.53-1.04 2.2-.82 2.2-.82.44 1.1.16 1.92.08 2.12.51.56.82 1.27.82 2.15 0 3.07-1.87 3.75-3.65 3.95.29.25.54.73.54 1.48 0 1.07-.01 1.93-.01 2.2 0 .21.15.46.55.38A8.01 8.01 0 0 0 16 8c0-4.42-3.58-8-8-8z"/></svg>Repo</a>
<a class="hidem ico" href="https://discord.gg/6mQYYfFMAn" target="_blank" rel="noopener"><svg viewBox="0 0 24 24" aria-hidden="true"><path fill="currentColor" d="M20.32 4.37a19.8 19.8 0 0 0-4.89-1.52.07.07 0 0 0-.08.04c-.21.38-.44.87-.6 1.25a18.3 18.3 0 0 0-5.49 0 12.6 12.6 0 0 0-.61-1.25.08.08 0 0 0-.08-.04 19.7 19.7 0 0 0-4.88 1.52.07.07 0 0 0-.03.03C.53 9.05-.32 13.58.1 18.06c0 .02.01.04.03.05a19.9 19.9 0 0 0 6 3.03.08.08 0 0 0 .08-.03c.46-.63.87-1.3 1.22-2a.08.08 0 0 0-.04-.11 13.1 13.1 0 0 1-1.87-.89.08.08 0 0 1-.01-.13c.13-.09.25-.19.37-.29a.07.07 0 0 1 .08-.01c3.93 1.79 8.18 1.79 12.06 0a.07.07 0 0 1 .08.01c.12.1.24.2.37.29a.08.08 0 0 1-.01.13c-.6.35-1.22.64-1.87.89a.08.08 0 0 0-.04.11c.36.7.77 1.36 1.22 2a.08.08 0 0 0 .08.03 19.8 19.8 0 0 0 6.02-3.03.08.08 0 0 0 .03-.05c.5-5.18-.84-9.67-3.55-13.66a.06.06 0 0 0-.03-.03zM8.02 15.33c-1.18 0-2.16-1.08-2.16-2.42s.96-2.42 2.16-2.42c1.21 0 2.18 1.1 2.16 2.42 0 1.34-.96 2.42-2.16 2.42zm7.97 0c-1.18 0-2.16-1.08-2.16-2.42s.96-2.42 2.16-2.42c1.21 0 2.18 1.1 2.16 2.42 0 1.34-.95 2.42-2.16 2.42z"/></svg>Community</a>
<a href="/search" aria-current="page" class="on">Tools</a>
<a href="/catalog">Catalog</a>
<!-- /search draws before the session is known: no Sign in that turns into Open dashboard. -->
<a v-if="authed" class="lnav-candy" href="/app">Open dashboard</a>
<template v-else-if="sessionChecked">
<a class="hidexs" href="/app?ref=search" @click.prevent="openSignin()">Sign in</a>
<button class="lnav-candy" type="button" @click="openSignin()">Start free</button>
</template>
</div>
</nav>
</div>
</template>
<style scoped>
.lnav-wrap{position:fixed;top:0;left:0;right:0;z-index:50}
.lnav{display:flex;align-items:center;gap:22px;padding:24px 40px}
.lnav-brand{display:flex;align-items:center;gap:9px;font-family:"DM Mono",ui-monospace,"SF Mono",Menlo,monospace;font-weight:600;font-size:15px;color:#1a1a1a;text-decoration:none}
.lnav-brand .brand-mark{width:28px;height:28px;border-radius:8px}
.lnav-links{margin-left:auto;display:flex;align-items:center;gap:28px}
.lnav-links a{font-family:var(--sans);font-size:13.5px;font-weight:500;color:#7c7c7c;text-decoration:none;cursor:pointer;white-space:nowrap}
.lnav-links a:hover,.lnav-links a.on{color:#1a1a1a}
.lnav-links a.ico{display:inline-flex;align-items:center;gap:6px}
.lnav-links a.ico svg{width:15px;height:15px;display:block}
.lnav-links .lnav-candy{display:inline-block;font-family:var(--sans);font-weight:550;letter-spacing:.01em;
color:#f8f8f7;background:#1a1a1a;border:0;border-radius:999px;padding:8px 18px;font-size:12.5px;cursor:pointer;
box-shadow:0 1px 2px #00000014;transition:box-shadow .24s cubic-bezier(.2,.72,.25,1),transform .12s cubic-bezier(.22,1,.36,1)}
.lnav-links .lnav-candy:hover{color:#f8f8f7;transform:translateY(-1px);
box-shadow:0 1px 2px -1px #0000000a,0 4px 6px -1px #0000000f,0 8px 16px #0000000a}
[data-theme="dark"] .lnav-brand,[data-theme="dark"] .lnav-links a:hover,[data-theme="dark"] .lnav-links a.on{color:#f2efe8}
[data-theme="dark"] .lnav-links .lnav-candy{background:#f2efe8;color:#151412}
@media(max-width:640px){
.lnav{gap:12px;padding:14px 16px}.lnav-links{gap:12px}.lnav-links a{font-size:11px}
.lnav-links .lnav-candy{padding:9px 13px;font-size:11px}
.lnav-links a.hidem{display:none} /* `a.ico` sets display too; this has to outrank it */
.lnav-links .lnav-candy{white-space:nowrap}
}
/* Start free opens the same sign-in, so the narrowest phones keep one of the two. */
@media(max-width:400px){
.lnav-links a.hidexs{display:none}
}
</style>
frontend/src/main.ts
+6
View File
@@ -1,5 +1,11 @@
import { createApp } from 'vue'
import App from './App.vue'
// The two brand faces, from pinned npm packages (OFL-1.1) and bundled same-origin: Geist Pixel for
// page titles, DM Mono for figures and code. Body text uses the system font (DESIGN.md).
import '@fontsource/geist-pixel/latin-400.css'
import '@fontsource/dm-mono/latin-400.css'
import '@fontsource/dm-mono/latin-500.css'
import '@fontsource/dm-mono/latin-400-italic.css'
import './styles/base.css'
import '../../src/treg/web/media/redesign/dashboard.css'
frontend/src/pages/ActivityPage.vue
+2 -2
View File
@@ -21,8 +21,8 @@ export default { setup: useDashboard }
<div class="field" style="max-width:460px;margin-bottom:10px"><select class="msel" v-model="activityKey" @change="loadCalls"><option value="">All API keys</option><option v-for="k in apiKeys" :key="k.id" :value="String(k.id)">{{k.identity}} — {{k.name}}</option></select></div>
<table><tr><th>When</th><th>Who</th><th>Key</th><th v-if="anyTagged">Tagged</th><th>Tool</th><th>Action</th><th>Status</th><th style="text-align:right">Cost</th></tr>
<tr v-for="a in activityShown" :key="a.kind+'-'+a.id" :class="{'act-row':a.kind==='call'}" @click="a.kind==='call'&&openCall(a)" :title="a.kind==='call'?(a.has_result?'Show request and response':'Show call details'):''"><td class="muted">{{when(a.created_at)}}</td><td>{{activityWho(a)}}<span v-if="activityOwner(a)" class="chip" style="margin-left:6px" :title="'Agent owner: '+activityAgentKey(a).created_by">owner: {{activityOwner(a)}}</span><span v-if="a.client && a.client!=='cli'" class="chip" style="margin-left:6px" :title="'reported by the runtime — attribution, not authentication'">via {{a.client}}</span></td><td><span v-if="a.api_key_name" class="chip">{{a.api_key_name}}<span v-if="a.api_key_prefix" class="muted mono"> · {{a.api_key_prefix}}</span></span><span v-else class="muted">—</span></td><td v-if="anyTagged"><template v-if="a.tags"><span v-for="(v,k) in a.tags" :key="k" class="chip" style="margin-right:4px" :title="'X-Treg-Meta '+k+'='+v">{{k}}={{v}}</span></template><span v-else class="muted">—</span></td><td>{{a.tool}}</td><td><span v-if="a.kind==='run'" class="chip" style="margin-right:6px" :title="a.where==='local'?'ran on this member\'s machine':'ran on the registry server'">{{a.where||'run'}}</span>{{a.action}}<!-- A generation task's artifact, once it succeeded: the provider's time-limited URL, or the CLI command that retrieves it (treg never downloads media). --><template v-if="a.task"><a v-if="a.task.result_url" :href="a.task.result_url" target="_blank" rel="noopener" style="margin-left:8px" :title="taskArtifactTitle(a.task)" @click.stop>result ↗</a><span v-else-if="a.task.fetch_command" class="chip" style="margin-left:8px" :title="'retrieve it from the CLI: '+a.task.fetch_command">result via CLI</span><span v-if="a.task.result_url||a.task.fetch_command" class="muted" style="margin-left:6px;font-size:.85em">{{a.task.ttl_note?'expires in '+a.task.ttl_note:'time-limited link'}}</span></template></td><td><span class="badge" :class="a.ok?'ok':'invalid'">{{a.status}}</span><span v-if="a.task" class="chip" style="margin-left:6px" :title="taskStateTitle(a.task)">{{taskStateLabel(a.task)}}</span><span v-if="a.has_result" class="act-view">result ›</span></td><td style="text-align:right;white-space:nowrap" class="muted" :title="a.held?'reserved - settles when the task finishes, refunded if it fails':(a.tier==='platform'?'charged to team balance':(a.cost!=null?'estimated — billed to your own provider key':''))"><span v-if="a.held" style="font-size:.85em">hold </span>{{a.cost!=null?money(a.cost):'—'}}<span v-if="a.cached" class="chip cached" style="margin-left:6px" title="Served from treg's archive instead of calling the provider.">Cached</span></td></tr></table>
<p v-if="!loading && !activityRows.length" class="sub">No activity yet.</p>
<p v-else-if="!loading && !activityShown.length" class="sub">No successful calls yet — <a href="#" @click.prevent="actOkOnly=false">show all {{activityRows.length}}</a>.</p>
<p v-if="callsLoaded && !activityRows.length" class="sub">No activity yet.</p>
<p v-else-if="callsLoaded && !activityShown.length" class="sub">No successful calls yet — <a href="#" @click.prevent="actOkOnly=false">show all {{activityRows.length}}</a>.</p>
</template>
<template v-if="canAdmin && actTab==='usage'">
frontend/src/pages/CatalogPage.vue
+33 -3
View File
@@ -1,12 +1,13 @@
<script>
import { useDashboard } from '../state/context'
export default { setup: useDashboard }
import FindAnswer from '../components/FindAnswer.vue'
export default { components: { FindAnswer }, setup: useDashboard, beforeUnmount(){ this.findUnschedule(); } }
</script>
<template>
<div class="tut-head">
<div><h1>2,800+ tools for agents</h1><p class="sub" style="margin:0">Connect an account once. treg holds the credential server-side and injects it on every call — nothing lands on your machine.</p></div>
<div><h1>{{toolCountText ? toolCountText+' tools' : 'Tools'}} for agents</h1><p class="sub" style="margin:0">Connect an account once. treg holds the credential server-side and injects it on every call — nothing lands on your machine.</p></div>
<!-- (The balance pill lives in the side nav, above the account block.) -->
<div class="tut-actions">
<button class="btn sm" @click="openToolRequest()" title="Missing a tool or provider? Tell us — requests steer what gets added next"><svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M21 15a2 2 0 0 1-2 2H7l-4 4V5a2 2 0 0 1 2-2h14a2 2 0 0 1 2 2z"/><line x1="12" y1="7" x2="12" y2="13"/><line x1="9" y1="10" x2="15" y2="10"/></svg>Request a tool</button>
@@ -16,6 +17,25 @@ export default { setup: useDashboard }
<button class="btn sm primary" @click="publicCatalog ? openSignin() : goByok()" title="Register your own provider key — your key wins over treg's and those calls are never metered"><svg width="13" height="13" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="m21 2-2 2m-7.61 7.61a5.5 5.5 0 1 1-7.778 7.778 5.5 5.5 0 0 1 7.777-7.777Zm0 0L15.5 7.5m0 0 3 3L22 7l-3-3m-3.5 3.5L19 4"/></svg>Bring your own key</button>
</div>
</div>
<!-- One box, two questions: a platform name filters the shelves as you type, and the finder
answers whatever is typed once typing pauses, or at once on Enter (state/find.js).
Clearing the box is how you leave an answer. -->
<div class="cat-find" v-if="plats.list.length">
<svg class="cat-find-i" width="18" height="18" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true"><circle cx="11" cy="11" r="7"/><path d="m20 20-3.5-3.5"/></svg>
<input :ref="el => setElement('search', el)" v-model="q" aria-label="Search the catalog"
placeholder="Search a platform, or describe what your agent needs to do"
@input="findSchedule($event.target.value)"
@keydown.enter="q.trim() && findRun(q)" @keydown.esc="q=''; findExit()">
<button v-if="q" class="cat-find-x" type="button" aria-label="Clear the search" @click="q=''; findExit()">×</button>
</div>
<!-- A sentence is a job, not a name: say so where the eye already is, as one clickable row. -->
<!-- Enter searches every tool for whatever is typed. A name still filters the shelves as you
type, so for a short query the row is quieter; for a sentence it is the main action. -->
<button v-if="plats.list.length && q.trim() && !findActive" class="cat-find-suggest" :class="{quiet:!findIsJob(q)}" type="button" @click="findRun(q)">
<span class="cat-find-suggest-i" aria-hidden="true"><svg width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round"><path d="M5 12h14M13 6l6 6-6 6"/></svg></span>
<span class="cat-find-suggest-t">{{findIsJob(q) ? 'Find tools for' : 'Search all tools for'}} <b>“{{q.trim()}}”</b></span>
<kbd>Enter</kbd>
</button>
<div v-if="connErr" class="banner" style="margin-top:12px">{{connErr}}</div>
<div v-for="c in needSecondCred" :key="'n'+c.id" class="banner" style="margin-top:12px">
<div><b>{{c.name}}</b> is connected, but can't call the API on its own yet. {{c.extra_credential_note}}</div>
@@ -39,6 +59,10 @@ export default { setup: useDashboard }
but the last asks WHICH DATA you want (platform tiles, grouped by category), while
"Platform" is the original integration shelf — which ACCOUNT you hold. Data-first is
the default because that is the question an agent actually arrives with. -->
<!-- A described job (the search box's Enter, see state/find.js) is answered here, above the
shelves rather than instead of them: the shelves stay, lit where the answer landed. -->
<FindAnswer v-if="findActive" />
<div class="mk-tabs-wrap" v-if="plats.list.length">
<div class="mk-tabs" role="tablist" aria-label="Catalog">
<button v-for="t in mkTabs" :key="t.key" role="tab" :aria-selected="mkTabActive===t.key"
@@ -60,8 +84,10 @@ export default { setup: useDashboard }
what the name and category already said. The summary survives as the hover
title, so nothing is lost for the one visitor who wants it. -->
<button v-for="pl in g.items" :key="pl.slug" class="pt-card"
:class="{'find-hit':findHits[pl.slug], 'find-dim':find.phase==='done' && findGroups.length && !findHits[pl.slug]}"
:title="pl.summary ? pl.label+' — '+pl.summary : pl.label"
:aria-label="'Open '+pl.label" @click="openPlatform(pl.slug)">
<span v-if="findHits[pl.slug]" class="pt-find">{{findHits[pl.slug]}} match{{findHits[pl.slug]===1?'':'es'}}</span>
<div class="pt-top">
<!-- A platform's OWN mark, not its providers': the card is the platform. Anything
we haven't drawn falls back to a generated initial tile, not a broken image. -->
@@ -111,7 +137,11 @@ export default { setup: useDashboard }
<span class="pt-more-a" aria-hidden="true">→</span>
</button>
</div>
<div v-if="!platCatGroups.length" class="mk-empty">
<!-- A query that names no platform is usually a JOB, not a typo: say what missed and offer
the finder, instead of implying the server has no catalog. -->
<p v-if="!platCatGroups.length && platNameQuery && plats.list.length && !findSoon" class="find-miss">
No platform is called that.</p>
<div v-else-if="plats.settled && !platCatGroups.length && !q.trim()" class="mk-empty">
No catalogued platforms{{mkTabActive==='all'?'':' in '+mkTabActive}} on this server yet — the
<b>Platform</b> tab lists every integration you can connect.
</div>
frontend/src/pages/GettingStartedPage.vue
+3 -3
View File
@@ -1,6 +1,6 @@
<script>
import { useDashboard } from '../state/context'
export default { setup: useDashboard }
export default { setup: useDashboard, mounted(){ this.loadPlatforms() } } // the catalog size in the copy
</script>
<template>
@@ -91,7 +91,7 @@ treg catalog, then open a PR.</pre>
</div>
<div v-show="buildTab==='platform'" class="rd-build-panel">
<p class="sub">Give <i>your</i> users 2,800+ tools without owning the keys — and bill each of your customers for what they used. Covers the call methods (HTTP, MCP, CLI), per-customer tagging, spend limits and invoicing.</p>
<p class="sub">Give <i>your</i> users {{toolCountText||'thousands of'}} tools without owning the keys — and bill each of your customers for what they used. Covers the call methods (HTTP, MCP, CLI), per-customer tagging, spend limits and invoicing.</p>
<div class="lc-codewrap" style="margin-top:8px">
<button class="lc-cp" @click="copyStart('Read '+proxy+'/integrate.md and integrate treg into our product, including per-customer usage tracking and billing.','intg')">{{startCopied==='intg'?'✓ copied':'copy'}}</button>
<pre>Read <span class="hl-str">{{proxy}}/integrate.md</span> and integrate treg into our
@@ -110,7 +110,7 @@ product, including per-customer usage tracking and billing.</pre>
<!-- TAB · Access the catalog — agent instruction first, then the manual CLI walkthrough -->
<div v-show="startTab==='access'" style="max-width:720px">
<p class="sub">2,800+ catalogued endpoints: SEO and backlinks, social and trends, people and company enrichment, ads. Find one by what it <i>does</i>, see its price, call it — no provider signup. New verified accounts get <b>$1.00 free credit once</b> on an eligible team, covering hundreds of calls.</p>
<p class="sub">{{toolCountText ? toolCountText+' catalogued endpoints' : 'Catalogued endpoints'}}: SEO and backlinks, social and trends, people and company enrichment, ads. Find one by what it <i>does</i>, see its price, call it — no provider signup. New verified accounts get <b>$1.00 free credit once</b> on an eligible team, covering hundreds of calls.</p>
<div class="lbl" style="margin-top:16px">▸ Set up with your agent <span class="muted" style="font-weight:400">— paste &amp; go</span></div>
<p class="sub">One line, token included — your agent reads llms.txt and does the rest: installs the CLI, signs in as you, and makes its first call.</p>
frontend/src/pages/PlatformPage.vue
+40 -15
View File
@@ -1,6 +1,27 @@
<script>
import { useDashboard } from '../state/context'
export default { setup: useDashboard }
// The ledger's section headings stick right under the filter bar, so their offset is the bar's
// live height. Written on the bar's parent, which also holds the table: the redesign shell
// redeclares --lsec-top on its own element, so a value on the document root never reached them.
let barObserver = null
let barHost = null
function stickLedgerBar(el) {
barObserver?.disconnect()
barHost?.style.removeProperty('--lsec-top')
barObserver = barHost = null
if (!el) return
barHost = el.parentElement
barObserver = new ResizeObserver(() => {
barHost.style.setProperty('--lsec-top', `calc(var(--lbar-top) + ${el.offsetHeight}px)`)
})
barObserver.observe(el)
}
export default {
// Added onto the bindings, never spread: each binding is a live getter onto the shared state.
setup() { return Object.assign(useDashboard(), { stickLedgerBar }) },
}
</script>
<template>
@@ -19,7 +40,7 @@ export default { setup: useDashboard }
aria-hidden="true" @error="platLogoBad[platSlug]=true">
<span v-else class="pt-i">{{platInitial({label:platLabel, slug:platSlug})}}</span>
</span>
<h1>{{platLabel}}</h1>
<h1>{{platLabel || '\u00a0'}}</h1>
</div>
<p class="sub plat-intro">Every endpoint treg knows for this platform, one ledger, filed by subject — jobs several
providers do sit on a single row, so you can compare price and coverage before you spend a call.</p>
@@ -42,25 +63,31 @@ export default { setup: useDashboard }
MERGED rows come first — a job several providers do, on one comparable line —
then the endpoints only one provider offers, each led by its own summary, because
"Get Showcase Product List" says more than the capability id ever could. -->
<div class="lbar">
<!-- The chips scroll rather than wrap: a bar that grows a second row as you filter
would shift the sticky section headings out from under it. -->
<!-- The bar is sticky and its chips WRAP: a scrolling strip with a hidden scrollbar cut the
last chip in half and left mouse users no way to reach the rest. Wrapping makes its
height vary with the platform and the filter, so the section headings that stick
under it read the measured height (`stickLedgerBar`) instead of assuming one row. -->
<div class="lbar" :ref="stickLedgerBar">
<div class="lctl">
<input class="lfind" v-model="platQ" placeholder="Filter, e.g. comments" aria-label="Filter this platform's tools">
<label class="lchk"><input type="checkbox" v-model="platVerifiedOnly"> verified only</label>
<!-- Two numbers only when they differ: a merged row is several endpoints, and that
is the one case where the row count understates the catalog. -->
<span class="lstat">{{platStats.rows}} row{{platStats.rows===1?'':'s'}}<template
v-if="platStats.eps!==platStats.rows"> · {{platStats.eps}} endpoint{{platStats.eps===1?'':'s'}}</template><span
v-if="platDomain || platQ || platVerifiedOnly"> · filtered <button class="lclear" @click="platClearFilters">clear</button></span></span>
</div>
<div class="lchips">
<button class="mk-chip" :class="{on:!platDomain}" @click="platDomain=''">All <span>{{platBrowseCount}}</span></button>
<button v-for="d in platDomainTabs" :key="d.domain" class="mk-chip"
:class="{on:platDomain===d.domain}"
@click="platDomain = platDomain===d.domain ? '' : d.domain">{{d.domain}} <span>{{d.n}}</span></button>
</div>
<label class="lchk"><input type="checkbox" v-model="platVerifiedOnly"> verified only</label>
<input class="lfind" v-model="platQ" placeholder="filter… e.g. comments, showcase">
</div>
<!-- Two numbers, because a merged row is several endpoints: what you are scrolling
through, and how much of the catalog that actually is. -->
<div class="lstat">{{platStats.rows}} row{{platStats.rows===1?'':'s'}} · {{platStats.eps}} endpoint{{platStats.eps===1?'':'s'}}<span
v-if="platDomain || platQ || platVerifiedOnly"> · filtered <button class="lclear" @click="platClearFilters">clear</button></span></div>
<div class="ttable-wrap lwrap" v-if="platLedger.length"><table class="ledger">
<thead><tr><th class="lth-w">What it does</th><th>Providers / route</th><th class="lth-p">Price</th><th class="lth-v">✓</th></tr></thead>
<thead><tr><th class="lth-w">What it does</th><th>Providers / route</th><th class="lth-p">Price</th><th class="lth-v"
title="Called for real against the live API, and the response captured">Verified</th></tr></thead>
<!-- A row, its section heading and its expanded detail are all table rows, so each
level rides a wrapper tag — a tbody per group would strip the row separators. -->
<template v-for="sec in platLedger" :key="sec.domain">
@@ -111,8 +138,7 @@ export default { setup: useDashboard }
</td>
<td class="lprice" :title="r.priceTitle">{{r.price}}<span
v-if="r.priceNative" class="cost-nat">({{r.priceNative}})</span></td>
<td><span v-if="r.verified" class="vmark" title="Called for real against the live API, and the response captured">✓</span><span
v-else class="xmark" title="Documented, but treg has not called it with a live key yet">·</span></td>
<td class="lver"><span v-if="r.verified" class="vmark" title="Called for real against the live API, and the response captured">✓</span></td>
</tr>
<tr v-if="platOpen[r.key]" :key="r.key+'::d'" class="cat-d">
<td colspan="4">
@@ -131,7 +157,6 @@ export default { setup: useDashboard }
<span class="chip">{{endpointAccessLabel(e)}}</span>
<span class="lsub-price" :title="costTitle(e.cost)">{{costShort(e.cost)}}</span>
<span v-if="e.verified" class="vmark" :title="'Called for real on '+e.verified">✓</span>
<span v-else class="xmark" title="Documented, but treg has not called it with a live key yet">·</span>
<span v-if="catEndpointConnected(e)" class="chip go" title="You have a connected account with an authorization method that can call this"><span class="godot"></span>connected</span>
<span class="lsub-path mono"><span class="cat-m">{{e.method}}</span>{{e.path}}</span>
</button>
frontend/src/pages/ReferralsPage.vue
+1 -1
View File
@@ -13,7 +13,7 @@ export default { setup: useDashboard }
everyone, team or not, so a partner without a team yet can still apply. -->
<h1>{{refTab==='partner' ? 'Affiliate partner' : 'Refer a friend'}}</h1>
<p class="sub" v-if="refTab==='partner'">Revenue share, paid in cash. By invitation.</p>
<p class="sub" v-else>They get {{money(ref.terms.referred_micro)}} when they add
<p class="sub" v-else :style="ref.loaded ? null : {visibility:'hidden'}">They get {{money(ref.terms.referred_micro)}} when they add
{{money(ref.terms.min_topup_micro)}}. You get {{money(ref.terms.referrer_micro)}},
{{ref.terms.hold_days}} days later.</p>
<div class="tabs" style="max-width:720px">
frontend/src/pages/SearchPage.vue
+514
View File
@@ -0,0 +1,514 @@
<script>
import { useDashboard } from '../state/context'
import { Pile, poseTransform, tileSize } from '../state/pile'
import { groupBest, jobGroups } from '../state/find.js'
// /search: every platform and every vendor in the catalog is a tile, dropped under gravity
// (state/pile.ts, Matter.js) into a pile on the floor of the page. A described job (GET
// /catalog/find, see state/find.js) makes the platforms and vendors its keyword recall touched hop
// while the judge reads them; the ones that fit then leave the pile and fly to their places on the
// answer cards. The next search drops them back in.
//
// The page is exactly one viewport tall: the pile's floor is the bottom of the screen, and a long
// answer scrolls inside its own panel, never the page.
//
// The examples are the jobs the landing and use-case pages sell (/people-search, /ugc, /use-cases/*),
// one each, shortened from those pages' own prompts. Keep each one a strong fit on /catalog/find.
const EXAMPLES = [
'Find heads of growth at US SaaS companies and their work emails',
'Show me every ad Notion is running on Meta right now',
'Keywords stripe.com ranks for on Google',
'Make a talking UGC video ad for my product',
'What is Reddit saying about our pricing?',
'Does ChatGPT mention my brand?',
]
const LAND = 'transform .8s cubic-bezier(.3,1.2,.4,1)'
export default {
setup: useDashboard,
data(){ return { text:'', examples:EXAMPLES, size:48, floor:220, reduced:false, landed:[], dragging:null, flung:null } },
computed: {
platforms(){ return this.plats.list.filter(p=>(p.category||'Other')!=='Other'); },
// The pile: a tile per platform and a tile per vendor on those platforms, keyed `p:`/`v:` because
// a slug can be both ("tiktok-ads"). A tile opens its platform, a vendor its busiest platform.
vendors(){
const home={};
for(const p of this.platforms) for(const s of p.providers||[]) home[s]=home[s]||p.slug; // busiest platform first
return Object.keys(home).sort().map(slug=>({slug, label:this.provName(slug), home:home[slug]}));
},
// `bad` is the tile's platLogoBad key, shared with every other place that draws that logo.
tiles(){
return [...this.platforms.map(p=>({key:'p:'+p.slug, slug:p.slug, label:this.platShort(p.label), home:p.slug,
src:'/logos/platforms/'+p.slug+'.svg', bad:p.slug})),
...this.vendors.map(v=>({...v, key:'v:'+v.slug, src:'/logos/'+v.slug+'.svg', bad:'v:'+v.slug}))];
},
// On a platform answer (a bare name) no vendor lands; the vendors on those platforms stay lit.
litVendors(){ return this.byVendor ? new Set() : new Set(this.find.rows.map(r=>'v:'+r.provider)); },
// A described job is answered by vendor: one card per vendor, best fit first, under the
// vendor's own logo, listing the jobs that vendor sells here. A platform's name ("google") asks
// what is on those platforms, so it is answered by platform; a vendor's name ("hunter") by vendor.
byVendor(){ return this.find.verdict!=='name' || this.find.named==='provider'; },
// Where tiles land (pile keys): `logo`, the card's logo place, takes the vendor's tile on a
// vendor card and the platform's on a platform card; `mark`, beside the platform name, takes
// the platform's tile on the first vendor card naming it. Later cards show a still copy.
cards(){
if(this.find.phase!=='done') return [];
const byVendor=this.byVendor, seen=new Set();
return groupBest(this.find.rows, r=>byVendor ? r.provider : r.platform, (r, slug)=>{
const platform_label=this.platShort(r.platform_label||r.platform);
return {slug, platform:r.platform, platform_label, label:byVendor ? r.provider_display||r.provider : platform_label};
}, 'rows').slice(0,12).map(c=>{
const mark=byVendor && !seen.has(c.platform) ? 'p:'+c.platform : null;
seen.add(c.platform);
return {...c, jobs:jobGroups(c.rows), logo:byVendor ? 'v:'+c.slug : 'p:'+c.platform, mark};
});
},
reading(){
return this.findBusy ? new Set([...this.findCandidatePlatforms.map(s=>'p:'+s), ...this.findCandidateVendors.map(s=>'v:'+s)]) : new Set();
},
readingList(){ return this.tiles.map(t=>t.key).filter(k=>this.reading.has(k)); },
},
watch: {
'plats.list'(){ this.$nextTick(()=>this.fit(true)); },
'find.phase'(v){
clearInterval(this.pokeTimer); clearInterval(this.scanTimer); this.scanTo(null);
if(v==='recall' || v==='idle') this.dropLanded();
if(v==='reading' && !this.reduced){
this.pokeTimer=setInterval(()=>{
const pick=this.readingList.filter(()=>Math.random()<0.35).slice(0,5);
this.pile?.poke(pick);
}, 420);
let i=-1;
this.scanTimer=setInterval(()=>{ const l=this.readingList; if(l.length) this.scanTo(l[i=(i+1)%l.length]); }, 120);
}
},
cards(){ this.$nextTick(this.land); },
},
created(){ this.els={}; this.slotAt={}; },
mounted(){
this.reduced=matchMedia('(prefers-reduced-motion: reduce)').matches;
document.documentElement.classList.add('sp-lock');
document.getElementById('prerender')?.remove(); // the server's placeholder (routers/web.py search_page)
this.pile=new Pile(poses=>{
for(const [id,p] of poses){ const el=this.els[id]; if(!el) continue; el.style.transform=poseTransform(p, this.size); el.style.visibility='visible'; }
});
this.onResize=()=>{ clearTimeout(this.resizeTimer); this.resizeTimer=setTimeout(()=>this.fit(false), 120); };
addEventListener('resize', this.onResize);
// Anything that moves the answer (its content growing, the meta line changing) re-seats the
// tiles already on their cards; the flight itself is not interrupted.
this.ro=new ResizeObserver(()=>{ if(this.landed.length) this.place(false); });
this.ro.observe(this.$el.querySelector('.sp-top'));
this.loadPlatforms();
this.mountField();
this.$nextTick(()=>this.fit(true));
const params=new URLSearchParams(location.search), q=params.get('q');
// Where visitors come to /search from: `?ref=` (the landing's Tools link says `landing-nav` or
// `landing-footer`), else the referring host. The person this merges into after sign-in is
// what the search-to-signup funnel reads (see interface/dashboard.md).
let referrer=''; try{ referrer=document.referrer ? new URL(document.referrer).hostname : ''; }catch(e){}
this.track('search_opened', {ref:params.get('ref') || (referrer && referrer!==location.hostname ? referrer : 'direct'),
has_q:!!q, signed_in:this.sessionChecked ? !!this.authed : null});
if(q){ this.text=q; this.ask(); }
},
beforeUnmount(){
document.documentElement.classList.remove('sp-lock');
removeEventListener('resize', this.onResize);
this.ro?.disconnect();
cancelAnimationFrame(this.fieldRaf); this.field?.dispose();
clearInterval(this.pokeTimer); clearInterval(this.scanTimer); clearTimeout(this.resizeTimer); clearTimeout(this.flungTimer);
cancelAnimationFrame(this.scrollRaf);
this.pile?.destroy();
},
methods: {
ask(){
// An empty box submits its placeholder: the example is an invitation, not decoration.
if(!this.text.trim()) this.text=this.examples[0];
const q=this.text.trim();
history.replaceState(history.state, '', '/search?q='+encodeURIComponent(q));
this.findRun(q);
},
pick(ex){ this.text=ex; this.ask(); },
// Back to the empty page: the answer goes, its tiles fall back into the pile.
exit(){
this.text='';
history.replaceState(history.state, '', '/search');
this.findExit();
this.$nextTick(()=>this.$el.querySelector('#sp-q')?.focus());
},
// The landing page's first-screen glyph field (/media/landing/hero-particles.js), mounted on
// this page area and driven from this page's frame loop, so both first screens share one
// implementation. Without WebGL the field declines and the gradient behind it remains.
mountField(){
const mount=()=>{
if(this._isUnmounted || !this.$refs.stage) return;
this.field=window.tregMountField?.(this.$refs.stage);
if(!this.field) return;
const loop=t=>{ this.field.tick(t); this.fieldRaf=requestAnimationFrame(loop); };
this.fieldRaf=requestAnimationFrame(loop);
};
if(window.tregMountField) return mount();
const script=document.createElement('script');
script.src='/media/landing/hero-particles.js'; script.onload=mount;
document.head.appendChild(script);
},
// The scan light moves several times a second while the judge reads: set on the element, not
// through a reactive field, so it does not re-render every tile on every step.
scanTo(key){
this.els[this.scanned]?.classList.remove('scan');
this.scanned=key;
this.els[key]?.classList.add('scan');
},
// The answer panel scrolls its landed tiles along; one re-seat per frame, however fast it scrolls.
onPanelScroll(){
if(this.scrollRaf) return;
this.scrollRaf=requestAnimationFrame(()=>{ this.scrollRaf=0; this.place(false); });
},
tileRef(key, el){
if(!el){ delete this.els[key]; return; }
if(this.els[key]===el) return;
this.els[key]=el;
const pose=this.pile?.poseOf(key);
if(pose){ el.style.transform=poseTransform(pose, this.size); el.style.visibility='visible'; }
},
// Tiles in the pile can be picked up and thrown; a press that barely moves is a click and
// opens the platform. Tiles on an answer card are plain links.
press(p, e){
if(e.button!==0) return;
const sb=this.$refs.stage.getBoundingClientRect();
const start={x:e.clientX, y:e.clientY};
const held=this.pile.grab(p.key, e.clientX-sb.left, e.clientY-sb.top);
let moved=false;
if(held){ e.preventDefault(); this.dragging=p.key; }
const move=ev=>{
if(Math.hypot(ev.clientX-start.x, ev.clientY-start.y)>5) moved=true;
if(held) this.pile.drag(ev.clientX-sb.left, ev.clientY-sb.top);
};
const up=()=>{
removeEventListener('pointermove', move); removeEventListener('pointerup', up); removeEventListener('pointercancel', up);
if(held){ this.pile.release(); this.flung=p.key; clearTimeout(this.flungTimer); this.flungTimer=setTimeout(()=>{ this.flung=null; }, 1500); }
this.dragging=null;
if(!moved){ this.findTrackClick('tile', p.home, p.key.startsWith('v:') ? {provider:p.slug} : {}); this.findGoDashboard(p.home); }
};
addEventListener('pointermove', move); addEventListener('pointerup', up); addEventListener('pointercancel', up);
},
// Size the page to the viewport, size the tiles to the page, and (re)build the pile. The first
// build drops the tiles from above so the pile forms on screen; a resize settles it unseen.
fit(first){
const stage=this.$refs.stage; if(!stage || !this.pile || !this.tiles.length) return;
if(first && this.built) return;
stage.style.height=Math.max(520, innerHeight-stage.getBoundingClientRect().top-scrollY)+'px';
const W=stage.clientWidth, H=stage.clientHeight;
// On a phone the pile is a band under the question box, not half the screen: smaller tiles
// and a smaller share, so the answer above keeps the height.
const size=W<640 ? tileSize(W, H, this.tiles.length, 0.08, 15) : tileSize(W, H, this.tiles.length, 0.25);
const rebuild=!this.built || size!==this.size;
this.size=size; this.pile.size=size;
// Room for the settled pile under the question box: the tiles' area, loosely packed, across the
// width. Small tiles on a narrow screen settle much denser.
this.floor=Math.round(this.tiles.length*size*size/((W<640 ? 1.1 : 0.62)*W) + size*0.8);
this.pile.bounds(W, H);
if(rebuild){
this.pile.clear();
const inPile=this.tiles.map(t=>t.key).filter(k=>!this.landed.includes(k));
if(this.reduced || this.built){ inPile.forEach(k=>this.pile.add(k)); this.pile.settle(); }
// An answer can land before the last tile has dropped (a shared ?q= link, a cached judge):
// a tile already on its card is not dropped into the pile behind it.
else inPile.forEach((k,i)=>setTimeout(()=>{ if(!this.landed.includes(k)) this.pile.add(k); }, i*10));
this.built=true;
}
this.$nextTick(()=>this.place(false));
},
// The answer arrived: each fitting tile leaves the pile from where it lies and flies to its
// place on a card (`cards`: logo and mark).
land(){
const slots=this.measureSlots(), flying=[];
// Every tile to its start pose first, one layout for all of them, then every flight at once.
this.cards.forEach((c,i)=>{
for(const key of [c.logo, c.mark]){
const el=this.els[key]; if(!key || !el || !slots[key] || this.landed.includes(key)) continue;
const pose=this.pile.remove(key);
const from=pose || {x:slots[key].x, y:this.$refs.stage.clientHeight, angle:0};
el.style.transition='none'; el.style.transform=poseTransform(from, this.size);
el.style.visibility='visible';
flying.push([el, i]);
this.landed.push(key);
}
});
if(flying.length) this.$refs.stage.getBoundingClientRect();
for(const [el, i] of flying){
el.style.transition=this.reduced ? 'none' : LAND;
el.style.transitionDelay=this.reduced ? '0s' : (100+i*70)+'ms';
}
this.place(true, slots);
},
// Keep landed tiles on their slots (after a scroll of the answer panel or a resize); a slot
// scrolled out of the panel hides its tile rather than letting it float over the page.
place(animated, slots=this.measureSlots()){
const stage=this.$refs.stage; if(!stage) return;
const panel=this.$refs.panel?.getBoundingClientRect(), sb=stage.getBoundingClientRect();
for(const key of this.landed){
const el=this.els[key], s=slots[key]; if(!el || !s) continue;
if(!animated){ el.style.transition='none'; el.style.transitionDelay='0s'; }
el.style.transform=`translate(${s.x}px,${s.y}px) scale(${s.w/this.size})`;
const inView=!panel || (s.y+sb.top>=panel.top-4 && s.y+sb.top+s.w<=panel.bottom+4);
el.style.visibility=inView ? 'visible' : 'hidden';
this.slotAt[key]=s;
}
},
measureSlots(){
const stage=this.$refs.stage, out={}; if(!stage) return out;
const sb=stage.getBoundingClientRect();
for(const el of stage.querySelectorAll('[data-slot]')){ const b=el.getBoundingClientRect(); out[el.dataset.slot]={x:b.left-sb.left, y:b.top-sb.top, w:b.width}; }
return out;
},
// A new question: the last answer's tiles fall back into the pile from where they sat.
dropLanded(){
for(const key of this.landed){
const el=this.els[key], s=this.slotAt[key];
if(el){ el.style.transition='none'; el.style.transitionDelay='0s'; }
this.pile.add(key, s ? s.x : undefined, s ? s.y : undefined);
}
this.landed=[];
},
tileClass(p){
const k=p.key;
return {read:this.reading.has(k), landed:this.landed.includes(k), held:this.dragging===k || this.flung===k,
dim:(this.findBusy && !this.reading.has(k))
|| (this.find.phase==='done' && this.cards.length && !this.landed.includes(k) && !this.litVendors.has(k))};
},
pct(p){ return p==null ? '' : Math.round(p*100)+'%'; },
openCard(c, i){
this.findTrackClick('card', c.platform, {provider:this.byVendor ? c.slug : undefined, rank:i+1});
this.findGoDashboard(c.platform);
},
openJob(c, i, g){
this.findTrackClick('job', g.platform, {provider:g.rows[0]?.provider, rank:i+1});
this.findGoDashboard(g.platform);
},
// A job line's corner: its price, and on a platform card how many vendors sell it.
jobMeta(g){
const n=this.findProviders(g).length;
return [!this.byVendor && n+' provider'+(n===1?'':'s'), this.findPrice(g)].filter(Boolean).join(' · ');
},
},
}
</script>
<template>
<div class="sp" :class="{answered:find.phase==='done' && cards.length}" ref="stage">
<section class="sp-top">
<div v-if="find.phase==='idle' || findBusy" class="sp-hero" :class="{quiet:findBusy}">
<span class="sp-count" :style="plats.settled ? null : {visibility:'hidden'}">{{platforms.length}} platforms · {{vendors.length}} providers<template v-if="toolCountText"> · {{toolCountText}} tools</template></span>
<h1 class="hero-h1">What does your agent<br><span>need to do?</span></h1>
</div>
<div v-else-if="find.phase==='error'" class="sp-panel sp-msg"><p>{{find.error}}</p>
<button class="btn sm" type="button" @click="ask()">Try again</button></div>
<div v-else-if="!cards.length" class="sp-panel sp-msg">
<p><b>Nothing in the catalog does this yet.</b>
<template v-if="find.verdict==='none'"> We read {{find.read}} candidate{{find.read===1?'':'s'}} and none fit.</template>
Tell us what you need and it steers what gets added next.</p>
<button class="btn sm primary" type="button" @click="findRequestTool()">Request this tool</button>
</div>
<div v-else class="sp-panel" ref="panel" @scroll.passive="onPanelScroll">
<!-- The answer's one action sits on its title line, where the eye lands when the cards appear. -->
<div class="sp-panel-h">
<span v-if="find.verdict==='strong'" class="sp-title">Best fit for <b>{{find.q}}</b></span>
<span v-else-if="find.verdict==='closest'" class="sp-title warn">No strong fit for <b>{{find.q}}</b>. Closest matches:</span>
<span v-else-if="find.verdict==='name'" class="sp-title">Tools for <b>{{find.q}}</b></span>
<span v-else class="sp-title">Keyword matches for <b>{{find.q}}</b>. Matching by meaning is unavailable right now.</span>
<span class="sp-actions">
<button class="sp-share" type="button" @click="findShare()">{{findCopied==='share'?'Link copied':'Share'}}</button>
<button class="sp-copy" type="button" @click="findCopyAll()">
<svg v-if="findCopied!=='all'" width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><rect x="9" y="9" width="12" height="12" rx="2"/><path d="M5 15V5a2 2 0 0 1 2-2h10"/></svg>
<svg v-else width="14" height="14" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.4" stroke-linecap="round" stroke-linejoin="round" aria-hidden="true"><path d="M20 6 9 17l-5-5"/></svg>
{{findCopied==='all' ? 'Copied. Paste it to your agent' : 'Copy for your agent'}}</button>
</span>
</div>
<div class="sp-cards">
<article v-for="(c, ci) in cards" :key="c.slug" class="sp-card" :class="{weak:findWeak(c)}">
<!-- A vendor card: the vendor's tile lands in the logo place, and its platform is named
under it. A platform card (a bare name): the platform's logo. -->
<header>
<span class="sp-slot sp-slot-lg" :data-slot="c.logo" aria-hidden="true"></span>
<button class="sp-plat" type="button" @click="openCard(c, ci)">
<span class="sp-vendor">{{c.label}}</span>
<small v-if="byVendor">
<span v-if="c.mark" class="sp-slot" :data-slot="c.mark" aria-hidden="true"></span>
<span v-else class="sp-slot sp-mark" aria-hidden="true">
<img v-if="!platLogoBad[c.platform]" :src="'/logos/platforms/'+c.platform+'.svg'" alt="" @error="platLogoBad[c.platform]=true">
<span v-else class="sp-i" :style="{background:platTileBg(c.platform)}">{{platInitial({label:c.platform_label, slug:c.platform})}}</span>
</span>{{c.platform_label}}</small></button>
<span v-if="c.p!=null" class="sp-fit">{{pct(c.p)}}</span>
</header>
<ul>
<li v-for="g in c.jobs.slice(0,3)" :key="g.key">
<button type="button" @click="openJob(c, ci, g)" :title="g.rows.map(r=>r.id).join(', ')">
<span class="sp-job">{{g.label}}</span>
<span class="sp-m">{{jobMeta(g)}}</span>
</button>
</li>
</ul>
<span v-if="c.jobs.length>3" class="sp-more">+{{c.jobs.length-3}} more</span>
</article>
</div>
</div>
</section>
<form class="sp-ask" @submit.prevent="ask()">
<div class="sp-in">
<label class="rd-sr-only" for="sp-q">Describe the job</label>
<!-- Esc here, not on the window: Esc on the sign-in dialog must close the dialog only. -->
<input id="sp-q" v-model="text" autocomplete="off" :placeholder="examples[0]" @keydown.esc="findActive && exit()">
<button v-if="findActive || text" class="sp-x" type="button" aria-label="Clear the search" @click="exit()">×</button>
<button class="sp-go" :class="{busy:findBusy}" type="submit" aria-label="Find tools">
<svg v-if="!findBusy" width="16" height="16" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2" aria-hidden="true"><path d="M5 12h14M13 6l6 6-6 6"/></svg>
</button>
</div>
<p class="sp-meta" aria-live="polite">
<template v-if="find.phase==='recall'">Looking through the catalog…</template>
<template v-else-if="find.phase==='reading'"><i class="dot"></i>{{find.candidates.length}} candidates on {{findCandidatePlatforms.length}} platforms from {{findCandidateVendors.length}} providers. Reading them for your job…</template>
</p>
<div class="sp-chips" v-if="find.phase==='idle' || find.phase==='done' && !cards.length">
<button v-for="ex in examples.slice(1)" :key="ex" class="sp-chip" type="button" @click="pick(ex)">{{ex}}</button>
</div>
</form>
<div class="sp-floor" :style="{height:floor+'px'}" aria-hidden="true"></div>
<button v-for="p in tiles" :key="p.key" :ref="el=>tileRef(p.key, el)" class="sp-tile" :class="tileClass(p)"
:style="{width:size+'px', height:size+'px'}" type="button" tabindex="-1" aria-hidden="true" :title="p.label" @pointerdown="press(p, $event)">
<img v-if="!platLogoBad[p.bad]" :src="p.src" alt="" draggable="false" @error="platLogoBad[p.bad]=true">
<span v-else class="sp-i" :style="{background:platTileBg(p.slug)}">{{platInitial(p)}}</span>
</button>
</div>
</template>
<style>
html.sp-lock,html.sp-lock body{overflow:hidden;overscroll-behavior:none}
/* The landing page's hero field, prepended into the page area by hero-particles.js. */
.sp > .hero-particles{position:absolute;inset:0;width:100%;height:100%;z-index:0;pointer-events:none;contain:strict}
</style>
<style scoped>
/* Same visual language as the landing page (src/treg/web/landing.html): warm paper, white surfaces
lifted by soft shadows rather than borders, pill controls, Geist Pixel display type. */
.sp{--l-bg:#f4f4f1;--l-surface:#fff;--l-ink:#1a1a1a;--l-muted:#7c7c7c;--l-muted2:#989898;--l-line:#2626231a;--l-line2:#25252233;
--l-inverse:#1a1a1a;--l-inverse-ink:#f8f8f7;--l-shadow-sm:0 1px 2px #00000014;
--l-shadow-md:0 1px 2px -1px #0000000a,0 4px 6px -1px #0000000f;
--l-shadow-lg:0 1px 2px -1px #0000000a,0 4px 6px -1px #0000000f,0 8px 16px #0000000a;--l-ease:cubic-bezier(.2,.72,.25,1);
position:relative;display:flex;flex-direction:column;align-items:center;gap:18px;overflow:hidden;padding:88px 20px 0;box-sizing:border-box;
min-height:520px;background:var(--l-bg);color:var(--l-ink);font-family:var(--sans)}
/* The landing hero's soft green light, under its glyph field. */
.sp::before{content:"";position:absolute;inset:0 0 auto;height:75%;pointer-events:none;z-index:0;
background:radial-gradient(ellipse at 50% 20%,#dce8d585,transparent 65%)}
[data-theme="dark"] .sp{--l-bg:#151412;--l-surface:#1e1d1b;--l-ink:#f2efe8;--l-muted:#a39f97;--l-muted2:#8a867e;--l-line:#ffffff14;--l-line2:#ffffff26;
--l-inverse:#f2efe8;--l-inverse-ink:#151412;--l-shadow-sm:0 1px 2px #0006;--l-shadow-md:0 1px 2px #0006,0 4px 10px -2px #0008;
--l-shadow-lg:0 1px 2px #0006,0 8px 18px -4px #000a}
.sp-top{position:relative;z-index:3;width:100%;max-width:1160px;flex:1;min-height:0;display:flex;align-items:center;justify-content:center}
.sp.answered .sp-top{align-items:flex-end}
.sp-hero{display:flex;flex-direction:column;align-items:center;gap:26px;text-align:center;transition:opacity .3s}
.sp-hero.quiet{opacity:.5}
.sp-count{display:inline-block;font-family:var(--mono);font-size:12px;letter-spacing:.08em;color:var(--l-muted);
border:1px solid var(--l-line2);border-radius:999px;padding:5px 15px;background:var(--l-surface)}
.sp-hero h1{margin:0;font-family:var(--display,"Geist Pixel",monospace);font-weight:400;letter-spacing:0;font-size:clamp(30px,4.4vw,56px);line-height:1.16}
.sp-hero h1 span{color:var(--l-muted)}
.sp-panel{width:100%;max-height:100%;overflow:auto;box-sizing:border-box;padding:6px 6px 10px;display:flex;flex-direction:column;gap:14px;overscroll-behavior:contain}
.sp-panel-h{display:flex;align-items:center;justify-content:space-between;gap:12px 20px;flex-wrap:wrap;font-size:14px;color:var(--l-muted)}
.sp-panel-h b{color:var(--l-ink);font-weight:500}
.sp-title{min-width:0}
.sp-title.warn{color:#ba6603}
.sp-actions{display:flex;align-items:center;gap:16px;flex:none}
.sp-share{border:0;background:none;padding:0;font:inherit;font-size:13.5px;font-weight:500;color:var(--l-muted);cursor:pointer}
.sp-share:hover{color:var(--l-ink)}
.sp-copy{display:inline-flex;align-items:center;gap:8px;border:0;border-radius:999px;padding:11px 22px;cursor:pointer;
background:var(--l-inverse);color:var(--l-inverse-ink);font:inherit;font-size:14px;font-weight:550;letter-spacing:.01em;
box-shadow:var(--l-shadow-sm);transition:box-shadow .24s var(--l-ease),transform .12s var(--l-ease)}
.sp-copy:hover{box-shadow:var(--l-shadow-lg);transform:translateY(-1px)}
.sp-msg{max-width:760px;flex-direction:row;align-items:center;justify-content:space-between;flex-wrap:wrap;gap:12px;
background:var(--l-surface);border-radius:15px;box-shadow:var(--l-shadow-md);padding:16px 20px;font-size:14px;color:var(--l-muted)}
.sp-msg p{margin:0}
.sp-msg b{color:var(--l-ink);font-weight:500}
.sp-msg .btn{border-radius:999px}
.sp-cards{display:grid;grid-template-columns:repeat(auto-fill,minmax(min(100%,290px),1fr));gap:14px}
/* Fade only: the landing tiles measure the card slots as the cards appear, so the cards must not move. */
.sp-card{background:var(--l-surface);border-radius:15px;box-shadow:var(--l-shadow-md);padding:14px 16px;display:flex;flex-direction:column;gap:8px;
transition:box-shadow .24s var(--l-ease);animation:sp-in .45s .3s both}
.sp-card:hover{box-shadow:var(--l-shadow-lg)}
@keyframes sp-in{from{opacity:0}}
.sp-card header{display:flex;align-items:center;gap:10px}
.sp-mark{flex:none;box-sizing:border-box;border:1px solid var(--l-line);background:#fff;display:grid;place-items:center;overflow:hidden;border-radius:22%}
.sp-mark img{width:58%;height:58%;object-fit:contain}
.sp-mark .sp-i{width:100%;height:100%;display:grid;place-items:center;color:#fff;font-weight:600;font-size:9px}
.sp-slot{flex:none;width:18px;height:18px}
.sp-slot-lg{width:40px;height:40px}
.sp-plat{flex:1;min-width:0;text-align:left;border:0;background:none;padding:0;font:inherit;color:var(--l-ink);cursor:pointer;
display:flex;flex-direction:column;gap:1px}
.sp-vendor{font-size:15px;font-weight:550;overflow-wrap:anywhere}
.sp-plat small{font-size:12px;color:var(--l-muted);display:flex;align-items:center;gap:6px}
.sp-plat:hover .sp-vendor{text-decoration:underline;text-underline-offset:3px}
.sp-more{font-size:12px;color:var(--l-muted)}
.sp-fit{font-family:var(--mono);font-size:11.5px;padding:2px 9px;border-radius:999px;background:var(--l-inverse);color:var(--l-inverse-ink);font-variant-numeric:tabular-nums}
.sp-card.weak .sp-fit{background:none;color:var(--l-muted);border:1px solid var(--l-line2)}
.sp-card ul{list-style:none;margin:0;padding:0;display:flex;flex-direction:column}
.sp-card li button{width:100%;display:flex;justify-content:space-between;gap:10px;align-items:baseline;text-align:left;border:0;border-top:1px solid var(--l-line);
background:none;padding:8px 0;font:inherit;font-size:13px;color:var(--l-ink);cursor:pointer}
.sp-card li button:hover .sp-job{text-decoration:underline;text-underline-offset:3px}
.sp-job{min-width:0}
.sp-m{flex:none;font-family:var(--mono);font-size:11px;color:var(--l-muted);white-space:nowrap}
.sp-ask{width:min(660px,100%);display:flex;flex-direction:column;align-items:center;gap:12px;position:relative;z-index:6}
.sp-in{width:100%;position:relative}
.sp-in input{width:100%;box-sizing:border-box;font:inherit;font-size:16px;color:var(--l-ink);background:var(--l-surface);
border:1px solid var(--l-line2);border-radius:999px;padding:15px 104px 15px 24px;box-shadow:var(--l-shadow-sm);
transition:border-color .12s var(--l-ease),box-shadow .24s var(--l-ease)}
.sp-in input::placeholder{color:var(--l-muted2)}
.sp-in input:hover{box-shadow:var(--l-shadow-md)}
.sp-in input:focus,.sp-in input:focus-visible{outline:none;border-color:var(--l-muted);box-shadow:var(--l-shadow-md)}
.sp-x{position:absolute;right:58px;top:50%;transform:translateY(-50%);width:32px;height:32px;border:0;border-radius:999px;background:none;
color:var(--l-muted);font-size:20px;line-height:1;cursor:pointer}
.sp-x:hover{color:var(--l-ink);background:var(--l-line)}
.sp-go{position:absolute;right:7px;top:50%;transform:translateY(-50%);width:40px;height:40px;border-radius:999px;border:0;
background:var(--l-inverse);color:var(--l-inverse-ink);display:grid;place-items:center;cursor:pointer;box-shadow:var(--l-shadow-sm)}
.sp-go.busy::after{content:"";width:15px;height:15px;border-radius:50%;border:2px solid color-mix(in srgb,currentColor 30%,transparent);border-top-color:currentColor;animation:sp-spin .8s linear infinite}
@keyframes sp-spin{to{transform:rotate(360deg)}}
.sp-meta{margin:0;min-height:34px;display:flex;align-items:center;justify-content:center;gap:10px;flex-wrap:wrap;font-family:var(--mono);font-size:12px;color:var(--l-muted)}
.sp-meta .dot{width:7px;height:7px;border-radius:50%;background:var(--l-ink);display:inline-block}
.sp-chips{display:flex;flex-wrap:wrap;gap:8px;justify-content:center;max-width:760px}
.sp-chip{font:inherit;font-size:13px;font-weight:500;border:1px solid var(--l-line2);background:var(--l-surface);color:var(--l-ink);border-radius:999px;padding:7px 15px;cursor:pointer;
transition:border-color .12s var(--l-ease),box-shadow .24s var(--l-ease)}
.sp-chip:hover{border-color:var(--l-muted);box-shadow:var(--l-shadow-sm)}
/* The floor the pile rests on is the bottom of the page; this spacer only keeps the question box
and its examples above where the pile settles. */
.sp-floor{flex:none;width:100%}
.sp-tile{position:absolute;left:0;top:0;z-index:2;visibility:hidden;box-sizing:border-box;padding:0;border-radius:22%;border:1px solid var(--l-line);background:#fff;
display:grid;place-items:center;cursor:grab;transform-origin:0 0;will-change:transform;user-select:none;touch-action:none;
box-shadow:var(--l-shadow-sm);transition:opacity .4s,filter .4s}
.sp-tile img{width:58%;height:58%;object-fit:contain;pointer-events:none}
.sp-tile .sp-i{width:100%;height:100%;border-radius:inherit;display:grid;place-items:center;color:#fff;font-weight:600}
.sp-tile.read{box-shadow:var(--l-shadow-lg)}
.sp-tile.scan{box-shadow:0 0 0 2px var(--l-bg),0 0 0 3.5px var(--l-ink)}
.sp-tile.dim{opacity:.4;filter:grayscale(1)}
.sp-tile.landed{z-index:7;box-shadow:none;cursor:pointer}
.sp-tile.held{z-index:8;cursor:grabbing;opacity:1;filter:none;box-shadow:var(--l-shadow-lg)}
@media (prefers-reduced-motion:reduce){.sp-card,.sp-hero{transition:none;animation:none}}
/* A phone: the question box sits low, just above the pile's band, so the hero or the answer gets
the screen above it; the examples are one row that scrolls sideways instead of a wall of pills. */
@media (max-width:640px){
.sp{padding:68px 14px 0;gap:10px}
.sp-hero{gap:14px}
.sp-count{font-size:10.5px;letter-spacing:.02em;white-space:nowrap}
.sp-hero h1{font-size:clamp(28px,8.4vw,36px)}
.sp-panel{padding:4px 2px 8px;gap:12px}
.sp-panel-h{gap:10px}
.sp-actions{width:100%;justify-content:space-between}
.sp-copy{padding:9px 16px;font-size:13px}
.sp-cards{gap:10px}
.sp-card{padding:12px 14px}
.sp-meta{min-height:22px;font-size:11px;text-align:center}
.sp-chips{flex-wrap:nowrap;justify-content:flex-start;max-width:100%;width:100%;overflow-x:auto;scrollbar-width:none;
padding:0 2px 2px;-webkit-mask-image:linear-gradient(90deg,#000 88%,transparent);mask-image:linear-gradient(90deg,#000 88%,transparent)}
.sp-chips::-webkit-scrollbar{display:none}
.sp-chip{flex:none;white-space:nowrap;font-size:12.5px;padding:6px 13px}
}
</style>
frontend/src/pages/TeamPage.vue
+2
View File
@@ -64,6 +64,8 @@ export default { setup: useDashboard }
<p class="sub" style="margin:-2px 0 8px;font-size:12px"><b>Tools</b> = which endpoints/CLIs a member may call or run. <b>Local run</b> = may run CLIs on their own machine (off = server only). The owner always has full access.</p>
<table>
<tr><th>Email</th><th>Role</th><th style="text-align:right">Today</th><th>Daily cap</th><th>Tools</th><th>Local run</th><th></th></tr>
<!-- An empty roster under its header read as "no members" until the request answered. -->
<tr v-if="!orgMembersLoaded"><td colspan="7" class="muted">Loading members…</td></tr>
<template v-for="m in rosterMembers" :key="m.key">
<tr v-if="m.is_observed">
<td><span style="opacity:.45">↳</span> <b>{{m.client}}</b> <span class="chip" title="seen in this member\'s traffic — the runtime reports itself; attribution, not authentication">detected</span>
frontend/src/pages/ToolsPage.vue
+2 -2
View File
@@ -1,6 +1,6 @@
<script>
import { useDashboard } from '../state/context'
export default { setup: useDashboard }
export default { setup: useDashboard, mounted(){ this.loadPlatforms() } } // the catalog size in the copy
</script>
<template>
@@ -66,7 +66,7 @@ export default { setup: useDashboard }
<p v-if="!loading && !hasAnyTools" class="sub">Nothing here{{q?' matches "'+q+'"':' yet'}}.<template v-if="!q && isPersonal(activeOrg) && myOrgs.some(o=>!isPersonal(o))"> This is your <b>personal</b> space - your team's tools live under another org. <a href="#" @click.prevent="jumpToTeam()">Switch to a team →</a></template></p>
<div v-if="!loading && !hasAnyTools && !q && canRegister" style="margin-top:14px">
<div style="max-width:660px;border:1px solid var(--line);border-radius:16px;padding:20px 22px;background:var(--panel2)">
<h3 style="margin:0 0 6px;font-size:15px">Nothing of your own yet — but you can already call 2,800+ tools</h3>
<h3 style="margin:0 0 6px;font-size:15px">Nothing of your own yet — but you can already call {{toolCountText||'thousands of'}} tools</h3>
<p class="sub" style="margin:0 0 12px">New verified accounts get <b>$1.00 of free credit once</b> when creating an eligible team, so the catalog works before you register anything: find a tool by what it does, see the price, call it. <a href="#" @click.prevent="go('connections')">Browse the catalog →</a><br>When you're ready to add your <i>own</i> keys and skills — your <span class="mono">.env</span> and skill folders — load them here and they become callable tools for the whole team.</p>
<div class="seg" style="margin-bottom:10px">
<button :class="{on:emptyTab==='agent'}" @click="emptyTab='agent'">Agent instruction</button>
frontend/src/state/activity.js
+2 -1
View File
@@ -50,5 +50,6 @@ pretty(text){ try{ return JSON.stringify(JSON.parse(text), null, 2); }catch(e){
isVideoUrl(u){ try{ return /\.(mp4|webm|mov|m4v)$/i.test(new URL(u).pathname); }catch(e){ return /\.(mp4|webm|mov|m4v)(\?|$)/i.test(u||''); } },
fmtBytes(n){ if(n==null) return '—'; if(n<1024) return n+' B'; if(n<1048576) return (n/1024).toFixed(1)+' KB'; return (n/1048576).toFixed(2)+' MB'; },
async copyCallBody(){ const t=this.callView&&this.callView.response&&this.callView.response.body_text; if(!t) return; if(await this.toClipboard(t)){ this.callCopied='Copied'; setTimeout(()=>{ this.callCopied=''; },1400); } },
async loadCalls(){ try{ if(!this.apiKeys.length)await this.loadApiKeys(); const q='?limit=100'+(this.activityKey?'&api_key_id='+encodeURIComponent(this.activityKey):''); const [calls,runs]=await Promise.all([this.api('/calls'+q), this.api('/runs'+q).catch(()=>[])]); this.calls=calls; this.runs=runs; }catch(e){ this.err='Failed to load activity.'; } }
async loadCalls(){ try{ if(!this.apiKeys.length)await this.loadApiKeys(); const q='?limit=100'+(this.activityKey?'&api_key_id='+encodeURIComponent(this.activityKey):''); const [calls,runs]=await Promise.all([this.api('/calls'+q), this.api('/runs'+q).catch(()=>[])]); this.calls=calls; this.runs=runs; }catch(e){ this.err='Failed to load activity.'; }
finally{ this.callsLoaded=true; } } // the empty-state line waits for this, not for the page's global `loading`
}
frontend/src/state/boot.js
+25 -7
View File
@@ -33,7 +33,21 @@ export default async function boot(){
if(v==='billing'){ this.orgTab='billing'; v='orgs'; }
if(['tools','orgs','activity','usage','admin','help','secrets','start','resources','connections','referrals','hub'].includes(v)) this.go(v, true);
});
this.meta = await fetch('/meta',{headers:{'ngrok-skip-browser-warning':'1'}}).then(r=>r.json()).catch(()=>this.meta);
// Catalog data does not depend on the session, so a view that shows it starts fetching now,
// alongside /meta and /auth/me, instead of after them (the shelves used to arrive last).
const catalogShelf=this.catalogFromPath(location.pathname)?.slug || this.platformFromHash();
if(this.catalogFromPath(location.pathname) || catalogShelf || location.hash==='#connections') this.loadPlatforms();
if(catalogShelf) this.prefetchPlatform(catalogShelf);
// /search needs no session to draw, so it does not wait for one: the page paints now, and the
// session (the top bar's buttons, a result clicked before sign-in) follows when /auth/me answers.
if(this.catalogFromPath(location.pathname)?.view==='find'){ this.publicCatalog=true; this.view='find'; this.bootReady=true; }
// /meta and /auth/me do not depend on each other or on the bundle, so index.html starts both
// before this code has even downloaded; a boot without them (tests, a remount) starts its own.
const early=window.__tregBoot||{}; delete window.__tregBoot;
const headers={'ngrok-skip-browser-warning':'1'};
const metaReq=early.meta||fetch('/meta',{headers}).then(r=>r.json());
const meReq=early.me||fetch('/auth/me',{credentials:'include',headers}).then(r=>r.ok?r.json():null);
this.meta = await metaReq.catch(()=>this.meta);
this.proxy = this.meta.public_url || location.origin;
this.initAnalytics();
// deploy detection: long-lived tabs learn about a new bundle on tab focus + a slow poll,
@@ -66,17 +80,21 @@ export default async function boot(){
if(route||mkRoute) history.replaceState(null,'',stashed);
} }
this._restoreAgent();
const me = await fetch('/auth/me',{credentials:'include',headers:{'ngrok-skip-browser-warning':'1'}}).then(r=>r.ok?r.json():null).catch(()=>null);
const me = await meReq.catch(()=>null);
this.sessionChecked=true;
if(me){ this.sessionMode=true; this.me=me.email; this.isAdmin=!!me.is_superadmin; this.onboarded=!!me.onboarded; this.icHash=me.intercom_user_hash||''; await this.loadAll(); this.analyticsIdentify(); this.initIntercom();
// Share-born arrival (/app/skills/x?invite_org=N from the invite email): accept silently and
// enter that team — the emailed "Sign in & accept" click was the consent. Otherwise the normal
// first-run / invite-banner flow.
// A result clicked on /search before sign-in: continue to it now.
if(this.findResume()){ this.maybeOnboard(); return; }
if(mkRoute){ this.maybeOnboard(); this.openProvider(mkRoute, true); return; }
if(runRoute){ this.maybeOnboard(); this.openRun(runRoute, true); return; }
// Signed in on a /catalog URL: the same views, but as a member — so `publicCatalog` is
// dropped and the shell comes back in full (vault, activity, try-it).
if(catRoute){ this.publicCatalog=false; this.maybeOnboard();
if(catRoute.slug) this.openPlatform(catRoute.slug, true); else this.go('connections', true);
// /search stays a public page for members too: it is a place to ask, not a dashboard view.
if(catRoute){ this.publicCatalog=catRoute.view==='find'; if(catRoute.view!=='find') this.maybeOnboard();
this.openCatalogRoute(catRoute);
return; }
const pfRoute=this.platformFromHash();
if(pfRoute){ this.maybeOnboard(); this.openPlatform(pfRoute, true); return; }
@@ -94,8 +112,8 @@ export default async function boot(){
// A token holder on a /catalog URL is a member, not a public visitor: same treatment as the
// session branch. Without this the route falls through to viewFromHash() — which is null for
// a path route — and a shelf link lands on Getting started instead.
if(catRoute){ this.publicCatalog=false;
if(catRoute.slug) this.openPlatform(catRoute.slug, true); else this.go('connections', true);
if(catRoute){ this.publicCatalog=catRoute.view==='find';
this.openCatalogRoute(catRoute);
return; }
const pfTok=this.platformFromHash();
if(runRoute){ this.openRun(runRoute, true); return; }
@@ -113,7 +131,7 @@ export default async function boot(){
// Platform tab (the provider shelf) fills for a signed-out visitor too — only /connections
// needs a session, and its failure is caught. Without this the tab reads "Platform 0" and
// renders blank in an incognito window.
if(catRoute.slug) this.openPlatform(catRoute.slug, true); else { this.view='connections'; this.loadConnections(); }
if(catRoute.view==='find'){ this.view='find'; this.loadPlatforms(); } else if(catRoute.slug) this.openPlatform(catRoute.slug, true); else { this.view='connections'; this.loadConnections(); }
return; }
if(!inv && !linkOrg && !route && !qs.get('invite_expired') && !ref && !oauthSignin){ location.replace('/'); return; } // logged-out plain visit → the marketing landing owns the front door. `ref` is a use-case page's CTA (/app?ref=p1), so keep that attribution while opening sign-in in place.
if(route){ this.shareGate=route; this.demo.signin=true; } // shared link while logged out: the focused gate (no sandbox mint, no tour); after sign-in the boot lands on it (email verify reloads in place; OAuth restores via the treg-next stash)
frontend/src/state/catalog.js
+27 -5
View File
@@ -1,3 +1,4 @@
import { markRaw } from 'vue'
export default {
// ---- endpoint catalog (/catalog/*) ----
@@ -5,10 +6,16 @@ export default {
// server that predates these routes shows no platform shelf rather than an error.
async loadPlatforms(){
if(this.plats.loaded || this.plats.loading) return;
// Public catalog pages carry the same body inline (routers/web.py `_spa_catalog_page`), so
// their first render already has the shelves; read it once and fall back to the request.
const inline=document.getElementById('catalog-platforms');
if(inline){ inline.remove();
try{ const d=JSON.parse(inline.textContent); this.plats.list=d.platforms||[]; this.plats.providers=d.providers||{}; this.plats.loaded=this.plats.settled=true; return; }
catch(e){} }
this.plats.loading=true;
try{ const d=await this.api('/catalog/platforms'); this.plats.list=(d&&d.platforms)||[]; this.plats.loaded=true; }
try{ const d=await this.api('/catalog/platforms'); this.plats.list=(d&&d.platforms)||[]; this.plats.providers=(d&&d.providers)||{}; this.plats.loaded=true; }
catch(e){ this.plats.list=[]; }
finally{ this.plats.loading=false; } },
finally{ this.plats.loading=false; this.plats.settled=true; } },
// Platform pages are hash routes (/app#platform/<slug>): unlike /app/marketplace/<service> there
// is no server route to serve the SPA on a hard reload of a /app/platforms/<slug> path.
platformFromHash(){ const m=/^#platform\/(.+)$/.exec(location.hash||''); return m?decodeURIComponent(m[1]):null; },
@@ -17,6 +24,7 @@ export default {
// Same Vue views as the signed-in marketplace — this is one UI, not a second implementation.
catalogFromPath(p){
if(p==='/catalog' || p==='/catalog/') return {view:'connections', slug:null};
if(p==='/search' || p==='/search/') return {view:'find', slug:null};
const m=/^\/catalog\/([^/]+)\/?$/.exec(p||'');
return m ? {view:'platform', slug:decodeURIComponent(m[1])} : null;
},
@@ -26,6 +34,10 @@ export default {
viewFromHash(){ let v=(location.hash||'').replace('#','');
if(v==='billing'){ this.orgTab='billing'; v='orgs'; }
return ['tools','orgs','activity','usage','admin','help','secrets','start','resources','connections','referrals','hub'].includes(v)?v:null; },
// Land on a public catalog URL (see catalogFromPath): the finder page, a platform shelf, or the index.
openCatalogRoute(r){
if(r.view==='find'){ this.view='find'; this.loadPlatforms(); return; }
if(r.slug) this.openPlatform(r.slug, true); else this.go('connections', true); },
openPlatform(slug, fromPop){ this.resetConfirms();
this.detail=null; this.platSlug=slug; this.view='platform'; this.platOpen={}; this.epOpen={}; this.epTab={}; this.platEx={}; this.platActionsOpen=false;
this.platClearFilters(); this.platCopied='';
@@ -40,12 +52,20 @@ openPlatform(slug, fromPop){ this.resetConfirms();
if(!this.publicCatalog && !this.providers.length) this.loadConnections();
this.loadPlatform();
window.scrollTo(0,0); },
// Start a shelf's request before boot has resolved the session, so it is already in flight when
// the view opens. `loadPlatform` takes it over instead of asking again.
prefetchPlatform(slug){
const request=this.api('/catalog/platforms/'+encodeURIComponent(slug)+'?include_hidden=1');
request.catch(()=>{}); // loadPlatform reports the failure; an unclaimed prefetch just drops it
this.platPrefetch=markRaw({slug, request}); },
async loadPlatform(){ if(!this.platSlug) return;
this.platErr=''; this.platLoading=true; this.platData=null;
const pre=this.platPrefetch; this.platPrefetch=null;
// include_hidden=1: pull the account/utility endpoints too. They render behind a per-section
// "N management endpoints" expander rather than in the main ledger — the page decides that,
// client-side, off each endpoint's `kind` (see platRowsAll / platLedger).
try{ this.platData=await this.api('/catalog/platforms/'+encodeURIComponent(this.platSlug)+'?include_hidden=1'); }
try{ this.platData=await (pre && pre.slug===this.platSlug ? pre.request
: this.api('/catalog/platforms/'+encodeURIComponent(this.platSlug)+'?include_hidden=1')); }
catch(e){ this.platErr = e.status===404
? 'No catalog for this platform on this server yet.'
: 'Could not load the endpoint catalog'+(e.detail?': '+e.detail:'.'); }
@@ -53,7 +73,9 @@ async loadPlatform(){ if(!this.platSlug) return;
// Tile furniture. Catalog labels carry a parenthetical or an em-dash gloss ("Google Search
// (SERPs, keyword data)") that reads as noise under a logo — the tile shows the name, the
// title attribute keeps the whole thing.
platShort(label){ return String(label||'').split(' — ')[0].split(' (')[0].trim(); },
// The name filter behind the Catalog search box, shared by the shelves and the tab counts.
platNameHit(p, q){ return ((p.label||'')+' '+(p.slug||'')+' '+(p.providers||[]).join(' ')).toLowerCase().includes(q); },
platShort(label){ return String(label||'').split(' — ')[0].split(' (')[0].trim(); },
platInitial(pl){ return (this.platShort(pl.label)||pl.slug||'?').slice(0,1).toUpperCase(); },
// Deterministic hue from the slug: an undrawn platform keeps the same colour across reloads
// and differs from its neighbours, with no colour table to maintain.
@@ -259,7 +281,7 @@ mkOauth(service){ const p=this.providers.find(x=>x.service===service); return !!
// every provider on a public shelf would render as its bare slug.
provName(service){ const p=this.providers.find(x=>x.service===service); if(p) return p.display_name;
const c=(this.platData&&this.platData.providers||{})[service];
return (c&&c.display_name) || service; },
return (c&&c.display_name) || this.plats.providers[service] || service; },
// Provider-wide facts, served once per provider on the platform response rather than copied
// onto every row.
provFact(service, key){ const p=(this.platData&&this.platData.providers||{})[service]; return (p&&p[key])||''; },
frontend/src/state/catalogComputed.js
+36 -7
View File
@@ -1,3 +1,4 @@
import { isJobQuery } from './find.js'
export default {
mkProvider(){ return this.providers.find(p=>p.service===this.mkService)||null; },
mkConns(){ return this.connections.filter(c=>c.provider===this.mkService); },
@@ -34,15 +35,31 @@ mkNeedsCred(){ return this.mkConns.filter(c=>c.extra_credential_note); },
const rest=Object.keys(by).filter(c=>!order.includes(c)).sort();
return known.concat(rest).map(c=>({category:c, hint:hints[c]||'', items:by[c]}));
},
// The catalog's size as a headline ("3,300+"), from the platform shelves already loaded: rounded
// down to the hundred so it never overstates, and empty until the list arrives so no stale
// number is ever shown.
toolCountText(){
const n=this.plats.list.reduce((a,p)=>a+(p.endpoints||0),0);
return n>=100 ? (Math.floor(n/100)*100).toLocaleString('en-US')+'+' : '';
},
// The platform-name filter the Catalog search box applies, lowercased; '' for none. A sentence is
// a job, not a name (the finder answers it), and a find answer lights shelves instead of
// filtering them, so neither filters anything.
platNameQuery(){ return this.findActive || isJobQuery(this.q) ? '' : this.q.trim().toLowerCase(); },
mkTabs(){
const out=[{key:'all', label:'All', n:this.platCategories.reduce((a,g)=>a+g.items.length,0)}];
for(const g of this.platCategories) out.push({key:g.category, label:g.category, n:g.items.length});
// With a name filter typed, each tab counts what it would SHOW; a tab reading "Social 33" over
// an empty result made the filter look broken.
const q=this.platNameQuery;
const n=g=>q ? g.items.filter(p=>this.platNameHit(p, q)).length : g.items.length;
const out=[{key:'all', label:'All', n:this.platCategories.reduce((a,g)=>a+n(g),0)}];
for(const g of this.platCategories) out.push({key:g.category, label:g.category, n:n(g)});
out.push({key:'platform', label:'Platform', n:this.providers.length});
return out;
},
// A build without /catalog has no tiles to show, so it falls back to the integration shelves
// rather than opening on an empty tab.
mkTabActive(){ return this.platCategories.length ? this.mkTab : 'platform'; },
// rather than opening on an empty tab. Only once the shelves have answered: falling back while
// they load flashed the integration list on every visit before the tiles replaced it.
mkTabActive(){ return this.platCategories.length || !this.plats.settled ? this.mkTab : 'platform'; },
// Shelves, with the long ones cut down to their featured tiles. A category of 14 platforms is a
// wall you scroll past rather than read, so past PLAT_SHELF_MAX only the catalog's `featured`
// ranks get a full tile and the tail collapses into one "See X, Y, and N more" row. Rank first,
@@ -52,9 +69,18 @@ mkTabs(){
: this.platCategories.filter(g=>g.category===this.mkTabActive);
// The top-nav search reaches here too: with a query, every match shows (no featured collapse —
// a hit hidden behind "N more" reads as no hit) and empty shelves drop away.
const q=this.q.trim().toLowerCase();
// A find answer (state/find.js) owns the box while it is showing: the sentence is not a name
// to filter by, so every shelf stays, platforms the answer landed on first and uncollapsed.
if(this.findActive){
const hits=this.findHits;
if(!Object.keys(hits).length) return groups.map(g=>({...g, rest:[], total:g.items.length}));
return groups.map(g=>{ const items=[...g.items].sort((a,b)=>(hits[b.slug]||0)-(hits[a.slug]||0) || (b.endpoints||0)-(a.endpoints||0));
return {...g, items, rest:[], total:items.length, hits:items.filter(p=>hits[p.slug]).length}; })
.sort((a,b)=>b.hits-a.hits);
}
const q=this.platNameQuery;
if(q){
const hit=p=>((p.label||'')+' '+(p.slug||'')+' '+(p.providers||[]).join(' ')).toLowerCase().includes(q);
const hit=p=>this.platNameHit(p, q);
return groups.map(g=>{ const items=g.items.filter(hit)
.sort((a,b)=>(b.endpoints||0)-(a.endpoints||0));
return {...g, items, rest:[], total:items.length}; }).filter(g=>g.items.length);
@@ -74,7 +100,10 @@ mkTabs(){
mkPlatforms(){ return this.plats.list.filter(pl=>(pl.providers||[]).includes(this.mkService)); },
platRow(){ return this.plats.list.find(pl=>pl.slug===this.platSlug)||null; },
platLabel(){ return (this.platData&&this.platData.platform&&this.platData.platform.label)
|| (this.platRow&&this.platRow.label) || this.platSlug || 'Platform'; },
|| (this.platRow&&this.platRow.label)
// The raw slug ("google-ads") only once nothing better can arrive: shown while loading, it
// read as a broken title that then corrected itself.
|| (this.plats.settled && !this.platLoading ? this.platSlug || 'Platform' : ''); },
platProviders(){ // providers with endpoints here, in catalog order
const seen=[]; for(const g of (this.platData&&this.platData.capabilities||[])) for(const e of (g.endpoints||[])) if(!seen.includes(e.provider)) seen.push(e.provider);
for(const e of (this.platData&&this.platData.extended||[])) if(!seen.includes(e.provider)) seen.push(e.provider);
frontend/src/state/connections.js
+1 -1
View File
@@ -3,6 +3,7 @@ export default {
// ---- connections (registry OAuth) ----
async loadConnections(){
this.connErr='';
this.loadPlatforms(); // fire-and-forget, and first: the catalog must neither hold up nor wait for the connect UI
try{
const [ps, cs]=await Promise.all([
fetch('/oauth/providers').then(r=>r.json()).catch(()=>[]),
@@ -10,7 +11,6 @@ export default {
]);
this.providers=ps||[]; this.connections=cs||[];
}catch(e){ this.connErr=String(e.message||e); }
this.loadPlatforms(); // fire-and-forget: the catalog must never hold up the connect UI
},
authorizationMethodSpec(providerName, methodName){
const provider=(this.providers||[]).find(item=>item.service===providerName);
frontend/src/state/controller.js
+7 -2
View File
@@ -27,6 +27,8 @@ import details from './details.js'
import admin from './admin.js'
import snippets from './snippets.js'
import tryTool from './tryTool.js'
import find from './find.js'
import findComputed from './findComputed.js'
import lifecycle from './lifecycle.js'
import hub from './hub.js'
import billingComputed from './billingComputed.js'
@@ -37,8 +39,8 @@ import onboardingComputed from './onboardingComputed.js'
import detailsComputed from './detailsComputed.js'
export default {
data,
computed: {...resourcesComputed, ...billingComputed, ...catalogComputed, ...sessionComputed, ...agentsComputed, ...onboardingComputed, ...detailsComputed},
methods: {...resources, setElement(name, element) { this.elements[name] = element }, ...session, ...team, ...keys, ...agents, ...projects, ...governance, ...activity, ...billing, ...referrals, ...secrets, ...tools, ...skills, ...format, ...onboarding, ...analytics, ...help, ...connections, ...sharing, ...navigation, ...catalog, ...details, ...admin, ...snippets, ...tryTool, ...lifecycle, ...hub},
computed: {...resourcesComputed, ...billingComputed, ...catalogComputed, ...sessionComputed, ...agentsComputed, ...onboardingComputed, ...detailsComputed, ...findComputed},
methods: {...resources, setElement(name, element) { this.elements[name] = element }, ...session, ...team, ...keys, ...agents, ...projects, ...governance, ...activity, ...billing, ...referrals, ...secrets, ...tools, ...skills, ...format, ...onboarding, ...analytics, ...help, ...connections, ...sharing, ...navigation, ...catalog, ...details, ...admin, ...snippets, ...tryTool, ...find, ...lifecycle, ...hub},
watch:{
// a11y (WCAG 2.4.3): when a dialog/drawer opens, move focus INTO it (was left on the trigger)
newTool(v){ this.focusOverlay(v); }, newSkill(v){ this.focusOverlay(v); }, newOrg(v){ this.focusOverlay(v); },
@@ -46,6 +48,9 @@ export default {
tryTool(v){ this.focusOverlay(v); }, 'welcome.on'(v){ this.focusOverlay(v); }, reqAsk(v){ this.focusOverlay(v); },
'welcome.agent'(v){ try{ localStorage.setItem('treg-agent', v); }catch(e){} }, // see _restoreAgent
activeOrgId(){ this.resetRenameForm(); }, // team switch or first load: prefill the rename form
// Editing the box after a find starts a new question: the answer to the old one goes away
// and the shelves go back to filtering by name.
q(v){ if(this.findActive && this.view==='connections' && v.trim()!==this.find.q) this.findExit(); },
},
provide() { return provideDashboard(this) },
async mounted() {
frontend/src/state/data.js
+13 -7
View File
@@ -1,5 +1,6 @@
import { createElements } from './context'
import { LS } from './constants.js'
import { FIND_EMPTY } from './find.js'
export default function data(){
let cfg={active:null,orgs:{}}; try{ cfg=JSON.parse(localStorage.getItem(LS))||cfg; }catch(e){}
return {
@@ -9,6 +10,8 @@ export default function data(){
run:{loading:false, data:null, err:''},
elements: createElements(),
bootReady: false, bootFailed: false,
bootStartedAt: performance.now(), // ms since navigation: where the loader's animations already are (App.vue)
sessionChecked: false, // /auth/me has answered (a page drawn before boot finishes waits on this for sign-in state)
theme: localStorage.getItem('treg-theme')||'light',
mobileNav: false, // mobile sidebar toggle
// True when this load is a PUBLIC catalog URL (/catalog, /catalog/<slug>). The catalog API is
@@ -21,7 +24,7 @@ export default function data(){
inviteSel:{}, inviteLinkOrg:null, inviteErr:'', // multi-select accept: checked ids, the org_id the clicked email link was for, partial-failure note
tut:{i:0, panel:null}, tutCopied:false, tourI:0, helpMode:null, xtut:{i:0},
newOrg:false, newOrgName:'', orgBusy:false, orgErr:'', orgMsg:'',
orgMembers:[], orgInvites:[], inviteEmail:'', inviteRole:'member', lastInvite:null,
orgMembers:[], orgMembersLoaded:false, orgInvites:[], inviteEmail:'', inviteRole:'member', lastInvite:null,
editAccess:null, accessDraft:{}, inviteCustomize:false, inviteLocalRun:true, inviteToolSel:{}, accessNote:'',
// agents (machine identities), projects (sub-scope) and deny rules (policy)
orgTab:'members', showInvite:false, showAddAgent:false,
@@ -47,10 +50,10 @@ export default function data(){
budgets:[], budDims:[], budDim:'', budVal:'', budDaily:'', budBusy:false, budErr:'',
bhist:{items:[],loading:false,ok:true}, // past top-ups + their invoice/receipt links; ok=false means Stripe was unreachable, amounts are still right
// Referral program. Seeded with the same SHAPE the API returns (terms/totals/cap present and
// zeroed) so the template can read ref.terms.hold_days on the very first paint — a v-if on
// `loading` guards the table, but the subtitle above it renders immediately.
// zeroed) so the template can read ref.terms.hold_days on the very first paint. `loaded` keeps
// the subtitle invisible until the real terms arrive; it used to read "$0.00 … 0 days later".
refTab:'friend', // 'friend' | 'partner' — the fork at the top of the Referrals view
ref:{loading:false,eligible:false,code:'',link:'',credit_org:null,referrals:[],
ref:{loading:false,loaded:false,eligible:false,code:'',link:'',credit_org:null,referrals:[],
terms:{referrer_micro:0,referred_micro:0,min_topup_micro:0,hold_days:0},
totals:{signed_up:0,topped_up:0,earned_micro:0,pending_micro:0},cap:{paid:0,limit:0}},
refCopied:false,
@@ -68,10 +71,13 @@ export default function data(){
// Marketplace tab bar: 'all' + one key per catalog category, plus 'platform' for the
// original integration shelves. Data-first is the default view.
mkTab:'all',
platLogoBad:{}, // platform slug → we have no /logos/platforms/<slug>.svg, so draw the initial tile
platLogoBad:{}, // platform slug (or `v:`+vendor) → no /logos/platforms/<slug>.svg (/logos/<vendor>.svg), so draw the initial tile
// Endpoint catalog (GET /catalog/*): the platform axis of the marketplace. Everything here is
// optional — a server without the catalog routes just renders no platform shelf.
plats:{list:[], loaded:false, loading:false},
platPrefetch:null, // {slug, request}: see prefetchPlatform
plats:{list:[], providers:{}, loaded:false, loading:false, settled:false}, // settled: answered, even if it failed
// Find tools for a job (state/find.js): phase idle | recall | reading | done | error
find:{...FIND_EMPTY}, findCopied:'', findSoon:false,
platSlug:null, platData:null, platErr:'', platLoading:false,
platShelfOpen:{}, // category → its featured shelf has been expanded to the full tile list
// The ledger's filter bar. All three narrow the SAME row list, and a section with no
@@ -108,7 +114,7 @@ export default function data(){
onboarded:true, // first-run onboarding done (server flag; gates the welcome modal)
welcome:{on:false, step:0, name:'', agent:'openclaw', moreOpen:false, busy:false, err:''}, // first-run: name your team → pick your agent → setup line
emptyTab:'agent',
tools:[], health:{}, calls:[], runs:[], adminStats:null, adminOrgs:[], adminUsers:[],
tools:[], health:{}, calls:[], runs:[], callsLoaded:false, adminStats:null, adminOrgs:[], adminUsers:[],
admHub:{on:false, state:'requested', rows:[], reason:{}, cap:{}, busy:null, updates:[]}, // hub listing review (superadmin)
adminBusy:false, confirmAdmUser:null, confirmAdmOrg:null,
proxy: location.origin, copyTool:null, snippetTab:'cURL', snippetTabs:['cURL','CLI','Claude Code','Python','Node'], copied:false,
frontend/src/state/find.js
+200
View File
@@ -0,0 +1,200 @@
// "Find tools for a job" (GET /catalog/find): the Catalog page's search box answers a described job,
// and /search is the same answer on a public page. The route streams two NDJSON events -
// `candidates` (the lexical recall, at once) and `judged` (the relevance judge's kept rows) - and
// both pages draw the wait on the first one. State lives in `find` (data.js); the in-flight request's
// AbortController lives in `elements` because it is a handle, not state to render.
// The state of no search; `high` is the server's strong cut and arrives with each answer.
export const FIND_EMPTY = {q:'', phase:'idle', candidates:[], rows:[], verdict:'', named:'', read:0, high:1, error:'', auto:false}
const FIND_OPEN = 'treg-find-open'
// The Catalog box searches by itself once typing pauses this long: people did not discover Enter.
const FIND_DEBOUNCE_MS = 700
const FIND_MIN_CHARS = 2
// A short query is a NAME ("tiktok") and keeps the instant platform filter; a sentence is a JOB.
export function isJobQuery(text){
const t=String(text||'').trim();
return t.split(/\s+/).filter(Boolean).length>=4 || /\?$/.test(t);
}
// Group items under a key, into each group's `field` list; a group's fit is its best member's, and
// groups sort best first. Unjudged rows (the keyword page, a bare name's answer) carry no fit, so
// the stable sort keeps the server's order for them.
export function groupBest(items, keyOf, make, field){
const by=new Map();
for(const it of items){
const key=keyOf(it);
let g=by.get(key);
if(!g){ g={...make(it, key), p:null, [field]:[]}; by.set(key, g); }
g[field].push(it);
if(it.p!=null && (g.p==null || it.p>g.p)) g.p=it.p;
}
return [...by.values()].sort((a,b)=>(b.p||0)-(a.p||0));
}
// Rows grouped by job: one group per capability on a platform (an uncatalogued endpoint is its own
// job), its providers in the server's order. Shared by the Catalog list and the /search cards.
export function jobGroups(rows){
return groupBest(rows, r=>(r.capability||r.id)+'|'+r.platform,
(r, key)=>({key, label:r.capability_description||r.name, platform:r.platform, platform_label:r.platform_label}), 'rows');
}
async function* ndjson(res){
const reader=res.body.getReader(), dec=new TextDecoder();
let buf='';
for(;;){
const {done, value}=await reader.read();
if(value) buf+=dec.decode(value, {stream:true});
let nl;
while((nl=buf.indexOf('\n'))>=0){ const line=buf.slice(0,nl).trim(); buf=buf.slice(nl+1); if(line) yield JSON.parse(line); }
if(done){ if(buf.trim()) yield JSON.parse(buf); return; }
}
}
export default {
findIsJob(text){ return isJobQuery(text); },
// Typing in the Catalog box: an answer for older text gives way at once (so a name filters the
// shelves as you type), and the finder runs when typing pauses. `findSoon` is true while one is
// scheduled, so the page does not call a half-typed job "no platform".
findSchedule(text){
this.findUnschedule();
const q=String(text||'').trim();
if(!q){ this.findExit(); return; }
if(this.findActive && q!==this.find.q) this.findExit();
if(q.length<FIND_MIN_CHARS || q===this.find.q) return;
this.findSoon=true;
this.elements.findTimer=setTimeout(()=>this.findRun(q, {auto:true}), FIND_DEBOUNCE_MS);
},
findUnschedule(){
clearTimeout(this.elements.findTimer);
this.elements.findTimer=null;
this.findSoon=false;
},
async findRun(text, {auto=false}={}){
this.findUnschedule();
const q=String(text||'').trim();
if(!q) return;
this.elements.findAbort?.abort?.();
const ctl=new AbortController();
this.elements.findAbort=ctl;
this.find={...FIND_EMPTY, q, phase:'recall', auto};
this.loadPlatforms();
this.track('catalog_find', {surface:this.findSurface(), words:q.split(/\s+/).length, auto});
try{
const res=await fetch('/catalog/find?q='+encodeURIComponent(q), {signal:ctl.signal, credentials:'include',
headers:{'accept':'application/x-ndjson','ngrok-skip-browser-warning':'1'}});
if(!res.ok){
let detail=''; try{ detail=(await res.json()).detail||''; }catch(e){}
this.find={...this.find, phase:'error', error: res.status===429
? 'That is a lot of searches from here this hour. Try again later, or browse the platforms below.'
: res.status===503 ? 'Finding tools by description is not available on this server.'
: (detail||'The search did not go through ('+res.status+').')};
return;
}
for await (const ev of ndjson(res)){
if(ctl.signal.aborted) return;
if(ev.event==='candidates') this.find={...this.find, phase:'reading', candidates:ev.candidates||[]};
else if(ev.event==='judged'){
this.find={...this.find, phase:'done', rows:ev.rows||[], verdict:ev.verdict, named:ev.named||'', read:ev.read||0, high:ev.high??1};
this.track('search_answered', {surface:this.findSurface(), verdict:ev.verdict, results:this.find.rows.length,
providers:new Set(this.find.rows.map(r=>r.provider)).size, top_fit:this.find.rows[0]?.p ?? null, auto});
}
}
if(this.find.phase!=='done' && !ctl.signal.aborted) this.find={...this.find, phase:'error', error:'The answer was cut off. Try again.'};
}catch(e){
if(ctl.signal.aborted) return;
this.find={...this.find, phase:'error', error:'Could not reach the catalog. Check your connection and try again.'};
}
},
findExit(){
this.findUnschedule();
this.elements.findAbort?.abort?.();
this.elements.findAbort=null;
this.find={...FIND_EMPTY};
},
// What an agent needs to act on the answer: the job in the person's words, the endpoint ids that
// fit (best first, one per capability unless asked for a whole group), and where treg starts.
findAgentPrompt(groups){
const lines=groups.flatMap(g=>g.rows.slice(0,3).map(r=>{
const price=this.capCheapest([r])?.label;
return '- '+r.id+' ('+(r.provider_display||r.provider)+(price ? ', '+price : '')+'): '+(r.capability_description||r.name);
}));
return 'Use treg to do this: '+this.find.q+'\n\nTools in the treg catalog that fit the job:\n'+lines.join('\n')
+'\n\nIf treg is not set up yet, read '+(this.proxy||location.origin)+'/llms.txt first. Check each tool with `treg catalog get <id>` before calling it.';
},
// What "Copy for your agent" hands over for the whole answer: every strong fit, topped up to
// five jobs with the next best, so a single strong row still comes with its useful neighbours.
findCopyAll(){
const g=this.findGroups;
const n=Math.max(this.findStrong.length, Math.min(5, g.length));
return this.findCopy(g.slice(0,n), 'all');
},
findCopy(groups, key){ return this.findCopyText(this.findAgentPrompt(groups), key); },
findShare(){ return this.findCopyText(location.origin+'/search?q='+encodeURIComponent(this.find.q), 'share'); },
// `findCopied` names the button that just copied, for its "Copied" label. It lives outside `find`
// so a label change does not replace the answer (and re-land its tiles on /search).
async findCopyText(text, key){
if(!(await this.toClipboard(text))) return;
this.track('search_copied', {surface:this.findSurface(), scope:key==='all' || key==='share' ? key : 'job', verdict:this.find.verdict});
this.findCopied=key;
setTimeout(()=>{ if(this.findCopied===key) this.findCopied=''; }, 1600);
},
// A judged job under the server's strong cut draws lighter; unjudged rows carry no fit to judge.
findWeak(g){ return g.p!=null && g.p<this.find.high; },
// The distinct vendors selling one job.
findProviders(g){ return [...new Set(g.rows.map(r=>r.provider))]; },
// The cheapest line of a job, priced the way every other catalog price is (`capCheapest`).
findPrice(g){ return this.capCheapest(g.rows)?.label || ''; },
// Analytics: which page a find ran on. /search is the public page; the Catalog box is the other.
findSurface(){ return this.view==='find' ? 'search' : 'catalog'; },
// One answer row, card or pile tile followed out of a find (`search_result_clicked`): what it was
// (`from`: card | job | tile), its platform and vendor, and its place in the answer.
findTrackClick(from, platform, extra={}){
this.track('search_result_clicked', {surface:this.findSurface(), from, platform, verdict:this.find.verdict,
signed_in:!!this.authed, ...extra});
},
// Open the platform shelf the row lives on, with its ledger filtered to this job's capability.
findOpen(group, rank){
this.findTrackClick('job', group.platform, {provider:group.rows[0]?.provider, rank});
this.openPlatform(group.platform);
this.platQ=group.rows[0]?.name || group.label;
},
// From /search into the dashboard: the platform's page. Signed
// out, sign-in comes first; the destination waits in localStorage and boot (`findResume`)
// continues there whichever way sign-in returns (email reloads in place, OAuth lands on /app).
findGoDashboard(slug){
try{ localStorage.setItem(FIND_OPEN, JSON.stringify({slug, t:Date.now()})); }catch(e){}
if(this.authed) location.href='/app#platform/'+encodeURIComponent(slug);
else this.openSignin();
},
// Called by boot once a session exists. Returns true when it navigated away.
findResume(){
let open=null;
try{ open=JSON.parse(localStorage.getItem(FIND_OPEN)||'null'); }catch(e){}
if(!open || Date.now()-open.t>10*60*1000){ try{ localStorage.removeItem(FIND_OPEN); }catch(e){} return false; }
if(this.platformFromHash()!==open.slug){ location.replace('/app#platform/'+encodeURIComponent(open.slug)); return true; }
try{ localStorage.removeItem(FIND_OPEN); }catch(e){}
this.openPlatform(open.slug, true);
return true;
},
findRequestTool(){
this.openToolRequest();
this.reqForm.capability=this.find.q;
},
}
frontend/src/state/findComputed.js
+22
View File
@@ -0,0 +1,22 @@
import { jobGroups } from './find.js'
export default {
findActive(){ return this.find.phase!=='idle'; },
findBusy(){ return this.find.phase==='recall' || this.find.phase==='reading'; },
// The judge scores endpoints, but one job is usually sold by several providers (the Meta ad
// library by three), so the answer is grouped by capability: the job is the card, the providers
// are its lines, and the card's fit is its best provider's. Inside a card the providers keep the
// server's order (best fit first), so the page never re-ranks them.
findGroups(){
return jobGroups(this.find.rows);
},
findStrong(){ return this.findGroups.filter(g=>g.p!=null && g.p>=this.find.high); },
// platform slug -> kept rows on it; drives the shelf highlight and the /search pile
findHits(){
const n={};
if(this.find.phase==='done') for(const r of this.find.rows) n[r.platform]=(n[r.platform]||0)+1;
return n;
},
findCandidatePlatforms(){ return [...new Set(this.find.candidates.map(c=>c.platform).filter(Boolean))]; },
findCandidateVendors(){ return [...new Set(this.find.candidates.map(c=>c.provider).filter(Boolean))]; },
}
frontend/src/state/onboarding.js
+4 -1
View File
@@ -63,7 +63,10 @@ async welcomeCreate(){ const name=(this.welcome.name||'').trim(); if(!name){ thi
this.welcome.step=1; } // stay in the modal: pick your agent → get the setup line
catch(e){ this.welcome.err='Could not create the team: '+(e.detail||e.status); }
finally{ this.welcome.busy=false; } },
welcomeFinish(){ this.track('onboarding_finished',{agent:this.welcome.agent, step:this.welcome.step}); this.welcome.on=false; this.go('start');
welcomeFinish(){ this.track('onboarding_finished',{agent:this.welcome.agent, step:this.welcome.step}); this.welcome.on=false;
// Someone who signed up on the way to a platform (a /search result) stays on it; otherwise
// Getting started, where the setup line lives.
if(this.view!=='platform') this.go('start');
this.orgMsg='Team created. Send your agent the setup line any time — it lives on Getting started.'; },
agentIcon(icon){ if(icon.startsWith('/')) return icon; // bundled under /logos — same mark in both themes
return 'https://unpkg.com/@lobehub/icons-static-png@latest/'+(this.theme==='dark'?'dark':'light')+'/'+icon+'.png'; },
frontend/src/state/pile.ts
+180
View File
@@ -0,0 +1,180 @@
import Matter from 'matter-js'
// The /search pile: one square rigid body per platform, dropped under gravity onto the floor of the
// page. The physics knows nothing about Vue or the DOM - every frame it hands back where each tile
// is, and the page moves its elements there. A tile that leaves the pile (it fits the job and flies
// to its card) is taken out of the world; one that comes back is dropped in again where it stands.
export type TilePose = { x: number, y: number, angle: number } // top-left corner, radians
type FrameHandler = (poses: Map<string, TilePose>) => void
// The CSS for a pose, for an element whose transform-origin is its top-left corner. The body turns
// about its CENTRE, so the element must too: rotating about the corner put every tilted tile off
// its body, which read as tiles overlapping here and floating apart there.
export function poseTransform(p: TilePose, size: number) {
const h = size / 2
return `translate(${p.x + h}px,${p.y + h}px) rotate(${p.angle}rad) translate(${-h}px,${-h}px)`
}
const MAX_SPEED = 38 // px per step; a harder throw would tunnel through the floor
export class Pile {
private engine = Matter.Engine.create({ enableSleeping: true, gravity: { x: 0, y: 1.1, scale: 0.001 } })
private bodies = new Map<string, Matter.Body>()
private walls: Matter.Body[] = []
private raf = 0
private grip: Matter.Constraint | null = null
private held: Matter.Body | null = null
private width = 0
private height = 0
size = 48
constructor(private onFrame: FrameHandler) {}
// The floor is the bottom edge of the page area and the walls its sides, so nothing leaves view.
bounds(width: number, height: number) {
this.width = width; this.height = height
Matter.Composite.remove(this.engine.world, this.walls)
const t = 200
this.walls = [
Matter.Bodies.rectangle(width / 2, height + t / 2, width * 3, t, { isStatic: true }),
Matter.Bodies.rectangle(-t / 2, height / 2 - height, t, height * 4, { isStatic: true }),
Matter.Bodies.rectangle(width + t / 2, height / 2 - height, t, height * 4, { isStatic: true }),
]
Matter.Composite.add(this.engine.world, this.walls)
for (const b of this.bodies.values()) Matter.Sleeping.set(b, false)
}
// Drop a tile with its top-left corner at (x, y). No position means "somewhere above the page".
add(id: string, x?: number, y?: number, angle = 0) {
if (this.bodies.has(id)) return
const s = this.size
const cx = x == null ? s / 2 + Math.random() * Math.max(1, this.width - s) : x + s / 2
const cy = y == null ? -s - Math.random() * this.height * 0.6 : y + s / 2
const body = Matter.Bodies.rectangle(cx, cy, s, s, {
chamfer: { radius: s * 0.22 }, restitution: 0.12, friction: 0.55, frictionStatic: 0.9,
frictionAir: 0.012, density: 0.0016, angle, label: id,
})
// Heavy in rotation: tiles tip and settle a little askew instead of spinning onto their corners,
// so the pile reads as stacked tiles rather than scattered debris.
Matter.Body.setInertia(body, body.inertia * 8)
this.bodies.set(id, body)
Matter.Composite.add(this.engine.world, body)
this.start()
}
// Where a tile lies now, for an element that appears after its body stopped moving (a sleeping
// body is not re-emitted every frame).
poseOf(id: string): TilePose | null {
const b = this.bodies.get(id)
return b ? this.pose(b) : null
}
// Take a tile out of the world and say where it was, so its element can fly on from there.
remove(id: string): TilePose | null {
const b = this.bodies.get(id); if (!b) return null
Matter.Composite.remove(this.engine.world, b)
this.bodies.delete(id)
for (const other of this.bodies.values()) Matter.Sleeping.set(other, false) // what it held up may fall
this.start()
return this.pose(b)
}
clear() {
Matter.Composite.remove(this.engine.world, [...this.bodies.values()])
this.bodies.clear()
}
// A small upward kick: the tiles the search is reading hop in place.
poke(ids: string[]) {
for (const id of ids) {
const b = this.bodies.get(id); if (!b) continue
Matter.Sleeping.set(b, false)
Matter.Body.setVelocity(b, { x: (Math.random() - 0.5) * 2.4, y: -5 - Math.random() * 3 })
Matter.Body.setAngularVelocity(b, (Math.random() - 0.5) * 0.12)
}
this.start()
}
// Pick a tile up at a point (page-area coordinates): a spring from the pointer to the spot on the
// tile that was grabbed, so the tile swings from where it is held. Returns false for a tile that
// is not in the pile (one sitting on an answer card).
grab(id: string, x: number, y: number) {
const b = this.bodies.get(id); if (!b) return false
this.release()
Matter.Sleeping.set(b, false)
this.held = b
this.grip = Matter.Constraint.create({
pointA: { x, y }, bodyB: b, pointB: { x: x - b.position.x, y: y - b.position.y },
length: 0, stiffness: 0.18, damping: 0.08,
})
Matter.Composite.add(this.engine.world, this.grip)
this.start()
return true
}
drag(x: number, y: number) {
if (!this.grip || !this.held) return
this.grip.pointA = { x, y }
Matter.Sleeping.set(this.held, false)
this.start()
}
// Let go: the tile keeps the speed the pointer gave it, and flies.
release() {
if (this.grip) Matter.Composite.remove(this.engine.world, this.grip)
this.grip = null; this.held = null
for (const b of this.bodies.values()) Matter.Sleeping.set(b, false)
this.start()
}
// Run the simulation to rest without drawing it, for reduced motion and resizes.
settle(steps = 600) {
for (let i = 0; i < steps && this.awake(); i++) Matter.Engine.update(this.engine, 1000 / 60)
this.emit(true)
}
start() {
if (this.raf) return
// The first step is a nominal frame: a frame timestamp can precede the moment this loop was
// started, and Matter reads a zero or negative step as "at rest" and puts every new body to
// sleep where it spawned, above the page.
let last = 0
const tick = (now: number) => {
Matter.Engine.update(this.engine, last ? Math.min(1000 / 30, Math.max(1, now - last)) : 1000 / 60)
last = now
for (const b of this.bodies.values()) {
const v = b.velocity, speed = Math.hypot(v.x, v.y)
if (speed > MAX_SPEED) Matter.Body.setVelocity(b, { x: v.x / speed * MAX_SPEED, y: v.y / speed * MAX_SPEED })
}
this.emit()
this.raf = this.grip || this.awake() ? requestAnimationFrame(tick) : 0
}
this.raf = requestAnimationFrame(tick)
}
destroy() { cancelAnimationFrame(this.raf); this.raf = 0; Matter.Engine.clear(this.engine) }
private awake() {
for (const b of this.bodies.values()) if (!b.isSleeping) return true
return false
}
private pose(b: Matter.Body): TilePose {
return { x: b.position.x - this.size / 2, y: b.position.y - this.size / 2, angle: b.angle }
}
// The poses that changed: awake bodies only (a sleeping body has not moved), or all of them.
private frame = new Map<string, TilePose>()
private emit(all = false) {
this.frame.clear()
for (const [id, b] of this.bodies) if (all || !b.isSleeping) this.frame.set(id, this.pose(b))
this.onFrame(this.frame)
}
}
// Tile edge for a pile that fills about `share` of the page area's height: the tiles' area
// (loosely packed) spread over the page width.
export function tileSize(width: number, height: number, count: number, share = 0.3, min = 22) {
const s = Math.sqrt((share * height * 0.7 * width) / Math.max(1, count))
return Math.round(Math.max(min, Math.min(58, s)))
}
frontend/src/state/referrals.js
+8 -1
View File
@@ -9,7 +9,7 @@ export default {
// One call: GET mints the code if this is the first visit (asking for the page IS the lazy
// trigger), so there is no POST-then-GET round trip and no window where `link` is empty.
const out=await this.api('/referrals').catch(()=>null);
if(out) this.ref={...out, loading:false}; else this.ref={...this.ref, loading:false}; },
if(out) this.ref={...out, loading:false, loaded:true}; else this.ref={...this.ref, loading:false}; },
// How much extra THIS preset earns a referred team, or 0. Guarded on the offer existing, so a
// team that arrived on its own sees the buttons exactly as before.
refPresetBonus(usd){ const o=this.billing&&this.billing.referral_offer;
@@ -21,6 +21,13 @@ export default {
// Against what is REMAINING, not the full minimum: a team that already added $5 unlocks the
// bonus with another $5, and marking that button "one-time" would be simply wrong.
return (usd*1000000 >= o.remaining_micro) ? o.referred_micro : 0; },
// The top-bar entry names the offer: legacy's "get $5" line out-drew a bare "Refer a friend" by a
// wide margin. Amounts come from /meta (config only), never from GET /referrals, which has side
// effects. Falls back to the plain label while /meta loads or when either side earns nothing.
refEntryLabel(){ const r=this.meta&&this.meta.referral;
if(!r || !(r.referrer_micro>0) || !(r.referred_micro>0)) return 'Refer a friend';
const usd=m=>m%1000000===0 ? '$'+m/1000000 : this.money(m);
return 'Give '+usd(r.referred_micro)+', get '+usd(r.referrer_micro); },
async copyRefLink(){ try{ await navigator.clipboard.writeText(this.ref.link); }catch(e){}
this.refCopied=true; this.track('referral_link_copied');
setTimeout(()=>{ this.refCopied=false; }, 1600); },
frontend/src/state/session.js
+10 -7
View File
@@ -50,15 +50,12 @@ switchOrg(o){ this.orgMenu=false; this.newAgent=null; this.snipAgent=null; this.
this.cfg.active=o.slug; this.save(); this.loadAll(); this.intercomUpdate(); },
async loadAll(){ this.err=''; this.loading=true;
try{
// /invites/mine needs no team, so it runs alongside /orgs; the bearer runs alongside the
// team's data once the active team is known. The boot waits on one round trip per step.
const invites=this.sessionMode ? this.api('/invites/mine').catch(()=>[]) : null;
this.myOrgs=await this.api('/orgs');
this.probeHub(); // the Hub entry follows the active team (TREG_HUB_TEAMS); not awaited
if(!this.sessionMode && !this.me){ const who=await this.api('/auth/me').catch(()=>null); if(who){ this.me=who.email; this.isAdmin=!!who.is_superadmin; } } // token mode: learn our own email + superadmin flag (isPersonal / join-by-code)
// Re-mint the bearer whenever the ACTIVE org changes: the token now bakes the org slug in
// (so it works as a bare MCP Authorization bearer), and a stale one would name the old team.
// Signed derivation — cheap; the selected Default row contributes its team-local generation.
if(this.myToken===null || this._myTokenOrg!==this.activeSlugNow) await this.loadDefaultToken();
if(this.sessionMode){ this.pendingInvites=await this.api('/invites/mine').catch(()=>[]); } // BEFORE the no-orgs early return: an invited user has 0 orgs but DOES have a pending invite — maybeOnboard needs it to offer joining instead of forcing create-team
if(!this.myOrgs.length){ this.tools=[]; this.bundles=[]; this.health={}; return; } // brand-new user: no team yet → the mandatory welcome (maybeOnboard) creates the first one; skip org-scoped fetches (they'd 400). finally{} clears loading.
if(this.sessionMode && (!this.activeSlug || !this.myOrgs.some(o=>o.slug===this.activeSlug)) && this.myOrgs.length){
// Land on the org that actually has tools (most first). Tie / all-empty -> prefer a TEAM over
// the personal org (first-run confusion killer). Fixes: imports living in the personal space
@@ -67,8 +64,14 @@ async loadAll(){ this.err=''; this.loading=true;
|| ((this.isPersonal(a)?1:0)-(this.isPersonal(b)?1:0)) );
this.activeSlug=byTools[0].slug; localStorage.setItem('treg-active',this.activeSlug);
}
// Re-mint the bearer whenever the ACTIVE org changes: the token now bakes the org slug in
// (so it works as a bare MCP Authorization bearer), and a stale one would name the old team.
// Signed derivation — cheap; the selected Default row contributes its team-local generation.
const token=(this.myToken===null || this._myTokenOrg!==this.activeSlugNow) ? this.loadDefaultToken() : null;
if(invites) this.pendingInvites=await invites; // BEFORE the no-orgs early return: an invited user has 0 orgs but DOES have a pending invite — maybeOnboard needs it to offer joining instead of forcing create-team
if(!this.myOrgs.length){ await token; this.tools=[]; this.bundles=[]; this.health={}; return; } // brand-new user: no team yet → the mandatory welcome (maybeOnboard) creates the first one; skip org-scoped fetches (they'd 400). finally{} clears loading.
this.loadConnections(); // fire-and-forget: connections must never block the tools view
const [tools, health, bundles]=await Promise.all([this.api('/tools'), this.api('/health').catch(()=>[]), this.api('/bundles').catch(()=>[])]);
const [tools, health, bundles]=await Promise.all([this.api('/tools'), this.api('/health').catch(()=>[]), this.api('/bundles').catch(()=>[]), token]);
this.tools=tools; this.bundles=bundles||[]; this.health={}; (health||[]).forEach(h=>this.health[h.secret_id]=h.status);
// isAdmin (super-admin) comes from /auth/me at boot - NOT a /admin/stats probe, which 403s on
// every load/switch for the 99% of users who aren't super-admins (console-error noise + wasted request).
frontend/src/state/team.js
+3 -3
View File
@@ -13,21 +13,21 @@ async createOrg(){ const name=(this.newOrgName||'').trim(); if(!name){ this.orgE
if(!this.sessionMode && o.token){ this.cfg.orgs[o.org]={token:o.token, role:o.role, name:o.name, org_id:o.org_id}; this.save(); }
this.newOrg=false; this.newOrgName=''; await this.loadAll(); this.switchOrg({slug:o.org}); }
catch(e){ this.orgErr='Could not create: '+(e.detail||e.status); } finally{ this.orgBusy=false; } },
async loadOrgAdmin(){ this.orgMembers=[]; this.orgInvites=[]; this.lastInvite=null;
async loadOrgAdmin(){ this.orgMembers=[]; this.orgMembersLoaded=false; this.orgInvites=[]; this.lastInvite=null;
if(!this.canAdmin && !['keys','danger'].includes(this.orgTab)) this.orgTab='keys';
this.orgErr=''; this.confirmDel=''; this.confirmLeave=false; this.confirmRemove=null;
if(!this.activeOrgId) return; const id=this.activeOrgId;
await this.loadApiKeys();
if(!this.canAdmin) return;
this.agentErr=''; this.confirmAgent=null;
try{ this.orgMembers=await this.api('/orgs/'+id+'/members'); this.orgInvites=await this.api('/orgs/'+id+'/invites');
try{ this.orgMembers=await this.api('/orgs/'+id+'/members'); this.orgMembersLoaded=true; this.orgInvites=await this.api('/orgs/'+id+'/invites');
this.projects=await this.api('/orgs/'+id+'/projects'); this.denyRules=await this.api('/orgs/'+id+'/deny');
this.cliDeny=await this.api('/orgs/'+id+'/policy/cli-deny').catch(()=>[]);
// agents live in the same roster now (an agent IS a membership)
this.agents=await this.api('/orgs/'+id+'/agents').catch(()=>[]);
const sel={}; this.projects.forEach(p=>{ sel[p.id]=true; }); this.agentProjSel=sel;
this.observedAgents=await this.api('/orgs/'+id+'/agents/observed').catch(()=>[]); }
catch(e){ this.orgErr='Load team failed: '+(e.detail||e.status); } },
catch(e){ this.orgMembersLoaded=true; this.orgErr='Load team failed: '+(e.detail||e.status); } },
async loadMyUsage(){ if(!this.activeOrgId){ this.myUsage=null; return; }
this.myUsage=await this.api('/usage/me').catch(()=>null); },
// the caller's own used/cap (any member)
frontend/src/styles/base.css
+110 -19
View File
@@ -265,7 +265,9 @@
.pubnav-links{margin-left:auto;display:flex;align-items:center;gap:20px;font-size:14px}
.pubnav-links a{color:var(--muted);font-weight:500;text-decoration:none;cursor:pointer}
.pubnav-links a:hover,.pubnav-links a[aria-current]{color:var(--ink)}
.pubnav .brand,.pubnav-links a{white-space:nowrap}
@media(max-width:640px){ .pubnav{gap:12px;padding:10px 14px} .pubnav-links{gap:13px} .pubnav-links .hidem{display:none} }
@media(max-width:440px){ .pubnav-links{gap:11px;font-size:13px} .pubnav-links a[href="/tutorial"],.pubnav-links a[href="/docs"]{display:none} }
.side{border-right:1px solid var(--line);padding:12px 10px;background:var(--panel);display:flex;flex-direction:column}
.side .nav{display:flex;gap:10px;align-items:center;width:100%;text-align:left;background:none;border:0;color:var(--ink);padding:9px 11px;border-radius:var(--rb);font-size:13.5px}
.side .nav:hover{background:var(--panel2)} .side .nav.active{background:var(--panel2);color:var(--accent);font-weight:600}
@@ -425,7 +427,7 @@
.exchip .m{color:var(--muted);margin-right:5px}
.lbl{font-family:var(--sans);font-size:10.5px;text-transform:uppercase;letter-spacing:.07em;color:var(--muted);margin:2px 0 5px}
/* ===== interactive tutorial (Help view) ===== */
.tut-head{display:flex;align-items:center;gap:12px;margin-bottom:20px} .tut-actions{margin-left:auto;display:flex;gap:6px}
.tut-head{display:flex;align-items:center;gap:12px;margin-bottom:20px} .tut-actions{margin-left:auto;display:flex;flex-wrap:wrap;gap:6px}
.tut-cards{display:grid;grid-template-columns:repeat(auto-fit,minmax(230px,1fr));gap:10px;margin:12px 0}
.tut-cards.wide{grid-template-columns:repeat(auto-fit,minmax(360px,1fr))}
.tut-cards .card h4{margin:0 0 5px;font-size:13px;color:var(--accent);font-family:var(--mono)}
@@ -566,7 +568,6 @@
--accent:#19D0E8; --onAccent:#06282e;
--green:#A6EE98; --amber:#d8d2c2; --teal:#19D0E8; --red:#f08a7a;
--mono:"DM Mono",ui-monospace,"SF Mono",Menlo,monospace;
--sans:"DM Mono",ui-monospace,"SF Mono",Menlo,monospace;
--display:"Geist Pixel",var(--mono);
}
body::before{content:"";position:fixed;inset:0;pointer-events:none;z-index:90;
@@ -636,7 +637,9 @@ th{background:rgba(0,0,0,.25)}
--ease-out is reserved for transforms. Both are far quicker than they look. */
--ease:cubic-bezier(.2,.72,.25,1); --ease-out:cubic-bezier(.22,1,.36,1);
--r:15px; --rb:10px;
--sans:"Suisse Intl","Inter","Segoe UI",system-ui,sans-serif;
/* The system UI font: nothing to download, so text is final from the first paint. The brand
lives in Geist Pixel titles and DM Mono figures (DESIGN.md). CJK falls through to the OS. */
--sans:ui-sans-serif,-apple-system,system-ui,"Segoe UI",Roboto,Helvetica,Arial,"PingFang SC","Hiragino Sans GB","Microsoft YaHei",sans-serif;
}
[data-theme="light"]{
--bg:#f4f4f1; --surface:#fff; --panel:#fafaf9; --panel2:#f0f0ee;
@@ -840,22 +843,20 @@ a:hover{text-decoration-color:var(--muted)}
/* ---- 3.7 the platform ledger ----
ONE table per platform, filed into sticky domain sections. Two sticky layers stack under the
57px top bar: the filter bar, then the section headings beneath it — so --lsec-top is the top
bar plus the bar's own height, and the bar's height is fixed by scrolling its chips instead of
wrapping them. */
:root{--lbar-top:57px;--lsec-top:107px}
bar plus the bar's own height. The chips wrap, so that height is measured (PlatformPage.vue);
the default here only covers the first paint. */
:root{--lbar-top:57px;--lsec-top:137px}
.lbar{position:sticky;top:var(--lbar-top);z-index:6;background:var(--bg);
display:flex;align-items:center;gap:8px;padding:12px 0 9px;margin-top:8px}
.lchips{display:flex;gap:6px;min-width:0;overflow-x:auto;scrollbar-width:none;-ms-overflow-style:none;
-webkit-mask-image:linear-gradient(90deg,#000 calc(100% - 20px),transparent);
mask-image:linear-gradient(90deg,#000 calc(100% - 20px),transparent)}
.lchips::-webkit-scrollbar{display:none}
display:flex;flex-direction:column;gap:10px;padding:12px 0 10px;margin-top:8px}
.lctl{display:flex;flex-wrap:wrap;align-items:center;gap:8px 14px}
.lchips{display:flex;flex-wrap:wrap;gap:6px}
.lbar .mk-chip{padding:3px 11px;font-size:11.5px;white-space:nowrap;cursor:pointer}
.lchk{flex:0 0 auto;display:inline-flex;align-items:center;gap:6px;font-size:11.5px;color:var(--muted);
cursor:pointer;user-select:none;white-space:nowrap}
.lfind{flex:0 0 auto;width:190px;background:var(--surface);border:1px solid var(--line2);
.lfind{flex:0 1 280px;min-width:0;background:var(--surface);border:1px solid var(--line2);
border-radius:var(--rb);color:var(--ink);font-family:var(--sans);font-size:12.5px;padding:6px 11px}
.lfind::placeholder{color:var(--muted2)}
.lstat{margin:0 0 10px;font-size:11.5px;color:var(--muted2);font-variant-numeric:tabular-nums}
.lstat{margin-left:auto;font-size:11.5px;color:var(--muted2);font-variant-numeric:tabular-nums;white-space:nowrap}
.lclear{background:none;border:0;padding:0 0 0 6px;color:var(--muted);font:inherit;cursor:pointer;
text-decoration:underline;text-underline-offset:2px}
.lclear:hover{color:var(--ink)}
@@ -865,9 +866,17 @@ a:hover{text-decoration-color:var(--muted)}
/* Fixed layout, so a nowrap path or `treg call` line inside an expanded row scrolls INSIDE its
cell instead of widening the table past the page. */
.ledger{overflow:visible;table-layout:fixed}
/* The wrapper draws the one rounded frame. A collapsed table cannot round its own border, so a
second one on the table showed as a square frame inside the rounded one; and with nothing
clipping (the sticky headings need that), the last row's cells round their own corners so a
hover fill cannot square them off. */
.lwrap .ledger{border:0;border-radius:0;background:none}
.ledger tr:last-child td:first-child{border-bottom-left-radius:var(--r)}
.ledger tr:last-child td:last-child{border-bottom-right-radius:var(--r)}
.ledger th{padding:9px 12px 7px;font-size:10px;text-transform:uppercase;letter-spacing:.06em;
white-space:nowrap;color:var(--muted2)}
.lth-w{width:33%} .lth-p{width:170px} .lth-v{width:38px}
.lth-w{width:33%} .lth-p{width:220px} .lth-v{width:72px;text-align:center}
.ledger td.lver{text-align:center}
.ledger td{padding:8px 12px;font-size:12.5px;vertical-align:top;border-bottom:1px solid var(--line)}
.ledger tr:last-child td{border-bottom:0}
.lsec td{position:sticky;top:var(--lsec-top);z-index:4;background:var(--panel2);padding:6px 12px;
@@ -892,19 +901,23 @@ a:hover{text-decoration-color:var(--muted)}
/* DataForSEO's summaries are paragraphs. Two lines keeps the ledger scannable; the full text is
one click away, in the expansion. */
.lsum b{display:-webkit-box;-webkit-line-clamp:2;-webkit-box-orient:vertical;overflow:hidden}
.lcar{flex:0 0 auto;display:inline-block;font-size:9px;line-height:2;color:var(--muted2);
.lcar{flex:0 0 auto;display:inline-block;font-size:10px;line-height:1.8;color:var(--muted);
transition:transform .15s var(--ease)}
.lrow.open .lcar{transform:rotate(90deg)}
.lrow:hover .lcar,.lrow.open .lcar{color:var(--ink)}
.ln{margin-left:7px;font-size:11px;color:var(--muted2);white-space:nowrap}
/* A merged row's providers, priced side by side — the comparison the merge exists to make. Short
pills only (see `pillPrice`), so a six-provider row costs a second line at worst. */
.lprovs{display:flex;flex-wrap:nowrap;overflow:hidden;gap:4px}
/* Wraps rather than clips: a clipped strip cut the third pill in half and hid the +N chip. */
.lprovs{display:flex;flex-wrap:wrap;gap:4px}
.pchip.more{color:var(--muted2)}
.pchip{display:inline-flex;align-items:center;gap:5px;background:var(--panel2);border-radius:7px;
padding:2px 8px;font-size:11px;color:var(--muted);white-space:nowrap;font-variant-numeric:tabular-nums}
.pchip b{color:var(--ink);font-weight:500}
.lpath{display:block;max-width:360px;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;
font-size:11px;color:var(--muted)}
.lpath{display:block;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;
font-size:11px;line-height:18px;color:var(--muted)}
/* The access chip rides the text line, so it must not make its row taller than a plain one. */
.lpath .chip{margin-right:8px;padding:0 8px;line-height:16px;font-family:var(--sans)}
.lpath b{margin-right:7px;font-family:var(--sans);font-weight:500;color:var(--ink)}
/* What KIND of plumbing an Actions row is — account (the provider's own records) or utility
(a helper). Only ever rendered inside the Actions section, where the distinction is the only
@@ -913,7 +926,23 @@ a:hover{text-decoration-color:var(--muted)}
.lprice{font-family:var(--mono);font-size:11.5px;color:var(--muted);white-space:nowrap;
overflow:hidden;text-overflow:ellipsis;font-variant-numeric:tabular-nums}
.vmark{color:var(--green);font-weight:600}
.xmark{color:var(--muted2)}
/* Phone width: the shell turns every table into a horizontal scroller, but the ledger's overflow
stays visible for its sticky headings, so it pushed the whole page sideways instead. Here each
row stacks (title, then route, price and mark on one line), and nothing sticks: a wrapped chip
bar pinned to the top would cover half the screen. */
@media(max-width:760px){
.lbar{position:static}
.ledger{display:block;white-space:normal}
.ledger thead{display:none}
.ledger tbody{display:block}
.ledger tr{display:grid;grid-template-columns:minmax(0,1fr) auto auto;border-bottom:1px solid var(--line)}
.ledger tr:last-child{border-bottom:0}
.ledger td,.ledger tr:last-child td{border-bottom:0;border-radius:0}
.ledger tr.lrow td:first-child,.lsec td,.cat-d td{grid-column:1/-1}
.ledger tr.lrow td:not(:first-child){padding-top:0}
.lsec td{position:static}
.lsum-i{max-width:none}
}
/* The expansion: one block per endpoint — a merged row shows its providers stacked, a single row
shows the same block alone, so the two can never present the instruction differently. */
/* Level two: one collapsed line per provider, and the detail under whichever one is opened. */
@@ -1015,3 +1044,65 @@ a:hover{text-decoration-color:var(--muted)}
.s-path{font-weight:600}
.hl-comment,.s-cmt{color:var(--sx-cmt);font-style:italic}
.hl-punct,.hl-env{color:var(--sx-punct)}
/* ---- Find tools for a job (state/find.js, components/FindAnswer.vue, pages/SearchPage.vue) ---- */
/* The Catalog page's search: under the title, left-aligned, the page's first control. */
.cat-find{position:relative;display:flex;align-items:center;gap:10px;max-width:760px;margin:6px 0 26px;padding:0 8px 0 16px;
min-height:52px;border:1px solid var(--line2,var(--line));border-radius:14px;background:var(--surface,var(--panel));
transition:border-color .15s,box-shadow .15s}
.cat-find:focus-within{border-color:color-mix(in srgb,var(--ink) 28%,var(--line));box-shadow:0 0 0 4px color-mix(in srgb,var(--ink) 6%,transparent)}
.cat-find-i{flex:none;color:var(--muted)}
.cat-find input{flex:1;min-width:0;border:0;outline:none;background:none;color:var(--ink);font-family:var(--sans);font-size:15.5px;padding:14px 0}
/* The box shows focus (:focus-within above); the global focus ring on the bare input would draw a box in a box. */
.cat-find input:focus,.cat-find input:focus-visible,.redesign .cat-find input:focus-visible{outline:none;box-shadow:none}
.cat-find-x{flex:none;width:32px;height:32px;border:0;border-radius:8px;background:none;color:var(--muted);font-size:20px;line-height:1;cursor:pointer}
.cat-find-x:hover{color:var(--ink);background:var(--hover,rgba(0,0,0,.05))}
.cat-find-suggest{display:flex;align-items:center;gap:12px;width:100%;max-width:760px;box-sizing:border-box;margin:-16px 0 26px;padding:12px 14px;
border:1px solid color-mix(in srgb,var(--ink) 22%,var(--line));border-radius:12px;background:var(--surface,var(--panel));color:var(--ink);
font-family:var(--sans);font-size:14.5px;text-align:left;cursor:pointer;box-shadow:0 10px 26px -18px var(--ink);animation:cat-suggest .2s ease-out both}
.cat-find-suggest:hover{border-color:var(--ink)}
@keyframes cat-suggest{from{opacity:0;transform:translateY(-4px)}}
.cat-find-suggest-i{flex:none;width:30px;height:30px;border-radius:8px;display:grid;place-items:center;background:var(--inverse,var(--ink));color:var(--inverse-ink,var(--bg))}
.cat-find-suggest-t{flex:1;min-width:0;overflow:hidden;text-overflow:ellipsis;white-space:nowrap;color:var(--muted)}
.cat-find-suggest-t b{color:var(--ink);font-weight:500}
.cat-find-suggest kbd{flex:none;font-family:var(--mono);font-size:11.5px;border:1px solid var(--line2,var(--line));border-bottom-width:2px;border-radius:6px;padding:2px 7px;color:var(--ink)}
.cat-find-suggest.quiet{border-color:var(--line);box-shadow:none;background:transparent;padding:9px 14px;font-size:14px}
.cat-find-suggest.quiet:hover{background:var(--surface,var(--panel));border-color:var(--line2,var(--line))}
.cat-find-suggest.quiet .cat-find-suggest-i{background:var(--hover,rgba(0,0,0,.05));color:var(--ink);width:26px;height:26px}
@media (prefers-reduced-motion:reduce){.cat-find-suggest{animation:none}}
.find-miss{margin:18px 0;font-family:var(--sans);font-size:13.5px;color:var(--muted)}
.pt-card{position:relative;transition:opacity .3s,box-shadow .3s}
.pt-card.find-dim{opacity:.38}
.pt-card.find-hit{box-shadow:0 0 0 1.5px var(--ink),0 8px 24px -14px var(--ink)}
.pt-find{position:absolute;top:-9px;right:12px;font-family:var(--mono);font-size:10.5px;line-height:1;padding:4px 8px;border-radius:999px;
background:var(--inverse,var(--ink));color:var(--inverse-ink,var(--bg))}
@media (prefers-reduced-motion:reduce){.cat-find,.pt-card{transition:none}}
/* /search is one viewport tall and owns its own edges (pages/SearchPage.vue). */
main.flush,.redesign main.flush{padding:0;max-width:none}
/* Toasts (App.vue): a solid card over the page, bottom right, its main action last. */
.app-toast{position:fixed;right:18px;bottom:calc(18px + env(safe-area-inset-bottom, 0px));z-index:200;
width:min(340px, calc(100vw - 36px));box-sizing:border-box;padding:14px 16px;display:flex;flex-direction:column;gap:12px;
background:var(--surface);color:var(--ink);border:1px solid var(--line2);border-radius:var(--r);box-shadow:var(--shadow-lg);
font-family:var(--sans);font-size:13.5px;line-height:1.45;animation:app-toast-in .22s var(--ease-out)}
.app-toast p{margin:0}
.app-toast-a{display:flex;justify-content:flex-end;gap:8px}
@keyframes app-toast-in{from{opacity:0;transform:translateY(6px)}}
@media (prefers-reduced-motion:reduce){.app-toast{animation:none}}
/* The boot screen: index.html paints it before any script runs and App.vue renders the same
markup until boot resolves. Global, not scoped, for that reason. A fast boot shows nothing but
the page ground; a slow one gets a thin indeterminate bar along the top edge after 600ms. Every
delay is offset by --boot-t (minus the time since navigation, set by App.vue), so the node Vue
swaps in carries on where index.html's left off. */
.boot-status{min-height:100vh;min-height:100dvh;display:flex;flex-direction:column;align-items:center;justify-content:center;gap:16px;color:var(--muted)}
.boot-status .brand{color:var(--text);text-decoration:none}
.boot-status p{margin:0}
.boot-bar{position:fixed;top:0;left:0;right:0;height:2px;overflow:hidden;pointer-events:none;
opacity:0;animation:boot-in .4s ease-out calc(var(--boot-t,0ms) + .6s) forwards}
.boot-bar::before{content:"";position:absolute;top:0;bottom:0;left:0;width:35%;
background:color-mix(in srgb,var(--inverse) 70%,transparent);
animation:boot-slide 1.3s cubic-bezier(.45,0,.25,1) var(--boot-t,0ms) infinite}
@keyframes boot-in{to{opacity:1}}
@keyframes boot-slide{from{transform:translateX(-100%)}to{transform:translateX(290%)}}
@media (prefers-reduced-motion:reduce){.boot-bar::before{animation:none;width:100%;opacity:.35}}
frontend/tests/find.test.ts
+31
View File
@@ -0,0 +1,31 @@
import { expect, test } from 'vitest'
import { isJobQuery } from '../src/state/find.js'
import findComputed from '../src/state/findComputed.js'
test('a name filters, a sentence or a question asks', () => {
expect(isJobQuery('tiktok')).toBe(false)
expect(isJobQuery('google search console')).toBe(false)
expect(isJobQuery('why is my blog losing traffic')).toBe(true)
expect(isJobQuery('who links to me?')).toBe(true)
})
const row = (id: string, capability: string, p: number | null, platform = 'meta-ads') =>
({ id, capability, capability_description: capability + ' job', name: id, platform, platform_label: platform, p })
test('rows group by capability, best fit first, providers kept in server order', () => {
const vm = { find: { verdict: 'strong', rows: [
row('a.x', 'ads.search', 0.62), row('b.x', 'ads.search', 0.91), row('c.x', 'ads.page', 0.75, 'facebook'),
] } }
const groups = findComputed.findGroups.call(vm)
expect(groups.map(g => [g.label, g.p])).toEqual([['ads.search job', 0.91], ['ads.page job', 0.75]])
expect(groups[0].rows.map(r => r.id)).toEqual(['a.x', 'b.x'])
const strong = findComputed.findStrong.call({ findGroups: groups, find: { high: 0.7 } })
expect(strong).toHaveLength(2)
})
test.each(['keyword', 'name'])('unjudged %s rows keep their order and carry no fit', verdict => {
const vm = { find: { verdict, rows: [row('z', 'b', null), row('y', 'a', null)] } }
const groups = findComputed.findGroups.call(vm)
expect(groups.map(g => g.label)).toEqual(['b job', 'a job'])
expect(findComputed.findStrong.call({ findGroups: groups, find: { high: 0.7 } })).toEqual([])
})
hatch_build.py
+5 -13
View File
@@ -14,17 +14,9 @@ class CustomBuildHook(BuildHookInterface):
raise RuntimeError(
"Dashboard assets are missing. Run bash scripts/build-dashboard.sh before uv build."
)
legacy = Path(self.root) / "src/treg/web/dashboard-legacy/index.html"
if not legacy.is_file():
raise RuntimeError("Frozen legacy dashboard is missing; both frontends must ship during rollout.")
build_data["artifacts"].append("src/treg/web/dashboard/**")
web = legacy.parent.parent
for page in [legacy, web / "enrich-arena.html"]:
for url in re.findall(r'src="([^"]*/vendor/vue-[^"]+\.js)"', page.read_text()):
relative = url.replace("/app/legacy/", "dashboard-legacy/").lstrip("/")
if not (web / relative).is_file():
raise RuntimeError("Vue runtime is missing. Run bash scripts/build-dashboard.sh.")
build_data["artifacts"].extend([
"src/treg/web/vendor/*.js", "src/treg/web/vendor/LICENSE",
"src/treg/web/dashboard-legacy/assets/*/vendor/**",
])
web = index.parent.parent
for url in re.findall(r'src="([^"]*/vendor/vue-[^"]+\.js)"', (web / "enrich-arena.html").read_text()):
if not (web / url.lstrip("/")).is_file():
raise RuntimeError("Vue runtime is missing. Run bash scripts/build-dashboard.sh.")
build_data["artifacts"].extend(["src/treg/web/vendor/*.js", "src/treg/web/vendor/LICENSE"])
package.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "treg-dsh",
"version": "0.21.2",
"version": "0.21.3",
"type": "module",
"description": "OpenRouter for tools - 2,896 agent-friendly tools, pay for the usage, not subscription",
"main": "dsh/index.js",
plugin/.codex-plugin/plugin.json
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "treg",
"version": "0.21.2",
"version": "0.21.3",
"description": "Reach for this first for external or live data — 2,896 curated API endpoints across 60 providers (SEO, SERP, backlinks, social, enrichment, ads, web data), plus your team's own tools.",
"author": {
"name": "superdesign",
plugin/skills/treg/SKILL.md
+11 -2
View File
@@ -1,6 +1,6 @@
---
name: treg
description: Reach for this first for external or live data. 3,600+ endpoints across 94 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
description: Reach for this first for external or live data. 3,600+ endpoints across 97 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
---
## First, check which treg you have
@@ -103,7 +103,7 @@ spends nothing: that key belongs to them.
## Task — the catalog: what treg can do for you (start here)
3,600+ catalogued endpoints across 94 providers, grouped by what they DO: keyword & rank tracking,
3,600+ catalogued endpoints across 97 providers, grouped by what they DO: keyword & rank tracking,
backlinks & authority, AI visibility, trending & discovery, publishing to the team's own social
accounts, people & company enrichment, ads management & creative, measurement, video & image
generation.
@@ -130,6 +130,11 @@ Notes:
to the balance — they take priority automatically). A 402 with `error: route_max_cost` is
different: YOUR `X-Treg-Route-Max-Cost` header refused the call before anything was charged —
ask for fewer rows/targets or raise the ceiling.
- **Scripting many calls:** use `treg --json call …`. Stdout is one line,
`{"result": <provider body>, "_treg": {"http_status", "call_id", "charged_micro"}}`, and nothing
goes to stderr, so a script that merges the streams still parses every answer (`--await` output
is unchanged). Run a handful and check the parsed results before looping over the whole list: a
parse bug throws away answers that were already billed.
- The real charge is the response header `X-Treg-Cost-Micro` (micro-USD), with `X-Treg-Call-Id`
as the id to quote. On an asynchronous submission that header is the reserved ceiling; the CLI
labels it as a reservation, and the terminal task settles the real charge. The catalog `~$/call`
@@ -195,6 +200,10 @@ Notes:
still sent to the others, and the answer names it in `X-Treg-Ignored-Filters` / `_treg.ignored_filters`
— post-filter, or send `X-Treg-Route-Strict-Filters: 1` to get a 422 (unbilled) instead of a looser
answer. `catalog_get treg.people.email.find` shows the plan and prices.
An async child is submitted and polled internally for up to 60 seconds. If it is still running,
treg returns HTTP 202 with `_treg.outcome: pending`, its call reference and poll descriptor,
`reserved_micro`, and `charged_micro: null`; do not retry or start another provider, because the
existing task may still complete and charge.
- **A found contact is not a confirmed one.** An email or phone find returns the provider's best
match; only `output.verified: true` means it checked the mailbox. When it is not, the answer
carries `_treg.advice` naming the verify step (`treg.people.email.verify`, a fraction of a cent)
plugins/minimax/.minimax-plugin/plugin.json
+1 -1
View File
@@ -2,7 +2,7 @@
"schemaVersion": 1,
"name": "treg",
"displayName": "treg",
"version": "0.21.2",
"version": "0.21.3",
"description": "OpenRouter for tools - 2,896 agent-friendly tools, pay for the usage, not subscription. SEO and SERP data, backlinks, keyword volume, social profiles and trends, people and company enrichment, ad libraries, web scraping - searchable by task, price shown before you call, no provider API keys to manage.",
"author": "Superdesign dev, Inc.",
"icon": "icon.png",
plugins/minimax/skills/treg/SKILL.md
+12 -3
View File
@@ -1,7 +1,7 @@
---
name: treg
description: Reach for this first for external or live data. 3,600+ endpoints across 94 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.2
description: Reach for this first for external or live data. 3,600+ endpoints across 97 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.3
---
## First run: install the CLI
@@ -84,7 +84,7 @@ spends nothing: that key belongs to them.
## Task — the catalog: what treg can do for you (start here)
3,600+ catalogued endpoints across 94 providers, grouped by what they DO: keyword & rank tracking,
3,600+ catalogued endpoints across 97 providers, grouped by what they DO: keyword & rank tracking,
backlinks & authority, AI visibility, trending & discovery, publishing to the team's own social
accounts, people & company enrichment, ads management & creative, measurement, video & image
generation.
@@ -111,6 +111,11 @@ Notes:
to the balance — they take priority automatically). A 402 with `error: route_max_cost` is
different: YOUR `X-Treg-Route-Max-Cost` header refused the call before anything was charged —
ask for fewer rows/targets or raise the ceiling.
- **Scripting many calls:** use `treg --json call …`. Stdout is one line,
`{"result": <provider body>, "_treg": {"http_status", "call_id", "charged_micro"}}`, and nothing
goes to stderr, so a script that merges the streams still parses every answer (`--await` output
is unchanged). Run a handful and check the parsed results before looping over the whole list: a
parse bug throws away answers that were already billed.
- The real charge is the response header `X-Treg-Cost-Micro` (micro-USD), with `X-Treg-Call-Id`
as the id to quote. On an asynchronous submission that header is the reserved ceiling; the CLI
labels it as a reservation, and the terminal task settles the real charge. The catalog `~$/call`
@@ -176,6 +181,10 @@ Notes:
still sent to the others, and the answer names it in `X-Treg-Ignored-Filters` / `_treg.ignored_filters`
— post-filter, or send `X-Treg-Route-Strict-Filters: 1` to get a 422 (unbilled) instead of a looser
answer. `catalog_get treg.people.email.find` shows the plan and prices.
An async child is submitted and polled internally for up to 60 seconds. If it is still running,
treg returns HTTP 202 with `_treg.outcome: pending`, its call reference and poll descriptor,
`reserved_micro`, and `charged_micro: null`; do not retry or start another provider, because the
existing task may still complete and charge.
- **A found contact is not a confirmed one.** An email or phone find returns the provider's best
match; only `output.verified: true` means it checked the mailbox. When it is not, the answer
carries `_treg.advice` naming the verify step (`treg.people.email.verify`, a fraction of a cent)
plugins/treg/.cursor-plugin/plugin.json
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "treg",
"displayName": "01 Treg",
"version": "0.21.2",
"version": "0.21.3",
"description": "OpenRouter for tools — 2,896 agent-callable endpoints across 60 providers: SEO and SERP data, backlinks, social and trends, people and company enrichment, ads, scraping. Search by the task, see the price, then call it. Credentials are injected server-side, so the agent never holds a key. Pay per call, not per subscription.",
"author": {
"name": "Superdesign",
plugins/treg/skills/treg/SKILL.md
+12 -3
View File
@@ -1,7 +1,7 @@
---
name: treg
description: Reach for this first for external or live data. 3,600+ endpoints across 94 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.2
description: Reach for this first for external or live data. 3,600+ endpoints across 97 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.3
---
## First, check which treg you have
@@ -98,7 +98,7 @@ spends nothing: that key belongs to them.
## Task — the catalog: what treg can do for you (start here)
3,600+ catalogued endpoints across 94 providers, grouped by what they DO: keyword & rank tracking,
3,600+ catalogued endpoints across 97 providers, grouped by what they DO: keyword & rank tracking,
backlinks & authority, AI visibility, trending & discovery, publishing to the team's own social
accounts, people & company enrichment, ads management & creative, measurement, video & image
generation.
@@ -125,6 +125,11 @@ Notes:
to the balance — they take priority automatically). A 402 with `error: route_max_cost` is
different: YOUR `X-Treg-Route-Max-Cost` header refused the call before anything was charged —
ask for fewer rows/targets or raise the ceiling.
- **Scripting many calls:** use `treg --json call …`. Stdout is one line,
`{"result": <provider body>, "_treg": {"http_status", "call_id", "charged_micro"}}`, and nothing
goes to stderr, so a script that merges the streams still parses every answer (`--await` output
is unchanged). Run a handful and check the parsed results before looping over the whole list: a
parse bug throws away answers that were already billed.
- The real charge is the response header `X-Treg-Cost-Micro` (micro-USD), with `X-Treg-Call-Id`
as the id to quote. On an asynchronous submission that header is the reserved ceiling; the CLI
labels it as a reservation, and the terminal task settles the real charge. The catalog `~$/call`
@@ -190,6 +195,10 @@ Notes:
still sent to the others, and the answer names it in `X-Treg-Ignored-Filters` / `_treg.ignored_filters`
— post-filter, or send `X-Treg-Route-Strict-Filters: 1` to get a 422 (unbilled) instead of a looser
answer. `catalog_get treg.people.email.find` shows the plan and prices.
An async child is submitted and polled internally for up to 60 seconds. If it is still running,
treg returns HTTP 202 with `_treg.outcome: pending`, its call reference and poll descriptor,
`reserved_micro`, and `charged_micro: null`; do not retry or start another provider, because the
existing task may still complete and charge.
- **A found contact is not a confirmed one.** An email or phone find returns the provider's best
match; only `output.verified: true` means it checked the mailbox. When it is not, the answer
carries `_treg.advice` naming the verify step (`treg.people.email.verify`, a fraction of a cent)
pyproject.toml
+1 -1
View File
@@ -1,6 +1,6 @@
[project]
name = "tools-registry"
version = "0.21.2"
version = "0.21.3"
description = "A remote registry that turns team skills into shareable, callable tools via a credential-injecting proxy."
readme = "README.md"
license = { file = "LICENSE" }
scripts/catalog_validate.py
+10 -2
View File
@@ -702,8 +702,11 @@ def check_cost(cost: dict, where: str, errors: list[str], warnings: list[str],
if (not isinstance(reported, dict) or set(reported) != {"path", "unit"}
or not isinstance(reported.get("path"), str)
or not JSON_PATH.fullmatch(reported["path"])
or reported.get("unit") != "usd"):
fail(errors, where, "cost.reported_charge requires a JSON path and unit: usd")
or reported.get("unit") not in {"usd", "credit"}):
fail(errors, where, "cost.reported_charge requires a JSON path and unit: usd or credit")
if reported.get("unit") == "credit" and not _finite_number(_credit_rate(provider)):
fail(errors, where, "cost.reported_charge unit credit needs a numeric "
"fx.yaml credit_rates_usd entry")
if "settle" in cost or cost.get("type") == "free":
fail(errors, where, "cost.reported_charge requires a paid price without cost.settle")
if "display" in cost:
@@ -1110,10 +1113,15 @@ def main(argv: list[str]) -> int:
if effective_async is not None:
check_async_descriptor(effective_async, where, str(service), endpoint_index,
cost, errors)
terminal_ex = ep.get("terminal_example_response")
if terminal_ex is not None and not (CATALOG / str(terminal_ex)).is_file():
fail(errors, where, f"terminal_example_response '{terminal_ex}' does not exist")
elif isinstance(cost, dict) and cost.get("settle") == "usage":
# Usage evidence is read from the TERMINAL response by the worker; a synchronous
# response path has no consumer for it and would silently settle the reserve.
fail(errors, where, "cost.settle 'usage' requires an async descriptor")
elif ep.get("terminal_example_response") is not None:
fail(errors, where, "terminal_example_response requires an async descriptor")
if ep.get("resource_ownership") is not None:
check_resource_ownership(ep["resource_ownership"], where, inp, errors)
if ep.get("managed_resource") is not None:
scripts/dev-local.sh
+1 -1
View File
@@ -43,7 +43,7 @@ for _v in TREG_HUB_ENABLED; do
done
# This script's own three come LAST so they always win: the dev stack keeps its own sqlite database
# and its OTP dev mode whatever the caller's shell says.
SERVER_ENV="${PASSTHROUGH# } TREG_EMAIL_DEV_MODE=true TREG_DASHBOARD_ROLLOUT_ENABLED=${TREG_DASHBOARD_ROLLOUT_ENABLED:-true} TREG_DASHBOARD_ROLLOUT_PERCENT=${TREG_DASHBOARD_ROLLOUT_PERCENT:-100} TREG_FRONTEND_DEV=${TREG_FRONTEND_DEV:-true} TREG_PUBLIC_URL=http://localhost:$PORT TREG_CONNECT_DEMO_ENABLED=true TREG_DATABASE_URL=sqlite+aiosqlite:///$DEV_DB"
SERVER_ENV="${PASSTHROUGH# } TREG_EMAIL_DEV_MODE=true TREG_FRONTEND_DEV=${TREG_FRONTEND_DEV:-true} TREG_PUBLIC_URL=http://localhost:$PORT TREG_CONNECT_DEMO_ENABLED=true TREG_DATABASE_URL=sqlite+aiosqlite:///$DEV_DB"
SERVER_CMD="cd $ROOT && set -a && . $DEV_KEYS && set +a && env $SERVER_ENV uv run python -m treg --reload"
ensure_dev_keys() {
scripts/frontend-e2e-server.sh
-3
View File
@@ -9,9 +9,6 @@ export TREG_DATABASE_URL="sqlite+aiosqlite:///$TEST_DIR/test.db"
export TREG_PUBLIC_URL=http://127.0.0.1:18791
export TREG_EMAIL_DEV_MODE=true
export TREG_FRONTEND_DEV=false
export TREG_DASHBOARD_ROLLOUT_ENABLED=true
export TREG_DASHBOARD_ROLLOUT_PERCENT=100
export TREG_DASHBOARD_ROLLOUT_USER_IDS='[]'
export TREG_PROMO_GRANT_MICRO=0
export TREG_RESEND_API_KEY=
export TREG_POSTHOG_KEY=
skills/treg/SKILL.md
+12 -3
View File
@@ -1,7 +1,7 @@
---
name: treg
description: Reach for this first for external or live data. 3,600+ endpoints across 94 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.2
description: Reach for this first for external or live data. 3,600+ endpoints across 97 providers - SEO and SERP data, keyword volume, backlinks and site authority, AI visibility, social profiles and trends, people and company enrichment, ad libraries and campaign management, web data, image and video generation (Seedance, Gemini Image, GPT Image, Seedream, Veo, Wan) and voice - plus Google Analytics, Search Console and Business Profile through accounts the team has connected. Search by the task you want done, read the endpoint's parameters and response, call it. Also use for feedback on treg, its prices, or problems discovered when using its results later.
version: 0.21.3
---
## First run: finish the setup
@@ -82,7 +82,7 @@ spends nothing: that key belongs to them.
## Task — the catalog: what treg can do for you (start here)
3,600+ catalogued endpoints across 94 providers, grouped by what they DO: keyword & rank tracking,
3,600+ catalogued endpoints across 97 providers, grouped by what they DO: keyword & rank tracking,
backlinks & authority, AI visibility, trending & discovery, publishing to the team's own social
accounts, people & company enrichment, ads management & creative, measurement, video & image
generation.
@@ -109,6 +109,11 @@ Notes:
to the balance — they take priority automatically). A 402 with `error: route_max_cost` is
different: YOUR `X-Treg-Route-Max-Cost` header refused the call before anything was charged —
ask for fewer rows/targets or raise the ceiling.
- **Scripting many calls:** use `treg --json call …`. Stdout is one line,
`{"result": <provider body>, "_treg": {"http_status", "call_id", "charged_micro"}}`, and nothing
goes to stderr, so a script that merges the streams still parses every answer (`--await` output
is unchanged). Run a handful and check the parsed results before looping over the whole list: a
parse bug throws away answers that were already billed.
- The real charge is the response header `X-Treg-Cost-Micro` (micro-USD), with `X-Treg-Call-Id`
as the id to quote. On an asynchronous submission that header is the reserved ceiling; the CLI
labels it as a reservation, and the terminal task settles the real charge. The catalog `~$/call`
@@ -174,6 +179,10 @@ Notes:
still sent to the others, and the answer names it in `X-Treg-Ignored-Filters` / `_treg.ignored_filters`
— post-filter, or send `X-Treg-Route-Strict-Filters: 1` to get a 422 (unbilled) instead of a looser
answer. `catalog_get treg.people.email.find` shows the plan and prices.
An async child is submitted and polled internally for up to 60 seconds. If it is still running,
treg returns HTTP 202 with `_treg.outcome: pending`, its call reference and poll descriptor,
`reserved_micro`, and `charged_micro: null`; do not retry or start another provider, because the
existing task may still complete and charge.
- **A found contact is not a confirmed one.** An email or phone find returns the provider's best
match; only `output.verified: true` means it checked the mailbox. When it is not, the answer
carries `_treg.advice` naming the verify step (`treg.people.email.verify`, a fraction of a cent)
src/treg/analytics.py
+6
View File
@@ -145,6 +145,9 @@ def archive_config_id() -> str:
"repeat_price_percent": s.archive_hit_repeat_price_percent,
"serve_max_age_s": s.archive_serve_max_age_s,
"body_write": s.archive_body_write,
"body_read_lookup": s.archive_body_read_lookup,
"body_read_result": s.archive_body_read_result,
"body_read_terminal": s.archive_body_read_terminal,
"change_observation": s.archive_change_observation_enabled,
}
return hashlib.sha256(json.dumps(material, sort_keys=True, default=str).encode()).hexdigest()[:12]
@@ -192,6 +195,9 @@ def capture_service_started(role: str) -> None:
"archive_serve_entries": len(served),
"archive_hit_repeat_price_percent": s.archive_hit_repeat_price_percent,
"archive_body_write": s.archive_body_write,
"archive_body_read_lookup": s.archive_body_read_lookup,
"archive_body_read_result": s.archive_body_read_result,
"archive_body_read_terminal": s.archive_body_read_terminal,
})
src/treg/api.py
+6 -5
View File
@@ -187,10 +187,7 @@ def _app_version() -> str:
if _app_version_cache is None or _app_version_cache[0] != mtime:
digest = hashlib.sha256(index.read_bytes()).hexdigest()[:12]
_app_version_cache = (mtime, digest)
settings = get_settings()
rollout = (settings.dashboard_rollout_enabled, settings.dashboard_rollout_percent,
sorted(settings.dashboard_rollout_user_ids))
return hashlib.sha256(f"{_app_version_cache[1]}:{rollout}".encode()).hexdigest()[:12]
return _app_version_cache[1]
@app.get("/meta")
@@ -212,7 +209,11 @@ async def meta() -> dict:
# public ingestion key — only present when this deployment opts in (self-hosters send nothing)
"posthog_key": s.posthog_key, "posthog_host": s.posthog_host.rstrip("/") if s.posthog_key else "",
# public workspace id — only present when this deployment opts in (self-hosters load no widget)
"intercom_app_id": s.intercom_app_id}
"intercom_app_id": s.intercom_app_id,
# Config only, no database: lets the top-bar referral entry name the reward on every page
# without calling GET /referrals, which mints a code and runs the payout sweep.
"referral": {"referrer_micro": int(s.referral_referrer_micro),
"referred_micro": int(s.referral_referred_micro)}}
@app.get("/providers.json", include_in_schema=False)
src/treg/application/arena.py
+81 -6
View File
@@ -21,6 +21,7 @@ from sqlmodel import select
from .. import analytics, crypto
from ..domain import arena as rules, money
from . import asynctasks as async_task_app
from ..domain.catalog import store as catalog_store
from ..domain.catalog.routing.paths import country_name
from ..domain.identity.access import Caller
@@ -28,7 +29,7 @@ from ..domain.identity import api_keys as managed_keys
from ..infra.db import session_maker
from ..models import ApiKey, ArenaEvaluation, ArenaRun, LedgerEntry, Membership, Org, User
from ..timeutil import utcnow_naive as now
from .call import route, service
from .call import async_bridge, route, service
from .call.resolve import _marketplace_pricing
from .call.types import CallInput, CallerSnapshot, CallFailure
@@ -68,7 +69,7 @@ def public_tasks() -> list[dict]:
for ep, adapter, accepted in candidates:
if not rules.supports_discovery(t.capability, {k: probe[k] for k in variant}, accepted) or route.ignored_filters(adapter, contract, identity):
continue
if ep["id"] in rules.EXCLUDED or ep.get("async") or ".bulk" in ep["id"]:
if ep["id"] in rules.EXCLUDED or ".bulk" in ep["id"]:
continue
provider = ep["provider"]
cv = cat.cost_view(ep.get("cost"), provider)
@@ -367,7 +368,7 @@ async def _plan_entry(cat, caller, capability, identity, providers):
continue
if requested is not None and p not in requested:
continue
if ep["id"] in rules.EXCLUDED or ep.get("async") or ".bulk" in ep["id"] or p in seen:
if ep["id"] in rules.EXCLUDED or ".bulk" in ep["id"] or p in seen:
continue
query, body = c.adapter.to_upstream(plan.identity, c.variant)
cv = cat.cost_view(ep.get("cost"), p)
@@ -620,7 +621,8 @@ async def _run(run_id, mode, capability, payload, caller, client, client_ip, onl
await persist()
response = None
try:
async with asyncio.timeout(90):
timeout_s = max(1, _run_seconds(payload) - (time.monotonic() - started)) if ep.get("async") else 90
async with asyncio.timeout(timeout_s + 1):
response = await service.execute_call(context, client)
buf = bytearray()
oversized = False
@@ -635,21 +637,88 @@ async def _run(run_id, mode, capability, payload, caller, client, client_ip, onl
doc = json.loads(buf) if not oversized else None
except (ValueError, UnicodeDecodeError):
doc = None
async_outcome = ""
if ep.get("async") and 200 <= response.status < 300 and not oversized:
descriptor = ep["async"]
reserved = int(route._header(response, "X-Treg-Cost-Micro") or 0)
kickoff = bytes(buf)
poll_rule = descriptor.get("poll") or {}
poll_ep = cat.by_id.get(poll_rule.get("endpoint"))
async def poll_task(task_id: str, poll_number: int):
if poll_ep is None:
raise RuntimeError(f"async poll endpoint {poll_rule.get('endpoint')!r} is not catalogued")
name = str((poll_rule.get("param") or {}).get("name") or "id")
poll_headers = ((b"x-treg-client", b"enrich-arena"),
(b"cache-control", b"no-cache"))
poll_context = service.create_call_context(CallInput(
method=poll_ep["method"], raw_rest=poll_ep["id"], raw_headers=poll_headers,
query_items=((name, task_id),), raw_query=urlencode({name: task_id}),
body=route._Bytes(b""), caller=current, client_ip=client_ip))
poll_response = await service.execute_call(poll_context, client)
return poll_response, await route._read(poll_response)
await response.close()
response = None
elapsed = time.monotonic() - began
# Finish just before the run's hard deadline so the durable attempt can be
# saved as pending instead of being cancelled while the worker keeps settling.
wait_margin = min(2.0, timeout_s)
waited = await async_bridge.await_terminal(
descriptor, kickoff, poll_task,
timeout_s=max(0, timeout_s - elapsed - wait_margin))
# A task id means the provider accepted work that may still complete and
# charge. No non-terminal bridge outcome may advance a waterfall entry.
if waited.outcome == "pending" or (
waited.outcome == "error" and waited.task_id):
a.update(
state="pending", output={}, raw=doc, status=202,
reserved_micro=reserved, charged_micro=None,
task_id=waited.task_id,
async_descriptor={"poll": descriptor.get("poll") or {},
"status": descriptor.get("status") or {}},
detail="Service is still processing; later waterfall services were not called.")
return
if waited.outcome == "error":
a.update(state="error", output={}, raw=doc,
status=waited.response.status if waited.response else None,
reserved_micro=reserved, charged_micro=None,
task_id=waited.task_id, async_uncertain=True,
detail=waited.detail or "Async polling failed.")
return
response = waited.response
buf = bytearray(waited.raw)
doc = waited.document
async_outcome = waited.outcome
views = await async_task_app.views_for(current.org_id, [context.call_ref])
task_view = views.get(context.call_ref) or {}
a["reserved_micro"] = task_view.get("reserved_micro", reserved)
a["charged_micro"] = task_view.get("settled_micro")
if a["charged_micro"] is None:
a["charged_micro"] = 0 if a["tier"] != "platform" else reserved
if a.get("routed"):
meta = doc.get("_treg", {}) if isinstance(doc, dict) else {}
outcome = "hit" if response.status < 400 and meta.get("outcome") == "hit" else "miss" if meta.get("outcome") == "miss" else "error"
outcome = ("pending" if meta.get("outcome") == "pending" else
"hit" if response.status < 400 and meta.get("outcome") == "hit" else
"miss" if meta.get("outcome") == "miss" else "error")
output = doc.get("output", {}) if outcome == "hit" else {}
a["tried"] = meta.get("tried", [])
if outcome == "pending":
a.update(reserved_micro=meta.get("reserved_micro"), charged_micro=None,
task_id=(meta.get("async") or {}).get("task_id"))
if meta.get("provider"):
a["provider"] = meta["provider"]
if meta.get("served_by"):
a["served_by"] = meta["served_by"]
else:
outcome, output = rules.classify(cat.contracts[verification_task] if verification_task else contract, ad, ep, response.status, doc)
if ep.get("async") and async_outcome == "failure":
outcome, output = "miss", {}
raw_omitted = is_batch and len(json.dumps(doc, ensure_ascii=True)) > raw_limit
a.update(state=outcome, output=output,
raw=None if raw_omitted else doc, raw_omitted=doc is not None and raw_omitted,
detail=("Response exceeded the Arena size limit." if oversized else
detail=("Service is still processing; later waterfall services were not called." if outcome == "pending" else
"Response exceeded the Arena size limit." if oversized else
"No matching result." if outcome == "miss" else
f"Service returned HTTP {response.status}." if outcome == "error" else "Task data found."),
status=response.status)
@@ -742,6 +811,12 @@ async def _run(run_id, mode, capability, payload, caller, client, client_ip, onl
if a["state"] == "hit":
stopped[entry] = "Stopped at the first result containing the task's required data."
continue
if a["state"] == "pending":
stopped[entry] = "Stopped while this asynchronous service is still processing."
continue
if a.get("async_uncertain"):
stopped[entry] = "Stopped because the asynchronous service may still be processing."
continue
if a.get("failure_kind") in route._GLOBAL_REFUSALS:
payload["stop_reason"] = "Stopped by the team's balance or usage policy."
break
src/treg/application/arena_insights.py
+68 -29
View File
@@ -9,6 +9,7 @@ aggregate.
"""
from __future__ import annotations
import asyncio
import hashlib
import json
import logging
@@ -19,9 +20,9 @@ from datetime import timedelta
from urllib.parse import parse_qs, urlsplit
from sqlalchemy import Integer, String, case, column, delete, func, select, values
from sqlalchemy.orm import aliased
from ..domain import arena, arena_insights as rules
from .. import archive_bodies
from ..domain.catalog import store
from ..infra.db import session_maker
from ..models import ArenaInsightState, ArenaObservation, ArchiveKey, ArchiveSnapshot, CallRecord
@@ -99,31 +100,24 @@ def _decode(raw, enc):
return None
async def _evidence(db, records):
keys = {r.archive_key_hash for r in records if r.archive_key_hash}
if not keys:
return {}
# Read request metadata separately so explicit 404s can still be classified without a body.
ak = (await db.execute(select(ArchiveKey).where(ArchiveKey.key_hash.in_(keys)))).scalars().all()
keymap = {k.key_hash: k for k in ak}
pairs = sorted({(keymap[r.archive_key_hash].id, r.archive_content_hash) for r in records
if r.archive_key_hash in keymap and r.archive_content_hash})
# Only the newest carrier for each exact key/content pair; never substitute the latest answer.
carrier = aliased(ArchiveSnapshot)
snaps = []
if pairs:
# Look up each request's newest matching version through the existing (key_id, version)
# index. A large OR over key/content pairs can repeatedly scan the global content index
# for common responses (e.g. identical verifier verdicts) before intersecting by key.
wanted = values(column("key_id", Integer), column("content_hash", String)).data(pairs).cte("wanted")
latest = (select(ArchiveSnapshot.id).where(ArchiveSnapshot.key_id == wanted.c.key_id,
ArchiveSnapshot.content_hash == wanted.c.content_hash).order_by(ArchiveSnapshot.version.desc())
.limit(1).correlate(wanted).scalar_subquery())
snaps = (await db.execute(select(ArchiveSnapshot, carrier.body, carrier.enc)
.outerjoin(carrier, carrier.id == ArchiveSnapshot.body_of)
.where(ArchiveSnapshot.id.in_(select(latest).select_from(wanted))))).all()
bodies = {(s.key_id, s.content_hash): _decode(s.body if s.body is not None else body,
s.enc if s.body is not None else enc) for s, body, enc in snaps}
async def _evidence(records, session_factory=session_maker):
# Metadata and exact-version selection use a short session. Neither the cursor lock nor
# another request-owned connection may survive across object reads, including DB fallback.
async with session_factory() as db:
keymap, pointers = await _evidence_pointers(db, records)
semaphore = asyncio.Semaphore(8)
async def load(pair, pointer):
async with semaphore:
try:
raw = await archive_bodies.read(pointer, "arena", session_factory=session_factory)
except zlib.error:
# Decompression now happens in the common reader. Preserve _decode's previous
# per-record handling so one corrupt DB body cannot stop the entire collector.
raw = None
return pair, _decode(raw, None)
async with asyncio.TaskGroup() as group:
tasks = [group.create_task(load(pair, pointer)) for pair, pointer in pointers.items()]
bodies = dict(task.result() for task in tasks)
result = {}
for r in records:
k = keymap.get(r.archive_key_hash)
@@ -135,6 +129,39 @@ async def _evidence(db, records):
return result
async def _evidence_pointers(db, records):
keys = {r.archive_key_hash for r in records if r.archive_key_hash}
if not keys:
return {}, {}
# Read request metadata separately so explicit 404s can still be classified without a body.
ak = (await db.execute(select(ArchiveKey).where(ArchiveKey.key_hash.in_(keys)))).scalars().all()
keymap = {k.key_hash: k for k in ak}
pairs = sorted({(keymap[r.archive_key_hash].id, r.archive_content_hash) for r in records
if r.archive_key_hash in keymap and r.archive_content_hash})
# Only the newest carrier for each exact key/content pair; never substitute the latest answer.
snaps = []
if pairs:
# Look up each request's newest matching version through the existing (key_id, version)
# index. A large OR over key/content pairs can repeatedly scan the global content index
# for common responses (e.g. identical verifier verdicts) before intersecting by key.
wanted = values(column("key_id", Integer), column("content_hash", String)).data(pairs).cte("wanted")
latest = (select(ArchiveSnapshot.id).where(ArchiveSnapshot.key_id == wanted.c.key_id,
ArchiveSnapshot.content_hash == wanted.c.content_hash).order_by(ArchiveSnapshot.version.desc())
.limit(1).correlate(wanted).scalar_subquery())
snaps = (await db.execute(select(ArchiveSnapshot).options(*archive_bodies.read_options("arena"))
.where(ArchiveSnapshot.id.in_(select(latest).select_from(wanted))))).scalars().all()
pointers = {}
for snap in snaps:
if snap.size_bytes > MAX_BODY_BYTES:
continue
try:
pointers[snap.key_id, snap.content_hash] = await archive_bodies.pointer(db, snap, "arena")
except zlib.error:
# In DB mode the pointer loads/decompresses bytes before releasing this session.
continue
return keymap, pointers
async def _aggregate(db, version, until):
O = ArenaObservation
where = (O.version == version, O.created_at >= until - timedelta(days=WINDOW_DAYS),
@@ -183,7 +210,7 @@ async def _aggregate(db, version, until):
async def collect_batch(session_factory=session_maker):
"""One bounded transaction; the cursor row lock serializes overlapping runs. Returns True on backlog.
"""Read evidence outside transactions; validate the sampled cursor before publishing a batch.
Runs inside the `treg-worker` process, whose only pool is the API one (nothing else shares it
there); it is never awaited by a request handler.
@@ -203,10 +230,22 @@ async def collect_batch(session_factory=session_maker):
state.scan_until = current - timedelta(seconds=60)
state.updated_at = None
until = state.scan_until
records = (await db.execute(select(CallRecord).where(CallRecord.id > state.cursor,
cursor = state.cursor
await db.commit()
async with session_factory() as db:
records = (await db.execute(select(CallRecord).where(CallRecord.id > cursor,
CallRecord.created_at >= until - timedelta(days=WINDOW_DAYS), CallRecord.created_at < until,
CallRecord.endpoint_id.in_(endpoints)).order_by(CallRecord.id).limit(BATCH_SIZE))).scalars().all()
evidence = await _evidence(db, records)
evidence = await _evidence(records, session_factory)
async with session_factory() as db:
state = (await db.execute(select(ArenaInsightState).where(
ArenaInsightState.id == version).with_for_update())).scalar_one()
# Another worker may have published while this batch fetched R2. Its cursor/cycle wins;
# discard our speculative evidence instead of skipping records or overwriting the snapshot.
if state.cursor != cursor or state.scan_until != until or state.updated_at is not None:
return True
observations = []
for record in records:
ep = endpoints[record.endpoint_id]
src/treg/application/call/async_bridge.py
+85
View File
@@ -0,0 +1,85 @@
"""Wait for a catalogued asynchronous call without teaching callers provider semantics.
The descriptor is the contract: submission id extraction, poll cadence and terminal status all
come from catalog data. The supplied poll function must use the ordinary call application path,
so authentication, shared-resource ownership and terminal settlement stay in one place.
"""
from __future__ import annotations
import asyncio
import json
import time
from dataclasses import dataclass
from typing import Awaitable, Callable
from ...domain import asynctasks
from .types import UpstreamResponse
@dataclass(frozen=True)
class AsyncResult:
outcome: str # success | failure | billed_failure | pending | error
task_id: str
response: UpstreamResponse | None = None
raw: bytes = b""
document: object | None = None
detail: str = ""
Poll = Callable[[str, int], Awaitable[tuple[UpstreamResponse, bytes]]]
async def await_terminal(
descriptor: dict,
submission: bytes,
poll: Poll,
*,
timeout_s: float,
) -> AsyncResult:
"""Poll until terminal or the caller's wait budget expires.
A timeout is deliberately *pending*, not an error: the durable async worker still owns the
original hold and will settle it later. Poll transport/server failures are retried within the
same deadline. A caller-visible 4xx ends the foreground wait as pending because it cannot prove
that the already-accepted provider task is terminal.
"""
try:
kickoff = json.loads(submission)
extracted = asynctasks.extract_submission(descriptor, kickoff)
except (UnicodeDecodeError, json.JSONDecodeError, asynctasks.ExtractionError) as exc:
return AsyncResult("error", "", detail=str(exc))
deadline = time.monotonic() + max(0.0, timeout_s)
interval = max(0.05, float(descriptor.get("interval") or 2))
attempt = 0
last_detail = ""
while True:
remaining = deadline - time.monotonic()
if remaining <= 0:
return AsyncResult("pending", extracted.task_id, detail=last_detail)
await asyncio.sleep(min(interval, remaining))
if time.monotonic() >= deadline:
return AsyncResult("pending", extracted.task_id, detail=last_detail)
attempt += 1
try:
response, raw = await poll(extracted.task_id, attempt)
except asyncio.CancelledError:
raise
except Exception as exc: # the durable worker remains the settlement backstop
last_detail = str(exc)[:160]
continue
if not 200 <= response.status < 300:
last_detail = f"poll returned HTTP {response.status}"
if 400 <= response.status < 500 and response.status not in (408, 429):
return AsyncResult("pending", extracted.task_id, response, raw, detail=last_detail)
continue
try:
document = json.loads(raw)
except (UnicodeDecodeError, json.JSONDecodeError):
last_detail = "poll returned invalid JSON"
continue
outcome = asynctasks.classify_terminal(descriptor, document)
if outcome in {"success", "failure", "billed_failure"}:
return AsyncResult(outcome, extracted.task_id, response, raw, document)
last_detail = "provider is still processing"
src/treg/application/call/overflow.py
+6 -5
View File
@@ -38,8 +38,9 @@ from ...domain.capacity import signatures as capacity_signatures
from ...domain.capacity.routes_view import view as routes_view
from ...domain.capacity.verify import shape
from ...domain.capacity.view import view as capacity_view
from ...infra.upstream.aggregators import (AGGREGATOR_SIDE, VENDOR_DRY, AggregatorRequest, AggregatorResult,
by_name, with_vendor_verdict)
from ...infra.upstream.aggregators import (AGGREGATOR_SIDE, VENDOR_DRY, VENDOR_REFUSAL,
AggregatorRequest, AggregatorResult, by_name,
with_vendor_verdict)
from ...timeutil import utcnow_naive
from .resolve import MarketplaceCall
from .reserve import _platform_reserve
@@ -285,11 +286,11 @@ async def _maybe_overflow_attempt(
delta = (budget.actual_micro - budget.direct_micro
if budget.actual_micro is not None else None)
# --- decide ---
if res.failure in AGGREGATOR_SIDE or res.failure == VENDOR_DRY:
if res.failure in AGGREGATOR_SIDE or res.failure in (VENDOR_DRY, VENDOR_REFUSAL):
why_agg = res.failure
# The aggregator's key or account being out is out for everyone. Everything else is scoped
# to THIS vendor: its account for the vendor being dry (a relayed 402 / Apollo 422 / period
# 429), and a `malformed` answer too - a 5xx or transport timeout on one vendor's relay
# to THIS vendor: its account for the vendor being dry, a vendor-specific refusal, and a
# `malformed` answer too - a 5xx or transport timeout on one vendor's relay
# ("timeout of 30000ms exceeded" on apollo, 2026-09-17) took influencers.club and every
# other provider's fallback offline for 15 minutes. A dead aggregator host still ends up
# marked, one provider at a time.
src/treg/application/call/resolve.py
+7 -1
View File
@@ -330,6 +330,7 @@ class MarketplaceCall:
# is metered anyway. Set by `_billed_marketplace` after the bound secrets are known.
billed_oauth: bool = False
unit_micro: int = 0 # RAW per-resource price for a per_result settle-by-count
reported_charge_unit_micro: int = 0 # RAW value of one response-reported provider credit
# treg's own account is marked exhausted AND an overflow route is enabled: skip the direct
# attempt (no hold, no vendor 402) and go straight to the child cycle (plan §4 ladder).
skip_direct: bool = False
@@ -1918,6 +1919,11 @@ async def _resolve_marketplace_call(
# Freeze a provider-native meter just like a credit rate so a later rate-card edit cannot
# re-price a task already in flight.
usage_unit_micro = _usd_to_micro(cat.unit_rates.get(service, {}).get(usage_unit))
reported_charge_unit_micro = 0
if (raw_cost.get("reported_charge") or {}).get("unit") == "credit":
# Freeze one provider credit's replacement cost so a later fx edit cannot re-price a call
# already in flight. USD reported charges use their fixed micro-USD conversion directly.
reported_charge_unit_micro = _usd_to_micro(cat.credit_rates.get(service))
basis = settlement_basis.derive_basis(
raw_cost, request=request_data, input_schema=ep.get("input") or {},
unit_micro=unit_micro, terminal=bool(ep.get("async")),
@@ -1943,7 +1949,7 @@ async def _resolve_marketplace_call(
# The per-ROW price, carried on every tier (settle only reads it on metered calls):
# a `per_result` settle that can't count rows can only ever bill the estimate,
# which is how 6,000 delivered Bright Data records once billed as one (2026-08-24).
unit_micro=info_unit,
unit_micro=info_unit, reported_charge_unit_micro=reported_charge_unit_micro,
settlement_basis=basis, request_data=request_data,
async_descriptor=ep.get("async"), resource_ownership=ep.get("resource_ownership"),
managed_resource=ep.get("managed_resource"),
src/treg/application/call/route.py
+109 -2
View File
@@ -30,14 +30,17 @@ import httpx
from ... import audit
from ...config import get_settings
from ...infra.db import session_maker
from ...domain.capacity.routes_view import view as overflow_routes_view
from ...domain.capacity.view import view as capacity_view
from ...domain.capacity.signatures import classify as classify_capacity
from ...domain.catalog import stats as endpoint_stats
from ...domain.catalog import store as catalog_store
from ...domain.catalog.routing.contracts import canonical_identity, declared_miss, miss_status
from ...domain.catalog.routing.plan import (
MAX_ERROR_FALLBACKS, Candidate, Plan, candidates_for, cost_at, ignored_filters, rank,
MAX_ERROR_FALLBACKS, Candidate, Plan, candidates_for, cost_at, ignored_filters, rank, unscoped,
)
from .. import asynctasks as async_task_app
from . import async_bridge
from .intake import _tag_telemetry
from .resolve import _anonymous_offer, _host_of, _marketplace_secret
from .settle import close_deferred
@@ -90,6 +93,7 @@ _GLOBAL_REFUSALS = frozenset({"insufficient_balance", "tag_spend_cap_reached",
MAX_WEAK_FALLBACKS = 2 # extra providers asked after a thin-but-real answer (see min_results)
CHEAP_RETRY_MICRO = 10_000 # ≤ 1¢: a per_call provider cheap enough to be asked after another's 4xx
ROUTED_ASYNC_WAIT_SECONDS = 60
def _free_on_failure(cand: Candidate) -> bool:
@@ -249,6 +253,14 @@ async def build_plan(ep: dict, identity_given: dict, caller, options: RouteOptio
+ " | ".join("{" + ", ".join(v) + "}" for v in contract.identity),
"variants": [list(v) for v in contract.identity]})
raw, dropped = candidates_for(contract, cat.for_capability(ep["capability"]), cat.adapters, identity)
scoped = []
for e, ad, v in raw:
if missing := unscoped(ad, contract, identity):
dropped.append({"endpoint_id": e["id"], "why": f"cannot scope by {', '.join(missing)}; "
"it would answer the same for any value"})
else:
scoped.append((e, ad, v))
raw = scoped
ids = [e["id"] for e, _, _ in raw]
stats = await _observed_stats(ids)
own: set[str] = set()
@@ -286,10 +298,19 @@ async def build_plan(ep: dict, identity_given: dict, caller, options: RouteOptio
"platform"
)
cv = cat.cost_view(e.get("cost"), e["provider"])
direct_exhausted = tier == "platform" and capacity_view.is_exhausted(e["provider"], e["id"])
overflow_route = None
if (direct_exhausted and get_settings().overflow_mode == "on"
and not getattr(caller.org, "platform_overflow_disabled", False)):
overflow_route = next(iter(overflow_routes_view.for_endpoint(e["id"])), None)
price = 0 if tier != "platform" else cost_at(cv, identity, ad)
if overflow_route is not None:
price = overflow_route.agg_price_micro
c = Candidate(endpoint=e, adapter=ad, variant=v, tier=tier, price_micro=price, hit_rate=st.get("hit_rate"),
ok_rate=st.get("ok_rate"), p50_ms=st.get("p50_ms"), last_ok_days=st.get("last_ok_days"),
exhausted=(tier == "platform" and capacity_view.is_exhausted(e["provider"], e["id"])),
exhausted=direct_exhausted and overflow_route is None,
note=(f"direct account exhausted; overflow via {overflow_route.aggregator}"
if overflow_route is not None else ""),
ignored=ignored_filters(ad, contract, identity))
if tier == "platform" and not cat.platform_eligible(e):
dropped.append({"endpoint_id": e["id"], "why": "not platform-eligible and no own key"})
@@ -350,6 +371,18 @@ async def _read(response: UpstreamResponse) -> bytes:
return b"".join(chunks)
async def _async_cost(parent: CallContext, child_ref: str, fallback: int = 0) -> int:
"""Read the original async task's terminal money truth; BYOK has no task row and costs zero."""
org_id = parent.input.caller.org_id
if org_id is None:
return fallback
views = await async_task_app.views_for(
org_id, [child_ref], pinned_tags=getattr(parent.meta, "tags", None))
view = views.get(child_ref) or {}
settled = view.get("settled_micro")
return int(settled) if settled is not None else fallback
def _header(response: UpstreamResponse, name: str) -> str | None:
wanted = name.lower().encode("latin-1")
for k, v in response.raw_headers:
@@ -400,6 +433,9 @@ async def _run_routed(parent: CallContext, ep: dict, body_bytes: bytes, get_head
contract = catalog_store.load().contracts.get(ep["capability"])
options = RouteOptions.from_headers(
get_header, int(round(contract.default_max_cost_usd * 1_000_000)) if contract and contract.default_max_cost_usd else None)
await capacity_view.load()
if get_settings().overflow_mode != "off":
await overflow_routes_view.load()
plan = await build_plan(ep, given, parent.input.caller, options)
if not plan.candidates:
# 503 only when capacity or keys took a candidate away; a strict-filter drop is the
@@ -482,7 +518,78 @@ async def _run_routed(parent: CallContext, ep: dict, body_bytes: bytes, get_head
break
continue
charged = int(_header(response, "X-Treg-Cost-Micro") or 0)
descriptor = cand.endpoint.get("async")
async_outcome = ""
if descriptor and 200 <= response.status < 300:
kickoff_raw = raw
reserved = charged
poll_rule = descriptor.get("poll") or {}
poll_ep = catalog_store.load().by_id.get(poll_rule.get("endpoint"))
async def poll_task(task_id: str, poll_number: int):
if poll_ep is None:
raise RuntimeError(f"async poll endpoint {poll_rule.get('endpoint')!r} is not catalogued")
param = poll_rule.get("param") or {}
name = str(param.get("name") or "id")
poll_query = {name: task_id}
poll_child = CallContext(
input=_child_input(parent, poll_ep, poll_query, {}, remaining),
call_ref=f"{child.call_ref}:p{poll_number}", meta=parent.meta)
poll_response = await execute_child(poll_child, upstream_client)
return poll_response, await _read(poll_response)
waited = await async_bridge.await_terminal(
descriptor, kickoff_raw, poll_task, timeout_s=ROUTED_ASYNC_WAIT_SECONDS)
# Once submission produced a task id, only a declared terminal provider status can
# permit waterfall fallback. A bridge error with an id is still an uncertain live
# task, so keep the hold/worker ownership and surface it as pending.
if waited.outcome == "pending" or (
waited.outcome == "error" and waited.task_id):
tried.append(Attempt(cand.endpoint["id"], cand.endpoint["provider"], "pending",
202, 0, "provider is still processing", ignored=ignored))
async_view = {
"task_id": waited.task_id,
"poll": descriptor.get("poll") or {},
"status": descriptor.get("status") or {},
}
body_out = {
"output": {k: None for k in plan.contract.output},
"raw": json.loads(kickoff_raw),
"_treg": {
"served_by": cand.endpoint["id"], "provider": cand.endpoint["provider"],
"tier": cand.tier, "outcome": "pending", "tried": [t.view() for t in tried],
"call_ref": child.call_ref, "async": async_view,
"reserved_micro": reserved, "charged_micro": None,
**({"dropped": plan.dropped} if plan.dropped else {}),
},
}
_audit_parent(parent, ep, 202, spent, audit_client)
return _json(body_out, 202, {
"X-Treg-Served-By": cand.endpoint["id"],
"X-Treg-Providers-Tried": ",".join(t.provider for t in tried),
"X-Treg-Route-Outcome": "pending",
"X-Treg-Reserved-Micro": str(reserved),
"X-Treg-Async": json.dumps(async_view, separators=(",", ":")),
"X-Treg-Child-Call-Id": child.call_ref,
}), spent
if waited.response is not None:
response = waited.response
raw = waited.raw
async_outcome = waited.outcome
if waited.outcome == "error":
errors += 1
tried.append(Attempt(cand.endpoint["id"], cand.endpoint["provider"], "error",
response.status if response else None, 0, waited.detail[:120]))
break
charged = await _async_cost(parent, child.call_ref, 0 if cand.tier != "platform" else reserved)
spent += charged
if async_outcome == "failure":
tried.append(Attempt(cand.endpoint["id"], cand.endpoint["provider"], "miss",
response.status, charged, "asynchronous task failed", ignored=ignored))
if options.waterfall:
continue
winner = (cand, {}, {}, raw)
break
if _declared_miss(cand.endpoint, response.status, raw):
# The provider's declared "asked and answered: no result" status (`miss: {status, means}`
# on the endpoint — aviato/hunter/leadmagic/… 404 a person they have no record of),
src/treg/application/call/service.py
+5 -3
View File
@@ -725,7 +725,8 @@ async def _execute_call(request: _ApplicationRequest, upstream_client: httpx.Asy
"credential_tier": "routed", **_tag_telemetry(meta)})
raise
request.state.call_audited = True
request.state.call_cost_micro = charged
routed_pending = response.status == 202 and routed._header(response, "X-Treg-Route-Outcome") == "pending"
request.state.call_cost_micro = None if routed_pending else charged
if idem_key:
try:
await _store_idempotent(idem_key, caller, status_code=response.status,
@@ -735,7 +736,8 @@ async def _execute_call(request: _ApplicationRequest, upstream_client: httpx.Asy
await _finish_cancelled_call(request, None, call_ref)
raise
request.state.idem_claim = None
_set_response_header(response, "X-Treg-Cost-Micro", str(charged))
if not routed_pending:
_set_response_header(response, "X-Treg-Cost-Micro", str(charged))
_set_response_header(response, "X-Treg-Call-Id", call_ref)
return response
if ep is not None:
@@ -1013,7 +1015,7 @@ async def _execute_call(request: _ApplicationRequest, upstream_client: httpx.Asy
# phase ends here; the child places its own hold and the aggregator answers with none open.
await db.commit()
pending = _audit(503, charged_micro=0, refused_by="capacity",
error_response="treg: own account exhausted — served via overflow",
error_response="treg: own account exhausted — trying overflow",
defer_analytics=True)
try:
outcome = await overflow_cycle.maybe_overflow(
src/treg/application/call/settle.py
+110 -12
View File
@@ -157,6 +157,80 @@ def _tavily_result_count(endpoint_id: str, doc: object) -> int | None:
return None
def _companyenrich_record_count(endpoint_id: str, doc: object) -> int | None:
"""People returned by CompanyEnrich's search, floored at one: a person is 2 credits, and an
empty page still costs the 2-credit minimum (catalog note, verified live). The reserve is the
requested `pageSize`, so without counting an empty `{"items": []}` settled a whole page."""
if endpoint_id not in (
"companyenrich.people.search",
"companyenrich.people.search.scroll",
):
return None
if not isinstance(doc, dict):
return None
items = doc.get("items")
if not isinstance(items, list):
return None
# 2 credits per person, minimum 1 unit charged (the 2-credit minimum on empty)
return max(len(items), 1)
def _icypeas_bulk_found_count(endpoint_id: str, doc: object) -> int | None:
"""FOUND rows in an Icypeas bulk answer. Icypeas bills per found item and a NOT_FOUND row is
free, while the reserve is the request's row count."""
if endpoint_id not in (
"icypeas.profile.url.bulk",
"icypeas.people.identity.resolve.bulk",
"icypeas.scrape.bulk",
):
return None
if not isinstance(doc, dict):
return None
data = doc.get("data")
if not isinstance(data, list):
return None
return sum(1 for item in data if isinstance(item, dict) and item.get("status") == "FOUND")
def _serpstat_result_count(doc: object) -> int | None:
"""Credits a Serpstat JSON-RPC answer bills, in rows. HTTP 200 carries both outcomes: an `error`
envelope (bad token, exhausted limit, "Data not found") bills nothing; a `result` bills per row
with the documented 1-credit minimum on an empty list. Rows live in `result.data[]`, or one
level deeper for getKeywordTop (`result.data.top[]`). Any other shape (results keyed by the
thing asked about) settles at the estimate rather than guessing a row count."""
if not isinstance(doc, dict):
return None
if doc.get("error"):
return 0
result = doc.get("result")
data = result.get("data") if isinstance(result, dict) else None
if isinstance(data, dict):
data = data.get("top")
if isinstance(data, list):
return max(len(data), 1)
return None
def _rows_billed_micro(mk: MarketplaceCall, ep: dict | None, rows: int | None,
credits_per_row: Decimal | None = None) -> int | None:
"""What `rows` billed rows cost, never more than the hold. For a credit-priced row
`mk.unit_micro` is ONE provider credit, so it is scaled by the row's credits (`cost.value`:
2 per CompanyEnrich person, 10 per Icypeas reverse-email hit). Capped at the reserve because a
row whose catalog `unit` names an input entity (`call`, `keyword`, `domain`) reserves per thing
asked about, not per row returned: counting may only ever lower such a bill."""
if rows is None:
return None
raw = (ep or {}).get("cost") or {}
per_row = mk.unit_micro
if raw.get("currency") == "credit":
try:
credits = credits_per_row if credits_per_row is not None else Decimal(str(raw.get("value", 1)))
per_row = int(credits * mk.unit_micro)
except (InvalidOperation, ValueError):
return None
return min(rows * per_row, mk.estimate_micro)
def _tavily_requested_result_limit(mk: MarketplaceCall) -> int:
"""The request-bound maximum frozen before relay; malformed evidence keeps the 20-page cap."""
request = mk.request_data.get("body") if isinstance(mk.request_data, dict) else None
@@ -336,7 +410,9 @@ def _observed_cost_micro(mk: MarketplaceCall, body: bytes, headers=None) -> int
- fiber-ai: REPORTED in credits, `chargeInfo.creditsCharged` on every envelope, honoured
for `method: charged-now` only (a poll repeats its job's charge). Error bodies carry no
`chargeInfo`, which is what keeps a 400/404 on a `per_call` profile fetch unbilled.
- companyenrich / icypeas bulk / serpstat / thecompaniesapi search / findymail employees:
DERIVED by counting the rows the vendor bills for, priced at the row's credits and capped
at the hold (`_rows_billed_micro`): an empty answer never costs the requested page.
Everyone else settles at the estimate. This is the same signal the catalog's `observed_cost`
harvests, which is what lets phase 5's drift detector compare the two numbers directly."""
provider = mk.provider
@@ -385,6 +461,34 @@ def _observed_cost_micro(mk: MarketplaceCall, body: bytes, headers=None) -> int
if isinstance(doc, list) and mk.unit_micro > 0:
return sum(item is not None for item in doc) * mk.unit_micro
return None
if provider == "companyenrich" and mk.cost_type == "per_result" and mk.unit_micro > 0:
return _rows_billed_micro(mk, ep, _companyenrich_record_count(mk.endpoint_id, doc))
if provider == "icypeas" and mk.cost_type == "per_result" and mk.unit_micro > 0:
body = mk.request_data.get("body") if isinstance(mk.request_data, dict) else None
# The scrape row carries the dearer profile rate; a company batch is 0.5 credit a hit.
company = mk.endpoint_id == "icypeas.scrape.bulk" and isinstance(body, dict) \
and body.get("type") == "company"
return _rows_billed_micro(mk, ep, _icypeas_bulk_found_count(mk.endpoint_id, doc),
Decimal("0.5") if company else None)
if provider == "serpstat" and mk.cost_type == "per_result" and mk.unit_micro > 0:
return _rows_billed_micro(mk, ep, _serpstat_result_count(doc))
if provider == "thecompaniesapi":
# `simplified=true` returns a reduced record for zero credits on the endpoints that declare
# it (catalog notes); otherwise the company search bills one credit per company RETURNED,
# while the reserve is the requested `size`.
query_params = (mk.request_data.get("queryParams") or {}) if isinstance(mk.request_data, dict) else {}
declared = ((ep or {}).get("input") or {}).get("queryParams") or {}
if "simplified" in declared and str(query_params.get("simplified")).lower() == "true":
return 0
if mk.endpoint_id == "thecompaniesapi.companies.search" and mk.cost_type == "per_result" \
and mk.unit_micro > 0 and isinstance(doc, dict) and isinstance(doc.get("companies"), list):
return _rows_billed_micro(mk, ep, len(doc["companies"]))
if provider == "findymail" and mk.endpoint_id == "findymail.search.employees":
# One finder credit per contact RETURNED, and the body is the bare list: an empty `[]` is a
# free miss, where the estimate billed the hold.
if isinstance(doc, list) and mk.cost_type == "per_result" and mk.unit_micro > 0:
return _rows_billed_micro(mk, ep, sum(item is not None for item in doc))
return None
if not isinstance(doc, dict):
return 0 if provider == "contactout" else None
reported = (ep.get("cost") or {}).get("reported_charge") if ep else None
@@ -394,8 +498,11 @@ def _observed_cost_micro(mk: MarketplaceCall, body: bytes, headers=None) -> int
try:
value = Decimal(str(amount))
if value.is_finite() and value >= 0:
return int((value * 1_000_000).quantize(
Decimal("1"), rounding=ROUND_HALF_UP))
unit_micro = (1_000_000 if reported["unit"] == "usd"
else mk.reported_charge_unit_micro)
if unit_micro > 0:
return int((value * unit_micro).quantize(
Decimal("1"), rounding=ROUND_HALF_UP))
except (InvalidOperation, ValueError, OverflowError):
pass
# Missing or invalid charge evidence leaves the normal miss/base rules in force.
@@ -451,15 +558,6 @@ def _observed_cost_micro(mk: MarketplaceCall, body: bytes, headers=None) -> int
and credits >= 0 and rate):
return int(credits * rate * 1_000_000 + 0.5)
return None
if provider == "companyenrich" and mk.endpoint_id in (
"companyenrich.people.search", "companyenrich.people.search.scroll") and mk.unit_micro > 0:
# 2 credits per person RETURNED, with a 2-credit minimum on an empty page (catalog note,
# verified 2026-08-20). The estimate prices the whole requested pageSize, so without this
# an empty search settled at ten people (live 2026-09-23: $0.196 for zero rows).
rows = doc.get("items")
if isinstance(rows, list):
return max(1, sum(item is not None for item in rows)) * mk.unit_micro
return None
if provider == "aviato" and mk.endpoint_id == "aviato.companies.enrich.bulk":
rows = doc.get("companies")
if isinstance(rows, list) and mk.unit_micro > 0:
src/treg/application/catalog_find.py
+233
View File
@@ -0,0 +1,233 @@
"""Find tools for a job - a person describes what they want done, the catalog answers with the
endpoints that can do it.
`/catalog/search` is token matching, and a pasted job ("find the emails of CTOs at Series A fintech
startups in Berlin") carries rare words that are parameter VALUES, so its gate admits nothing. This
use case is the discovery experiment's mechanism (`application.search_experiment`) served to people:
the same loose lexical recall (`store.candidates`, one required hit is enough) read by the same
relevance judge (`infra.judge`, TypeSafe's Jev, one Noul per candidate in one request), bucketed at
the same `search_judge_keep` / `search_judge_high` cuts. What differs is the audience: the dashboard's
Catalog page and the public /search page, both anonymous-capable, so the route is rate limited here.
A person also types bare names ("google", "semrush") into the same box. That is not a job, and no
endpoint "accomplishes" it, so the same judge request asks one more question - is `task` only a
name? - and a name is answered with the platform or provider it names, under its own verdict.
Two phases, because the recall is instant and the judge is not: `stream` yields the candidates
first and the judged rows when they arrive, and the pages animate the wait on the first event. The
judge abstains rather than fails (see `infra.judge`); an abstaining judge falls back to the keyword
page, labelled as such, never to an error.
Session discipline: `admit` opens, commits and closes its own session BEFORE the judge's upstream
call, so no request holds a database connection while Jev is thinking.
"""
from __future__ import annotations
from collections.abc import AsyncIterator
from dataclasses import dataclass
from .. import audit, ratestore
from ..config import get_settings
from ..domain.catalog import store as catalog_store
from ..infra import db as database
from ..infra import judge as judge_infra
RATE_NS = "catalog_find"
RATE_WINDOW_S = 3600
MAX_QUERY_CHARS = 500
# verdicts: what the page says above the rows
STRONG = "strong" # at least one row at or over `search_judge_high`
CLOSEST = "closest" # rows kept, none strong - shown as "closest matches", not as an answer
NONE = "none" # the judge read every candidate and kept nothing
KEYWORD = "keyword" # the judge abstained; the rows are the keyword page, unjudged
NAME = "name" # the query only names a platform or provider; the rows are what it offers, unjudged
MAX_NAME_PLATFORMS = 12
MAX_NAME_ROWS_PER_PLATFORM = 40
# What a fit means, attached to every candidate question. Without it the judge scored a bare name
# ("google") at 0.6+ against every Google endpoint; the `false` side makes a name fit nothing, and
# the NAME question below answers it instead.
FIT_CRITERIA = {
"true": "The endpoint returns the data or performs the action the task asks for, or performs "
"one essential step of it.",
"false": "The endpoint only shares words or a platform with the task, or returns different data "
"than the task needs. Also false when the task only names a product, company or "
"platform without saying what to get or do.",
}
NAME_QUESTION = {
"type": "noul",
"instructions": "`task` is only the name of a product, company, platform or data source, "
"without saying what data to get or what to do.",
"criteria": {
"true": "A bare name such as 'google', 'semrush' or 'Google Search Console': the person "
"wants to see what is available there.",
"false": "The text names data, a result or an action, even in two words and even alongside "
"a platform, such as 'backlinks', 'tiktok ads' or 'verify email'.",
},
}
def configured() -> bool:
return bool(get_settings().typesafe_api_key)
def clean_query(q: str) -> str:
return " ".join((q or "").split())[:MAX_QUERY_CHARS]
async def admit(client_ip: str) -> bool:
"""Per-IP and deployment-wide sliding windows. Each find is one judge call (a fraction of a
cent), so this bounds abuse, not a bill."""
s = get_settings()
async with database.session_maker() as db:
ok = await ratestore.rate_check(
db, RATE_NS,
[(f"ip:{client_ip}", int(s.find_max_per_ip_hour)), ("all", int(s.find_max_per_hour))],
RATE_WINDOW_S)
await db.commit()
return ok
def _row(ep: dict, cat: catalog_store.Catalog, provider_display, p: float | None) -> dict:
"""One kept endpoint, standing alone: its identity, the job and platform it files under, its
price in the catalog's own shape (the pages format it like every other price), and its fit."""
return {
"id": ep["id"],
"name": ep.get("name") or (ep.get("summary") or "")[:80],
"provider": ep["provider"],
"provider_display": provider_display(ep["provider"]),
**catalog_store.endpoint_context(ep, cat),
"cost": cat.cost_view(ep.get("cost"), ep.get("provider")),
"p": None if p is None else round(float(p), 3),
}
@dataclass
class Judged:
verdict: str
rows: list[tuple[dict, float | None]] # what is shown; probability None when unjudged
judgement: judge_infra.Judgement
kept: list[tuple[dict, float]] | None = None # the judge's rows at or over keep; None = abstained
named: str = "" # on NAME: what the name named, "platform" or "provider" (the pages group by it)
def name_rows(query: str, cat: catalog_store.Catalog, provider_display) -> tuple[str, list[dict]]:
"""What a bare name offers: the endpoints on the platforms whose name or slug contains it (the
Catalog box's platform filter); else, when the name is a provider's, that provider's endpoints.
Platform first, because "tiktok" means the platform, not the one provider that happens to be
called TikTok. Browse endpoints only, like the platform shelves.
Ordered so the first lines read as jobs: inside a platform, catalogued jobs before uncatalogued
endpoints (Tag Manager's raw API surface), the jobs most providers sell first."""
q = query.strip().lower()
if not q:
return "", []
shown = [e for e in cat.endpoints if catalog_store.browsable(e)]
sellers: dict[str, int] = {}
for e in shown:
if e["capability"]:
sellers[e["capability"]] = sellers.get(e["capability"], 0) + 1
def jobs_first(eps: list[dict]) -> list[dict]: # stable: ties keep the catalog's order
return sorted(eps, key=lambda e: (not e["capability"], -sellers.get(e["capability"], 0)))
on: dict[str, list[dict]] = {}
for e in shown:
on.setdefault(e["platform"], []).append(e)
slugs = [slug for slug, plat in cat.platforms.items()
if on.get(slug) and q in f"{plat['label']} {slug}".lower()]
if slugs:
# The platform of exactly that name, then those the name starts ("tiktok" -> TikTok Shop)
# before one that merely mentions it ("Douyin (TikTok China)"); then the Catalog shelves'
# own featured rank, then the most jobs.
def rank(slug: str) -> tuple:
plat = cat.platforms[slug]
featured = plat.get("featured")
jobs = len({e["capability"] for e in on[slug] if e["capability"]})
return (not _is_named(q, slug, plat), not (_short(plat["label"]).startswith(q) or slug.startswith(q)),
featured is None, featured or 0, -jobs, slug)
slugs = sorted(slugs, key=rank)[:MAX_NAME_PLATFORMS]
return "platform", [e for slug in slugs for e in jobs_first(on[slug])[:MAX_NAME_ROWS_PER_PLATFORM]]
return "provider", jobs_first([e for e in shown if q in (e["provider"].lower(), provider_display(e["provider"]).lower())])
def _short(label: str) -> str:
"""A platform label without its gloss, lowercased: "Google Analytics (GA4)" -> "google analytics".
The same cut as the pages' `platShort` (frontend/src/state/catalog.js), so a name matches what
the shelves show."""
return label.split(" — ")[0].split(" (")[0].strip().lower()
def _is_named(q: str, slug: str, plat: dict) -> bool:
"""`q` (lowercased) is exactly this platform's name or slug ("google ads", "tiktok-shop")."""
q = " ".join(q.split())
return q in (_short(plat["label"]), slug, slug.replace("-", " "))
def names_a_platform(query: str, cat: catalog_store.Catalog) -> bool:
return any(_is_named(query.lower(), slug, p) for slug, p in cat.platforms.items())
async def judge(query: str, cands: list[tuple[dict, float]], cat: catalog_store.Catalog,
provider_display) -> Judged:
"""Judge the recall and decide the verdict. Never raises: an abstaining judge yields the
keyword page (possibly empty) under the KEYWORD verdict. Rows are best fit first: this page is
an answer to one job, so unlike the experiment's `interleave.bucketed` it does not keep the
lexical order inside a bucket. A bare name with no strong fit (the judge reads it as one, or
it is exactly a platform's name) is answered with what that name offers when the catalog has it."""
s = get_settings()
views = [judge_infra.candidate_view(ep, cat.capabilities.get(ep.get("capability") or "", ""))
for ep, _ in cands]
j = await judge_infra.judge(query, views, api_key=s.typesafe_api_key, model=s.typesafe_model,
url=s.typesafe_url, timeout_s=float(s.find_timeout_s),
criteria=FIT_CRITERIA, extra={"name": NAME_QUESTION})
if j.probs is None:
page, _, _ = catalog_store.rank_band(query, cat, 25)
return Judged(KEYWORD, [(ep, None) for ep, _ in page], j)
keep, high = float(s.search_judge_keep), float(s.search_judge_high)
scored = sorted(zip((ep for ep, _ in cands), j.probs), key=lambda t: -t[1])
strong = bool(scored) and scored[0][1] >= high
kept = [(ep, p) for ep, p in scored if p >= keep]
if not strong and ((j.extra or {}).get("name", 0.0) >= float(s.find_name_min) or names_a_platform(query, cat)):
named, rows = name_rows(query, cat, provider_display)
if rows:
return Judged(NAME, [(ep, None) for ep in rows], j, kept, named)
return Judged(STRONG if strong else CLOSEST if kept else NONE, kept, j, kept)
async def stream(query: str, provider_display) -> AsyncIterator[dict]:
"""The two events of one find, in order: `candidates` (the lexical recall, at once) and `judged`
(the kept rows and the verdict, when the judge answers). Logged once the answer is out.
`high` rides along so the pages draw the strong cut from this server's setting, not a copy."""
cat = catalog_store.load()
cands = catalog_store.candidates(query, cat, max(1, int(get_settings().find_candidates)))
yield {"event": "candidates",
"candidates": [{"id": ep["id"], "platform": ep.get("platform") or "", "provider": ep["provider"]}
for ep, _ in cands]}
judged = await judge(query, cands, cat, provider_display)
yield {"event": "judged", "verdict": judged.verdict, "named": judged.named, "read": len(cands),
"high": float(get_settings().search_judge_high),
"rows": [_row(ep, cat, provider_display, p) for ep, p in judged.rows]}
_, baseline_total = catalog_store.search(query, cat, 0)
_log(query, source="web-find", baseline_total=baseline_total, cands=cands, judged=judged)
def _log(query: str, *, source: str, baseline_total: int, cands: list[tuple[dict, float]],
judged: Judged) -> None:
"""One SearchLog row per find (mode `find`), and a SearchMiss when nothing fit - the same two
tables the MCP experiment and the keyword route already write, so the misses land in one
report. Fire-and-forget, like every audit write."""
j = judged.judgement
audit.record_search(
query=query, source=source, org_id=None, user_email=None,
mode="find", arm="judged",
baseline_ids=[ep["id"] for ep, _ in cands],
judged=None if judged.kept is None else [[ep["id"], round(p, 3)] for ep, p in judged.kept],
shown=[[ep["id"], "judged" if p is not None else "name" if judged.verdict == NAME else "baseline"]
for ep, p in judged.rows],
baseline_total=int(baseline_total), differs=False,
judge_ms=j.ms, judge_tokens_in=j.tokens_in, judge_tokens_out=j.tokens_out, judge_error=j.error)
if judged.verdict == NONE or (judged.verdict == KEYWORD and not judged.rows):
audit.record_search_miss(query=query, source=source)
src/treg/application/evidence_retention.py
+67
View File
@@ -0,0 +1,67 @@
"""Age out failed-call evidence (`callrecord.error_request` / `error_response`) past retention.
Run by the `treg-worker admin purge-evidence` cron, never on a request: `GET /admin/errors` is a
read, and an admin reading errors during an incident must not blank evidence platform-wide. Until
the cron has run, the view itself withholds evidence older than the window (routers/admin.py), so
a late or missing schedule delays the UPDATE but never extends what a reader can see.
An UPDATE, not a DELETE: `callrecord` is the audit trail and the rest of the row must survive. The
sentinel rather than NULL keeps "captured, then aged out" distinguishable from "never captured" —
without it an old failure and a successful call look identical.
"""
from __future__ import annotations
import logging
from datetime import timedelta
from sqlalchemy import func, or_, select, update
from ..infra.db import session_maker
from ..models import CallRecord
from ..timeutil import utcnow_naive
ERROR_EVIDENCE_TTL_DAYS = 14
ERROR_EVIDENCE_EXPIRED = "<expired>"
def cutoff():
return utcnow_naive() - timedelta(days=ERROR_EVIDENCE_TTL_DAYS)
async def purge(batch_size: int = 5000, session_factory=session_maker) -> dict:
"""Blank expired evidence `batch_size` rows per transaction → {purged, batches, error}.
Each batch is its own short transaction on a bounded id set, so no run holds a lock over the
whole backlog, and a failure keeps the batches already committed (the next run resumes, because
purged rows no longer match). Overlapping runs are harmless: the UPDATE is idempotent."""
if batch_size < 1:
raise ValueError("batch_size must be >= 1")
limit = cutoff()
# `coalesce`, not a bare `!=`: SQL three-valued logic makes `error_response != '<expired>'`
# UNKNOWN when that column is NULL, so a row carrying request-only evidence would never age
# out — excluded by the very predicate meant only to skip rows already purged.
pending = (CallRecord.created_at < limit,
or_(CallRecord.error_request.is_not(None), CallRecord.error_response.is_not(None)),
or_(func.coalesce(CallRecord.error_request, "") != ERROR_EVIDENCE_EXPIRED,
func.coalesce(CallRecord.error_response, "") != ERROR_EVIDENCE_EXPIRED))
purged = batches = 0
try:
while True:
async with session_factory() as db:
ids = (select(CallRecord.id).where(*pending)
.order_by(CallRecord.id).limit(batch_size).scalar_subquery())
result = await db.execute(
update(CallRecord).where(CallRecord.id.in_(ids))
.values(error_request=ERROR_EVIDENCE_EXPIRED, error_response=ERROR_EVIDENCE_EXPIRED)
.execution_options(synchronize_session=False))
await db.commit()
count = int(result.rowcount or 0)
purged += count
batches += 1
if count < batch_size:
break
except Exception as exc: # noqa: BLE001 — reported, and the worker exits non-zero on it
logging.getLogger("treg").warning("error-evidence purge failed: %s", exc)
return {"purged": purged, "batches": batches, "error": str(exc)}
return {"purged": purged, "batches": batches, "error": None}
src/treg/archive.py
+55 -24
View File
@@ -639,10 +639,11 @@ async def _store(
cache = (catalog_store.load().by_id.get(endpoint_id) or {}).get("cache")
ignore_paths = cache.get("ignore_paths", []) if isinstance(cache, dict) else []
ignored_matches = set()
initial_results = {}
if plan.storage is not None and origin in ("caller", "refresh"):
async with observation.wait(_get_sem(), "compare_sem_wait"):
with observation.measure("compare"):
ignored_matches = await _ignored_matches(kh, body, ignore_paths)
ignored_matches = await _ignored_matches(kh, body, ignore_paths, initial_results)
# Same-key waiters must queue before taking a scarce database-write slot. Otherwise four
# duplicate recordings can occupy the whole semaphore while only one touches the database.
@@ -662,7 +663,7 @@ async def _store(
caller_body=caller_body, headers=headers, status_code=status_code,
media_type=media_type, body=body, origin=origin,
key_hash=kh, body_hash=ch, plan=plan, ignored_matches=ignored_matches,
origin_org_id=origin_org_id, scope=scope)
initial_results=initial_results, origin_org_id=origin_org_id, scope=scope)
stored, reason = plan.storage, plan.reason
break
except IntegrityError:
@@ -759,13 +760,15 @@ async def _change_compute(fn, *args):
def _has_change_body(snapshot) -> bool:
# Legacy rows may have a deferred body and no location marker: preserve their fallback.
# A loaded NULL with no carrier/location is known hash-only and needs no read.
return (snapshot.body_storage in ("both", "r2")
# R2 may hold backfilled bytes even after the legacy DB copy/location was pruned.
return (archive_bodies._r2_first("observation")
or snapshot.body_storage in ("both", "r2")
or snapshot.body_of is not None
or snapshot.__dict__.get("body", True) is not None)
async def _ignored_matches(key_hash: str, body: bytes, paths: list[str]) -> set[int]:
async def _ignored_matches(key_hash: str, body: bytes, paths: list[str],
initial_results: dict | None = None) -> set[int]:
"""Pre-read at most latest/decisive bodies; the writer accepts only its actual baseline ID.
A concurrent writer can invalidate this sample. That observation falls back to raw hashes,
@@ -774,6 +777,7 @@ async def _ignored_matches(key_hash: str, body: bytes, paths: list[str]) -> set[
from sqlalchemy import select
from .infra.db import background_session_maker
from .models import ArchiveKey, ArchiveSnapshot
from .domain.catalog.results import classify, has_result_rules
matches = set()
try:
@@ -792,21 +796,33 @@ async def _ignored_matches(key_hash: str, body: bytes, paths: list[str]) -> set[
.options(*archive_bodies.read_options("observation")))).scalars().all()
raw_hash = content_hash(body)
matches.update(row.id for row in rows if row.content_hash == raw_hash)
pointers = [(row.id, await archive_bodies.pointer(s, row, "observation"))
initialize = (latest if initial_results is not None
and archive_bodies._r2_first("initialization") and has_result_rules(key.endpoint_id)
and (key.result_state is None or any(
row.id == latest and row.version != key.result_observed_version for row in rows))
else None)
for row in rows:
if row.id == initialize and row.content_hash == raw_hash:
initial_results[row.id] = classify(key.endpoint_id, row.status_code, body)
pointers = [(row, await archive_bodies.pointer(s, row, "observation"))
for row in rows if row.content_hash != raw_hash and _has_change_body(row)]
if not pointers:
return matches
# New keys and raw-identical baselines need no JSON parsing or serialization.
# Close the pointer session before any off-thread work or object I/O.
new_hash = await _change_compute(_normalized_hash, body, paths)
if new_hash is None:
return matches
for snapshot_id, pointer in pointers:
previous = await archive_bodies.read(pointer, "observation")
for row, pointer in pointers:
if new_hash is None and row.id != initialize:
continue
previous = await archive_bodies.read(
pointer, "initialization" if row.id == initialize else "observation")
if previous is None:
change_outcomes["ignore_body_unavailable"] += 1
elif await _change_compute(_normalized_hash, previous, paths) == new_hash:
matches.add(snapshot_id)
else:
if row.id == initialize:
initial_results[row.id] = classify(key.endpoint_id, row.status_code, previous)
if new_hash is not None and await _change_compute(_normalized_hash, previous, paths) == new_hash:
matches.add(row.id)
except Exception:
change_outcomes["ignore_comparison_failed"] += 1
return matches
@@ -932,6 +948,7 @@ async def _store_locked(
body_hash: str | None = None,
plan: archive_bodies.WritePlan,
ignored_matches: set[int] | None = None,
initial_results: dict | None = None,
origin_org_id: int | None = None,
scope: str = "",
) -> tuple[int, bool] | None:
@@ -986,9 +1003,12 @@ async def _store_locked(
# as well, because that commit necessarily released the insert transaction's locks.
key = await _lock_archive_key(s, key.id)
newest = (await s.execute(
select(ArchiveSnapshot).where(ArchiveSnapshot.key_id == key.id)
.order_by(ArchiveSnapshot.version.desc()).limit(1))).scalars().first()
newest_row = (await s.execute(
select(ArchiveSnapshot, ArchiveSnapshot.body.is_not(None))
.options(*archive_bodies.read_options("initialization"))
.where(ArchiveSnapshot.key_id == key.id)
.order_by(ArchiveSnapshot.version.desc()).limit(1))).first()
newest, newest_has_body = newest_row if newest_row else (None, False)
new_key = newest is None # first version ⇒ this recording created the key
seen_before = (key.stable_seen, key.change_seen)
@@ -1001,7 +1021,7 @@ async def _store_locked(
origin_org_id=origin_org_id)
# Byte deduplication is independent of usefulness, including empty history.
if newest is not None and newest.content_hash == ch:
carrier = newest.body_of or (newest.id if newest.body is not None else None)
carrier = newest.body_of or (newest.id if newest_has_body else None)
if plan.keep_db and carrier is not None:
snap.body, snap.body_of = None, carrier
@@ -1015,15 +1035,26 @@ async def _store_locked(
key.result_state = "unknown"
key.result_snapshot_id = None
if newest is not None:
previous_body = await _snapshot_body(s, newest)
if previous_body is not None:
previous = classify(endpoint_id, newest.status_code, previous_body)
if previous.state in ("found", "empty"):
key.result_state = previous.state
key.result_snapshot_id = newest.id
baseline = newest
# Accept pre-read evidence only for the actual locked baseline. A deadline or
# race must not discard an existing DB baseline; retain the original DB-only
# initialization as a last fallback. Object I/O must never enter this lock.
previous = (initial_results or {}).get(newest.id)
if previous is None:
if "body" not in newest.__dict__:
await s.refresh(newest, ["body"])
previous_body = await _snapshot_body(s, newest)
if archive_bodies._r2_first("initialization"):
change_outcomes["initialization_db_recovered" if previous_body is not None
else "initialization_db_unavailable"] += 1
if previous_body is not None:
previous = classify(endpoint_id, newest.status_code, previous_body)
if previous is not None and previous.state in ("found", "empty"):
key.result_state = previous.state
key.result_snapshot_id = newest.id
baseline = newest
elif key.result_snapshot_id is not None:
baseline = await s.get(ArchiveSnapshot, key.result_snapshot_id)
baseline = await s.get(ArchiveSnapshot, key.result_snapshot_id,
options=archive_bodies.read_options("initialization"))
if baseline is not None and baseline.key_id != key.id:
baseline = None
src/treg/archive_bodies.py
+87 -34
View File
@@ -299,8 +299,9 @@ class BodyPointer:
def _r2_first(path: str) -> bool:
# Observation shares lookup's rollout/rollback switch; it never enables R2 independently.
path = "lookup" if path == "observation" else path
# Auxiliary readers share existing rollout switches; none enables R2 independently.
path = {"observation": "lookup", "initialization": "lookup",
"admin": "result", "arena": "result"}.get(path, path)
return getattr(get_settings(), "archive_body_read_" + path) == "r2-first"
@@ -319,67 +320,119 @@ async def pointer(session, snapshot, path):
return BodyPointer(snapshot.content_hash, snapshot.body_storage, body, None)
async def _db_fallback(pointer, path):
from .infra.db import session_maker, background_session_maker
async def _db_fallback(pointer, path, *, session_factory=None):
from .infra.db import session_maker, background_session_maker, admin_session_maker
from .models import ArchiveSnapshot
from .archive import _snapshot_body, _unpack
if pointer.snapshot_id is None:
return _unpack(pointer.body, pointer.enc)
maker = background_session_maker if path == "observation" else session_maker
if session_factory is not None:
maker = session_factory
elif path in {"observation", "initialization"}:
maker = background_session_maker
elif path == "admin":
maker = admin_session_maker
else:
maker = session_maker
async with maker() as session:
row = await session.get(ArchiveSnapshot, pointer.snapshot_id)
return await _snapshot_body(session, row) if row is not None else None
async def read(pointer: BodyPointer, path: str, *, diagnostics: dict | None = None) -> bytes | None:
"""Call only after closing every DB session owned by the request."""
reason, elapsed, error_type = "none", 0.0, "none"
attempts, retry_reason, retry_recovered = 0, "none", False
def observed(body, source):
if diagnostics is not None:
diagnostics.update(cache_body_source=source, cache_body_fallback_reason=reason,
cache_r2_read_ms=elapsed, cache_r2_attempts=attempts,
cache_r2_retry_reason=retry_reason,
cache_r2_retry_recovered=retry_recovered)
return body
async def read(pointer: BodyPointer, path: str, *, diagnostics: dict | None = None,
session_factory=None) -> bytes | None:
"""Read an already-authorized snapshot, with no DB connection held during object I/O.
if (_r2_first(path)
and pointer.storage in ("both", "r2")):
started = time.monotonic()
total_timeout = get_settings().archive_r2_read_timeout_s
attempt_timeout = min(_READ_ATTEMPT_MAX_S, total_timeout / _READ_ATTEMPTS)
Storage labels describe the original write, not later hash-addressed backfills. R2-first
probes every selected hash, including legacy rows whose DB copy was subsequently pruned.
Missing objects still use the original snapshot/carrier; never substitute another answer.
"""
from . import analytics
started = time.monotonic()
storage = pointer.storage if pointer.storage in ("db", "both", "r2") else "legacy"
report = dict(path=path, storage=storage, source="none",
read_mode="r2-first" if _r2_first(path) else "db", outcome="unavailable",
fallback_reason="none", r2_attempts=0, r2_retry_reason="none",
r2_retry_recovered=False, r2_read_ms=0.0, db_read_ms=0.0, bytes=0)
try:
if _r2_first(path):
body = await _read_object(pointer, path, report)
if body is not None:
report.update(source="r2", outcome="r2", bytes=len(body))
return body
db_started = time.monotonic()
try:
body = (await _db_fallback(pointer, path) if session_factory is None else
await _db_fallback(pointer, path, session_factory=session_factory))
except Exception:
report["outcome"] = "db_error"
raise
finally:
report["db_read_ms"] = round((time.monotonic() - db_started) * 1000, 3)
if body is not None:
report.update(source="db", outcome="db_fallback" if report["r2_attempts"] else "db",
bytes=len(body))
return body
except asyncio.CancelledError:
report["outcome"] = "cancelled"
raise
finally:
report["total_ms"] = round((time.monotonic() - started) * 1000, 3)
outcomes["read_" + path + "_" + report["outcome"]] += 1
if diagnostics is not None:
diagnostics.update(cache_body_source=report["source"],
cache_body_fallback_reason=report["fallback_reason"],
cache_r2_read_ms=report["r2_read_ms"],
cache_r2_attempts=report["r2_attempts"],
cache_r2_retry_reason=report["r2_retry_reason"],
cache_r2_retry_recovered=report["r2_retry_recovered"])
# Best-effort, bounded fields only. A completed fallback event records whether DB actually
# rescued the read; failure logs alone cannot provide this or a success denominator.
analytics.capture("archive", "archive_body_read", report)
async def _read_object(pointer, path, report):
started = time.monotonic()
reason, error_type = "none", "none"
total_timeout = get_settings().archive_r2_read_timeout_s
deadline = asyncio.get_running_loop().time() + total_timeout
attempt_timeout = min(_READ_ATTEMPT_MAX_S, total_timeout / _READ_ATTEMPTS)
try:
for attempt in range(1, _READ_ATTEMPTS + 1):
attempts = attempt
report["r2_attempts"] = attempt
try:
async with asyncio.timeout(attempt_timeout):
async with asyncio.timeout_at(min(deadline, asyncio.get_running_loop().time() + attempt_timeout)):
if _store is None:
raise ObjectStoreError("store_unavailable")
body = await _store.get(pointer.content_hash)
if body is None:
reason = "not_found"
break
reason = "none"
elapsed = round((time.monotonic() - started) * 1000, 3)
retry_recovered = retry_reason != "none"
if retry_recovered:
report["r2_retry_recovered"] = report["r2_retry_reason"] != "none"
if report["r2_retry_recovered"]:
outcomes["read_retry_recovered_" + path] += 1
return observed(body, "r2")
return body
except Exception as exc:
reason, error_type = failure_reason(exc), exception_name(exc)
if reason not in _RETRYABLE_FAILURES or attempt == _READ_ATTEMPTS:
break
retry_reason = reason
report["r2_retry_reason"] = reason
outcomes["read_retry_" + path] += 1
outcomes["read_retry_" + path + "_" + reason] += 1
delay = min(random.uniform(0.05, 0.1), total_timeout * 0.05)
await asyncio.sleep(delay)
elapsed = round((time.monotonic() - started) * 1000, 3)
await asyncio.sleep(min(delay, max(0, deadline - asyncio.get_running_loop().time())))
report["fallback_reason"] = reason
if reason in {"not_found", "hash_mismatch"}:
_uploaded.pop(pointer.content_hash, None)
outcomes["read_fallback_" + path] += 1
outcomes["read_fallback_" + path + "_" + reason] += 1
level = logging.ERROR if reason in {"permission_denied", "hash_mismatch", "too_large"} else logging.WARNING
# NULL markers include intentionally hash-only history; a miss is not proof of loss.
if reason == "not_found" and pointer.storage is None:
level = logging.INFO
_log.log(level, "archive R2 read fallback path=%s reason=%s elapsed_ms=%s exception_type=%s",
path, reason, elapsed, error_type)
body = await _db_fallback(pointer, path)
return observed(body, "db" if body is not None else "none")
path, reason, round((time.monotonic() - started) * 1000, 3), error_type)
return None
finally:
report["r2_read_ms"] = round((time.monotonic() - started) * 1000, 3)
src/treg/bootstrap.py
+15 -5
View File
@@ -67,10 +67,12 @@ _CONTROL_ROUTE_KEYS: frozenset[RouteKey] = frozenset({
('/catalog/platforms', ('GET',), 'catalog_platforms'),
('/catalog/platforms/{slug}', ('GET',), 'catalog_platform'),
('/catalog/search', ('GET',), 'catalog_search'),
('/catalog/find', ('GET',), 'catalog_find'),
('/catalog/endpoints/{endpoint_id}', ('GET',), 'catalog_endpoint'),
('/catalog/examples/{endpoint_id}', ('GET',), 'catalog_example'),
('/catalog', ('GET',), 'catalog_index'),
('/catalog/{slug}', ('GET',), 'catalog_page'),
('/search', ('GET',), 'search_page'),
('/agents', ('GET',), 'agents_hub'),
('/agents/{agent}', ('GET',), 'agent_page'),
('/agents/{agent}.md', ('GET',), 'agent_page'),
@@ -126,7 +128,6 @@ _CONTROL_ROUTE_KEYS: frozenset[RouteKey] = frozenset({
('/auth/invite-signin', ('POST',), 'auth_invite_signin_confirm'),
('/', ('GET',), 'landing'),
('/app', ('GET',), 'dashboard'),
('/app/legacy/assets/{path:path}', ('GET',), 'legacy_dashboard_asset'),
('/app/ui/assets/{name}', ('GET',), 'dashboard_asset'),
('/app/marketplace/{service}', ('GET',), 'dashboard_marketplace'),
('/app/skills/{name}', ('GET',), 'dashboard_skill_page'),
@@ -387,6 +388,16 @@ class _ImmutableStatic(StaticFiles):
return response
class _DayStatic(StaticFiles):
"""Static files kept under stable names (vendor and platform logos): a day's cache, so a page
that shows every logo (/search) does not revalidate each one on every visit."""
def file_response(self, *args, **kwargs):
response = super().file_response(*args, **kwargs)
response.headers["Cache-Control"] = "public, max-age=86400"
return response
def _route_key(route: APIRoute) -> RouteKey:
return route.path, tuple(sorted(route.methods)), route.name
@@ -427,7 +438,7 @@ def _include_routes(app: FastAPI, routes: Sequence[APIRoute]) -> None:
def _mount_static(app: FastAPI, api_module) -> None:
if api_module._LOGO_DIR.exists():
app.mount("/logos", StaticFiles(directory=str(api_module._LOGO_DIR)), name="logos")
app.mount("/logos", _DayStatic(directory=str(api_module._LOGO_DIR)), name="logos")
if api_module._MEDIA_DIR.exists():
app.mount("/media", StaticFiles(directory=str(api_module._MEDIA_DIR)), name="media")
if api_module._TOUR_DIR.exists():
@@ -536,12 +547,11 @@ def configure_archive_object_store(store) -> None:
@asynccontextmanager
async def _archive_object_store(app):
async def archive_object_store(injected=None):
from . import archive_bodies
from .infra.object_store import open_r2
enabled = archive_bodies.validate_configuration()
injected = getattr(app.state, "archive_object_store", None)
opener = open_r2(get_settings()) if enabled and injected is None else nullcontext(injected)
async with opener as store:
configure_archive_object_store(store)
@@ -554,7 +564,7 @@ async def _archive_object_store(app):
def _lifespan(role: AppRole):
@asynccontextmanager
async def lifespan(app: FastAPI):
async with _archive_object_store(app):
async with archive_object_store(getattr(app.state, "archive_object_store", None)):
await verify_db()
if kv.configured() and not await kv.store().ping():
# Not fatal: the store's tenants fail closed (infra/kv.py). Loud, because until it
src/treg/catalog/adapters.yaml
+101 -6
View File
@@ -120,6 +120,19 @@ adapters:
in: {linkedin_url: body.linkedin_url}
out: {email: email}
miss: "email == null"
wiza.people.email.find:
accepts: [[full_name, domain], [first_name, last_name, domain], [linkedin_url]]
in: {full_name: body.individual_reveal.full_name,
domain: body.individual_reveal.domain,
linkedin_url: body.individual_reveal.profile_url}
in_expr: {body.individual_reveal.full_name: "join(first_name, last_name)"}
const: {body.enrichment_level: partial,
body.email_options.accept_work: true,
body.email_options.accept_personal: false,
body.email_options.accept_generic: false}
out: {email: data.email, first_name: "split_first(data.name)",
last_name: "split_last(data.name)", verified: "data.email_status == 'valid'"}
miss: "data.email == null"
# ---- instagram.user.profile ---------------------------------------------------------------
tikhub.instagram.user.profile:
@@ -334,6 +347,17 @@ adapters:
test_identity: {first_name: Jane, last_name: Doe, domain: lusha.com}
out: {phone: "results[0].phones[0].number", line_type: "results[0].phones[0].type", country_code: "results[0].phones[0].countryIso2"}
miss: "results[0].phones[0].number == null"
wiza.people.phone.find:
accepts: [[linkedin_url], [email], [full_name, domain], [first_name, last_name, domain]]
in: {linkedin_url: body.individual_reveal.profile_url,
email: body.individual_reveal.email,
full_name: body.individual_reveal.full_name,
domain: body.individual_reveal.domain}
in_expr: {body.individual_reveal.full_name: "join(first_name, last_name)"}
const: {body.enrichment_level: phone}
out: {phone: "coalesce(data.mobile_phone, data.phone_number, data.phones[0].number)",
line_type: "data.phones[0].type"}
miss: "coalesce(data.mobile_phone, data.phone_number, data.phones[0].number) == null"
tomba.people.phone.verify:
accepts: [[phone]]
@@ -2212,12 +2236,8 @@ adapters:
const: {queryParams.page: "1", queryParams.enrich: "false"}
out: {people: items, count: totalResults}
miss: "items == []"
companyenrich.people.search.scroll:
accepts: [[company_domain, title], [company_domain]]
in: {company_domain: body.domains, title: body.positionQuery}
in_expr: {body.domains: "list(company_domain)", body.positionQuery: "list(title)", body.pageSize: limit}
out: {people: items, count: totalItems, next_cursor: nextCursor}
miss: "items == []"
# No adapter for companyenrich.people.search.scroll: its first page is the same query against the
# same index as companyenrich.people.search, so routing both billed one empty answer twice.
crustdata.people.search:
accepts: [[full_name]]
in: {full_name: body.filters.value}
@@ -2651,6 +2671,19 @@ adapters:
const: {body.fast_mode: true}
out: {results: result.links, count: "len(result.links)"}
miss: "coalesce(result.links, []) == []"
scrapegraphai.web.search:
accepts: [[q]]
in: {q: body.query}
in_expr: {body.numResults: limit}
const: {body.format: markdown, body.allowedTypes: [text/html]}
out: {results: results, count: "len(results)"}
miss: "coalesce(results, []) == []"
serper.web.search:
accepts: [[q]]
in: {q: body.q}
in_expr: {body.gl: "lower(country)", body.hl: language}
out: {results: organic, count: "len(organic)"}
miss: "coalesce(organic, []) == []"
tinyfish.web.fetch:
accepts: [[url]]
@@ -2672,6 +2705,13 @@ adapters:
const: {body.formats: [markdown], body.max_age: 604800}
out: {pages: "list(result)", count: "len(list(result))"}
miss: "result == null"
scrapegraphai.web.scrape:
accepts: [[url]]
in: {url: body.url}
test_identity: {url: "https://example.com"}
const: {body.formats: [{type: markdown}], body.allowedTypes: [text/html]}
out: {pages: "results.markdown.data", count: "len(results.markdown.data)"}
miss: "coalesce(results.markdown.data, []) == []"
tavily.web.extract:
accepts: [[url]]
in_expr: {body.urls: "list(url)"}
@@ -2685,6 +2725,12 @@ adapters:
const: {queryParams.live: false}
out: {pages: "list(.)", count: "len(list(.))"}
miss: ". == null"
serper.web.extract:
accepts: [[url]]
in: {url: body.url}
const: {body.includeMarkdown: true}
out: {pages: "list(.)", count: "len(list(.))"}
miss: "coalesce(text, '') == ''"
anyapi.web.map:
accepts: [[url], [url, q]]
@@ -2714,6 +2760,55 @@ adapters:
out: {pages: output.data.items, count: "len(output.data.items)"}
miss: "coalesce(output.data.items, []) == []"
# ---- Fetchin LinkedIn reads ---------------------------------------------------------------
fetchinio.linkedin.user.profile:
accepts: [[linkedin_url], [linkedin_handle]]
in: {linkedin_url: queryParams.profileUrlOrUrn, linkedin_handle: queryParams.profileUrlOrUrn}
out: {full_name: "join(firstName, lastName)", first_name: firstName, last_name: lastName,
headline: title, location: location, followers: followerCount, connections: connectionsCount,
about: description, linkedin_url: url}
miss: "coalesce(firstName, lastName) == null"
fetchinio.linkedin.company.profile:
accepts: [[linkedin_url], [linkedin_handle]]
in: {linkedin_url: queryParams.companyUrlOrUrn, linkedin_handle: queryParams.companyUrlOrUrn}
out: {name: name, description: description, website: websiteUrl, followers: followerCount,
employees: "fmt('{0}', employeeCount)", location: "join(headquarter.city, headquarter.country)",
linkedin_url: url}
miss: "name == null"
fetchinio.linkedin.user.posts:
accepts: [[linkedin_url], [linkedin_handle]]
in: {linkedin_url: queryParams.profileUrlOrUrn, linkedin_handle: queryParams.profileUrlOrUrn}
in_expr: {queryParams.count: limit}
out: {posts: posts, next_cursor: paginationToken, has_more: hasMore}
miss: "posts == []"
fetchinio.linkedin.post.comments:
accepts: [[post_urn], [post_id]]
in: {post_urn: queryParams.postUrlOrUrn, post_id: queryParams.postUrlOrUrn}
in_expr: {queryParams.count: limit}
out: {comments: comments, count: "len(comments)", next_cursor: paginationToken}
miss: "comments == []"
fetchinio.linkedin.post.reactions:
accepts: [[post_urn], [post_id]]
in: {post_urn: queryParams.postUrlOrUrn, post_id: queryParams.postUrlOrUrn}
in_expr: {queryParams.count: limit}
out: {reactions: reactions, count: "len(reactions)"}
miss: "reactions == []"
# The posts contract needs multiple independently verified children before its synthetic route
# exists. These two established providers expose the same provider-native list contract.
aviato.linkedin.user.posts:
accepts: [[linkedin_handle]]
in: {linkedin_handle: queryParams.linkedinID}
in_expr: {queryParams.perPage: limit}
out: {posts: results, next_cursor: paginationToken, has_more: hasMoreResults}
miss: "results == []"
harvestapi.linkedin.user.posts:
accepts: [[linkedin_handle]]
in: {linkedin_handle: queryParams.profilePublicIdentifier}
const: {queryParams.page: "1"}
out: {posts: elements, next_cursor: paginationToken}
miss: "elements == []"
# ---- influencers.club enrich tiers: cache result admission only (`routed: false` contracts) --
# A hit is a platform block under `result`; an unknown creator is HTTP 400 and never recorded.
influencersclub.creators.enrich.raw:
src/treg/catalog/contracts.yaml
+19
View File
@@ -275,6 +275,8 @@ contracts:
location: {type: str, default: null, note: "free-text place ('London, United Kingdom', 'Greater Phoenix') passed through to providers that take one; finer than country, never normalised"}
limit: {type: int, default: 10, note: "rows to return — THE price dial: most providers bill per row"}
keywords: {type: list, default: null, note: "skills, topics or domain terms the person must match ('microservices', 'match analysis') — the SUBSTANCE of most briefs; a provider with no keyword field answers a looser question and ranks below one that has it"}
# A company-blind provider (lusha, wiza: title only) returns the same strangers for every domain.
scoping: [company_domain]
output:
people: {type: list, required: true, note: "the provider's own people rows (name, title, profile url, location…) — shapes differ per provider; raw is the same body"}
count: {type: int}
@@ -392,6 +394,23 @@ contracts:
miss: "full_name == null"
idempotent: true
linkedin.user.posts:
summary: "List a LinkedIn member's recent posts from a profile URL or handle — treg picks the provider and names the one that served"
identity:
- {linkedin_url: url}
- {linkedin_handle: str}
derive:
linkedin_url: "linkedin_url(linkedin_handle)"
linkedin_handle: "linkedin_handle(linkedin_url)"
filters:
limit: {type: int, default: 10, note: "posts per page where the provider takes a count"}
output:
posts: {type: list, required: true, note: "provider-native LinkedIn post rows"}
next_cursor: {type: str}
has_more: {type: bool}
miss: "posts == []"
idempotent: true
google.keywords.volume:
summary: "Monthly Google search volume (and CPC where the provider has it) for a list of keywords — one provider-native row per keyword in `rows`"
identity:
src/treg/catalog/examples/fetchinio.linkedin.company.profile.json
+49
View File
@@ -0,0 +1,49 @@
{
"id": "urn:li:fsd_company:1035",
"companyId": "1035",
"name": "Microsoft",
"publicIdentifier": "microsoft",
"url": "https://www.linkedin.com/company/microsoft/",
"websiteUrl": "https://news.microsoft.com/",
"domain": "news.microsoft.com",
"industry": "Software Development",
"industryV1": "Computer Software",
"employeeCount": 231791,
"employeeCountRange": {
"start": 10001
},
"headquarter": {
"line1": "1 Microsoft Way",
"city": "Redmond",
"geographicArea": "Washington",
"postalCode": "98052",
"country": "US",
"headquarter": true
},
"locations": [
{
"line1": "Pradiareň 1900",
"line2": "Svätoplukova 2A",
"city": "Bratislava",
"postalCode": "821 08",
"country": "SK"
},
{
"line1": "Thames Valley Park Drive",
"city": "Reading",
"geographicArea": "Berkshire",
"postalCode": "RG6 1WG",
"country": "GB"
},
"… 43 more item(s) truncated"
],
"followerCount": 29160306,
"logoUrl": "https://media.licdn.com/dms/image/v2/D560BAQH32RJQCl3dDQ/company-logo_400_400/B56ZYQ0mrGGoAc-/0/1744038948046/microsoft_logo?e=1792022400&v=beta&t=7--yYnKRaITVGjZz-B2sKfI-UdhRxFcWyimbEhiuNtU",
"coverImageUrl": "https://media.licdn.com/dms/image/v2/D4E3DAQGWOa_gAs1qXg/image-scale_325_1920/B4EZ8ZTX5SJEAI-/0/1782835937311/microsoft_cover?e=1790877600&v=beta&t=0qdKCqqRENZJw3YCIDJy_YpxHs-E0EijObn7AT7vLdw",
"description": "Every company has a mission. What's ours? To empower every person and every organization to achieve more. We believe technology can and should be a force for good and that meaningful innovation contributes to a brighter world in the future and today. Our culture doesn’t just encourage curiosity; it embraces it. Each day we make progress together by showing up as our authentic selves. We show up with a learn-it-all mentality. We show up cheering on others, knowing their success doesn't diminish o… [747 chars total]",
"specialities": [
"Business Software",
"Developer Tools",
"… 21 more item(s) truncated"
]
}
src/treg/catalog/examples/fetchinio.linkedin.post.comments.json
+43
View File
@@ -0,0 +1,43 @@
{
"comments": [
{
"urn": "urn:li:comment:(activity:7508596155145043968,REDACTED)",
"text": "[redacted public comment]",
"createdAt": "2026-09-23T18:47:08.212Z",
"reactionCount": 8,
"author": {
"id": "urn:li:fsd_profile:REDACTED",
"name": "Example Person",
"headline": "[redacted]",
"profilePictureUrl": "https://example.com/profile-picture.jpg",
"profileUrl": "https://example.com/profile/redacted",
"publicId": "redacted",
"profileId": "REDACTED"
},
"replies": [
{
"urn": "urn:li:comment:(activity:7508596155145043968,REDACTED_REPLY)",
"text": "[redacted public reply]",
"createdAt": "2026-09-23T18:49:35.456Z",
"reactionCount": 2,
"author": {
"id": "urn:li:fsd_profile:REDACTED_REPLY_AUTHOR",
"name": "Example Person Two",
"headline": "[redacted]",
"profilePictureUrl": "https://example.com/profile-picture-2.jpg",
"profileUrl": "https://example.com/profile/redacted-2",
"publicId": "redacted-2",
"profileId": "REDACTED_REPLY_AUTHOR"
},
"replies": [],
"permalink": "https://www.linkedin.com/feed/update/urn:li:activity:7508596155145043968",
"pinned": false
}
],
"permalink": "https://www.linkedin.com/feed/update/urn:li:activity:7508596155145043968",
"pinned": false
}
],
"paginationToken": "REDACTED_CURSOR",
"hasMore": true
}
src/treg/catalog/examples/fetchinio.linkedin.post.engagement.json
+40
View File
@@ -0,0 +1,40 @@
{
"post": "urn:li:activity:7508596155145043968",
"comments": [
{
"urn": "urn:li:comment:(activity:7508596155145043968,REDACTED)",
"text": "[redacted public comment]",
"createdAt": "2026-09-23T18:47:08.212Z",
"reactionCount": 8,
"author": {
"id": "urn:li:fsd_profile:REDACTED",
"name": "Example Person",
"headline": "[redacted]",
"profilePictureUrl": "https://example.com/profile-picture.jpg",
"profileUrl": "https://example.com/profile/redacted",
"publicId": "redacted",
"profileId": "REDACTED"
},
"replies": [],
"permalink": "https://www.linkedin.com/feed/update/urn:li:activity:7508596155145043968",
"pinned": false
}
],
"commentsPaginationToken": "REDACTED_CURSOR",
"commentsHasMore": true,
"reactions": [
{
"reactionType": "LIKE",
"actor": {
"urn": "urn:li:fsd_profile:REDACTED",
"name": "Example Person",
"headline": "[redacted]",
"profileUrl": "https://example.com/profile/redacted",
"profilePictureUrl": "https://example.com/profile-picture.jpg",
"publicId": "redacted",
"profileId": "REDACTED"
}
}
],
"reactionsHasMore": true
}
src/treg/catalog/examples/fetchinio.linkedin.post.reactions.json
+17
View File
@@ -0,0 +1,17 @@
{
"reactions": [
{
"reactionType": "LIKE",
"actor": {
"urn": "urn:li:fsd_profile:REDACTED",
"name": "Example Person",
"headline": "[redacted]",
"profileUrl": "https://example.com/profile/redacted",
"profilePictureUrl": "https://example.com/profile-picture.jpg",
"publicId": "redacted",
"profileId": "REDACTED"
}
}
],
"hasMore": true
}

Some files were not shown because too many files have changed in this diff Show More