Jeremy Alvis
41c5f01968
Fixes #997 ## Summary Give `ateapi` separate PostgreSQL connections for schema ownership and normal application access. Migrations and outbox partition maintenance use the owner connection, allowing application reads and writes to run with restricted permissions. - Each connection assumes its configured role with `SET ROLE`, keeping object ownership and grants separate from login credentials. - Configure owner and read/write connections through `ate-setup` and the installation manifests. - Default the schema to `substrate` and document provisioning for external PostgreSQL and Cloud SQL, including one IAM login with membership in both roles. - Keep bundled PostgreSQL using its existing `postgres` account and role for both pools. Restricted runtime access requires operator-provisioned roles. See `docs/postgres.md` for configuration details. ## Scope This is the base PR for two separate follow-ups: [file-based credentials](https://github.com/agent-substrate/substrate/pull/1979) and [automatic development database bootstrap](https://github.com/agent-substrate/substrate/pull/2025). Neither feature is included here. Operators provision external database roles, login memberships, schema ownership, and grants before startup. Table migrations still run automatically through the owner connection. The documented external database layout assumes a fresh database; this PR does not migrate existing table ownership or grants. Both connections remain inside `ateapi`, so this separates database permissions without isolating owner credentials from the application process. ## Validation - `env -u NO_COLOR make verify` passed. - PostgreSQL integration tests cover restricted runtime access, owner migrations and partition maintenance, and OpenFGA table permissions. - Installer tests cover bundled PostgreSQL, separate external logins, and Cloud SQL configuration.
2.9 KiB
Database Configuration
Overview
Substrate supports separate connections for schema ownership (DDL) and normal reads and writes (DML). Migrations and partition maintenance use the owner connection; application queries use the read/write connection. Each connection assumes its configured PostgreSQL role.
The bundled development database uses its existing postgres account and role for both pools. For restricted runtime access, provision separate owner and read/write roles in an external database. A single login can also be used if it has membership in both roles.
BYO DB Configuration
A custom database can be provided to Substrate.
ATE_API_POSTGRES_SCHEMA (or --postgres-schema when running ateapi directly) selects the schema, defaulting to substrate. The operator must provision the schema and its grants before starting Substrate. ateapi then uses the schema for both connection pools and migrations, overriding any search_path in the connection strings.
Operator Provisioned BYO DB
For production systems, the recommendation is to fully provision an external Postgres Database and provide credentials to Substrate to interact with that database. The ATE_API_POSTGRES_OWNER_CONNECTION_STRING and ATE_API_POSTGRES_READ_WRITE_CONNECTION_STRING are used to configure Substrate's owner and runtime connection pools.
Create an owner role with schema ownership and a read/write role with schema usage. Grant the runtime role SELECT, INSERT, UPDATE, and DELETE on tables; USAGE, SELECT, and UPDATE on sequences; EXECUTE on routines; and USAGE on types. Configure the owner's default privileges so newly migrated objects receive these grants. Grant the respective roles to the login users. The Cloud SQL setup example shows the SQL for one login with both roles; use separate logins for stronger isolation.
Set ATE_API_POSTGRES_OWNER_ROLE and ATE_API_POSTGRES_READ_WRITE_ROLE to the roles you created, and leave role settings out of the connection strings: ateapi runs SET ROLE after connecting.
User credentials are passed as connection strings through ATE_API_POSTGRES_OWNER_CONNECTION_STRING and ATE_API_POSTGRES_READ_WRITE_CONNECTION_STRING. When running ateapi directly, pass the DSNs as flags or use @env flags to read these environment variables; the installer manifest already uses @env.
ATE_API_POSTGRES_POOL_MAX_CONNS (or --postgres-pool-max-conns when running ateapi directly) sets the read/write pool limit after the connection string is loaded. When unset, a pool_max_conns value in the DSN or the pgxpool default applies. This setting does not affect the owner or watch pools; they are capped at 2 and 3 connections, respectively.
ateapi uses the same @env flag pattern for role names and schema.
CloudSQL Configuration
CloudSQL setup information can be found in the dedicated guide here.