mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-11-10 10:13:50 +08:00
Jeremy Alvis
41c5f01968
Fixes #997 ## Summary Give `ateapi` separate PostgreSQL connections for schema ownership and normal application access. Migrations and outbox partition maintenance use the owner connection, allowing application reads and writes to run with restricted permissions. - Each connection assumes its configured role with `SET ROLE`, keeping object ownership and grants separate from login credentials. - Configure owner and read/write connections through `ate-setup` and the installation manifests. - Default the schema to `substrate` and document provisioning for external PostgreSQL and Cloud SQL, including one IAM login with membership in both roles. - Keep bundled PostgreSQL using its existing `postgres` account and role for both pools. Restricted runtime access requires operator-provisioned roles. See `docs/postgres.md` for configuration details. ## Scope This is the base PR for two separate follow-ups: [file-based credentials](https://github.com/agent-substrate/substrate/pull/1979) and [automatic development database bootstrap](https://github.com/agent-substrate/substrate/pull/2025). Neither feature is included here. Operators provision external database roles, login memberships, schema ownership, and grants before startup. Table migrations still run automatically through the owner connection. The documented external database layout assumes a fresh database; this PR does not migrate existing table ownership or grants. Both connections remain inside `ateapi`, so this separates database permissions without isolating owner credentials from the application process. ## Validation - `env -u NO_COLOR make verify` passed. - PostgreSQL integration tests cover restricted runtime access, owner migrations and partition maintenance, and OpenFGA table permissions. - Installer tests cover bundled PostgreSQL, separate external logins, and Cloud SQL configuration.
28 lines
2.9 KiB
Markdown
28 lines
2.9 KiB
Markdown
# Database Configuration
|
|
|
|
## Overview
|
|
Substrate supports separate connections for schema ownership (DDL) and normal reads and writes (DML). Migrations and partition maintenance use the owner connection; application queries use the read/write connection. Each connection assumes its configured PostgreSQL role.
|
|
|
|
The bundled development database uses its existing `postgres` account and role for both pools. For restricted runtime access, provision separate owner and read/write roles in an external database. A single login can also be used if it has membership in both roles.
|
|
|
|
## BYO DB Configuration
|
|
A custom database can be provided to Substrate.
|
|
|
|
`ATE_API_POSTGRES_SCHEMA` (or `--postgres-schema` when running `ateapi` directly) selects the schema, defaulting to `substrate`. The operator must provision the schema and its grants before starting Substrate. `ateapi` then uses the schema for both connection pools and migrations, overriding any `search_path` in the connection strings.
|
|
|
|
### Operator Provisioned BYO DB
|
|
For production systems, the recommendation is to fully provision an external Postgres Database and provide credentials to Substrate to interact with that database. The `ATE_API_POSTGRES_OWNER_CONNECTION_STRING` and `ATE_API_POSTGRES_READ_WRITE_CONNECTION_STRING` are used to configure Substrate's owner and runtime connection pools.
|
|
|
|
Create an owner role with schema ownership and a read/write role with schema usage. Grant the runtime role SELECT, INSERT, UPDATE, and DELETE on tables; USAGE, SELECT, and UPDATE on sequences; EXECUTE on routines; and USAGE on types. Configure the owner's default privileges so newly migrated objects receive these grants. Grant the respective roles to the login users. The [Cloud SQL setup example](../tools/setup-gcp/cloud-sql.md#2-one-time-roles-and-schema-privileges) shows the SQL for one login with both roles; use separate logins for stronger isolation.
|
|
|
|
Set `ATE_API_POSTGRES_OWNER_ROLE` and `ATE_API_POSTGRES_READ_WRITE_ROLE` to the roles you created, and leave role settings out of the connection strings: `ateapi` runs `SET ROLE` after connecting.
|
|
|
|
User credentials are passed as connection strings through `ATE_API_POSTGRES_OWNER_CONNECTION_STRING` and `ATE_API_POSTGRES_READ_WRITE_CONNECTION_STRING`. When running `ateapi` directly, pass the DSNs as flags or use `@env` flags to read these environment variables; the installer manifest already uses `@env`.
|
|
|
|
`ATE_API_POSTGRES_POOL_MAX_CONNS` (or `--postgres-pool-max-conns` when running `ateapi` directly) sets the read/write pool limit after the connection string is loaded. When unset, a `pool_max_conns` value in the DSN or the pgxpool default applies. This setting does not affect the owner or watch pools; they are capped at 2 and 3 connections, respectively.
|
|
|
|
`ateapi` uses the same `@env` flag pattern for role names and schema.
|
|
|
|
### CloudSQL Configuration
|
|
CloudSQL setup information can be found in the dedicated guide [here](../tools/setup-gcp/cloud-sql.md).
|