Fixes#997
## Summary
Give `ateapi` separate PostgreSQL connections for schema ownership and
normal application access. Migrations and outbox partition maintenance
use the owner connection, allowing application reads and writes to run
with restricted permissions.
- Each connection assumes its configured role with `SET ROLE`, keeping
object ownership and grants separate from login credentials.
- Configure owner and read/write connections through `ate-setup` and the
installation manifests.
- Default the schema to `substrate` and document provisioning for
external PostgreSQL and Cloud SQL, including one IAM login with
membership in both roles.
- Keep bundled PostgreSQL using its existing `postgres` account and role
for both pools. Restricted runtime access requires operator-provisioned
roles.
See `docs/postgres.md` for configuration details.
## Scope
This is the base PR for two separate follow-ups: [file-based
credentials](https://github.com/agent-substrate/substrate/pull/1979) and
[automatic development database
bootstrap](https://github.com/agent-substrate/substrate/pull/2025).
Neither feature is included here.
Operators provision external database roles, login memberships, schema
ownership, and grants before startup. Table migrations still run
automatically through the owner connection.
The documented external database layout assumes a fresh database; this
PR does not migrate existing table ownership or grants. Both connections
remain inside `ateapi`, so this separates database permissions without
isolating owner credentials from the application process.
## Validation
- `env -u NO_COLOR make verify` passed.
- PostgreSQL integration tests cover restricted runtime access, owner
migrations and partition maintenance, and OpenFGA table permissions.
- Installer tests cover bundled PostgreSQL, separate external logins,
and Cloud SQL configuration.
### Description
This PR introduces native, secure support for using Cloud SQL as the
PostgreSQL store backend for `ate-api-server`.
To ensure the highest level of security and ease of use in GCP
environments, this integration leverages the Cloud SQL Auth Proxy
sidecar with automatic IAM database authentication. This means transport
security (TLS 1.3 tunnel) is handled automatically, and database
sessions are authenticated using Workload Identity via short-lived OAuth
tokens, completely eliminating the need for database passwords.
### Key Changes
* **Cloud SQL Auth Proxy Sidecar:** Added
`manifests/ate-install/cloudsql-proxy-patch.yaml` to patch the sidecar
into the `ate-api-server` deployment when a Cloud SQL instance is
configured.
* **Automated Provisioning:** Extended `tools/setup-gcp` with a new
`cloudsql` command. This handles the idempotent creation of the Cloud
SQL instance, Google Service Accounts (GSA), IAM bindings, and Workload
Identity bindings.
* **Installation Script Updates:** Updated `hack/install-ate.sh` to
parse new environment variables (e.g.,
`ATE_API_POSTGRES_CLOUDSQL_INSTANCE`, `ATE_API_POSTGRES_CLOUDSQL_GSA`)
and correctly synthesize the passwordless DSN and ConfigMaps for the
proxy.
* **Security & Documentation:**
* Added extensive documentation in `tools/setup-gcp/cloud-sql.md`
covering provisioning, schema privileges, deployment, and database
scaling.
* Updated `docs/threat-model.md` to reflect the new Cloud SQL egress
flows and Auth Proxy tunnel mechanics.
* **Dependencies:** Vendored required Google API clients (`sqladmin/v1`,
`servicenetworking/v1`, `iam/v1`) for the GCP setup tool.
- [X] Tests pass
- [X] Appropriate changes to documentation are included in the PR
Jeremy Alvis