mirror of
https://github.com/agent-substrate/substrate.git
synced 2026-10-26 07:23:59 +08:00
Yufan Su
10ba865ea7
Adds a credential provider for egress credential injection backed by Google Cloud Secret Manager. **It lives in its own Go module under `plugins/gcp-secret-manager`, temporally hosted here until it moves to a repository of its own.** **What it does** - Serves `credproviderpb.CredentialProvider` over mTLS and admits only the egress gateway's identity (`--injector-identity`). - Resolves global and regional secrets, optionally picking one key out of a JSON payload: `ate-secret://secretmanager.googleapis.com/projects/<project>[/locations/<location>]/secrets/<secret>/versions/<version>[/keys/<key>]` - Enforces a default-deny atespace→project policy (`--project-policy-file`), the counterpart of the Kubernetes provider's namespace policy. - Returns a retryable 503 only for transient Secret Manager failures, and caps each read with `--fetch-timeout` (default 3s). **Repository changes** - New top-level `plugins/` directory for self-contained plugins, documented in `docs/dev/code-layout.md` and `AGENTS.md`. The module imports only substrate's public `pkg/` packages; a test enforces this. - CI runs the module's tests, `make verify` and golangci-lint. govulncheck scans the module too, with the action pinned by SHA. - `docs/egress-credential-injection.md` describes each provider's credential URI format. The plugin's README covers installing and using it. - [ ] Tests pass - [ ] Appropriate changes to documentation are included in the PR
106 lines
4.7 KiB
Markdown
106 lines
4.7 KiB
Markdown
# Code Layout
|
|
|
|
This document explains the top-level directory structure of this repository and
|
|
the rationale behind where code lives. When in doubt about where to put a new
|
|
file, read this first.
|
|
|
|
## Top-Level Directories
|
|
|
|
```
|
|
substrate/
|
|
├── cmd/ # Binary entry points (one subdirectory per binary)
|
|
├── internal/ # Shared Go packages, not importable outside this module
|
|
├── pkg/ # Shared Go packages, intentionally importable by external users
|
|
├── docs/ # Design documents and developer guides
|
|
├── hack/ # Scripts used during development, CI, and cluster management
|
|
├── manifests/ # Kubernetes manifests for deploying Substrate
|
|
├── demos/ # Self-contained demo applications
|
|
├── benchmarking/ # Load testing tools and workloads
|
|
├── tools/ # Standalone Go tools (runnable with `go run ./tools/<name>`)
|
|
└── bin/ # Vendored or pinned tool binaries (e.g. protoc)
|
|
```
|
|
|
|
## Decision Rules
|
|
|
|
### `pkg/` vs `internal/`
|
|
|
|
**`pkg/`** is for Go packages with an external API contract — code that users or
|
|
third-party tools may import directly from
|
|
`github.com/agent-substrate/substrate/pkg/...`. Putting a package here signals
|
|
a commitment to backwards-compatibility and discoverability.
|
|
|
|
**`internal/`** is for Go packages that are shared across multiple binaries
|
|
within this module but are not part of any external API. The Go toolchain
|
|
enforces that nothing outside `github.com/agent-substrate/substrate` can import
|
|
these. The exception is [`internal/plugins/`](../../internal/plugins/README.md):
|
|
self-contained plugins, each a Go module of its own, in a temporary location
|
|
that nothing else in the repo should depend on or import.
|
|
|
|
> **Use `pkg/` sparingly.** Once the project reaches GA, any exported type,
|
|
> function, or field in `pkg/` becomes subject to a compatibility guarantee —
|
|
> removing or renaming it is a breaking change for external consumers. Prefer
|
|
> keeping new code in `internal/` until its API has stabilized. The bar for
|
|
> adding to `pkg/` is: "I am confident external users need to import this, and I
|
|
> am prepared to maintain its API indefinitely."
|
|
|
|
### `cmd/<binary>/`
|
|
|
|
Each subdirectory of `cmd/` corresponds to one compiled binary:
|
|
|
|
| Directory | Binary / Purpose |
|
|
|----------------------|-------------------------------------------------------|
|
|
| `cmd/ateapi` | Control-plane API server (gRPC) |
|
|
| `cmd/atecontroller` | Kubernetes controller for WorkerPools (and network/egress resources) |
|
|
| `cmd/atelet` | Node supervisor (DaemonSet) |
|
|
| `cmd/atenet` | Network proxy / Envoy external-processing server |
|
|
| `cmd/ateom-gvisor` | In-pod gVisor container image entry point |
|
|
| `cmd/ateom-microvm` | In-pod kata + cloud-hypervisor micro-VM container image entry point |
|
|
| `cmd/kubectl-ate` | `kubectl` plugin for interacting with Substrate |
|
|
| `cmd/podcertcontroller` | Controller that issues pod TLS certificates |
|
|
|
|
Each `cmd/<binary>/` contains:
|
|
- `main.go` — the entry point, kept thin (flag parsing, wiring, signal handling)
|
|
- `internal/` — packages private to that binary; **not** shared with other binaries
|
|
|
|
If a package is only used by one binary, put it under `cmd/<binary>/internal/`.
|
|
If two or more binaries share a package, move it to `internal/` (or `pkg/` if
|
|
external consumers need it too).
|
|
|
|
### `hack/`
|
|
|
|
Scripts for humans: setting up dev environments, running verifiers, generating
|
|
code, managing clusters. Nothing in `hack/` is imported as a Go package. Prefer
|
|
shell scripts here; for more complex automation, put a Go tool in `tools/`.
|
|
|
|
### `tools/`
|
|
|
|
Dev/CI tools written in Go. These are build-time or
|
|
operations tools that are part of the repository but not compiled into any
|
|
shipped binary. Example: `tools/setup-gcp` provisions GCP resources.
|
|
|
|
Each tool must have its own dedicated `go.mod` (and `go.sum`).
|
|
|
|
### `manifests/`
|
|
|
|
Kubernetes YAML for deploying Substrate components.
|
|
|
|
### `demos/`
|
|
|
|
Example applications showing how to build on Agent Substrate.
|
|
|
|
### `benchmarking/`
|
|
|
|
Load-testing tooling and benchmark workloads.
|
|
|
|
## Placement Checklist
|
|
|
|
When adding a new Go package, ask:
|
|
|
|
1. **Is it only used by one binary?** → `cmd/<binary>/internal/<pkg>`
|
|
2. **Is it shared by multiple binaries but not meant for external import?** → `internal/<pkg>`
|
|
3. **Is it a deliberately public API for external users?** → `pkg/<pkg>`
|
|
4. **Is it a protobuf-generated package?**
|
|
- Public gRPC API (control plane) → `pkg/proto/<name>`
|
|
- Internal gRPC API (atelet, ateom) → `internal/proto/<name>`
|
|
5. **Is it a script or dev tool?** → `hack/` (shell) or `tools/<name>` (Go)
|