AX is undergoing a significant redesign, moving away from a single CLI with
an embedded Python harness toward a general-purpose orchestration layer for
agentic tasks. The repository is reorganized around three binaries: ax-server
exposes a gRPC API for submitting Task manifests, ax-controller runs as a
horizontally scaled reconciler that consumes work from Redis Streams, and
ax-task-runner executes tasks inside sandboxed workers. Task state now lives
in Redis rather than Kubernetes CRDs so the system can handle millions of
short-lived tasks without straining etcd. The legacy Python harness, ATE
client, SQL event log, and skill examples are removed, and new design,
concepts, manifest, networking, runner, and sandbox docs plus Kubernetes
deploy manifests describe the new architecture.
We want AX to be compatible with the Interactions API protocol as much as possible. This PR aligns the proto with Steps and with the Interactions semantics. We will follow up with more changes to make it more compatible with Interactions while preserving a `conversation_id` based history model.
`ConversationService` is removed for now until we know what kind of data operations we want to provide for event log management.
google-antigravity 0.1.7's _connect_websocket retries both "localhost"
and "127.0.0.1" when dialing localharness (local_connection.py:993), so
the sidecar no longer needs to patch /etc/hosts for substrate/gVisor
environments where localhost may not resolve.
Ref #325
antigravity: guard conversation_id for safe save_dir use (#312)
harness_server uses the caller-supplied conversation_id directly as a
per-conversation storage directory name (save_dir = state_dir /
conversation_id) before the Antigravity harness ever sees it, so a value
like "../escape" could traverse outside state_dir. Reject an empty id or
one containing a path separator or a "." / ".." component at the boundary
in _run_turn, returning an INVALID_ARGUMENT end frame.
This is intentionally only a path-safety guard, not the id-format
contract. The Antigravity harness owns that: it validates the forwarded
cascade_id (length and character set) per cl/948016352. Keeping the
format rule in one place avoids the two layers drifting.
chore: remove stale docs, gitignore, and Makefile entries (#329)
- CONTRIBUTING.md: drop 'make run-remote' (no such Makefile target)
- .gitignore: remove orphaned entries (local/remote_agent, sandbox-router,
tasklog, axepp, google-cloud-sdk, test-sandbox-config.yaml)
- Makefile: add missing deps, clean-logs to .PHONY
antigravity: remove vertex env-var override (superseded by AGY 0.1.7) (#331)
* antigravity: remove vertex env-var override (superseded by AGY 0.1.7)
google-antigravity 0.1.7 reads GOOGLE_GENAI_USE_VERTEXAI /
GOOGLE_GENAI_USE_ENTERPRISE + GOOGLE_CLOUD_{PROJECT,LOCATION} natively,
so the sidecar no longer needs _vertex_kwargs_from_env to thread them
into LocalAgentConfig.
- Delete _vertex_kwargs_from_env + VertexKwargs.
- _build_default_config: bare LocalAgentConfig(system_instructions=...).
- _has_credentials: read project/location from env (SDK hydrates them
onto VertexEndpoint, not LocalAgentConfig).
- Update tests accordingly.
Requires the 0.1.7 bump. Credential resolution is startup-only and
does not touch the save_dir/state_dir resume path.
Ref #325
* antigravity: drop stale _has_credentials docstring note about vertex env override
Interactions: persist interactions harness with durable dir and resume from it (#327)
interactions: make WorkDir authoritative for run_command (#335)
The interactions harness executed run_command relative to the process's
ambient working directory, which is not reliably the agent's workspace: with
no Cwd argument the command ran wherever the process cwd happened to be, and a
model-supplied Cwd (including "/") was honored blindly. As a result tool calls
often ran from "/" instead of the configured workspace (#332).
Make the workspace directory authoritative in the executor rather than relying
on ambient process state:
- Add AntigravityInteractionsConfig.WorkDir (defaults from AX_HARNESS_WORKDIR).
- run_command now resolves its directory via resolveRunDir: no Cwd -> WorkDir;
relative Cwd -> joined under WorkDir; absolute Cwd -> honored as-is. Empty
WorkDir preserves the previous process-cwd behavior.
- Add WorkspaceSystemInstruction to orient the agent about its working
directory so it emits workspace-relative paths, and wire it in cmd/ax.
Fixes#332
Remove the binary (#334)
* Remove the binary
* chore: add e2e directory to .gitignore
* refactor: migrate binary file check to a dedicated workflow file
* fix: restrict binary file check to added, copied, or modified files
The 17 tests in python/antigravity/harness_server_test.py have never
run in CI. .github/workflows/go.yml only runs 'go test ./...' and the
Makefile 'test' target likewise only runs Go. That let PR #244 land
with 2 stale assertions unnoticed (fixed in #291).
Adds:
- .github/workflows/python.yml -- dedicated Python workflow (kept
separate from the Go-only go.yml so each can evolve independently).
- Makefile 'test-python' target for local parity.
Test deps are inlined in the workflow rather than added as a separate
requirements-test.txt -- the diff is 2 packages ('pytest>=7.0',
'pytest-timeout>=2.0'), not worth its own file. Version floors match
the upstream google-antigravity SDK's dev extras so this doesn't
drift as pytest evolves.
--timeout=30 --timeout-method=thread guards against hung gRPC servers
so a stuck test can't eat the whole workflow budget.
* Cleanup the repo and move to controller2
* Migrate controller and registry to internal/controller2
- Remove the legacy harness build tag
* refactor: move GeminiConfig definition from internal/config to internal/gemini package
* test: replace controller registry with dummy implementation and update dependency constraints in go.mod
* chore: remove harness build and test steps from GitHub Actions workflow
* Move axharness service to ax binary
This is for the simplicity of building everything into the same binary. In Substrate deployments, ax harness will be executed as an actor.
* feat: add harness implementation to support AGY and other service harnesses
* Added internal/controller2/: as a copy of the existing controller.
* Added a new deployment manifest which will replace the previous one, internal/manifests/ax-deployment2.yaml.
* Added HarnessService to proto as a placeholder.
* Added internal/SUBSTRATE_REFACTOR_CLEANUPS.md for some cleanups after this refactor is done.
* Added SubstrateHarness that can create/resume axharness as an actor with conversation ID.
* Remove unnecessary ate build tags because Substrate is now public.
Updates #3.
So we don't accidentally promote this option. There are also rough edges
in this implementation that we don't have time to fix, e.g. the use of
kubectl.
This change introduces an EPP to create/resume actor the incoming
conversation_id. We also introduce an ax-shell image and a demo pod for
testing purposes.
Follow up changes:
- Suspend the AX server automatically after the request is sent.
- Explore failure cases and how we can ensure suspension safely even if
AX server crashes.
- EventLogService.Fork shouldn't autogenerate a dest_conversation_id.
dest_conversation_id should always be supplied from the client for
routing.
This change also removes the RegisterAgent RPC. Actor
resumption/suspension requires a valid conversation_id for all RPCs and
RegisterAgent no longer fits into this new constraint.
We allow any encoding for sub agent configuration after
https://github.com/google-gemini/ax/pull/248. This change is removing
the proto encoding requirement for the Gemini and the test agents.
This pull request introduces initial support for integrating
experimental ATE (SubstrATE) agents. It also includes a complete example
with manifests and a sample agent implementation.
The changes are primarily focused on enabling optional ATE agent support
behind a build tag. CreateActor isn't always working but we will use
this experimental feature as a starting point.
ATE agents start and suspend actors for execution at every agent call.
In the future, we would like to make changes to the controller to make
native resumption of agents from the snapshots possible. For now, we are
creating and suspending at every agent call.
We will rename ATE once we have an external name for the project.
Inputs should only be appended to the history once the execution is
complete. Otherwise, they will cause duplicates in task event log.
Also remove Rehydrate option. There isn't a clear good use case for it
yet.
We are replacing the current loop executor with a new internal execution
framework. This update refactors the majority of the project's core
logic to improve stability and performance.
We introduce a new Task Executor, moving away from simple
request-response agent loops toward a more structured, stateful
execution model. The core of this update is the addition of a Task
struct and a TaskExecutor interface, which allow agents to trigger
sub-tasks and manage complex workflows. This change enables better
recursion and delegation between different agents, such as the testing
coding agent and specialized worker agents. The new executor framework
supports confirmation propagation and leaves the confirmation handling
to the agent implementation.
The update introduces a new persistent Event Log system to track every
step of a task's execution. By recording inputs, outputs, and state
changes to a newline-delimited JSON file, the system can now "rehydrate"
sessions at any point. This allows the executor to resume from a
consistent state after a crash or restart, and provides a durable
history of agent actions.
The change refactors the Controller and various agent implementations to
support these new interfaces. It includes updates to the CLI trigger
tool to handle interactive confirmation prompts more effectively and
manage conversation history across task boundaries. Comprehensive tests
for the new Task Manager, including fan-out execution and resume
capabilities, are also included to ensure reliability in concurrent
environments.
The change also introduces a "gemini" agent to make it easier to make
Gemini calls without having to write any new code. We default to the
Flash models but can follow up with some configuration options to
override the default settings.
Trigger now also allows starting execution from any available agent, `$
trigger --agent <agent_id>`.
The new capabilities can be explorable and testable with the
internal/testagent.
Follow up changes after this change is merged:
- Fixing Gemini planner and skills support
- gar trigger will be rename to gar exec
- We will remove the session IDs and replace them with IDs/task IDs
- Exec over the wire, changes to AgentService
- Task log in SQLite
- Trace view tool to be able to inspect the logs
### Description
This PR introduces end-to-end support for dynamically provisioning,
managing, and routing to isolated K8s Sandbox environments on GKE for
secure agent execution as part of GAR
### Key Features & Components
#### 1. Kubernetes Resource Orchestration (`pkg/sandboxclient`)
Implemented a robust `dynamic` K8s client to programmatically interact
with the complete suite of Custom Resources required for GKE Agent
Sandbox:
* **`SandboxTemplate`**: Defined via manifest
(`examples/sandbox_agent/sandbox-template.yaml`) and injected via
`sandbox_template_ref` in `gar.yaml` to serve as the blueprint for the
container environment.
* **`SandboxWarmPool`**: Client APIs added
(`CreateWarmPool`/`DeleteWarmPool`) for pre-booting "warm" sandboxes to
eliminate cold-start latency.
* **`SandboxClaim`**: Client APIs added (`CreateClaim`/`DeleteClaim`) to
provision ephemeral execution environments and bind them exclusively to
a GAR `sessionID`.
* **`Sandbox`**: Polling APIs added (`WaitForSandbox`) to read the
resulting pod's `selector` and `status.service` IP once the Claim is
successfully fulfilled.
#### 2. Sandbox Agent Interface (`agent/sandbox.go`)
Added the `sandbox` agent type implementation.
* Deterministically maps a GAR `sessionID` to a K8s `SandboxClaim` name
for session reusability.
* Requests the lease, waits for the pod to become ready, establishes the
gRPC streaming connection via the Router, and handles automatic cleanup
of the K8s pod during `agent.Close()`.
#### 3. In-Cluster Sandbox Router (`cmd/sandbox-router`)
* Added a lightweight, high-performance gRPC proxy that runs natively
inside GKE.
* Securely bridges external `kubectl port-forward` traffic from the GAR
CLI directly to the isolated backend gVisor pods by parsing
`x-sandbox-id` headers.
#### 4. Enhancements & Fixes
* **Resiliency**: Enhanced `agent/remote.go` to support automatic
retries during internal network or port-forward disconnects.
* **Stability**: Fixed a panic in `internal/controller/health.go`
occurring during the health monitor shutdown sequence.
* **Example**: Provided a complete end-to-end sandbox example (Go
uppercase processor, Dockerfile, and K8s manifest) in
`examples/sandbox_agent/`.
* **Documentation**: Added comprehensive tutorials to `README.md` for
deploying the router, configuring warm pools, and testing the sandbox
examples. Includes a new `make build-router` target for easy proxy
cross-compilation.
*
Tested manually with GKE agent sandbox
**1. Compiled and Deployed the newly-coded K8s Router & Sandbox
Template**
```bash
$ make build-router
$ kubectl apply -f cmd/sandbox-router/sandbox-router.yaml
$ kubectl wait --for=condition=Ready pod -l app=sandbox-router --timeout=60s
$ kubectl apply -f examples/sandbox_agent/sandbox-template.yaml
```
``
**2. Built and Started the GAR Controller CLI**
```bash
$ make build
$ ./bin/gar serve --config gar.yaml > serve.log &
2026/03/07 02:28:18 Starting GAR server at :8494...
```
**3. Executed the Trigger**
```bash
$ ./bin/gar trigger --server localhost:8494 --input "test sandbox agent by converting 'hello from tty' to uppercase"
```
**4. The Result**
```
Session: 1338459a-40d2-45df-b0c5-a07512c3506a
⏺ test sandbox agent by converting 'hello from tty' to uppercase
hey im your sandbox agentHERE IS YOUR UPPERCASE TEXT: TEST SANDBOX AGENT BY CONVERTING 'HELLO FROM TTY' TO UPPERCASE
```
**5. Check the sandbox log
```
$ kubectl logs gar-claim-uppercase-1338459a40d245dfb0c5 -n default
2026/03/07 10:33:11 🟢 upper_case_agent listening on :8494
2026/03/07 10:33:15 📥 Received request text: "test sandbox agent by converting 'hello from tty' to uppercase"
2026/03/07 10:33:15 📤 Sending response: "TEST SANDBOX AGENT BY CONVERTING 'HELLO FROM TTY' TO UPPERCASE"
```
JBD