Self-Hosting AI: Why We Chose to Run in Your Infrastructure

Self-Hosting Fails When a Sandbox Needs Your Database Password

An AI agent that can write code eventually needs to query data. Giving that code a database connection string makes the sandbox a second holder of a secret that can reach far beyond the question being answered.

That is the boundary we care about in a self-hosted deployment. The backend, database, and model endpoint can sit in infrastructure operated by the team, while generated code still has a narrow way to ask for data.

The distinction matters during a compromise as much as during normal operation. A leaked database password may remain usable from any network location and with any privileges it permits; a sandbox token still has to pass the backend's live authorization check before a query runs.

The Shipping Topology Uses a Nested Docker Runner

The current runnable Dokploy Compose topology starts a dedicated Docker-in-Docker runner, a bootstrap service, and the application backend. The bootstrap service prepares the sandbox environment and artifacts store before the backend begins creating sandboxes.

The older browser-importable template covers the database, migrations, and backend, but it does not include the runner or bootstrap pieces required by sandbox-dependent agents. A fresh import of that template therefore needs reconciliation before those agents can run.

The nested runner keeps application Docker calls away from the host daemon. It also creates a boundary that ordinary Compose service discovery cannot cross, which becomes an explicit operating constraint later in the deployment.

Sandboxes Get a Scoped Gateway and Token

When the backend creates a sandbox, it injects SANDBOX_GATEWAY_URL and a newly minted SANDBOX_TOKEN. The token is signed with HS256 and identifies a chat, an agent, and a user. It contains neither a database ID nor a connection secret.

Database credentials stay in the backend process. Only after the gateway authorizes a request does the backend load the selected data source, build its adapter configuration, and let that adapter connect.

This is capability scoping enforced by the request path, rather than a request for generated code to behave. Its SQL surface exposes text2sql.validate and text2sql.run over bearer-authenticated POST JSON-RPC.

Each SQL Request Rechecks the Agent’s Current Source Assignment

The token proves who created the sandbox, but it does not grant permanent access to one particular database. On every SQL request, the gateway verifies the token and checks the agent's current data-source assignment for the requested source.

Removing a source from an agent takes effect on the sandbox's next request. The sandbox might still hold a valid token, yet the gateway rejects a source that no longer belongs to that agent's live allowlist.

1. 1

   Request. The sandbox sends a source identifier and SQL to the gateway with its Bearer token.

2. 2

   Authenticate. The gateway verifies the token signature and reads the chat, agent, and user scope.

3. 3

   Authorize. The backend confirms that the scoped agent still has the requested source assigned, then resolves that source's credentials internally.

4. 4

   Execute. The adapter applies SQL preprocessing and runs the allowed request using the backend-held configuration.

Integration coverage exercises this boundary: a token cannot query an unassigned source from the same team, while missing or modified tokens receive an unauthorized response. That is the useful property here, the authorization decision follows the current assignment rather than a secret copied into a sandbox at startup.

Read-Only Has a Shared Policy and Different Engine Boundaries

Every adapter runs registered preprocessors before it calls a database driver. The shared read-only validator rejects empty input, comments, multiple statements, non-SELECT forms, write PRAGMAs, data-modifying CTEs, dangerous MySQL file operations, and additional unsafe constructs handled by the shared validator.

That policy catches a broad set of mistakes and hostile queries before a driver sees them. The deeper protection depends on the database engine and adapter.

SQLite adds mode=ro&immutable=1 and opens its database in read-only mode. MySQL and MariaDB set transaction_read_only for each query session. These are meaningful second boundaries after parsing.

PostgreSQL intends to initialize pooled connections with default_transaction_read_only=on, but that initializer is asynchronous and can fail after a connection is returned. The parser still runs, yet the current implementation cannot claim a universal database-enforced read-only guarantee on that path.

SQL Server currently relies on parser validation in its query path. BigQuery performs a dry run for validation, so its durable write boundary depends on the IAM permissions provisioned outside the application. Teams should choose data-source credentials and roles as part of the boundary, rather than treat an AI-facing validator as the entire control.

Team-Owned Provider Rows Remove a Platform-Key Fallback

Model credentials follow the same ownership principle. Each team can save an enabled provider record with an API key and base URL, and OpenAI-compatible providers require both values when first configured.

When a submitted API key is verified for an OpenAI-compatible configuration, the backend probes the configured endpoint's /models route before saving. During sandbox execution, configuration accepts only an enabled provider belonging to the same team; a missing or disabled selection stops the request instead of substituting a platform credential.

That arrangement keeps the model endpoint and billing relationship under the deployer's control. It also means availability is theirs to operate: a provider outage or incorrect endpoint prevents the agent from completing the turn.

Pinned Addresses and Privileged DinD Are an Operating Contract

Nested Docker containers cannot use the outer Compose network's DNS. The current runnable topology therefore pins the database to 10.89.0.10 and the backend to 10.89.0.20, then gives sandboxes the backend gateway URL at that address.

The privileged runner is an honest trade-off, not a marketing detail. A sandbox escape reaches a privileged inner Docker engine, and the separately documented rootless orchestration route remains untested. Teams with a stronger isolation requirement should evaluate that gap against their own threat model before deployment.

The Checked-In Pulumi Project Does Not Cover This Sandbox Path

The checked-in Pulumi deployment implementation creates Cloud Run, Cloud SQL, a migration job, and a GCS-backed SQLite mount. It uses a password database user and leaves backup, point-in-time recovery, deletion protection, and bucket lifecycle choices to its declared configuration.

As checked in, it does not provide the sandbox runner, bootstrap service, Docker endpoint, or required sandbox gateway environment. The backend validates its gateway URL and secret at boot, so this configuration can fail before the application is available.

The result is a real distinction for evaluators: the Dokploy Compose topology is the runnable path for sandbox-required agents today. The Pulumi project is useful evidence of deployment work, but it is not an equivalent deployment guide for that product path.

Gateway Tokens, Secret Storage, and Recovery Need a Lifecycle

The gateway limits what a sandbox can ask the backend to do. It does not solve every secret and recovery concern around a self-hosted deployment.

The next hardening work is concrete: add token expiry and a rotation path, decide how secrets should be protected at rest, and make each deployment topology intended to run sandbox-required agents carry the gateway and runner it needs. Until then, “runs in your infrastructure” describes a narrow, implemented boundary: backend-held database credentials, live source authorization, and team-owned model configuration, with operational responsibility left plainly with the deployer.