Tunnels and shells
Three commands and a console terminal give your machine access to one environment’s private services. Your machine does not join the environment network, and no container runs an SSH server.
pl db connectforwards a managed database tolocalhost.pl tunnelforwards one port of a running service tolocalhost.pl execruns a command, or opens a shell, in a running service.pl cpcopies a file or directory to or from a running service.- The Shell button on the console’s Services page opens a terminal in a running service.
You need the Admin or Owner role in the organization. A Member runs pl exec, pl cp, the console’s shell, pl tunnel and pl db connect in preview environments (Who). Built 2026-11-04; the console’s shell and pl exec without a command added 2026-11-06; pl cp, the shell’s instance choice, the sessions list and the database Connect panel added 2026-11-06; exec for Members in preview environments decided and built 2026-11-13; tunnels and database connections for Members in preview environments on 2026-11-14.
Connect to a database
-
Find the database ID:
pl db list -
Forward it.
plprints the connection string on standard output, and the rest on standard error:$ pl db connect 0193f2c4-8a1e-7c3b-9d40-5e6f7a8b9c0d postgres://[email protected]:54012/shop The password is not shown: the API never reads database credentials back. Use the one in the role's DATABASE_URL (or PG*/CLICKHOUSE_URL/REDIS_URL) variables. Forwarding until you press Ctrl-C. Session 0193f2c5-…; a new one takes over before its four-hour limit. -
In a second terminal, connect with the printed address and the password from the role’s
DATABASE_URL:psql "postgres://[email protected]:54012/shop" -
Press Ctrl-C in the first terminal when you are done.
plprintsSession ended: closed.
Use --port <local port> for a fixed local port, for example to keep a saved connection in a database GUI. A port that is in use fails before a session opens.
| Engine | Port forwarded | Printed address |
|---|---|---|
| PostgreSQL | 5432 | postgres://<login>@127.0.0.1:<port>/<database> |
| ClickHouse | 8123 (HTTP) | http://<login>@127.0.0.1:<port>/?database=<database> |
| Valkey | 6379 | redis://<login>@127.0.0.1:<port>/0 |
The address has the manifest login and its database, but no password: the API never reads database credentials back from Vault. Take the password from the role’s connection variables (DATABASE_URL, PG*, CLICKHOUSE_URL or REDIS_URL). When an instance has several logical databases, the address names the first. The tunnel reaches all of them: change the name in the address. Without a ready login with migrate, the address has no <login>@. The database must be ready or degraded in pl db list; otherwise the API answers 409.
In the console. A browser cannot forward a port. So a database’s page on the Data page has a Connect panel instead. It shows the pl db connect <id> command with a copy button, and the database and login the address will name. It never shows a password. It names the secret that holds it: the role’s DATABASE_URL (or PG*), CLICKHOUSE_URL or REDIS_URL.
Open a shell
From a checkout, open a shell in the web role of the checkout’s preview environment:
$ pl exec web
/app #
Without a command, pl exec runs sh. Add --production for production, or --project and --environment for another environment. Exit the shell to end the session.
Run one command and use its output and exit code in a script. Put the command after --:
$ pl exec web --production -- ls /app
package.json
server.js
$ echo $?
0
pl exec passes standard input, output and error, and exits with the command’s exit code. When standard input and output are a terminal, the command gets a terminal of the same size, and a resize follows. Without a terminal, you can pipe into it: pl exec web -- sh -c 'cat > /tmp/seed.sql' < seed.sql. When the session ends before the command exits, pl prints Session ended: <reason>. and exits with 255.
In the console. On the Services page, an Admin or Owner sees a Shell button on each web, private or worker service. A Member sees it in a preview environment. The service needs at least one ready instance. When it has several ready instances, a list next to the button picks one, by ordinal and short ID (web.1 · 0193bbbb). With one, there is no list. It opens a terminal below the services: bash -l where the image has bash, else sh -l, with TERM=xterm-256color. The terminal is ghostty-web, Ghostty’s terminal emulator compiled to WebAssembly, loaded only when a shell opens. The console and pl exec open the same exec session, with the same checks and record. When the shell exits, the panel shows the exit code. Close ends the session.
Changes you make in a shell stay in that one instance until its next deployment. Put lasting changes in the image or the manifest.
Choose an instance
A service with replicas has several instances. pl exec, pl cp and pl tunnel take --instance <id>. The IDs are under Technical details on the service’s card, and in the console’s shell list. Without --instance, the command picks the first ready instance of the newest live release.
Copy files
Copy a local directory into the web role, and a log file back:
$ pl cp ./fixtures web:/app/tmp/fixtures
$ pl cp web:/var/log/app.log ./
Name one side role:/absolute/path. The other side is a local path. Write ./a:b for a local name with a colon. When the destination is a directory that exists, the copy goes into it under its own name. Otherwise it gets the destination’s name, as with cp -r.
pl cp sends a tar archive over an exec session. It runs sh and tar in the instance, so the image needs both. Without tar, it stops with pl cp: the image has no tar. It takes the same options as pl exec: --instance, --project, --environment, --production and --path. It exits with the remote command’s exit code. The archive streams through the session, so a copy of any size holds only a small buffer in memory. A failed download may leave part of the copy, and pl cp says so.
Forward a port
Forward port 9090 of the web role, for example an admin page that is not public:
$ pl tunnel web 9090
127.0.0.1:54013
Forwarding until you press Ctrl-C. Session 0193f2c6-…; a new one takes over before its four-hour limit.
Then open http://127.0.0.1:54013 in a browser. A private role works the same way: pl tunnel search 7700. Use --local-port <port> for a fixed local port.
Options and limits
pl tunnelandpl exectake the environment likepl logs: the checkout’s project and branch by default, or--project,--environment,--productionand--path.pl db connecttakes only the database ID.- Without
--instance <id>,pl tunnel,pl execandpl cppick a ready instance of the newest live release. The console’s shell does the same unless you pick an instance (Choose an instance). - The role must be long-running (
web,privateorworker).execinto a database is refused. - A tunnel carries up to 32 connections at once.
- A session lasts up to four hours. It also ends after 30 minutes without traffic. When a session ends,
plprints why (When a session ends). pl tunnelandpl db connectkeep the local port until you press Ctrl-C, across sessions. Ten minutes before a session’s four-hour limit,plopens a new session for the new connections. The connections that are open finish on the old session until its limit. When a session ends early, for example when the node’s agent updates or after 30 idle minutes, the next connection opens a new session. Only the connections that were open on the ended session are lost. A revoked session, or a role that no longer allows it, ends the tunnel.pl execandpl cpend with their session. Run the command again.
Session access
The Admin or Owner role in the organization, with a user credential: a console or pl login session, or a user’s API token. Viewers, read-only tokens and service tokens are refused.
A Member opens sessions in preview environments only: pl exec, pl cp, the console’s shell, pl tunnel and pl db connect. A Member gets 403 for each of them in production. A shell in an instance can read that environment’s variables and secrets, and a tunnel or database connection reaches what the app reaches. A Member can already deploy code to a preview environment, so these sessions give no new access there.
| Session | Admin, Owner | Member |
|---|---|---|
exec, tunnel, database in a preview environment | Yes | Yes |
exec, tunnel, database in production | Yes | No |
An open session ends when its creator’s role no longer allows it: an Admin loses the role, a Member leaves the organization, or the environment is no longer a preview.
Recorded
Every session is a developer_sessions row (migration 0130). The row records who opened it, the environment and the target. A target is a database, a role, instance and port, or a role, instance and command. The row also records the node, when it was created, when pl connected, and when and why it ended. Opening and revoking are audit events (developer_session_opened, developer_session_revoked).
See and revoke sessions
Settings in the console has a Developer sessions panel for Admins and Owners. It lists the open sessions first, then the last 50 that ended. Each row shows the kind, the target, the project and environment, who opened it, and when it opened, connected and ended. An ended row shows why. An open row has a Revoke button. The relay then closes both sides within five seconds.
API
POST /v1/sessions with {"kind":"database","database":<id>}, {"kind":"tunnel","project","environment","role","port","instance"?} or {"kind":"exec","project","environment","role","instance"?,"command":[…],"tty"} answers 201 with the session, its one-time token, the relay URL and the connection details. GET /v1/sessions lists the organization’s open sessions, then the last 50 ended, with the project and environment slugs and the creator’s email. GET /v1/sessions/{id} shows a session and why it ended. The list needs the Admin role, because a target can name a command line. A Member reads only the sessions they opened. DELETE /v1/sessions/{id} revokes it: the relay closes both sides within five seconds, and a job that no node has leased yet fails.
Internals
Agents get work only by polling (POST /v1/agent/lease, agent gateway), so there is no channel the control plane can push a byte stream over. All three commands use one relay:
plasks the API for a session. The API checks the role, finds the target and the node that runs it, records the session and queues oneopen_sessionjob for that node. The job carries the one target the session may reach. It runs once (max_attempts1), so a session is never retried or renewed. The API answers409when that node’s agent has no session support yet (it does not reportsession_lane).plopens the relay with the session’s one-time token.- The node’s agent leases the job in a lane of its own (
session). An open tunnel or shell never holds up deployments, database work or name updates in the other execution lanes. The lane runs up to eight sessions at once per node. The agent checks the target and opens its own side of the relay with its node certificate and the lease. It renews the lease while the session is open. The relay pairs the two sides and copies bytes both ways. - On the node, the agent connects to the target. For each connection
plopens, it makes a TCP connection to the container’s address on the environment bridge at the session’s port. Forexec, it runsdocker execin the container.
pl and the agent exchange small frames over the relay: a kind, a channel and a length. They carry each tunnel connection on its own channel, and for exec the standard input, output and error, the terminal size and the exit code. The relay never reads them.
Relay placement
Decided 2026-11-04 while building it: in the agent gateway process, not in the API and not as a new service.
- The two sides must meet in one process to be paired in memory. The agent side already has its trust model in the gateway. That model is mutual TLS with the node certificate, checked against the database on each request, and the job lease. The API authenticates people and tokens, never nodes.
plcannot reach the gateway’s agent listener: it is on the private network only and requires a client certificate. The gateway gets a third listener for theplside, plain HTTP on loopback (PUSHLEFT_SESSION_RELAY_BIND,127.0.0.1:8449in the examples). Only the control-plane TLS proxy reaches it, ashttps://api.<base>/v1/sessions/{id}/relay. The registry token realm uses the same pattern: the proxy also sends it to the gateway. The gateway refuses a non-loopback address for it.- The
plside’s only credential is the session’s token: 256 random bits, issued by the API in thePOSTanswer. The API stores only its SHA-256. The token works once, within two minutes of issue, and only while the session has not ended. Any other token, a second use, an expired or a revoked session is401. - The agent side is
GET /v1/agent/jobs/{id}/relayon the mTLS listener with the headerx-pushleft-lease-token. It needs the current lease of that node’sopen_sessionjob for a session that has not ended; otherwise409. - Both sides are HTTP/1.1 upgrades (
Upgrade: pushleft-session). The relay is then a byte copy, and no new library is needed: axum, hyper and reqwest already do upgrades. The TLS proxy passes an upgrade through as it does a WebSocket. - A browser cannot send that upgrade, so the
plside also takes a WebSocket (decided 2026-11-06, for the console’s shell). A browser cannot setAuthorizationon a WebSocket, and a token in the URL would reach access logs. The token is therefore the second subprotocol:Sec-WebSocket-Protocol: pushleft-session, pls_…. The relay answers withpushleft-session. The frames travel in binary messages, in any chunking, and the relay turns them back into the byte stream the agent reads. The console’s host routes/v1/sessions/*/relayto the relay, as the API’s host does. - The first side waits up to 60 seconds for the other. A session whose agent does not join in that time ends as
not_connected.
Reachable addresses
The target is fixed when the API creates the session, and checked three times. Nothing in the byte stream names an address, a port or a container.
- API. The database, role or instance must belong to the caller’s organization; otherwise
404. A role must be long-running (web,privateorworker) with a ready instance in a live release. A database must bereadyordegraded. The task names one container (pushleft-db-<runtime name>, orpl-app-<instance>-g<generation>), the environment’s bridge (pl_<segment>) and, for a tunnel, one port.execis refused for databases. - Gateway, at lease. The session must be live and issued less than two minutes ago. The bridge must be the environment’s. The target must still be on this node: the database’s active runtime, or the instance at that deployment and generation, ready. Otherwise the job fails and the session ends as
not_connected. A customer-owned node leases only its own organization’s sessions, as for every job. - Agent. The container must be running and carry the Pushleft labels of the target. For applications, the labels are managed, the organization, the environment, deployment, instance and generation. For databases, they are managed, the organization, the database runtime and bridge. Its address is the one Docker gives it on the task’s bridge, and it must be inside the environment networks (
10.128.0.0/9). A tunnel dials only that address at the task’s port. The host reaches it the way the node’s ingress reaches its upstreams.execruns through Docker’s API in that container only.
Session lifetime
| Reason | When |
|---|---|
client_closed | pl closed its side (Ctrl-C, or the command ended). |
agent_closed | The agent closed its side, for example after the command exited. |
idle | 30 minutes without traffic in either direction. |
lease_lost | The agent’s lease of the job is no longer current. |
revoked | An admin revoked the session. |
max_duration | Four hours after the session was created: the lease protocol’s cap for one attempt. A new session is a new job, checked and recorded again. |
not_connected | The agent did not join within 60 seconds, or the lease was refused (see above). |
access_removed | The creator’s role no longer allows the session (Who). |
The relay checks the conditions every five seconds and records the reason before it closes either side.
A relay that stops before it records an end leaves the session open, for example when the agent gateway restarts. The scheduler closes such a session within a minute or two:
| Reason | When the scheduler records it |
|---|---|
not_connected | Nobody joined within five minutes. |
lease_lost | The agent’s lease of the job is gone. |
max_duration | Four hours and five minutes after the session was created. |
The tailnet alternative
The Tailscale client would have to trust the coordinator’s internal certificate authority. The environment network admits every member to every service. A coordinator that moves starts with fresh state, so every joined machine would have to enroll again. pl is Rust, and there is no embedded Tailscale client for it. A long-lived device that joins with per-user access rules stays a later option.
Limits
- Traffic goes through the control plane, so the tunnels suit administration and debugging. Use
pl dbbackups and imports for bulk transfers. - One gateway process. Both sides must reach the same gateway process. With more than one gateway behind the proxy, the relay needs sticky routing by session or pairing through the database (gap).
- A gateway restart ends every open session (its 30-second drain).
- A tunnel’s address is taken when the session starts. If the instance is replaced meanwhile, new connections fail; open a new session.
- The data path through a tunnel is tested on Linux only; Docker Desktop does not route the host to container addresses.
Rollout (operators)
Deploy in this order:
- Migration
0130(the API runs it at start). Then run theGRANT ... ON ALL TABLESstatements again (database roles), so the API and the gateway can usedeveloper_sessions. From migration0142, the API also needsGRANT EXECUTE ON FUNCTION node_runs_sessions(uuid) TO pushleft_api. - Gateway. Set
PUSHLEFT_SESSION_RELAY_BIND=127.0.0.1:8449and add the proxy route for/v1/sessions/*/relayon the API host (deploy/install-control-plane.shdoes both). The gateway must know thesessionlane before an agent asks for it. An older gateway answers400to that lane, and the agent logs the error every five seconds. - API. Until agents report
session_lane, it answers409for every session. - Agents. The general lane then reports
session_lane. An agent from before the lane asks forgeneraland never sees a session job. Only agents from before the durability lane (allorworkload, before migration0095) would lease one and fail it as an unsupported kind. The API never targets them, because they do not reportsession_lane. - CLI. A
plwithdb connect,tunnelandexec.pl cpneeds only a newpl. - Console shell. The gateway with WebSocket support, the console’s Caddy route for
/v1/sessions/*/relay(deploy/install-control-plane.sh), then the console build. - Sessions list. The API with
GET /v1/sessions, then the console build. No migration.
Tests
- Unit: the task’s structure and refusals (
pushleft-coresession::tests: no exec into a database, no other bridge, unknown fields refused), the frames, and a tunnel carrying several connections to its one target. - Unit: the agent’s target check (
pushleft-agentsession::tests: another organization’s, environment’s or generation’s container, a stopped one, and addresses outside the environment networks are refused). - Against migrated PostgreSQL, the API (
sessions_tests: viewers, read-only and service tokens refused; a Member opensexec, a tunnel and a database in a preview environment and gets403for each in production, and reads only the sessions they opened; another organization’s database, role or instance404; a node without the lane409; the session, job, lane and audit; revoke; the list, open first, and only for admins). - Against migrated PostgreSQL, the gateway (
sessions_tests: the job leases only in thesessionlane; the relay refuses another node, a wrong lease and a revoked session; a stale session or a moved target fails at lease; a customer-owned node of another organization never sees it; the token works once, only within two minutes, never for another session or a revoked one; a session ends withaccess_removedwhen its creator’s role no longer allows it). - End to end:
scripts/developer-access-e2e.shruns the API, the gateway with its relay listener, the real agent andplagainst a labelled container on an environment bridge. It checkspl execoutput, errors, exit code, standard input and a terminal;pl cpof a directory and a file both ways, and a missing path failing; the console’s shell code (src/shell.js, throughscripts/console-shell-check.mjs) joining as a WebSocket, with standard input, output and the exit code; a spent token refused;pl tunnel, with bytes through it on Linux; a revoke ending a running tunnel, reported bypl. Passed on macOS (Docker Desktop) on 2026-11-04, without the tunnel’s data path. Withpl cp, passed on macOS on 2026-11-06.