Scheduled jobs
A scheduled job runs a script from your repository on a schedule, in a throwaway VM that can run Docker. Pushleft sends mail when the script fails. A run that succeeds sends no mail.
Scheduled jobs and cron roles
A cron role and a scheduled job both run on a five-field UTC schedule. Neither starts a second run while one is active. After an outage, both start one run for the latest tick.
| Need | cron role | Scheduled job |
|---|---|---|
| Code | The image of the live release | A checkout of the production branch at one commit |
| Docker | No | Yes |
| Time limit | 1 hour | Your timeout, up to 6 hours |
| Files between runs | No | The paths in cache: |
| Failure report | The last lines, on the deployment’s events | Mail with the failed lanes and tests |
| Runs on | The application nodes | The workspace pool |
Use a cron role for a small task inside a release, such as a cleanup query. Use a scheduled job for a test suite, a nightly gate or other work that needs Docker or a long run.
pl schedules list shows only scheduled jobs. To see your cron roles, use pl cron list:
pl cron list --production
It prints one line per cron role: its schedule, its next run, and the start and result of its last run. Use --environment <slug> for another environment. pl logs <deployment-id> shows the output of each run.
The manifest
Add a schedules: block to .pl.yaml. A project can have 20 schedules. The key is the schedule’s name, a DNS-safe slug.
schedules:
nightly-gate:
cron: "0 1 * * *"
branch: main
run: GLUON_RUN_GATE=1 scripts/push
runner: { cpu: 14, memory: 32GiB, disk: 200GiB }
timeout: 3h
cache: [~/.cargo, target, ~/.local/share/pnpm, ~/.cache/ms-playwright]
environment: ci
secrets: [ANTHROPIC_API_KEY]
report:
to: [owners]
summary: reports/gate-summary.json
| Field | Required | Meaning |
|---|---|---|
cron | Yes | Five fields, UTC. |
branch | Yes | The production branch of the project. |
run | Yes | A bash command of up to 4096 bytes. It starts in the checkout. |
runner.cpu | Yes | 1 to 64 vCPUs. |
runner.memory | Yes | 1GiB to 512GiB, in whole GiB. |
runner.disk | Yes | 10GiB to 2048GiB, in whole GiB. |
timeout | Yes | 1m to 6h, written as 90s, 45m or 3h. |
cache | No | Up to 16 paths that stay between runs. A path starts with ~/ or is relative to the checkout. See the cache. |
environment | With secrets | The environment that holds the secrets. |
secrets | No | Names of the secrets the job receives. |
report.to | No | [owners], the default and the only value. |
report.summary | No | The path of the summary file, relative to the checkout. |
pl plan checks this block. When you are signed in, it also checks the branch, the environment and the secrets on the server.
The production branch
A schedule runs only on the production branch. The job runs code from its branch, so anyone who can push to that branch can read the job’s secrets.
A hosted repository or a repository with a webhook names its production branch. For any other repository, the production branch is the branch of the production deployment. An admin who runs pl up --production from another branch moves it.
At each tick, Pushleft reads the head commit of the branch and records that commit on the run. This works when deploy_on_push is off.
Changes to a schedule
A production deploy checks its schedules. An invalid schedule refuses the deploy. The schedules take effect when the production release goes live. A release that fails leaves the old schedules in place.
Repository access
Pushleft reads the repository when each run starts: the head of the branch, as an archive. A production deploy with schedules needs a read token for the repository’s host:
pl git credential add <host> --provider <github|gitlab> --token-stdin
A push webhook is not enough. Deploy from a checkout with an origin remote, so that Pushleft knows the repository.
The checkout is the git host’s archive of that commit: every tracked file, symlinks included, and nothing that git does not track. It is at most 2 GiB compressed.
The runner
Each run gets one Kata VM with the runner size. Pushleft destroys the VM when the run ends.
- The script runs as root in the VM.
- Docker is available. The script starts its own databases and other services, and they end with the VM.
- Rust, C and C++, Node.js and pnpm are installed.
- The VM reaches the public internet only.
Secrets and the environment
A job receives only the secrets it lists, as environment variables. They come from the one environment in its environment field. A job gets production secrets only when the field says production.
pl plan and the API refuse these:
secretswith noenvironment;- an environment that does not exist;
- a secret that the environment does not hold.
The cache
cache: is opt-in. Without it, a run gets one work disk, and nothing stays after the run.
With cache:, the listed paths stay between runs:
- The volume takes half of
runner.disk. The work disk gets the rest. - One volume exists per schedule and branch on a node. Stale files are the job’s concern.
- The volume stays on the node that made it. The scheduler picks that node first when the node qualifies.
- When that node doesn’t qualify, the run goes to another node with an empty cache. The run is slower, and the console marks it
Cold cache. A later run uses the newest volume. - A volume unused for 30 days is deleted.
pl schedules cache-clear <name>deletes the volumes of a schedule. The next run starts empty.
Cost
A run uses the workspace prices of your plan. CPU and memory count only during a run.
A kept cache volume is billed for the whole month as reserved disk. On your own server, runs and cache cost nothing. A budget with hard_stop stops new runs.
The summary file
The script can write this JSON file at the path in report.summary:
{
"lanes": [
{ "name": "rust", "status": "passed", "failed_tests": [] },
{ "name": "browser", "status": "failed", "failed_tests": ["checkout pays with a saved card"] }
]
}
Pushleft keeps 100 lanes, 200 failed tests per lane and 200 characters per text value. It ignores other fields. It doesn’t read the file after a timeout.
Mail goes to the owners of the organization. It links to the run in the console.
| Result | |
|---|---|
failed | Yes |
timed_out | Yes |
no_capacity | Yes |
succeeded | No |
skipped, the first in a row | No |
skipped, the second in a row | One mail, because the job runs longer than its schedule |
With a summary file, the mail lists the failed lanes and tests. Without it, the mail shows the exit code and the last 50 lines. Each mail names the commit and the duration of the run.
Run results
| Status | Meaning |
|---|---|
pending | The run waits for a node. Pushleft gets the commit when the run starts. |
waiting | No node has room yet. The scheduler checks again every minute. |
running | The script runs. |
succeeded | The script exited with 0. |
failed | The script exited with another code, or the run could not start. |
timed_out | The run reached timeout, and Pushleft stopped the VM. |
no_capacity | No node had room for 1 hour. |
skipped | The previous run was still active. Runs never queue. |
cancelled | Someone cancelled the run. No mail is sent. |
To stop a run that is stuck or no longer needed, select Cancel run on its page in the console, or run pl schedules cancel <name>. Members and above can cancel. The run stops within a minute: Pushleft removes its VM and keeps the cache. A run whose VM stops by itself ends as failed within 30 seconds.
Output and limits
- A run can store 200 MiB of output. Past that limit, the log keeps the first and last 100 MiB and marks the cut.
- Output is stored with your tier’s log retention of 7, 14 or 30 days.
- Output doesn’t count against the daily output limit of services.
- The Deployments page and
pl logsdon’t list scheduled runs.
The Schedules page of the console lists each schedule and its runs. The page of one run shows the lanes, the failed tests and the log. You can search the whole log and download it.
Commands
Run these in the directory that holds .pl.yaml.
| Command | Result |
|---|---|
pl schedules list | Each schedule, its next run and its last result |
pl cron list [--production | --environment <slug>] | Each cron role of the environment, its next run and its last result |
pl schedules run <name> | Starts a run now. Viewers can’t. A schedule with an active run refuses it |
pl schedules logs <name> [--run <id>] [-f] | The output of the newest run, or of the run you name. -f keeps reading while the run is active |
pl schedules cache-clear <name> | Deletes the cache volumes of the schedule |
pl schedules cancel <name> [--run <id>] | Stops the active run, or the run you name. Viewers can’t. No mail is sent |
Hosts
The scheduler picks the host when each run starts. No manifest or setting names a host. Only nodes of the workspace pool in your organization qualify. Application and database nodes never run a job.
A node qualifies when these are true:
- its free capacity fits the
runnersize; - its heartbeat is fresh;
- its CPU and memory use over the last 15 minutes leaves room for the runner.
Among those nodes, the scheduler takes the one with the most free CPU.
To use your own server:
- Prepare a workspace host. The host needs KVM. Enrollment for the
workspacepool stops without/dev/kvm. - Set
PUSHLEFT_RUNNER_IMAGEin/etc/pushleft/agent.envto the runner image, and restart the agent.