fix: allow graceful runner shutdown during stack maintenance
This commit is contained in:
@@ -52,19 +52,58 @@ It uses `actions-rust-lang/setup-rust-toolchain` which caches `~/.cargo` and
|
||||
|
||||
## Manage
|
||||
|
||||
```bash
|
||||
docker compose up -d # apply config changes
|
||||
docker compose down # stop (registration persists in volume)
|
||||
```
|
||||
|
||||
⚠️ **Check for running jobs before recreating the runner.** Recreating it orphans
|
||||
any job in flight — Gitea marks them failed and you have to re-run:
|
||||
The wrapper operates only this project's Compose stack:
|
||||
|
||||
```bash
|
||||
docker exec gitea-runner-dind docker ps --format '{{.Names}}' | grep -v buildkit
|
||||
./runner-stack.sh up -d # start the stack
|
||||
./runner-stack.sh stop # stop runner before DinD; retain containers and volumes
|
||||
./runner-stack.sh down # remove containers; retain registration and caches
|
||||
```
|
||||
|
||||
Wait until that prints nothing (`buildx_buildkit_builder-*` is a helper, not a job).
|
||||
To apply an update on another existing runner, first wait for jobs to finish:
|
||||
|
||||
```bash
|
||||
docker exec gitea-runner-dind docker ps --format '{{.Names}}'
|
||||
```
|
||||
|
||||
Only persistent BuildKit helpers should remain. Then run:
|
||||
|
||||
```bash
|
||||
git pull --ff-only
|
||||
./runner-stack.sh stop
|
||||
./runner-stack.sh up -d --force-recreate
|
||||
```
|
||||
|
||||
Recreation loads changes in the bind-mounted runner YAML as well as Compose
|
||||
settings. It retains registration and cache volumes. Keep that machine's `.env`
|
||||
and intentional local capacity/label settings; resolve any pull conflicts while
|
||||
retaining `runner.shutdown_timeout: 5m`. A new installation still requires the
|
||||
per-machine registration setup above. Running the wrapper without arguments
|
||||
prints usage and does not change the stack.
|
||||
|
||||
The runner waits up to five minutes for jobs to finish before cancelling them.
|
||||
Compose allows six minutes before SIGKILL, leaving additional time for cleanup.
|
||||
This ordering applies to Compose operations, not power loss or independently
|
||||
stopping DinD. Any other clients sharing the daemon must be stopped separately
|
||||
before stack maintenance.
|
||||
|
||||
For changes only to `config.yaml`, restart the runner to load the file:
|
||||
|
||||
```bash
|
||||
./runner-stack.sh restart runner
|
||||
```
|
||||
|
||||
### Network allocation
|
||||
|
||||
DinD allocates /24s from `10.210.0.0/16`: 256 subnet slots, including one for
|
||||
its default bridge. Keep this setting when updating the daemon, and verify
|
||||
that this range does not overlap routes in another machine's environment.
|
||||
Host Docker has separate allocation state.
|
||||
|
||||
The larger pool provides headroom; interrupted jobs can still leave networks
|
||||
behind. Workflow-created networks need their own teardown on success, failure
|
||||
and cancellation. Inspect leftover networks inside DinD during idle maintenance;
|
||||
do not schedule blanket pruning while jobs are creating their stacks.
|
||||
|
||||
### Housekeeping
|
||||
|
||||
|
||||
Reference in New Issue
Block a user