pilotlight

pilotlight

A small, fast process manager for the machines you actually ssh into.

Describe your processes in one TOML file. pilotlight start hands them to a supervisor daemon that keeps them alive, captures their output, and stays running after you close the terminal.

curl -fsSL https://pilotlight.sh/install.sh | sh

[ install ] [ source ]

Pre-release. No version is tagged yet, so the install commands on this page will not resolve until v0.1.0 ships.

~/srv/app 
$ pilotlight start
  ✔ migrate    exited 0 (312ms)
  ✔ api        4 instances  pid 4412 4413 4414 4415
  ✔ worker     2 instances  pid 4416 4417
  ✔ caddy      1 instance   pid 4418
 
$ pilotlight status
  NAME     ID  PID   STATUS   ↺  CPU   MEM      UPTIME
  api      0   4412  running  0  0.4%  48.2 MB  2h 14m
  api      1   4413  running  0  0.3%  47.9 MB  2h 14m
  worker   0   4416  running  0  1.1%  61.0 MB  2h 14m
  caddy    0   4418  running  0  0.1%  12.3 MB  2h 14m
 
$ exit  # your processes do not care

What it replaces

Every long-lived process on a box eventually gets the same three-part treatment: a nohup, a redirect, and a sticky note about which pid was which. pilotlight is the file you write instead.

before 
$ nohup node server.js > out.log 2>&1 &
[1] 4411
$ nohup node worker.js > wrk.log 2>&1 &
[2] 4416
$ # ...four hours later
$ ps aux | grep node
cmoreside 4416  0.0  node worker.js
$ # where did 4411 go?
   and nothing restarted it.
after 
$ pilotlight start
  ✔ api        4 instances
  ✔ worker     2 instances
$ # ...four hours later
$ pilotlight status
  api     0  4412  running   0  2h 14m
  api     1  4413  running   0  2h 14m
  worker  0  4416  running   1  6m 02s
   worker.0 crashed once and came back.

Features

  • One file, whole stack

    A pilotlight.toml committed next to your code. Reviewed like code, identical on your laptop and on the box.

  • Outlives your shell

    A daemon owns the processes. Closing the terminal — or losing the ssh connection — changes nothing.

  • Instances are a number

    instances = 4 runs four copies. {instance} templating gives each one its own port.

  • Restarts have a policy

    Exponential backoff, a restart budget inside a rolling window, and a process parked as errored instead of crash-looping forever.

  • Logs are captured

    Per-instance files on disk, plus a merged logs -f that prefixes and colorizes each source.

  • Rolling reload

    reload restarts one instance at a time, so the rest of the pool keeps serving traffic.

  • Starts at boot

    pilotlight startup emits a launchd agent or a systemd user unit. Neither needs root.

  • Scriptable, and one binary

    Every command takes --json. Nothing to install first — no runtime, no package manager.

Configuration

Everything pilotlight does is driven by one file. It looks for pilotlight.toml in the current directory and walks up parent directories until it finds one.

pilotlight.toml 
# Inherited by every process below.
[defaults]
restart       = "on-failure"
restart_delay = "1s"   # doubles each retry
max_restarts  = 10
stop_signal   = "SIGTERM"
stop_timeout  = "10s" # then SIGKILL

# Runs to completion before anything that
# depends_on it is started.
[[process]]
name    = "migrate"
command = "./bin/migrate up"
oneshot = true
restart = "never"

[[process]]
name       = "api"
command    = "node dist/server.js"
cwd        = "./server"
instances  = 4
depends_on = ["migrate"]
env_file   = [".env", ".env.production"]

[process.env]
NODE_ENV = "production"
PORT     = "300{instance}"  # 3000..3003

Instance templating

Inside [process.env], {instance} expands to the zero-based index of each copy. It is the intended way to hand every instance its own port without repeating yourself.

Every child also gets these, whether you use templating or not:

PILOTLIGHT_PROCESS_NAME
api
PILOTLIGHT_INSTANCE_ID
2
PILOTLIGHT_INSTANCES
4

Start ordering

depends_on holds back a process until its dependencies are up — or, for a oneshot task like a migration, until it has exited cleanly. Dependency cycles are rejected by pilotlight config check rather than at runtime.

Merged logs

$ pilotlight logs -f
  api.0    | GET /healthz 200 1.2ms
  worker.1 | job 8812 complete
  caddy.0  | serving on :443
  api.2    | GET /v1/orders 200 8.4ms

stdout and stderr are captured to separate files per instance under ~/.pilotlight/logs. logs merges them back together.

Install

brew install cmoresid/tap/pilotlight

Works on macOS and Linuxbrew. Upgrades come through brew upgrade like everything else.

Then

~/srv/app 
$ pilotlight init          # writes a starter pilotlight.toml
$ $EDITOR pilotlight.toml
$ pilotlight start
$ pilotlight status

Commands

CommandWhat it does
pilotlight initWrite a starter pilotlight.toml into the current directory
pilotlight start [NAME…]Start everything, or only the named processes
pilotlight stop [NAME…]Send stop_signal, then SIGKILL after stop_timeout
pilotlight restart [NAME…]Stop, then start
pilotlight reload [NAME…]Rolling restart — one instance at a time
pilotlight statusName, id, pid, status, restarts, cpu, memory, uptime
pilotlight logs [NAME] -fTail captured output, prefixed and colorized per process
pilotlight scale NAME=NChange instance count without editing the config
pilotlight config checkValidate the config and print it fully resolved
pilotlight daemon start|stop|statusManage the supervisor daemon directly
pilotlight startupPrint — or install — a launchd or systemd unit for boot start

Global flags: -c, --config <PATH> · --json · --no-color · -v, --verbose. Full reference in the README.