Boot order
shep starts a flock all at once unless you tell it otherwise. Add depends_on to an app and it sorts the flock into stages instead, holding each one until the members something later waits on are up. An app that would have crash-looped through its restart budget waiting for a database waits for the database instead.
Naming a dependency
depends_on is a list of other sheep, or of dogs. It goes in the app's own Flockfile, because a requirement the app has belongs in the app's own repository.
[[app]] name = "db" script = "./db" [[app]] name = "api" script = "./api" depends_on = ["db"] [[app]] name = "web" script = "./web" depends_on = ["api"]
That sorts into three stages, db, then api, then web. Names inside a stage are sorted, so one flock always plans the same way. The whole start took 6.17 seconds, and the UPTIME column is where it went:
$ shep start Flockfile.toml ID NAME STATUS PID RESTARTS EXIT CFG CPU MEM UPTIME FOLD SMIT 1 api online 21899 0 - - - 1.2M 3s - - 0 db online 21886 0 - - - 1.2M 6s - - 2 web online 21907 0 - - - 1.2M 0s - -
Names only. web:2 is refused with the app-level form named, since a dependency on one instance of a load-balanced app is not a claim about availability. A dependency on a multi-instance app waits for every instance of it.
What ready means, and what it costs
A stage advances when its members have settled. An app with wait_ready or a readiness_probe settles the moment it says so, and an app that exits settles at once rather than holding its stage for the full wait.
An app with neither has nothing to report. shep holds it starting for its own listen_timeout, three seconds by default, then calls it up and moves on. That is where the 6.17 seconds went: db cost one wait and api cost another. web is last, so nothing waits on it and it costs nothing. Only apps something depends on pay this.
Configuring a real signal takes it back off. Two apps, the database behind a TCP probe polled every hundred milliseconds, and the start finishes in 0.14 seconds:
[[app]]
name = "db"
script = "./db"
readiness_probe = { kind = "tcp", target = "127.0.0.1:5432", interval = "100ms" }
[[app]]
name = "api"
script = "./api"
depends_on = ["db"]$ shep start probed.toml ID NAME STATUS PID RESTARTS EXIT CFG CPU MEM UPTIME FOLD SMIT 4 api online 21994 0 - - - 1.2M 0s - - 3 db online 21993 0 - - - 9.5M 0s - -
interval is doing real work. A probe polls every ten seconds by default and the first poll runs at spawn, so an app needing fifty milliseconds to bind its port fails that first poll and has nothing left to answer with before listen_timeout elapses. Measured on the flock above: 0.14 seconds at an interval of 100ms, and the full 3.05 at the default. Set an interval shorter than the app's own startup time or the probe buys you nothing at boot.Dogs run last, unless you move one
A dog is a node in the same graph, and by default it lands in a final stage after every sheep. That is the right place for the metrics dog: it must not answer for a flock that is not up yet.
A log-rotation dog is the opposite case, since it has to be running before a sheep starts writing. Name it in [daemon] boot_first_dogs and it runs ahead of everything:
[daemon] enabled_dogs = ["metrics", "log-rotate"] boot_first_dogs = ["log-rotate"]
log-rotate promoted, so it spawns before the restore
stage 0 db
stage 1 api
stage 2 web
metrics spawns after every stageA name here that is not in enabled_dogs is inert: nothing enables a dog by promoting it. A sheep may also name a dog in its own depends_on, but at a boot that does not move the dog. The shepherd spawns dogs in two groups, the promoted ones before the restore and the rest after it, and neither group sits at a stage boundary. A sheep depending on an unpromoted dog is warned about and started anyway.
When the graph is wrong
What happens depends on who is watching. An operator typing shep start or shep add is there to fix a typo. A machine rebooting at 4am is not, and stranding its whole flock over one is worse than starting it in a bad order.
Every "warned" above is a line in the shepherd's own log, not a notice on the reply, so shep start and shep add print nothing about the graph and exit 0. They are in $SHEP_HOME/logs/shepd.err.log. The autostart = false warning is narrower still: it lives on the boot path, so a typed start does not report that case at all.
The cycle refusal names the actual cycle, not the set of apps caught in one, because a path is the thing you can break:
$ shep start cycle.toml error[invalid_config]: the daemon reported InvalidConfig: dependency cycle: api -> db -> cache -> api
The graph it checks spans the batch and the registered flock together, so a cycle can close through a sheep the file never mentions. Only a cycle this batch is part of refuses it; a knot standing elsewhere in the registry is left to whichever request drew it.
An unknown name warns rather than refusing on both sides. A dependency on an app whose Flockfile lives in another repository is legitimate, and refusing it would make cross-repository dependencies unusable.
autostart wins. A dependency that sets autostart = false stays stopped, and the sheep that depends on it starts in its stage as though the edge were satisfied. A boot writes a warning naming both to the shepherd's log; a typed shep start writes nothing. depends_on orders what is already being started; it does not decide what gets started. An explicit statement in one app's own config is not overridden by a field in another app's file.Shutdown runs backwards
A shepherd going down walks the stages in reverse, so web gets its SIGTERM, then api, then db. A worker draining its queue drains against a database that is still answering. Without ordering both get signalled in the same millisecond.
A stage's members overlap, so a stage costs the longest kill_timeout in it rather than the sum, 1.6 seconds by default. Five stages is eight seconds against systemd's default TimeoutStopSec of ninety, so shep startup's unit needs nothing extra.
Dogs are not in the reverse stages. They stop after every sheep, because monitoring should outlive what it monitors and a strict reverse would kill the bark dog before the flock it reports on.
listen_timeout clears systemd's default TimeoutStartSec of ninety seconds, and three or four do at a listen_timeout of half a minute. Past it systemd kills the shepherd mid-restore and starts it again on RestartSec, which is a loop rather than a slow boot. A flock that deep wants a TimeoutStartSec of its own in the [Service] section; shep startup writes no value, so the default applies.shep restart and shep reload against a selector matching several sheep walk the stages forward instead, dependencies first. Not reverse-stop then forward-start: that sounds more correct and behaves worse, since it puts the whole fold down at once in the middle.
Both verbs ask the shepherd once per app, so an app already being reloaded, or one that left the flock after the walk was planned, is refused on its own and the rest of the fold goes ahead. shep reload and shep restart name the app they went around on stderr and exit non-zero, and under --format json the same names come back under refused in the one envelope: a row missing from the table is not something a deploy script can read.
A restart that came back errored is a different failure and is reported separately, with its own exit code. That sheep was reached and the restart was not refused; it is the child that could not start. It also empties stdout, as every verb's failure does, so a restart that hits both at once prints no envelope: the refused names are appended to that error's own sentence on stderr instead, and the refused key above is what a run reports when nothing came back errored.
What a staged start leaves behind
Checks shep can make in advance still refuse a batch before anything is registered. Once spawning has begun there is nothing to make them against: stage 0 is running by the time stage 1 proves unstartable, and none of it is rolled back, because rolling back means stopping apps that came up fine on a guess about what you wanted. The refusal names them instead:
$ shep start fail.toml error[spawn_failed]: the daemon reported SpawnFailed: nothing in this batch was registered; 1 of 1 apps cannot start: api: no such file: /private/tmp/shepdocs-boot/app/./not-a-real-binary (this start already has children running and leaves them alone: db)
shep stop db if you want it down. Nothing after the failing stage is attempted in that same invocation.
Two things this does not do
Ordering applies to a batch. shep start api on its own starts api, immediately, whatever it depends on, because there is nothing else in that request to order it against.
Promoting a dog fixes the boot window and only the boot window. A shep start web typed at noon, while log-rotate happens to be down, is a supervision problem. Boot ordering has no opinion about it.
Reading it back
shep describe lists a sheep's dependencies, which is what answers "why did web start six seconds in":
$ shep describe web ID NAME STATUS PID RESTARTS EXIT CFG CPU MEM UPTIME FOLD SMIT 2 web online 21907 0 - - - 1.2M 0s - - Depends on for web: api
There is no shep flock column for it. The field names other sheep rather than saying anything about this row's own status, and that table already drops columns under pressure. Under --format json every sheep carries a depends_on array.