Stopping and replacing a sheep
Four verbs end a sheep's current process: stop, restart, reload and delete. All four take the same selector grammar as everything else (a name, an id, fold:<name>, all, and the rest are on the folds page), and three of them run through the exact same kill ladder underneath. What they don't share is what they leave behind: a stopped sheep is still in the flock, a deleted one is gone, and a reload can end either as a clean handover or as a crash you get to watch happen. That is what this page is for.
stopkilled, through the ladder belowstays, entry marked stoppedrestartkilled, then a fresh one at oncestays, marked online againreloadone instance replaced, old or new order belowunchanged, still the same appdeletekilled if running, through the same ladderremoved, for goodshep thatlldo is a hidden, working alias for stop, "graceful stop" in Getting started's own alias grid. It isn't in --help, but it runs the same code as the row above.
One ladder, two grace periods
stop, restart and delete all end a live process the same way. First, either the app's configured kill_signal goes to the sheep's whole process group (lambs included), or, if the app set shutdown_with_message, a {"kind":"shutdown"} message goes over the shepherd channel instead, no signal at all. Then shep waits. If the process is still alive when the grace period runs out, that same process group gets SIGKILL, a signal nothing can trap, and that's the end of it either way.
The grace period is where the two verbs that also drive a reload diverge from the two that don't. stop, restart and delete wait kill_timeout, 1.6 seconds by default. A reload's drain of the OLD instance waits graceful_timeout instead, 8 seconds by default and configured separately, because a reload's whole premise is that the instance being replaced might be mid-request and deserves more time to finish than an ordinary stop does. See the field table for both, and Examples for a live run watching a sheep that ignores its stop signal get escalated to SIGKILL.
A signal stop sends is never mistaken for something else afterward. Whatever the app's own autorestart says, an exit that ends a manual stop is always recorded as a clean stop, never as something worth restarting for. That's also why an operator's command always wins a race against the daemon's own: a cron occurrence or a watched file landing on a sheep mid-kill-ladder gets dropped rather than reviving it out from under you.
On Windows there is nothing SIGTERM-shaped to send, so stop waits out the whole of kill_timeout with no signal delivered and then terminates the process directly, unless the app opted into shutdown_with_message. See What's not built yet.
What stays, what goes: the muster roll
shep keeps a file at $SHEP_HOME/flock.json, the muster roll, recording every registered app's config and whether it was running the last time the roll was written. A shepherd that starts back up reads it and restores what was up. Surviving reboots is the full account of save and muster; this section is only the one thing that page doesn't say: what stop leaves in that file, and what delete takes out of it.
shep stop <selector>...Runs the kill ladder against every matched sheep and marks it stopped. The sheep is not removed: it stays a member of the flock, still shows up in shep flock, and is still there for the next restart or delete to find.
$ shep stop parked ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79454 0 - - 1.4M 6s - - 1 parked stopped - 0 SIGTERM - - 0s - - 4 probed online 79458 0 - - 1.5M 6s - - 2 web online 79456 0 - - 1.4M 6s - - 3 worker online 79457 0 - - 1.4M 6s - - $ shep save FILE APPS /tmp/shepdocs-lifecycle/home/flock.json 5 $ shep kill PID SOCKET_REMOVED 79433 true $ shep muster ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 1 keeper online 79631 0 - - - 0s - - 0 parked stopped - 0 - - - 0s - - 2 probed starting 79632 0 - - - 0s - - 3 web online 79633 0 - - - 0s - - 4 worker online 79634 0 - - - 0s - -
parked was stopped, then the roll was saved, then the whole shepherd went down and came back through muster. It's back, registered, still called parked, still stopped. Nothing restarted it, because it wasn't running when the roll was written and muster's rule is "was it up when we saved," not "does it exist."
shep delete <selector>...Runs the same kill ladder if the sheep is running, then removes the registration entirely. In table format the listing that follows is the whole remaining flock, the same as every other lifecycle verb; the ids it removed are named on stderr because their own rows are gone by the time anything could print them.
$ shep delete parked notice[delete]: deleted 1 sheep, id 0 ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 1 keeper online 79631 0 - - 1.4M 5s - - 2 probed online 79632 0 - - 1.5M 5s - - 3 web online 79633 0 - - 1.4M 5s - - 4 worker online 79634 0 - - 1.4M 5s - - $ shep save FILE APPS /tmp/shepdocs-lifecycle/home/flock.json 4 $ shep kill PID SOCKET_REMOVED 79610 true $ shep muster ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79791 0 - - - 0s - - 1 probed starting 79792 0 - - - 0s - - 2 web online 79793 0 - - - 0s - - 3 worker online 79794 0 - - - 0s - -
Same sheep, same save-kill-muster sequence, different verb. This time parked is not in the restored flock at all, not even as a stopped row. Deleting it took it out of the registry immediately, so it was never a candidate for the roll save wrote afterward, and muster can only restore what the roll remembers. There's no undelete: the config is gone unless you kept the Flockfile that produced it.
whistle's four control tools mirror three of these verbs (start_sheep, stop_sheep, restart_sheep, reload_sheep), deliberately with no delete_sheep. See whistle.
restart: back immediately
shep restart <selector>...Runs the kill ladder, then spawns a fresh process in the same slot at once, whatever autorestart or stop_exit_codes would otherwise decide. The restart budget is reset the same way a spec-mandated manual action always resets it, and the daemon records that a person asked for this one, which is what lets shep describe and the bus tell an operator's restart apart from one the daemon raised on its own.
$ shep restart keeper ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79867 1 - - 640.0K 0s - - 1 probed online 79792 0 - - 1.5M 5s - - 2 web online 79793 0 - - 1.4M 5s - - 3 worker online 79794 0 - - 1.4M 5s - -
keeper's pid changed and its RESTARTS column went from 0 to 1. Everything else in the flock is untouched: a lifecycle verb's listing always shows the whole flock, not only what it touched, which is why the other three rows are here too. See Terminal output.
reload: one instance at a time, in one of two orders
A reload overlaps the old instance and the new one by default: the replacement is spawned and confirmed before the old one drains, so the app is never short an instance. Whether that overlap is actually seamless is a property of the app, not of shep. shep never binds an app's listening socket and never sets SO_REUSEPORT on one; if the app hasn't set that option on its own listener, the second instance has nowhere to go but EADDRINUSE, the moment it tries to bind the port the first one is still holding.
shep reload <selector>...Replaces each matched sheep's instances one at a time with a fresh process, so a clustered app is never short more than one instance mid-swap. Answers as soon as the shepherd accepts the reload, printing the flock as it stands at that moment; the swaps themselves land afterward and are reported on the bus.
Before anything is killed, shep has to decide the order, and it picks from the app's own config: an app with a readiness_probe and no reuse_port is reloaded serially, old instance fully gone before the new one starts. Every other app overlaps, new instance spawned and confirmed before the old one drains: that's an app with no probe at all, one using wait_ready (its channel belongs to one instance, so an outgoing instance has no way to answer on the replacement's behalf, and wait_ready wins over a configured probe if an app sets both), or a probed app that also sets reuse_port.
The reason is what a readiness_probe is actually asking. It polls an address, and an address can't say which of two processes answered it. Overlap two instances of a probed app and the outgoing one is still bound when the probe lands, so it answers for the incoming one and shep calls a release ready that may never have come up at all. Serializing removes the ambiguity by removing the overlap: there's only ever one instance to ask.
Overlap, done right
worker below has no probe, so it always overlaps. It writes to its own log file and binds nothing, so there's nothing for two instances to fight over:
$ shep reload worker ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79867 1 - - 1.4M 2s - - 1 probed online 79792 0 - - 1.5M 7s - - 2 web online 79793 0 - - 1.4M 7s - - 3 worker:0 stopping 79794 0 - - 1.4M 7s - - 4 worker:0 starting 79907 0 - - - 0s - - $ shep flock ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79867 1 - - 1.4M 4s - - 1 probed online 79792 0 - - 1.5M 10s - - 2 web online 79793 0 - - 1.4M 10s - - 4 worker online 79907 0 - - 1.4M 2s - -
reload's own printed table is the proof: it shows two rows for worker at once, id 3 stopping on the old pid and id 4 starting on the new one. Both are real, live processes for that moment, which is the entire point of overlapping. A moment later the drainee is gone and the replacement has taken the name back.
Overlap, when the app doesn't cooperate
web also has no probe, so it also overlaps, but it binds a port and does nothing to share it. shep never sets SO_REUSEPORT on an app's behalf; that has to happen inside the child, on a socket shep never sees. Without it, two instances cannot hold the same port at once:
$ shep reload web ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79867 1 - 0.0% 1.4M 10s - - 1 probed online 79792 0 - 0.0% 1.5M 15s - - 2 web:0 stopping 79793 0 - 0.0% 1.4M 15s - - 5 web:0 starting 80029 0 - - - 0s - - 4 worker online 79907 0 - 0.0% 1.4M 7s - - $ shep bleats web --no-follow --lines 4 web | http_server pid=79456 listening on 127.0.0.1:19293 web | http_server pid=79633 listening on 127.0.0.1:19293 web | http_server pid=79793 listening on 127.0.0.1:19293 web | web | thread 'main' (54715137) panicked at examples/src/bin/http_server.rs:31:33: web | could not bind 127.0.0.1:19293: Address already in use (os error 48) web | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace $ shep flock ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT 0 keeper online 79867 1 - 0.0% 1.4M 12s - - 1 probed online 79792 0 - 0.0% 1.5M 17s - - 2 web online 79793 0 - 0.0% 1.4M 17s - - 4 worker online 79907 0 - 0.0% 1.4M 10s - -
The replacement spawns, tries to bind the port the drainee is still holding, and panics with Address already in use. The reload is abandoned right there: the drainee was never told to drain, so it's still the one answering requests, still on its original pid, and RESTARTS never moves, because the failure belongs to a replacement that was never registered as the sheep. This exits 0. The daemon accepted the reload and the acceptance is what the exit code answers for; the failed swap is reported only on the bus, under process.reload_abandoned.
reuse_port = true doesn't make an app share its socket. It tells shep the app already does, on its own, so shep will pick the overlap order for it. Setting the field without the app actually calling the option doesn't fail at config time; it fails exactly like the run above, at reload time, with the same panic. Getting this right is entirely on the app.
Serial: the gap you can see
probed has a readiness_probe and no reuse_port, so it reloads serially. Watch the flock table right after the command, and watch the address itself while the swap runs:
$ shep reload probed
ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT
0 keeper online 79867 1 - 0.0% 1.4M 3m 42s - -
7 probed stopping 80436 0 - 0.0% 0B 2m 56s - -
2 web online 79793 0 - 0.0% 1.4M 3m 48s - -
4 worker online 79907 0 - 0.0% 1.4M 3m 40s - -
$ while sleep 0.3; do curl -m 0.5 -s -o /dev/null -w "%{http_code}\n" http://127.0.0.1:19294/; done
000
000
000
000
000
000
000
000
000
200
200
200
$ shep flock
ID NAME STATUS PID RESTARTS EXIT CPU MEM UPTIME FOLD SMIT
0 keeper online 79867 1 - 0.0% 1.4M 3m 46s - -
8 probed starting 82373 0 - - - 3s - -
2 web online 79793 0 - 0.0% 1.4M 3m 51s - -
4 worker online 79907 0 - 0.0% 1.4M 3m 44s - -reload's own table shows only one row for probed, stopping, no replacement id yet: a serial reload doesn't spawn one until the old instance is fully gone. curl's 000 is what it prints when there's nobody to connect to at all, not a real status code, and nine of them at roughly 300ms apiece is about the 2.7 seconds the drain and the new instance's own startup delay together take here, before the probe passes and the address answers again. An overlapping reload never produces that gap, because the old instance is still bound the entire time. A serial one always will, on any app whose replacement takes real time to become ready.
SIGKILLing a sheep's own process never reaches every lamb it forked; see Folds for what a stop actually kills versus what a sheep's process tree contains.