Logs

A sheep never sees its own log file. It is spawned with its stdout and stderr on pipes, and the shepherd does the file writing on the far side of them. That one fact is why every verb on this page is cheap: swapping the shepherd's handle, or emptying the file under it, is invisible across the process boundary. No signal to the app, no fd surgery, no restart, and no gap in the pipe.

Where the lines land

Two files per sheep, under $SHEP_HOME/logs/, named <name>-<instance>-out.log and <name>-<instance>-err.log. An app can point either somewhere else with out_file and err_file, or fold both streams into one file with merge_logs. A clustered app gets one pair per instance unless it says otherwise, which is why an explicit out_file on a multi-instance app is refused unless it carries {{instance}} or the app sets merge_logs: without one of those, every instance would be appending to the same path by accident rather than on purpose.

Every handle is opened append-only, and a symlink sitting at the path is refused rather than written through. Append matters more than it looks: a handle that tracked its own offset would seek back to where it had got to and write past the end of a file a rotator had just replaced, leaving a sparse hole the size of everything rotated away, after every rotation.

Reading them

Following prints that history first and then subscribes, rather than showing only new lines. A sheep that already crashed has said everything it is going to say, and a follow that skipped the tail showed an empty screen while the reason sat in the file, which is exactly how a boot-looping sheep came to look like it had logged nothing at all. --lines 0 is how you ask for the old behaviour.

$ shep bleats noisy --no-follow
noisy | to stdout
noisy | to stderr

$ shep bleats noisy --no-follow --err
noisy | to stderr

$ shep bleats --no-follow --lines 1
echoer | got: reload
noisy | to stdout
noisy | to stderr

A sheep's own output always goes to stdout, both streams of it, because both are the data you asked for. Only shep's own diagnostics, the dropped-event count and the shutdown notice, go to stderr, so shep bleats > file captures everything the flock said and nothing shep said about it. Interleaving a sheep's stderr into shep's would silently lose half the output of that redirect and make --err produce an empty file.

The tail is bounded twice: --lines lines, found within the last 256 KiB of the file, read one file at a time. Peak memory for that path is one window regardless of how large the flock or the log is.

Handing them to a rotator

$ mv /tmp/shepdocs/home/logs/web-0-out.log /tmp/shepdocs/home/logs/web-0-out.log.1
$ mv /tmp/shepdocs/home/logs/web-1-out.log /tmp/shepdocs/home/logs/web-1-out.log.1
$ shep reopen web
ID  NAME   STATUS  PID    RESTARTS  EXIT  CPU  MEM  UPTIME  FOLD  SMIT
0   web:0  online  54244  0         -     -    -    43s     -     -
1   web:1  online  54338  1         -     -    -    37s     -     -

A zero exit is a promise worth relying on: every matched sheep's log pump holds a handle on the recreated path, so a postrotate that waits for this command knows no live pump is still filling the archive it just renamed. That holds because the shepherd reaches every writer to a path it is rotating, not only the sheep the selector named. Several instances can share one file, and one of them left unasked would go on filling the archive.

A matched sheep that is not running has no pump and nothing to reopen, and is reported alongside the rest rather than as a failure. A pump that could not open its path again fails the command instead, naming the sheep and the path on stderr. The rename is still safe to act on, because the old handle was closed either way, but that sheep is writing its stream nowhere until the path can be opened, and exiting 0 there would be the silent failure this verb exists to end.

/home/shepherd/.shep/logs/*.log {
    daily
    rotate 7
    missingok
    notifempty
    sharedscripts
    postrotate
        su shepherd -c 'shep reopen'
    endscript
}

Both halves of that stanza are more literal than they look. logrotate globs the pattern with glob(3) and never asks for tilde expansion, so ~/.shep/logs/*.log looks for a directory actually named ~ and matches nothing even when the real one is sitting there. With missingok set, that failure is silent. Write the shepherd account's absolute log directory, and substitute a custom SHEP_HOME if the account sets one.

postrotate then runs as whoever runs logrotate, usually root, and shep reopen finds its shepherd through that user's $SHEP_HOME. A root job would look under /root/.shep and report no shepherd running, so it has to become the account that owns the flock first.

Emptying them

$ shep flush worker
ID  NAME    OUT_FILE                                  ERR_FILE
2   worker  /tmp/shepdocs/home/logs/worker-0-out.log  /tmp/shepdocs/home/logs/worker-0-err.log

$ shep flush
error: the following required arguments were not provided:
  <SELECTOR>

$ shep flush --daemon
STREAM  FILE                                   RESULT
stdout  /tmp/shepdocs/home/logs/shepd.out.log  emptied
stderr  /tmp/shepdocs/home/logs/shepd.err.log  emptied

The selector is required here and defaulted everywhere else on this page. bleats and reopen default to all because neither destroys anything; this one follows stop, restart and delete in demanding a target, because it is the one command on this page whose slip of the finger cannot be undone. shep flush all is a short thing to type when it is meant.

The table is one row per sheep carrying both of its paths, rather than the lifecycle columns every other flock-shaped verb answers with. Those keep out_file and err_file out for being too wide, and here they are the answer: a verb that empties files an operator may have mistyped, and then reports uptime and restart counts it did not touch, has said nothing about what it destroyed.

A stopped sheep is emptied like any other, since the shepherd truncates recorded paths rather than open handles, and a stopped sheep's logs are still readable with shep bleats --no-follow. Several sheep can share one path, and each distinct path is truncated once, so the same path can appear in two rows. A sharing sheep the selector skipped has that file emptied under it all the same, with its pump flushed first so none of its pending lines lands in the file afterwards. It is not a row here because it is not a sheep you named, and that is the one thing this table cannot show you.

--daemon empties the shepherd's own two files instead, and takes no selector because no selector can reach them. Those files belong to the CLI: the launcher creates them before the shepherd exists and hands them over as plain fds 1 and 2, so the shepherd never learns their paths and holds no handle it could be asked about. The useful consequence is that shep flush --daemon needs no running shepherd at all.

Where to go next