Reading logs

A sheep never sees its own log file. It is spawned with its stdout and stderr on pipes, and the shepherd does the file writing on the far side of them. That one fact is why every verb on this page is cheap: swapping the shepherd's handle, or emptying the file under it, is invisible across the process boundary. No signal to the app, no fd surgery, no restart, and no gap in the pipe.

Where the lines land

Two files per sheep, under $SHEP_HOME/logs/, named <name>-<instance>-out.log and <name>-<instance>-err.log. An app can point either somewhere else with out_file and err_file, or fold both streams into one file with merge_logs. A relative path in either lands under the app's cwd, so a Flockfile can keep its logs beside the app it runs. A path starting with {{SHEP_HOME}} is absolute already and stays under the shepherd's home; written out that way the default is {{SHEP_HOME}}/logs/{{name}}-{{instance}}-out.log, which is what shep init --all scaffolds. A clustered app gets one pair per instance unless it says otherwise, which is why an explicit out_file on a multi-instance app is refused unless it carries {{instance}} or the app sets merge_logs: without one of those, every instance would be appending to the same path by accident rather than on purpose.

Every line in those files starts with the time shep wrote it, in local time with the offset spelled out: 2026-09-02T14:22:31.412+02:00 the sheep said this. The stamp is always the same 30 characters wide, so anything that wants the sheep's own bytes back cuts the first 30.

It's a property of the file, not of the stream, and shep's own readers treat it that way: shep bleats strips it whether it reads the file or follows the bus, so a line means the same thing either way and --format json reports the sheep's own bytes. The stamp is there for tail, less, grep, and a rotated archive nobody can date any other way.

An app that already stamps its own lines can turn shep's off with log_timestamps = false in its Flockfile, or shep start --no-log-timestamps for a script. Its lines then land exactly as it wrote them, which is what pm2 does by default. A dog's log always keeps the stamp.

Every handle is opened append-only, and a symlink sitting at the path is refused rather than written through. Append matters more than it looks: a handle that tracked its own offset would seek back to where it had got to and write past the end of a file a rotator had just replaced, leaving a sparse hole the size of everything rotated away, after every rotation.

Reading them

Following prints that history first and then subscribes, rather than showing only new lines. A sheep that already crashed has said everything it is going to say, and a follow that skipped the tail showed an empty screen while the reason sat in the file, which is why a boot-looping sheep used to look like it had logged nothing at all. --lines 0 is how you ask for the old behaviour.

$ shep bleats noisy --no-follow
noisy | to stdout
noisy | to stderr

$ shep bleats noisy --no-follow --err
noisy | to stderr

$ shep bleats --no-follow --lines 1
echoer | got: reload
noisy | to stdout
noisy | to stderr

lookout reads the same files. b opens one sheep's feed full screen, with a stream, a minimum level and a text or regex filter stacked on top.

A sheep's own output always goes to stdout, both streams of it, because both are the data you asked for. Only shep's own diagnostics, the dropped-event count and the shutdown notice, go to stderr, so shep bleats > file captures everything the flock said and nothing shep said about it.

A log file that is not there is named on stderr rather than read as an empty tail, and the notice carries the path this command resolved. That matters most when out_file is relative: the shepherd resolves it against its own directory and shep bleats against yours, so the two are different files.

The tail is bounded twice: --lines lines, found within the last 256 KiB of the file, read one file at a time. Peak memory for that path is one window regardless of how large the flock or the log is.

Telling shep what a level looks like

lookout's minimum-level filter reads the level out of the line itself: a bare word like WARN near the start, or a level=warn pair. That covers logfmt and most Go and Rust services, and nothing at all for an app writing JSON or a Java line that puts its level sixth.

level_rules lets an app say what its own lines look like. Each rule is a regex and a level, tried in the order written, first match wins.

[[app]]
name = "api"
script = "./api"

level_rules = [
  { pattern = '"severity":"ERROR"', level = "error" },
  { pattern = '"severity":"WARN"', level = "warn" },
  { pattern = '"severity":"INFO"', level = "info" },
]

The pattern matches the whole line, exactly as written: (?i) folds case, and nothing is anchored for you. A pattern that will not compile is refused when the Flockfile loads, not when a line is drawn, and so is a list past 64 rules: lookout rebuilds it every redraw, so length is a per-frame cost. Five levels and a few spellings each stays far under it.

Handing them to a rotator

$ mv /tmp/shepdocs/home/logs/web-0-out.log /tmp/shepdocs/home/logs/web-0-out.log.1
$ mv /tmp/shepdocs/home/logs/web-1-out.log /tmp/shepdocs/home/logs/web-1-out.log.1
$ shep reopen web
ID  NAME   STATUS  PID    RESTARTS  EXIT  CFG  CPU  MEM  UPTIME  FOLD  SMIT
0   web:0  online  54244  0         -     -    -    -    43s     -     -
1   web:1  online  54338  1         -     -    -    -    37s     -     -

A zero exit is a promise worth relying on: every matched sheep's log pump holds a handle on the recreated path, so a postrotate that waits for this command knows no live pump is still filling the archive it just renamed. The shepherd reaches every writer to a path it is rotating, not only the sheep the selector named, so a sheep sharing that file but left unasked still gets moved off the old archive.

A matched sheep that is not running has no pump and nothing to reopen, and is reported alongside the rest rather than as a failure. A pump that could not open its path again fails the command instead, naming the sheep and the path on stderr. The rename is still safe to act on, because the old handle was closed either way, but that sheep is writing its stream nowhere until the path can be opened, and exiting 0 there would be the silent failure this verb exists to end.

/home/shepherd/.shep/logs/*.log {
    daily
    rotate 7
    missingok
    notifempty
    sharedscripts
    postrotate
        su shepherd -c 'shep reopen'
    endscript
}

Both halves of that stanza are more literal than they look. logrotate globs the pattern with glob(3) and never asks for tilde expansion, so ~/.shep/logs/*.log looks for a directory actually named ~ and matches nothing even when the real one is sitting there. With missingok set, that failure is silent. Write the shepherd account's absolute log directory, and substitute a custom SHEP_HOME if the account sets one.

postrotate then runs as whoever runs logrotate, usually root, and shep reopen finds its shepherd through that user's $SHEP_HOME. A root job would look under /root/.shep and report no shepherd running, so it has to become the account that owns the flock first.

Emptying them

$ shep flush worker
ID  NAME    OUT_FILE                                  ERR_FILE
2   worker  /tmp/shepdocs/home/logs/worker-0-out.log  /tmp/shepdocs/home/logs/worker-0-err.log

$ shep flush
error: the following required arguments were not provided:
  <SELECTOR>

$ shep flush --daemon
STREAM  FILE                                   RESULT
stdout  /tmp/shepdocs/home/logs/shepd.out.log  emptied
stderr  /tmp/shepdocs/home/logs/shepd.err.log  emptied

The selector is required here and defaulted everywhere else on this page. bleats and reopen default to all because neither destroys anything; this one follows stop, restart and delete in demanding a target, because it is the one command on this page whose slip of the finger cannot be undone. shep flush all is a short thing to type when it is meant.

The table is one row per sheep carrying both of its paths, rather than the lifecycle columns every other flock-shaped verb answers with. A verb that empties files an operator may have mistyped and then reports uptime and restart counts it never touched would say nothing about what it actually destroyed.

A stopped sheep is emptied like any other, since the shepherd truncates recorded paths rather than open handles, and a stopped sheep's logs are still readable with shep bleats --no-follow. Several sheep can share one path, and each distinct path is truncated once, so the same path can appear in two rows. A sharing sheep the selector skipped has that file emptied under it all the same, with its pump flushed first so none of its pending lines lands in the file afterwards. It is not a row here because it is not a sheep you named, and that is the one thing this table cannot show you.

--daemon empties the shepherd's own two files instead, and takes no selector because no selector can reach them. Those files belong to the CLI: the launcher creates them before the shepherd exists and hands them over as plain fds 1 and 2, so the shepherd never learns their paths. The useful consequence is that shep flush --daemon needs no running shepherd at all.