The shepherd channel

This is for anyone writing an app that runs under shep and wants to talk back: send readiness, custom metrics, or answer a shep trigger. It's language-agnostic: the channel is a plain file descriptor carrying JSON text, nothing Rust-specific about it.

Getting a channel

shep opens the channel (a socketpair, one end handed to your process as an extra file descriptor) only when your app's Flockfile config asks for one. Three fields open it, and any one is enough:

• channel = true: ask for it directly, no other behavior implied.
• wait_ready = true: implies a channel, because it needs one to receive your ready message.
• shutdown_with_message = true: also implies a channel, for the same reason in the other direction.

Leave all three unset and your app gets no fd 3 at all. This is opt-in on purpose: a channel is a socketpair plus two pump tasks running for the life of the process, and shep would rather not pay that for an app that never uses it.

[[app]]
name = "web"
script = "./server"
channel = true

On unix the descriptor is fd 3, also exported as the SHEP_CHANNEL_FD environment variable: read that instead of hardcoding 3 if you want to be robust to it changing later. It's a normal blocking file descriptor: a plain read() parks your process until the daemon has something to say, exactly like any other pipe. No event loop, non-blocking I/O, or polling required: a shell script doing read -r line <&3 works. The daemon also exports SHEP_CHANNEL_VERSION (currently 1) so a defensive app can notice a wire it's never seen rather than failing to parse a line with nothing to connect that failure to.

The wire format

Newline-delimited JSON, one complete message per line, in both directions. No framing header, no length prefix: just {...}\n{...}\n.... Read a line, parse it as JSON, act on it; build a JSON object, append \n, write it.

What you send (daemon reads this)

What you receive (daemon writes this)

A worked example

Two apps below, both answering the same Flockfile.toml shape (channel = true and wait_ready = true) and both actually run against a live shepherd for this page. The shell one is the whole contract in five lines; the Python one adds action replies.

#!/bin/sh
echo '{"kind":"ready"}' >&3
while read -r line <&3; do
  : # handle "shutdown" or "action" here
done

import os, json, sys

fd = int(os.environ.get("SHEP_CHANNEL_FD", "3"))
r, w = os.fdopen(fd, "r", buffering=1), os.fdopen(fd, "w", buffering=1)

w.write(json.dumps({"kind": "ready"}) + "\n")
w.flush()

for line in r:
    msg = json.loads(line)
    if msg["kind"] == "action":
        reply = {
            "kind": "action-reply",
            "action": msg["name"],
            "body": f"handled {msg['name']} params={msg.get('params')}",
            "id": msg["id"],
        }
        w.write(json.dumps(reply) + "\n")
        w.flush()
    elif msg["kind"] == "shutdown":
        sys.exit(0)

On Windows: open a named pipe instead

Windows has no fd-3 inheritance an app can rely on, so the daemon exports SHEP_CHANNEL_PIPE instead and your app opens that path like any other file. SHEP_CHANNEL_FD is deliberately not set there, so branch on which variable is present rather than on the platform. Everything below the handle is identical: same newline-delimited JSON, same message kinds, same correlation id.

Do not reconstruct the name: the trailing hex is random per spawn. The channel is readable by other accounts on the same machine, so do not put anything on it you would not show them.

import os, json, sys

pipe = os.environ.get("SHEP_CHANNEL_PIPE")
if pipe:                      # Windows: open the named pipe by name
    r = open(pipe, "r", buffering=1)
    w = open(pipe, "w", buffering=1)
else:                         # unix: fd 3, or whatever SHEP_CHANNEL_FD says
    fd = int(os.environ.get("SHEP_CHANNEL_FD", "3"))
    r, w = os.fdopen(fd, "r", buffering=1), os.fdopen(fd, "w", buffering=1)

w.write(json.dumps({"kind": "ready"}) + "
")
w.flush()
# ... the loop below is unchanged

Started under shep with channel = true and wait_ready = true, then triggered from another terminal, real output, not a mockup:

$ shep trigger chatty reload-config "verbose"
ID  NAME    OUTCOME  DETAIL
5   chatty  replied  handled reload-config params=verbose

$ shep trigger chatty ping --format json
{"schema_version":1,"command":"trigger","data":[{"id":5,"name":"chatty","outcome":{"kind":"replied","body":"handled ping params=None"}}]}

Custom actions: the part most worth reading closely

The action name is entirely yours. shep never validates it, never keeps a registry of known actions, and never inspects params. Whatever the operator typed after the selector is sent to you verbatim, for you to recognize or refuse. There is no way to ask the daemon "what actions does this app support": that documentation lives with your app, not with shep.

The reply body is what the operator actually sees. shep trigger's DETAIL column is your body, and --format json carries it whole and untouched: verified above, params=verbose came back exactly as sent. The table view is the only place it's ever altered, and only for display: capped at 80 characters with a trailing ... when cut, embedded newlines shown as the two-character escapes \n/\r. Neither limit exists on the wire or in JSON output.

Echo the id, and your reply is matched exactly. Every action message carries an id: an opaque number, unique for the life of the daemon. Put it back on your action-reply and shep hands your answer to that exact request. Do that and the fallback matching below stops applying to you.

If you don't echo it, shep matches your reply to a waiting trigger by action name and by order: the behavior every app written before id existed gets. It has a sharp edge: while a late reply is settling a previous timeout's debt, an unstamped reply to a live trigger of the same name is consumed as that debt payment instead, and the live trigger reports timed_out even though you answered it promptly. Echoing id is how you make that impossible.

Not every sheep has a channel, and that's not a silent failure

An app configured with none of channel/wait_ready/shutdown_with_message still gets a row back from shep trigger: verified above with a plain sheep, web, that never asked for a channel:

$ shep trigger web ping
ID  NAME  OUTCOME     DETAIL
0   web   no_channel  no shepherd channel — set channel = true, or wait_ready / shutdown_with_message, which imply it

A reload drainee (the old instance mid-swap-out) gets skipped instead of a wait, because an answer from a process on its way out would be worse than none. Neither of these costs the operator a timeout; both are refused immediately.

Everything you write here is also public on the bus

Every message you send on fd 3 (ready, metric, action-reply) is republished on the daemon's event bus under channel.* (channel.ready, channel.metric, channel.action_reply), its own topic alongside process.* and the log topics. Anyone subscribed to channel.* sees it, not just the operator who happened to send the trigger you were answering.

This isn't a security warning: nothing on this wire is a credential, which is why the event carries your message whole rather than a redacted stand-in. It's a reminder that a body you write for one operator's terminal is also a body a dashboard, a log aggregator, or a dog might render. Put in it what you'd be comfortable with any subscriber seeing.

Summary for the impatient

• Ask for a channel with channel = true (or get one free from wait_ready / shutdown_with_message).
• Read SHEP_CHANNEL_FD on unix, or SHEP_CHANNEL_PIPE on Windows, where the channel is a named pipe your app opens by path rather than an inherited fd. Exactly one is ever set, so branch on which one is present, not on the platform. Then read/write newline-delimited JSON.
• A plain blocking read works: no event loop required.
• Reply to every action message you receive, even ones you don't recognize, exactly once, promptly.
• Echo the id: one field, and it's what makes a slow action's answer land on the right trigger.
• What you put in body is what the operator reads back, and what anyone on channel.* reads too.

Where to go next