The shepherd channel

This is for anyone writing an app that runs under shep and wants to talk back: send readiness, custom metrics, or answer a shep trigger. It's language-agnostic: the channel is a plain file descriptor carrying JSON text, nothing Rust-specific about it.

SHEP_CHANNEL_VERSION is 1 today. It isn't negotiated: a mismatch only warns rather than refuses, because every wire change so far has been additive and an older reader ignores a field it doesn't know.

Getting a channel

shep opens the channel (a socketpair, one end handed to your process as an extra file descriptor) only when your app's Flockfile config asks for one. Three fields open it, and any one is enough:

• channel = true: ask for it directly, no other behavior implied.
• wait_ready = true: implies a channel, because it needs one to receive your ready message.
• shutdown_with_message = true: also implies a channel, for the same reason in the other direction.

Leave all three unset and your app gets no fd 3 at all. This is opt-in on purpose: a channel is a socketpair plus two pump tasks running for the life of the process, and shep would rather not pay that for an app that never uses it.

[[app]]
name = "web"
script = "./server"
channel = true

On unix the descriptor is fd 3, also exported as the SHEP_CHANNEL_FD environment variable: read that instead of hardcoding 3 if you want to be robust to it changing later. It's a normal blocking file descriptor: a plain read() parks your process until the daemon has something to say, exactly like any other pipe. No event loop, non-blocking I/O, or polling required: a shell script doing read -r line <&3 works.

Rust and Go apps do not have to speak this by hand. The shep-channel crate and github.com/shep-pm/shep-go/channel each implement everything on this page: discovering the descriptor, framing the JSON, and answering messages the app does not handle itself. JavaScript and Python libraries over the same contract are planned. Whatever language an app is written in, this page is still the contract it has to hold to.

Four working apps ship in the repository, one per language: chatty.rs over the crate, and hand-rolled Go, Node and Python under examples/polyglot/, each with both platform arms. See Examples.

Go types are generated from the Rust enums, not typed by hand. The committed file is crates/shep-channel/wire/channel.go in this repository, and github.com/shep-pm/shep-go/channel vendors the same bytes as channel/wire.go. Copy it rather than retyping it: every optional field is a pointer, because Go's omitempty on a plain value drops a metric of zero and an id of zero.

The wire format

Newline-delimited JSON, one complete message per line, in both directions. No framing header, no length prefix: just {...}\n{...}\n.... Read a line, parse it as JSON, act on it; build a JSON object, append \n, write it.

What you send (daemon reads this)

What you receive (daemon writes this)

A worked example

Two apps below, both answering the same Flockfile.toml shape (channel = true and wait_ready = true) and both actually run against a live shepherd for this page. The shell one is the whole contract in five lines; the Python one adds action replies.

#!/bin/sh
echo '{"kind":"ready"}' >&3
while read -r line <&3; do
  : # handle "shutdown" or "action" here
done

import os, json, sys

fd = int(os.environ.get("SHEP_CHANNEL_FD", "3"))
r, w = os.fdopen(fd, "r", buffering=1), os.fdopen(fd, "w", buffering=1)

w.write(json.dumps({"kind": "ready"}) + "\n")
w.flush()

for line in r:
    msg = json.loads(line)
    if msg["kind"] == "action":
        reply = {
            "kind": "action-reply",
            "action": msg["name"],
            "body": f"handled {msg['name']} params={msg.get('params')}",
            "id": msg["id"],
        }
        w.write(json.dumps(reply) + "\n")
        w.flush()
    elif msg["kind"] == "shutdown":
        sys.exit(0)

On Windows: open a named pipe instead

Windows has no fd-3 inheritance an app can rely on, so the daemon exports SHEP_CHANNEL_PIPE instead and your app opens that path like any other file. SHEP_CHANNEL_FD is deliberately not set there, so branch on which variable is present rather than on the platform. Everything below the handle is identical: same newline-delimited JSON, same message kinds, same correlation id.

Do not reconstruct the name: the trailing hex is random per spawn. The channel is readable by other accounts on the same machine, so do not put anything on it you would not show them.

import os, json, sys

pipe = os.environ.get("SHEP_CHANNEL_PIPE")
if pipe:                      # Windows: open the named pipe by name, ONCE
    r = w = open(pipe, "r+", buffering=1)
else:                         # unix: fd 3, or whatever SHEP_CHANNEL_FD says
    fd = int(os.environ.get("SHEP_CHANNEL_FD", "3"))
    r, w = os.fdopen(fd, "r", buffering=1), os.fdopen(fd, "w", buffering=1)

w.write(json.dumps({"kind": "ready"}) + "
")
w.flush()
# ... the loop below is unchanged

Started under shep with channel = true and wait_ready = true, then triggered from another terminal, real output, not a mockup:

$ shep trigger chatty reload-config "verbose"
ID  NAME    OUTCOME  DETAIL
5   chatty  replied  handled reload-config params=verbose

$ shep trigger chatty ping --format json
{"schema_version":1,"command":"trigger","data":[{"id":5,"name":"chatty","outcome":{"kind":"replied","body":"handled ping params=None"}}]}

That sheep is the sample above. The shipped examples/src/bin/chatty.rs knows three actions, so it answers this one with unknown action: reload-config.

Custom actions: the part most worth reading closely

The action name is entirely yours. shep never validates it, never keeps a registry of known actions, and never inspects params. Whatever the operator typed after the selector is sent to you verbatim, for you to recognize or refuse. There is no way to ask the daemon "what actions does this app support": that documentation lives with your app, not with shep.

The reply body is what the operator actually sees. shep trigger's DETAIL column is your body, and --format json carries it whole and untouched: verified above, params=verbose came back exactly as sent. The table view is the only place it's ever altered, and only for display: capped at 80 characters with a trailing ... when cut, embedded newlines shown as the two-character escapes \n/\r. Neither limit exists on the wire or in JSON output.

Echo the id, and your reply is matched exactly. Every action message carries an id: an opaque number, unique for the life of the daemon. Put it back on your action-reply and shep hands your answer to that exact request. Do that and the fallback matching below stops applying to you.

If you don't echo it, shep matches your reply to a waiting trigger by action name and by order: the behavior every app written before id existed gets. It has a sharp edge: while a late reply is settling a previous timeout's debt, an unstamped reply to a live trigger of the same name is consumed as that debt payment instead, and the live trigger reports timed_out even though you answered it promptly. Echoing id is how you make that impossible.

Not every sheep has a channel, and that's not a silent failure

An app configured with none of channel/wait_ready/shutdown_with_message still gets a row back from shep trigger: verified above with a plain sheep, web, that never asked for a channel:

$ shep trigger web ping
ID  NAME  OUTCOME     DETAIL
0   web   no_channel  no shepherd channel — set channel = true, or wait_ready / shutdown_with_message, which imply it

A reload drainee (the old instance mid-swap-out) gets skipped instead of a wait, because an answer from a process on its way out would be worse than none. A dog with no channel gets dog_no_channel: it has no Flockfile, and only the dog can ask. None of these costs the operator a timeout; all are refused immediately.

Naming your lambs

shep describe lists your child processes as a pid and an executable name, never their command lines, so four python workers read as four identical rows. Name them:

{"kind":"lamb-label","pid":4312,"label":"worker 1"}

describe then adds a LABEL column, --format json carries lambs[].label, and lookout's lamb line reads 4312 python (worker 1). Send it again to replace the label, or with an empty label to clear it.

At most 64 characters and no control characters: anything else is dropped as a malformed line. A label shows only while its pid is in your own tree, and goes within 15 seconds of the lamb exiting. It lives in the shepherd's memory, so a restart forgets it: send it when you spawn the lamb.

Everything you write here is also public on the bus

Every message you send on fd 3 (ready, metric, action-reply, lamb-label) is republished on the daemon's event bus under channel.* (channel.ready, channel.metric, channel.action_reply, channel.lamb_label), its own topic alongside process.* and the log topics. Anyone subscribed to channel.* sees it, not just the operator who happened to send the trigger you were answering.

This isn't a security warning: nothing on this wire is a credential, which is why the event carries your message whole rather than a redacted stand-in. It's a reminder that a body you write for one operator's terminal is also a body a dashboard, a log aggregator, or a dog might render. Put in it what you'd be comfortable with any subscriber seeing.