Flockfile reference

Getting started showed a two-field Flockfile and moved on. This page is what's underneath: every field AppConfig understands, the four formats and ten filenames config discovery will find on its own, and the grammar that turns a typo into a parse error instead of a surprise at 3am.

Or start from a generated one

# Manage your app in a Flockfile
# Add as many apps as you would like using TOML syntax

#[[app]]
# A convenient and unique name for shep to display
#name = "my-first-sheep"
# The script that shep should use to launch your app
#script = "./index.js"
# Restarts the process automatically when it exits unexpectedly
#autorestart = true
# Where the process runs. Without it, the daemon's own directory
#cwd = "/srv/app"

Almost every line is commented out, and uncommenting the ones you want is the whole workflow. The generator builds a real document first and comments it in a single pass afterwards, which is why the same scaffold exists in four languages instead of one: the comment marker never touches the structure. A test uncomments each format mechanically and parses the result, so this is a Flockfile that works rather than prose shaped like one.

--all writes every field the grammar has instead of the handful worth meeting on day one, which is 92 lines of TOML against the 12 above. --force replaces a Flockfile that is already there, keeping that file's format rather than the default one, so forcing over a .yaml writes YAML instead of dropping TOML into it. Without --force, an existing Flockfile is a refusal and nothing is written.

$ shep init deploy.json --all
error[usage]: JSON has no comment syntax, so a full scaffold would pin every default instead of explaining it; write a .json5 Flockfile for the same syntax with comments, or drop --all

Writing a second Flockfile beside an existing one is legal and said out loud rather than refused, since somebody naming a path explicitly may well be migrating. Discovery walks the fixed order below and stops at the first hit, so the new file can be one shep never reads; shep init prints an init_shadowed notice naming both files when it spots that.

Formats and discovery

TOML, YAML, JSON, and JSON5 all parse the same document shape: a list of app tables under one app key ([[app]] in TOML). shep start always takes an explicit <TARGET>: clap refuses with "the following required arguments were not provided" if you leave it off. Discovery belongs to shep runtime and shep dev, whose [TARGET] is optional: run either with no argument and it searches the current directory for these ten names, in order, and stops at the first one it finds:

Flockfile.toml
Flockfile.yaml
Flockfile.yml
Flockfile.json
Flockfile.json5
flockfile.toml
flockfile.yaml
flockfile.yml
flockfile.json
flockfile.json5

.js is a fifth format, and deliberately not on that list. Reading one means running node on it (arbitrary code execution), so discovery never selects a .js file no matter what's sitting in the directory. It's read only when you name it explicitly with --flockfile:

$ shep start flock.config.js --flockfile

shep gives node 30 seconds to hand the config back and exit, then kills it. Exporting the config is not enough by itself: a module that leaves a server listening or a timer armed keeps node's event loop alive, so node never exits. An unattended shep start ends in a refusal instead of waiting forever.

A pm2 ecosystem.config.js is not a Flockfile shep can read this way. shep never parses pm2's config format at all. That's what shep import pm2 is for: it reads a real dump.pm2 and writes a Flockfile, once, rather than teaching start a second config dialect. See Coming from pm2.

Strict grammars

Two field types are small parsers of their own rather than bare numbers, so a typo in either fails at parse time instead of silently meaning something else. Both accept plain digits with one optional trailing unit, nothing more:

512M and 30s parse. 512MB, 1.5G, and 30S all fail: a fractional size, a two-letter unit, and an uppercase duration suffix, each rejected by the same strict grammar rather than rounded, guessed, or silently dropped:

$ shep start Flockfile.toml
error[invalid_config]: invalid TOML Flockfile: TOML parse error at line 4, column 14
  |
4 | max_memory = "512MB"
  |              ^^^^^^^
memory size must be ASCII digits with an optional trailing G, M, or K

Unknown fields are a parse error, not a shrug

Every Flockfile field name is checked against the real set. A typo'd key is rejected at load, not ignored, and the message names the exact key including which app it sat under:

$ shep start Flockfile.toml
error[invalid_config]: Flockfile names unrecognized key: app.0.max_memory_restart

A key at the top level rather than inside an app is reported by the parser itself, so that one does name the alternatives: unknown field \`apps\`, expected one of \`$schema\`, \`dog\`, \`app\`. The per-app set is large enough that listing it would bury the key you actually mistyped, which is what the reference below is for.

A parsed document with zero [[app]] entries is its own distinct error, Flockfile declares no apps: an empty file isn't a no-op, since shep start with nothing to start almost always means the path was wrong.

Paths

Four fields hold one: script, cwd, out_file and err_file. All four expand a leading ~/ to your home directory, because a Flockfile is read from a file rather than typed at a shell, and nothing would otherwise expand it: shep would look for a directory literally named ~.

Deliberately narrow. ~user/ is refused rather than resolved, since answering it means a passwd lookup whose result depends on which user the daemon runs as rather than on who wrote the file. $VAR is never expanded: once a config file expands variables it has to decide whose environment it means, and there is no good answer to that.

Templates: {{instance}}, {{name}}, {{SHEP_HOME}} and {{secret:...}}

env values, args, out_file and err_file all accept doubled-brace tokens. Three are filled in at spawn time: {{instance}} and {{name}} from the sheep's own identity, {{SHEP_HOME}} from the directory the shepherd keeps its state in. Every one of the four fields takes all three:

[[app]]
name = "z-worker"
script = "./z-worker"
instances = 4
args = ["--metrics-port", "91{{instance}}"]
out_file = "{{SHEP_HOME}}/logs/{{name}}-{{instance}}-out.log"

[app.env]
Z_WORKER_ID = "z-{{instance}}"
Z_DEVICE_ID = "z-{{instance}}d"

That out_file line is the default written out, so uncommenting it changes nothing. Pair it with merge_logs and it does: an explicit path spelling {{instance}} keeps the instances in separate files anyway. shep init --all scaffolds the line, and a path built on {{SHEP_HOME}} moves with the shepherd rather than pinning the Flockfile to one machine's layout. {{SHEP_HOME}} is $SHEP_HOME when that is set and ~/.shep when it is not. It renders absolute, so unlike a relative path it is never joined onto cwd, and it renders the same for every instance, so a clustered app still needs {{instance}} or merge_logs in its log paths.

Casing says where a token reads from

An uppercase name is something from the environment, a lowercase one is something from the config. SHEP_HOME is an environment variable, so the token is {{SHEP_HOME}}; name and instance are fields of the app, so they stay lowercase. {{secret:API_KEY}} is both halves at once: secret: names the mechanism, and the key beside it is an environment-variable-shaped name. A secret key is matched as written, so the uppercase there is a convention rather than a rule.

The fourth token reads a value rather than the app's own identity: {{secret:NAME}} looks up NAME in the secrets store instead of putting a live value in the Flockfile at all. This one is for env values and args only: out_file and err_file refuse it, because a resolved log path becomes a filename on disk and is reported by shep flock and shep describe.

[app.env]
DB_PASSWORD = "{{secret:DB_PASSWORD}}"

Any other {{...}} is refused at shep start, not left to reach the child as a literal string:

error[invalid_config]: the daemon reported InvalidConfig: sheep `z-worker`, env.WORKER: `{{instnace}}` is not a template token: valid tokens are `{{instance}}`, `{{name}}`, `{{SHEP_HOME}}`, `{{secret:...}}`

An opening {{ that is never closed is refused the same way, rather than passed through as a literal:

error[invalid_config]: the daemon reported InvalidConfig: sheep `z-worker`, env.WORKER: a `{{` in this value is never closed by a `}}`

Doubling the braces escapes them: {{{{ and }}}} render as one literal {{ and }}. Single braces are left alone and never need escaping, so a JSON blob, a regex quantifier like {2,3}, or a Go or Helm template passed through as an arg all survive untouched.

SHEP_INSTANCE, SHEP_NAME and SHEP_ENVIRONMENT are set on every sheep's environment automatically, whether or not the Flockfile uses any template at all. Setting one yourself under env is refused rather than silently overwritten:

error[invalid_config]: the daemon reported InvalidConfig: sheep `z-worker` sets `SHEP_INSTANCE` in env, but shep injects it: use a different name, or `{{instance}}` in your own variable

Changing the count without editing the file

$ shep stock web 4
ID  NAME    STATUS  PID    RESTARTS  EXIT  CFG  CPU  MEM     UPTIME  FOLD  SMIT
0   web:0   online  62888  0         -     -    -    3.0M    0s      -     -
1   web:1   online  62889  0         -     -    -    3.0M    0s      -     -
3   web:2   online  62936  0         -     -    -    176.0K  0s      -     -
4   web:3   online  62937  0         -     -    -    144.0K  0s      -     -
2   worker  online  62890  0         -     -    -    3.0M    0s      -     -

$ shep stock web 2
ID  NAME    STATUS  PID    RESTARTS  EXIT  CFG  CPU  MEM   UPTIME  FOLD  SMIT
0   web:0   online  62888  0         -     -    -    3.0M  1s      -     -
1   web:1   online  62889  0         -     -    -    3.0M  1s      -     -
3   web:2   online  62936  0         -     -    -    0B    1s      -     -
4   web:3   online  62937  0         -     -    -    0B    1s      -     -
2   worker  online  62890  0         -     -    -    3.0M  1s      -     -

$ shep flock
ID  NAME    STATUS  PID    RESTARTS  EXIT  CFG  CPU  MEM   UPTIME  FOLD  SMIT
0   web:0   online  62888  0         -     -    -    3.0M  2s      -     -
1   web:1   online  62889  0         -     -    -    3.0M  2s      -     -
2   worker  online  62890  0         -     -    -    3.0M  2s      -     -

Stocking up fills the lowest free slots, and stocking down releases the highest, so stocking out and back gives you the same slot numbers, the same SHEP_INSTANCE values and the same log files you started with. Slots are not ids: web:2 above arrived with id 3, because an id is handed out once per sheep and a slot describes its place in the app.

The second command printed four instances because it answers the moment the shepherd accepts, and the two on their way out were still running their stop ladders at that point. They report themselves on the bus under process.delete, and a listing a moment later shows the flock settled. The new count goes into the muster roll, so a reboot keeps it.

The full field reference

Every field AppConfig accepts, grouped by what it's for. readiness_probe and liveness_probe each take a nested object: kind (http/tcp/exec), target, and optional interval (10s), timeout (5s), and failure_threshold (3). depends_on has a page of its own: see boot order.

Lookout's config pane builds its form straight off this same Flockfile JSON Schema, the way it builds one for a dog's own --schema answer. A row it cannot edit says why: = for read-only, like name, and ~ for a shape with no widget, like the probes. dogs opens a sub-screen instead, editing each dog's table through that dog's own schema rather than this one.

Get the schema directly

Everything above is generated from the same source the parser uses, so it can't drift from what shep actually accepts. shep schema prints it as JSON Schema, and the same output is committed at crates/shep-core/assets/flockfile.schema.json for an editor to point at:

$ shep schema | head -20
  {
    "$schema": "https://json-schema.org/draft/2020-12/schema",
    "title": "Flockfile",
    "type": "object",
    "properties": {
      "$schema": {
        "description": "The editor's schema hint, read and discarded.\n\nThis is the \"future schema key\" the comment above anticipated, added\nHERE explicitly rather than by relaxing `deny_unknown_fields`: a\ntypo'd key must still fail loudly, and exactly one more key is now\nlegal. shep does not validate against the named schema and makes no\npromise about it, it is a hint for the operator's editor, which is\nthe only consumer that ever reads it.\n\nTOML Flockfiles do not need it: taplo's `#:schema <url>` directive is\na comment, invisible to serde. JSON and JSON5 have no comment an\neditor agrees to look in, which is why this field exists at all.",
        "type": [
          "string",
          "null"
        ],
        "default": null
      },
      "app": {
        "type": "array",
        "default": [],
        "items": {
          "$ref": "#/$defs/AppConfig"
        }
      },

That's the real first twenty lines, unedited. head -20 stops just after "app" closes and before "dog" begins, and the "$schema" key's own description is one long escaped line explaining itself, not the tidied multi-line prose above. The full document closes "app", adds the "dog" key described below, then "additionalProperties": false and a "$defs" section with one entry per Flockfile field. Run the command yourself to see the rest.

The "dog" key is the other one shep accepts and ignores. A dog with per-app configuration has nowhere else to put it: it belongs beside the app it describes, in the repository the dog deploys. Before the key existed, a Flockfile carrying any of it was refused outright, so an operator could not register their app at all. shep does not read what is under dog, does not validate it, and promises nothing about it. The table itself is required, though: dog = 5 and dog = ["a"] are refused, because otherwise this would be the one key in the document where a typo does not fail loudly. What is inside it is the dog's: the dog that owns a key is the only thing that understands it, and shep refusing a document because it does not recognise another program's config is a coupling neither side wants. Everything else still fails loudly, which is why it is one nested table rather than loose top-level keys.

That "$schema" key is real and accepted: a JSON or JSON5 Flockfile can set it to a path and get editor completion, since neither format has a comment syntax an editor would look inside for a schema hint. shep itself reads the key and discards it; it never validates against whatever it points to. TOML doesn't need the key at all: a leading #:schema <path> comment (the taplo convention) does the same job, invisibly to the parser.