From 367b70cba1bc49c981c1fc481901abd946f76e30 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 11:56:47 -0700 Subject: [PATCH 01/31] docs: EGB-713 version-skew nudge plan --- .../2026-06-08-egb-713-version-skew-nudge.md | 300 ++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100644 docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md diff --git a/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md b/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md new file mode 100644 index 0000000..61d1ce5 --- /dev/null +++ b/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md @@ -0,0 +1,300 @@ +# EGB-713: Version-skew nudge Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans / subagent-driven-development. Steps use `- [ ]`. + +**Goal:** Warn (non-fatally) when the active store was last written by a newer `secrets` version than the running client, so a behind user is told to update — the loud counterpart to EGB-712's quiet forcing function. + +**Architecture:** Stamp the store with the highest writer `VERSION` seen (`$SECRETS_DIR/.secrets-writer-version`, committed, monotonic) on every store-committing write. On any store command, compare that stamp to the client's own `VERSION` (read from `$SCRIPT_DIR/VERSION`); if the stamp is newer, print a one-time stderr nudge. Legacy stores with no stamp are silent. + +**Tech Stack:** bash 3.2 (`secrets`); bats. Spec/idea: ticket **EGB-713**. + +## Background (verified against `main`, post-EGB-712) +- `SCRIPT_DIR` already defined at `secrets:21` (`$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)`). The repo-root `VERSION` file lives next to the script. +- Store-committing write sites (each does `git -C "$SECRETS_DIR" add -A`): `commit_and_push_secrets()` `secrets:1265` (push), rekey `secrets:1822`, migrate copy-forward `secrets:2215`, finalize no-v1 `secrets:2305`, finalize drop `secrets:2340`. Plus `cmd_init` (born store) and `cmd_rm`. +- `ensure_store_protections()` (`secrets:1144`) is shared with pull (read) — do NOT stamp there. +- `check_initialized()` (`secrets:54`) early-returns when `$SECRETS_DIR/.git` exists and is called by push/pull/list/rm/rekey/verify/migrate — the natural warning hook. +- `cmd_which()` (`secrets:1945`) prints `format: v$(_store_format)` — add the writer-version line here. +- `.gitignore` only ignores `key.txt`, so `.secrets-writer-version` commits normally. +- VERSION currently `0.7.0.0` → bump to `0.7.1.0`. +- Baseline: `bats test/` = 246 passing. New tests land in a new file `test/version.bats`. + +--- + +## Task 1: Version helpers + comparator (TDD) + +**Files:** `secrets`, `test/version.bats` (new). + +- [ ] Step 1: Create `test/version.bats` testing the comparator via a tiny harness that sources the script's functions is awkward (the script runs main). Instead test through observable behavior in later tasks; for the comparator, add a hidden debug subcommand is overkill. Use this approach: test `_version_gt` indirectly by exporting it is not possible. So test the comparator by adding the helpers and a **`secrets __vercmp `** internal is overkill too. Decision: test the comparator's *effect* in Task 3 (warning) and Task 2 (stamp monotonicity), which exercise it end-to-end. For Task 1, write the helpers and verify with a one-off `bash -c` sourcing guard. + +Add to `test/version.bats`: +```bash +load test_helper + +# Exercises the comparator through a bash subshell that defines the same logic +# the script uses, guarding the numeric (not lexical) ordering contract. +@test "version comparator orders 0.7.0.0 < 0.10.0.0 numerically" { + run bash -c ' + _version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done; return 1 + } + _version_gt 0.10.0.0 0.7.0.0 && echo "10gt7" + _version_gt 0.7.0.0 0.10.0.0 || echo "7not_gt_10" + _version_gt 0.7.0.0 0.7.0.0 || echo "equal_not_gt" + ' + [ "$status" -eq 0 ] + [[ "$output" == *"10gt7"* ]] || false + [[ "$output" == *"7not_gt_10"* ]] || false + [[ "$output" == *"equal_not_gt"* ]] || false +} +``` + +- [ ] Step 2: Run `bats test/version.bats` → PASS (pins the contract the script must match). + +- [ ] Step 3: Add the helpers to `secrets` (near `_store_format`, after `SCRIPT_DIR`/version constants — place after the `MANIFEST_VERSION=2` area or near `_store_format`): +```bash +# The running client's own version, read from the VERSION file shipped beside +# the script. Empty/"0.0.0.0" if absent (e.g. an odd install) — treated as +# "unknown/oldest" so a missing VERSION never triggers a spurious nudge. +_client_version() { + local v="" + [ -f "$SCRIPT_DIR/VERSION" ] && v=$(head -1 "$SCRIPT_DIR/VERSION" 2>/dev/null | tr -d '\r\n[:space:]') + printf '%s' "${v:-0.0.0.0}" +} + +# Numeric four-field (MAJOR.MINOR.PATCH.MICRO) compare. Returns 0 iff $1 > $2. +# Per-field numeric (so 0.10.0.0 > 0.7.0.0); missing/garbage fields → 0. +_version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done + return 1 +} + +WRITER_VERSION_FILE_NAME=".secrets-writer-version" +# Highest client version recorded as having written to the store (empty if the +# store predates this feature — "minus the initial builds", silent by design). +_store_writer_version() { + local f="$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + [ -f "$f" ] && head -1 "$f" 2>/dev/null | tr -d '\r\n[:space:]' +} +``` + +- [ ] Step 4: `bash -n secrets` parses; `bats test/` still 246 + 1 (the comparator test) = 247. +- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: version helpers + numeric comparator (EGB-713)"` + +--- + +## Task 2: Stamp the writer-version on write (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add tests: +```bash +@test "push stamps the store writer-version with the client version" { + init_with_remote + create_project_dir wvstamp + "$SECRETS_BIN" push wvstamp >/dev/null 2>&1 + [ -f "$SECRETS_DIR/.secrets-writer-version" ] + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "$(cat "$(dirname "$SECRETS_BIN")/VERSION")" ] +} + +@test "writer-version stamp is monotonic (a push never lowers a higher stamp)" { + init_with_remote + create_project_dir wvmono + printf '9.9.9.9\n' > "$SECRETS_DIR/.secrets-writer-version" + "$SECRETS_BIN" push wvmono >/dev/null 2>&1 + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "9.9.9.9" ] # not lowered to the client's version +} + +@test "writer-version stamp is committed, not gitignored" { + init_with_remote + create_project_dir wvcommit + "$SECRETS_BIN" push wvcommit >/dev/null 2>&1 + run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-writer-version" + [ "$status" -eq 0 ] +} +``` + +- [ ] Step 2: Run `bats test/version.bats -f "stamp"` → FAIL (no stamping yet). + +- [ ] Step 3: Add the stamp helper (after `_store_writer_version`): +```bash +# Raise the store's recorded writer-version to the client's version (monotonic; +# never lowers it). Called right before each store-committing `git add -A` so +# the stamp rides the same commit. Read paths (pull) never call this. +_stamp_writer_version() { + local cur cli + cur=$(_store_writer_version) + cli=$(_client_version) + if [ -z "$cur" ] || _version_gt "$cli" "$cur"; then + printf '%s\n' "$cli" > "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + fi +} +``` + +- [ ] Step 4: Call `_stamp_writer_version` immediately before each store-committing `git -C "$SECRETS_DIR" add -A`: + - `secrets:1265` (in `commit_and_push_secrets`, before `git add -A`) + - `secrets:1822` (rekey) + - `secrets:2215` (migrate copy-forward) + - `secrets:2305` (finalize, no-v1 path) + - `secrets:2340` (finalize, drop path) + Also in `cmd_init`, after the store repo is created and before its first commit (so a born store records its version), and in `cmd_rm` before its commit. + Each insertion is the single line ` _stamp_writer_version` at the matching indentation directly above the `git ... add -A` (or before the `git ... commit` where there's no add -A, e.g. rm/init — there, stamp then ensure it's staged via the existing add/commit). + +- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 250 (247 + 3). +- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: stamp store writer-version on write, monotonic (EGB-713)"` + +--- + +## Task 3: Skew warning on command (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add tests: +```bash +@test "a store written by a newer version warns on a command (non-fatal)" { + init_with_remote + create_project_dir skewwarn + "$SECRETS_BIN" push skewwarn >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] # non-fatal + [[ "$output" == *"newer"* || "$output" == *"update"* ]] || false +} + +@test "a store at the same/older version is silent" { + init_with_remote + create_project_dir noskew + "$SECRETS_BIN" push noskew >/dev/null 2>&1 # stamp == client version + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"update your secrets"* ]] || false +} + +@test "a store with no writer-version marker is silent (legacy store)" { + init_with_remote + create_project_dir legacynostamp + "$SECRETS_BIN" push legacynostamp >/dev/null 2>&1 + rm -f "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"update your secrets"* ]] || false +} +``` + +- [ ] Step 2: Run `bats test/version.bats -f "skew\|silent\|legacy"` → the "newer" test FAILS (no warning yet). + +- [ ] Step 3: Add the skew check (after `_stamp_writer_version`): +```bash +# Warn ONCE per invocation if the store was last written by a newer client than +# us. Non-fatal (read/write paths keep their exit codes). Silent when the store +# carries no writer-version (legacy) or is same/older than us. +_VERSION_SKEW_WARNED=0 +_check_store_version_skew() { + [ "$_VERSION_SKEW_WARNED" = 1 ] && return 0 + local sv cv + sv=$(_store_writer_version) + [ -n "$sv" ] || return 0 + cv=$(_client_version) + if _version_gt "$sv" "$cv"; then + _VERSION_SKEW_WARNED=1 + echo "NOTE: this store was last written by secrets v$sv; you're on v$cv." >&2 + echo " Update your secrets tool: git -C \"$SCRIPT_DIR\" pull" >&2 + fi + return 0 +} +``` + +- [ ] Step 4: Hook it into `check_initialized` — change `secrets:55-57`: +```bash + if [ -d "$SECRETS_DIR/.git" ]; then + return + fi +``` +to: +```bash + if [ -d "$SECRETS_DIR/.git" ]; then + _check_store_version_skew + return + fi +``` + +- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 253. +- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: warn on store version skew (once per invocation, EGB-713)"` + +--- + +## Task 4: `secrets which` surfaces the writer-version (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add test: +```bash +@test "which prints the store writer-version and a behind note" { + init_with_remote + create_project_dir whichwv + "$SECRETS_BIN" push whichwv >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" which + [ "$status" -eq 0 ] + [[ "$output" == *"written-by: v99.0.0.0"* ]] || false + [[ "$output" == *"behind"* || "$output" == *"update"* ]] || false +} +``` + +- [ ] Step 2: Run → FAIL (which doesn't print written-by). + +- [ ] Step 3: In `cmd_which`, after the `echo "format: v$(_store_format)"` line (`secrets:1951`), add: +```bash + local _wv; _wv=$(_store_writer_version) + if [ -n "$_wv" ]; then + local _cv; _cv=$(_client_version) + if _version_gt "$_wv" "$_cv"; then + echo "written-by: v$_wv (you're on v$_cv — behind; run: git -C \"$SCRIPT_DIR\" pull)" + else + echo "written-by: v$_wv" + fi + fi +``` +Note: `cmd_which` calls `resolve_store` but may not call `check_initialized`, so this also avoids double-printing the skew NOTE; the `which` line is the dedicated surface. + +- [ ] Step 4: `bats test/version.bats` → pass. `bats test/` → 254. +- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: secrets which shows store writer-version + behind note (EGB-713)"` + +--- + +## Task 5: Docs + version bump + +**Files:** `secrets` (cmd_help unchanged unless adding a note), `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`. + +- [ ] Step 1: `CLAUDE.md` — add to the "Store format" bullet a sentence on the writer-version: a committed `.secrets-writer-version` records the highest client `VERSION` that has written (monotonic, stamped on store-committing writes); commands warn once (stderr, non-fatal) when the store's stamp exceeds the running client, and `secrets which` shows `written-by: vN`. Legacy stores (no marker) are silent. (EGB-713.) +- [ ] Step 2: `README.md` — under the upgrading section, note that an out-of-date `secrets` prints a one-line "update" nudge when it touches a store newer than itself. +- [ ] Step 3: `VERSION` → `0.7.1.0`. +- [ ] Step 4: `CHANGELOG.md` — new `## [0.7.1.0] - 2026-06-08` with an Added entry for the version-skew nudge + `secrets which` writer-version line. +- [ ] Step 5: `bats test/` → all green (254). `./secrets which` against a scratch store renders (covered by tests). +- [ ] Step 6: Commit: `git add secrets CLAUDE.md README.md VERSION CHANGELOG.md && git commit -m "docs: version-skew nudge + writer-version; bump 0.7.1.0 (EGB-713)"` + +--- + +## Self-review vs ticket AC +- "Newer stamp → nudge; same/older → silent" → Task 3. +- "No marker → silent (legacy)" → Task 3 + `_store_writer_version` empty. +- "Monotonic, committed" → Task 2. +- "Numeric comparator (0.7.0.0 < 0.10.0.0)" → Task 1. +- "Non-fatal, never changes read/pull exit codes" → Task 3 (`_check_store_version_skew` always `return 0`). +- "`which` surfaces it" → Task 4. "Warn once per invocation" → `_VERSION_SKEW_WARNED` guard. +- bash 3.2: `read -a`, `local -a`, `10#`, parameter strips — all 3.2-safe. From 5461418c5d060dce20acc68de7cdc54bf28e2494 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 12:07:50 -0700 Subject: [PATCH 02/31] =?UTF-8?q?feat:=20version-skew=20nudge=20=E2=80=94?= =?UTF-8?q?=20stamp=20store=20writer-version,=20warn=20when=20behind=20(EG?= =?UTF-8?q?B-713)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- secrets | 81 +++++++++++++++++++++++++++++++++++ test/version.bats | 106 ++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 187 insertions(+) create mode 100644 test/version.bats diff --git a/secrets b/secrets index 52b6129..b2f1559 100755 --- a/secrets +++ b/secrets @@ -53,6 +53,7 @@ check_cmd() { check_initialized() { if [ -d "$SECRETS_DIR/.git" ]; then + _check_store_version_skew return fi if [ "$STORE_SOURCE" != "default" ]; then @@ -539,6 +540,72 @@ _store_format() { echo 1 } +# ─── Client/store version skew (EGB-713) ────────────────────────────── +# +# The running client's own version, read from the VERSION file shipped beside +# the script. Empty/"0.0.0.0" if absent — treated as "unknown/oldest" so a +# missing VERSION never triggers a spurious nudge. +_client_version() { + local v="" + [ -f "$SCRIPT_DIR/VERSION" ] && v=$(head -1 "$SCRIPT_DIR/VERSION" 2>/dev/null | tr -d '\r\n[:space:]') + printf '%s' "${v:-0.0.0.0}" +} + +# Numeric four-field (MAJOR.MINOR.PATCH.MICRO) compare. Returns 0 iff $1 > $2. +# Per-field numeric (so 0.10.0.0 > 0.7.0.0); missing/garbage fields → 0. +_version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done + return 1 +} + +WRITER_VERSION_FILE_NAME=".secrets-writer-version" +# Highest client version recorded as having written to the store (empty if the +# store predates this feature — "minus the initial builds", silent by design). +_store_writer_version() { + local f="$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + if [ -f "$f" ]; then + head -1 "$f" 2>/dev/null | tr -d '\r\n[:space:]' + fi + return 0 +} + +# Raise the store's recorded writer-version to the client's version (monotonic; +# never lowers it). Called right before each store-committing `git add -A` so +# the stamp rides the same commit. Read paths (pull) never call this. +_stamp_writer_version() { + local cur cli + cur=$(_store_writer_version) + cli=$(_client_version) + if [ -z "$cur" ] || _version_gt "$cli" "$cur"; then + printf '%s\n' "$cli" > "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + fi +} + +# Warn ONCE per invocation if the store was last written by a newer client than +# us. Non-fatal (read/write paths keep their exit codes). Silent when the store +# carries no writer-version (legacy) or is same/older than us. +_VERSION_SKEW_WARNED=0 +_check_store_version_skew() { + [ "$_VERSION_SKEW_WARNED" = 1 ] && return 0 + local sv cv + sv=$(_store_writer_version) + [ -n "$sv" ] || return 0 + cv=$(_client_version) + if _version_gt "$sv" "$cv"; then + _VERSION_SKEW_WARNED=1 + echo "NOTE: this store was last written by secrets v$sv; you're on v$cv." >&2 + echo " Update your secrets tool: git -C \"$SCRIPT_DIR\" pull" >&2 + fi + return 0 +} + # Resolve the on-disk path of an external blob for READING. Tries the v2 suffix # (.properties.age) first, then falls back to the v1 (.gradle-properties.age) for # `properties` externals, so an upgraded client finds the blob whichever format @@ -1262,6 +1329,7 @@ commit_and_push_secrets() { # store missing the key.txt line would stage and push the private key. ensure_store_protections + _stamp_writer_version git -C "$SECRETS_DIR" add -A if git -C "$SECRETS_DIR" diff --cached --quiet 2>/dev/null; then info "No changes to push (secrets unchanged)" @@ -1819,6 +1887,7 @@ cmd_rekey() { # Commit and push (heal .gitignore first so add -A can't stage key.txt) ensure_store_protections + _stamp_writer_version git -C "$SECRETS_DIR" add -A git -C "$SECRETS_DIR" commit -m "rekey all secrets" >/dev/null if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then @@ -1949,6 +2018,15 @@ cmd_which() { # EGB-700 (folded into EGB-703): surface the store format so users can tell # v1 from v2 during the migration window. v1 = legacy store, no format marker. echo "format: v$(_store_format)" + local _wv; _wv=$(_store_writer_version) + if [ -n "$_wv" ]; then + local _cv; _cv=$(_client_version) + if _version_gt "$_wv" "$_cv"; then + echo "written-by: v$_wv (you're on v$_cv — behind; run: git -C \"$SCRIPT_DIR\" pull)" + else + echo "written-by: v$_wv" + fi + fi # v2 manifest (.secrets.json): validate and summarize. Validation here # is deliberately fatal (symlink / malformed / unsupported version) so @@ -2212,6 +2290,7 @@ _migrate_project() { return 0 fi ensure_store_protections + _stamp_writer_version git -C "$SECRETS_DIR" add -A git -C "$SECRETS_DIR" commit -m "migrate: copy-forward v2 twins for $project" >/dev/null 2>&1 || true # Push the twins so a --finalize on another machine sees them (finalize @@ -2302,6 +2381,7 @@ $untwinned cd into each project and run 'secrets migrate', then re-run 'secrets # No v1 blobs at all — just stamp the marker (dotenv/file-only store). printf '2\n' > "$SECRETS_DIR/$STORE_FORMAT_FILE_NAME" ensure_store_protections + _stamp_writer_version git -C "$SECRETS_DIR" add -A git -C "$SECRETS_DIR" commit -m "migrate: finalize store format v2" >/dev/null 2>&1 || true git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 && git -C "$SECRETS_DIR" push >/dev/null 2>&1 || true @@ -2337,6 +2417,7 @@ $untwinned cd into each project and run 'secrets migrate', then re-run 'secrets done < <(find "$SECRETS_DIR" -type f -name '*.gradle-properties.age') ensure_store_protections + _stamp_writer_version git -C "$SECRETS_DIR" add -A git -C "$SECRETS_DIR" commit -m "migrate: finalize store format v2 (drop $v1count v1 blob(s))" >/dev/null 2>&1 || true git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 && git -C "$SECRETS_DIR" push >/dev/null 2>&1 || true diff --git a/test/version.bats b/test/version.bats new file mode 100644 index 0000000..fa12630 --- /dev/null +++ b/test/version.bats @@ -0,0 +1,106 @@ +#!/usr/bin/env bats +# EGB-713 version-skew nudge: writer-version stamp, numeric comparator, skew +# warning, `which` surface. bash 3.2: every standalone [[ ]] ends with || false. + +load test_helper + +VERSION_FILE() { echo "$(cd "$(dirname "$SECRETS_BIN")" && pwd)/VERSION"; } + +# ─── comparator contract ────────────────────────────────────────────── + +@test "version comparator orders 0.7.0.0 < 0.10.0.0 numerically (not lexically)" { + run bash -c ' + _version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done; return 1 + } + _version_gt 0.10.0.0 0.7.0.0 && echo "10gt7" + _version_gt 0.7.0.0 0.10.0.0 || echo "7not_gt_10" + _version_gt 0.7.0.0 0.7.0.0 || echo "equal_not_gt" + _version_gt 1.0.0.0 0.9.9.9 && echo "major_wins" + ' + [ "$status" -eq 0 ] + [[ "$output" == *"10gt7"* ]] || false + [[ "$output" == *"7not_gt_10"* ]] || false + [[ "$output" == *"equal_not_gt"* ]] || false + [[ "$output" == *"major_wins"* ]] || false +} + +# ─── stamp on write ─────────────────────────────────────────────────── + +@test "push stamps the store writer-version with the client version" { + init_with_remote + create_project_dir wvstamp + "$SECRETS_BIN" push wvstamp >/dev/null 2>&1 + [ -f "$SECRETS_DIR/.secrets-writer-version" ] + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "$(cat "$(VERSION_FILE)")" ] +} + +@test "writer-version stamp is monotonic (a push never lowers a higher stamp)" { + init_with_remote + create_project_dir wvmono + printf '9.9.9.9\n' > "$SECRETS_DIR/.secrets-writer-version" + "$SECRETS_BIN" push wvmono >/dev/null 2>&1 + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "9.9.9.9" ] +} + +@test "writer-version stamp is committed, not gitignored" { + init_with_remote + create_project_dir wvcommit + "$SECRETS_BIN" push wvcommit >/dev/null 2>&1 + run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-writer-version" + [ "$status" -eq 0 ] +} + +# ─── skew warning on command ────────────────────────────────────────── + +@test "a store written by a newer version warns on a command (non-fatal)" { + init_with_remote + create_project_dir skewwarn + "$SECRETS_BIN" push skewwarn >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" == *"last written by secrets v99.0.0.0"* ]] || false + [[ "$output" == *"Update your secrets tool"* ]] || false +} + +@test "a store at the same/older version is silent" { + init_with_remote + create_project_dir noskew + "$SECRETS_BIN" push noskew >/dev/null 2>&1 + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"Update your secrets tool"* ]] || false +} + +@test "a store with no writer-version marker is silent (legacy store)" { + init_with_remote + create_project_dir legacynostamp + "$SECRETS_BIN" push legacynostamp >/dev/null 2>&1 + rm -f "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"Update your secrets tool"* ]] || false +} + +# ─── which surface ──────────────────────────────────────────────────── + +@test "which prints the store writer-version and a behind note" { + init_with_remote + create_project_dir whichwv + "$SECRETS_BIN" push whichwv >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" which + [ "$status" -eq 0 ] + [[ "$output" == *"written-by: v99.0.0.0"* ]] || false + [[ "$output" == *"behind"* ]] || false +} From 50476e19fdabeda798f6c9b0e8ecde164a8148fb Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 12:07:51 -0700 Subject: [PATCH 03/31] docs: version-skew nudge + writer-version; bump 0.7.1.0 (EGB-713) --- CHANGELOG.md | 13 +++++++++++++ CLAUDE.md | 2 +- README.md | 2 ++ VERSION | 2 +- 4 files changed, 17 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index ee8c3b6..2325a2f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,19 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.1.0] - 2026-06-08 + +### Added + +- **Version-skew nudge (EGB-713)** — the store now records the highest `secrets` + version that has written to it (`.secrets-writer-version`, committed, + monotonic). When you run a command against a store last written by a *newer* + `secrets` than your own, you get a one-line non-fatal stderr nudge to update + your tool; `secrets which` shows the store's `written-by:` version (and flags + when you're behind). Stores written by older builds carry no stamp and stay + silent — no false alarms. The loud counterpart to EGB-712's quiet + forcing function. + ## [0.7.0.0] - 2026-06-08 ### Changed diff --git a/CLAUDE.md b/CLAUDE.md index 1a8dba4..7a0926c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -62,7 +62,7 @@ Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rek - Storage: Private git repo at `~/.secrets/` - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `/.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. -- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. +- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. - Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR//` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`. - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below - Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq` diff --git a/README.md b/README.md index 61a432d..532ba11 100644 --- a/README.md +++ b/README.md @@ -188,6 +188,8 @@ Store-format v2 is **additive** — an upgraded client reads either blob suffix "Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`. +And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent. + ### Automatic project detection When you run `secrets push` or `secrets pull` without specifying a project name, the tool figures out which project you're in by: diff --git a/VERSION b/VERSION index 7b86566..67085cc 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.0.0 +0.7.1.0 From 446256caf192df5918beb0ab5df82baf2b7dce09 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 13:56:45 -0700 Subject: [PATCH 04/31] feat: secrets list --json machine-readable output (EGB-699) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add a --json flag to `secrets list` that emits a structured object for tooling/CI instead of the human table — feeds the EGB-671 install scripts, which need to enumerate a cloned store programmatically. Contract: {"store", "projects":[{"name","entries":[...]}]}, each entry self-describing via a type discriminator — {type:dotenv,path} or {type:external,subtype:properties|file,path}. cmd_list_json mirrors the same recursive store walk as the human list (nested /.age + external/.age); jq assembles the JSON so paths escape correctly and stdout stays pure JSON (the non-default-store hint is suppressed; jq is a hard dep only in --json mode). Tests: 7 new bats cases (dotenv, nested relpath, external properties + file subtypes, empty store, pure-stdout-under-notice, store path). Full suite 261 pass / 0 fail. VERSION 0.7.1.0 -> 0.7.2.0; CHANGELOG/README/CLAUDE.md updated. --- CHANGELOG.md | 15 +++++++++ CLAUDE.md | 4 +-- README.md | 1 + VERSION | 2 +- secrets | 73 +++++++++++++++++++++++++++++++++++++++++- test/secrets.bats | 81 +++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 172 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2325a2f..4a51e6a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,21 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.2.0] - 2026-06-08 + +### Added + +- **`secrets list --json` (EGB-699)** — machine-readable listing for tooling and + CI. Emits a single JSON object on stdout: `{"store", "projects": [{"name", + "entries": [...]}]}`, where each entry self-describes via a `type` + discriminator — `{"type":"dotenv","path":}` or + `{"type":"external","subtype":"properties"|"file","path":}`. Reflects the + same recursive store walk as the human `list` (nested `/.age` + + `external/.age`). jq does the assembly so paths escape correctly; the + human store hint is suppressed so stdout stays pure JSON (notices → stderr). + jq is required only for `--json`. Feeds the EGB-671 install scripts, which need + to enumerate a cloned store programmatically instead of scraping the table. + ## [0.7.1.0] - 2026-06-08 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index 7a0926c..608040a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -75,7 +75,7 @@ Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rek secrets # CLI script (~2000 lines bash) hooks/pre-commit # Pre-commit hook template test/ - secrets.bats # bats-core test suite (133 tests) + secrets.bats # bats-core test suite (140 tests) manifest.bats # EGB-677 .secrets.json manifest tests (78 tests) migrate.bats # EGB-703 store-format-v2 migration tests (26 tests) test_helper.bash # Shared setup/teardown @@ -107,7 +107,7 @@ The active store directory is picked by `resolve_store()` using these rules, hig Key design decisions (all driven by /autoplan review): - **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel). -- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). +- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv`→`path` or `external`→`subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode). - **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `.secrets-bak` before each merge. - **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins. - **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped). diff --git a/README.md b/README.md index 532ba11..e345d64 100644 --- a/README.md +++ b/README.md @@ -167,6 +167,7 @@ secrets clear | `secrets clear` | Delete plaintext secret files from the current directory | | `secrets run ` | Pull secrets, run a command, then clear secrets when it exits | | `secrets list` | Show all projects that have stored secrets | +| `secrets list --json` | Same listing as a machine-readable JSON object (`{store, projects[].entries[]}`, each entry `dotenv`/`external`) for tooling and CI. JSON goes to stdout; notices to stderr | | `secrets rm ` | Delete a project's secrets from the store | | `secrets rekey` | Generate a new encryption key and re-encrypt everything | | `secrets verify [project]` | Check the current project's `.secrets.json` against the store (missing/orphaned blobs) and decrypt every blob. `[project]` overrides the store directory name; the manifest is still read from the current directory | diff --git a/VERSION b/VERSION index 67085cc..9872478 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.1.0 +0.7.2.0 diff --git a/secrets b/secrets index b2f1559..3ad6c40 100755 --- a/secrets +++ b/secrets @@ -1744,9 +1744,21 @@ cmd_pull_workspaces() { } cmd_list() { + local json=0 + case "${1:-}" in + --json) json=1 ;; + "") ;; + *) die "Usage: secrets list [--json]" ;; + esac + resolve_store check_initialized + if [ "$json" -eq 1 ]; then + cmd_list_json + return + fi + local found=0 for dir in "$SECRETS_DIR"/*/; do [ -d "$dir" ] || continue @@ -1782,6 +1794,64 @@ cmd_list() { fi } +# EGB-699: machine-readable listing for tooling/CI (feeds EGB-671 install +# scripts). Contract: a single JSON object on stdout — +# {"store": "", "projects": [{"name", "entries": [...]}]} +# where each entry is {"type":"dotenv","path":} or +# {"type":"external","subtype":"properties"|"file","path":}. Mirrors the +# recursive store walk the human `list` uses (nested /.age + +# external/.age). jq does the assembly so paths are escaped correctly; +# stdout stays pure JSON (the human store hint is suppressed in this mode). +cmd_list_json() { + check_cmd jq + + { + for dir in "$SECRETS_DIR"/*/; do + [ -d "$dir" ] || continue + local project + project=$(basename "$dir") + [[ "$project" == .* ]] && continue + # Marker line so a project with zero blobs still appears (mirrors the + # human header), grouped via jq below. + printf 'project\t%s\n' "$project" + local f rel name + while IFS= read -r f; do + [ -f "$f" ] || continue + rel=${f#"$dir"} + rel=${rel%.age} + case "$rel" in + external/*) + name=${rel#external/} + case "$name" in + *.file) printf 'entry\t%s\texternal\tfile\t%s\n' "$project" "${name%.file}" ;; + *.properties) printf 'entry\t%s\texternal\tproperties\t%s\n' "$project" "${name%.properties}" ;; + *.gradle-properties) printf 'entry\t%s\texternal\tproperties\t%s\n' "$project" "${name%.gradle-properties}" ;; + *) printf 'entry\t%s\texternal\tunknown\t%s\n' "$project" "$name" ;; + esac + ;; + *) + printf 'entry\t%s\tdotenv\t\t%s\n' "$project" "$rel" + ;; + esac + done < <(find "$dir" -type f -name '*.age' | sort) + done + } | jq -R -n --arg store "$SECRETS_DIR" ' + [inputs | split("\t")] as $lines + | ($lines | map(select(.[0] == "project") | .[1]) | unique) as $names + | { + store: $store, + projects: ($names | map(. as $p | { + name: $p, + entries: [ $lines[] + | select(.[0] == "entry" and .[1] == $p) + | if .[2] == "external" + then { type: "external", subtype: .[3], path: .[4] } + else { type: "dotenv", path: .[4] } + end ] + })) + }' +} + cmd_rm() { check_cmd git resolve_store @@ -2464,6 +2534,7 @@ Usage: secrets clear -w|--workspaces Clear secrets from all workspaces in package.json secrets run [-w] Pull secrets, run command, clear secrets on exit secrets list List all projects and their secret files + secrets list --json Same listing as machine-readable JSON (for tooling/CI) secrets rm Remove a project's secrets from the repo secrets rekey Re-encrypt all secrets with a new key secrets verify [project] Check the manifest against the store + decrypt every blob @@ -2636,7 +2707,7 @@ case "${1:-help}" in cmd_run "$@" ;; add) cmd_add "${2:-}" ;; - list) cmd_list ;; + list) shift; cmd_list "$@" ;; rm) cmd_rm "${2:-}" ;; rekey) cmd_rekey ;; verify) shift; cmd_verify "$@" ;; diff --git a/test/secrets.bats b/test/secrets.bats index a75c0e6..3a72910 100644 --- a/test/secrets.bats +++ b/test/secrets.bats @@ -1763,3 +1763,84 @@ file_project() { [[ "$output" == *"Extracted 1 key"* ]] || false [[ "$output" == *"Encrypted file"* ]] || false } + +# ─── EGB-699: `list --json` machine-readable output ────────────────────── + +@test "EGB-699: list --json emits valid JSON with project and dotenv entry" { + init_with_remote + create_project_dir jproj + "$SECRETS_BIN" push jproj >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + # entire stdout parses as JSON + echo "$output" | jq -e . >/dev/null + # project is present + echo "$output" | jq -e '.projects[] | select(.name == "jproj")' >/dev/null + # .env shows up as a dotenv entry + echo "$output" | jq -e '.projects[] | select(.name == "jproj") + | .entries[] | select(.type == "dotenv" and .path == ".env")' >/dev/null +} + +@test "EGB-699: list --json includes a nested dotenv relpath" { + init_with_remote + create_project_dir nestjson + mkdir -p packages/web + echo "N=nested" > packages/web/.env.development + "$SECRETS_BIN" add packages/web/.env.development >/dev/null + "$SECRETS_BIN" push >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + echo "$output" | jq -e '.projects[] | select(.name == "nestjson") + | .entries[] | select(.type == "dotenv" and .path == "packages/web/.env.development")' >/dev/null +} + +@test "EGB-699: list --json marks an external properties entry with subtype" { + init_with_remote + gradle_src $'beaconClerkPkTest=pk_test_abc\n' + gradle_project gjson beaconClerkPkTest + "$SECRETS_BIN" push gjson >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + echo "$output" | jq -e '.projects[] | select(.name == "gjson") + | .entries[] | select(.type == "external" and .subtype == "properties")' >/dev/null +} + +@test "EGB-699: list --json marks an external file entry with subtype" { + init_with_remote + file_src + local dir="$WORK_DIR/fjson"; mkdir -p "$dir" + printf 'file ~/keystores/upload.keystore\n' > "$dir/.secrets-files" + cd "$dir" + "$SECRETS_BIN" push fjson >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + echo "$output" | jq -e '.projects[] | select(.name == "fjson") + | .entries[] | select(.type == "external" and .subtype == "file")' >/dev/null +} + +@test "EGB-699: list --json on an empty store emits an empty projects array" { + "$SECRETS_BIN" init >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + echo "$output" | jq -e '.projects == []' >/dev/null +} + +@test "EGB-699: list --json keeps stdout pure JSON (notices go to stderr)" { + # The non-default-store hint normally prints to stdout in human mode; under + # --json it must not, or it would corrupt the document. Capture stdout only. + init_with_remote + create_project_dir purejson + "$SECRETS_BIN" push purejson >/dev/null 2>&1 + local json + json=$("$SECRETS_BIN" list --json 2>/dev/null) + echo "$json" | jq -e . >/dev/null +} + +@test "EGB-699: list --json reports the active store path" { + init_with_remote + create_project_dir storejson + "$SECRETS_BIN" push storejson >/dev/null 2>&1 + run "$SECRETS_BIN" list --json + [ "$status" -eq 0 ] + echo "$output" | jq -e --arg s "$SECRETS_DIR" '.store == $s' >/dev/null +} From 6319313ee40af73328b67b69f8f16df673a86367 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 16:23:50 -0700 Subject: [PATCH 05/31] feat: secrets join + init --remote + verified onboarding (EGB-671) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add second-machine onboarding as a first-class verb rather than a manual clone + key-copy sequence: - secrets join --remote --key : clone the vault, install the key at mode 600, then decrypt-test it before declaring success. An empty vault reports "nothing to verify yet" (never a false VERIFIED); a wrong key fails loudly. Reuses the audited core (resolve_store, get_pubkey, _verify_all) — no security logic re-implemented. - secrets init --remote : wire the remote and push the initial store so the upstream branch exists (fixes the commit_and_push_secrets pull --ff-only die against a brand-new empty remote). init also offers an interactive first-add of a project (default No; skipped under --yes / non-interactive). - cmd_push first-manifest scaffold writes an explicit committed options.autoAdd value, asked once when interactive (EGB-677 contract #2). - secrets pull now dies loudly when a blob fails to decrypt (all three decrypt paths) instead of warning and exiting 0 — a wrong key can't pass silently. - Interactive prompts gate on stdin AND stdout being ttys, so bats/CI never hang. - Dispatcher routes init/join args correctly; second-machine trap points at join. Tests: 20 new (join, autoAdd, pty-no-hang regression); 2 trap tests updated. --- secrets | 239 ++++++++++++++++++++++++++++++++++++++++----- test/join.bats | 145 +++++++++++++++++++++++++++ test/manifest.bats | 11 +++ test/secrets.bats | 13 +-- 4 files changed, 380 insertions(+), 28 deletions(-) create mode 100644 test/join.bats diff --git a/secrets b/secrets index 3ad6c40..219cfbf 100755 --- a/secrets +++ b/secrets @@ -1235,6 +1235,25 @@ ensure_store_protections() { cmd_init() { check_cmd age check_cmd git + + # EGB-671: flag parsing. --remote wires the encrypted-vault git remote and + # establishes an upstream branch (so the first project push won't hit the + # commit_and_push_secrets `pull --ff-only` die on a brand-new empty remote). + # --yes / non-interactive means init-only: skip the interactive first-add. + local remote="" assume_yes=false + while [ $# -gt 0 ]; do + case "$1" in + --remote) [ $# -ge 2 ] || die "--remote requires a URL" + case "$2" in --|-*) die "--remote value looks like a flag: $2" ;; esac + remote="$2"; shift 2 ;; + --remote=*) remote="${1#--remote=}" + [ -n "$remote" ] || die "--remote= requires a value"; shift ;; + --yes|-y) assume_yes=true; shift ;; + -*) die "Unknown init flag: $1. Usage: secrets init [--remote ] [--yes]" ;; + *) die "Unexpected argument to init: $1" ;; + esac + done + resolve_store if [ -d "$SECRETS_DIR/.git" ]; then @@ -1242,17 +1261,15 @@ cmd_init() { fi # Second-machine trap: a copied key.txt without a repo means the user - # should clone their existing secrets repo, not init a fresh one. - # Catch it BEFORE git init so we don't leave a half-initialized store. + # should join their existing vault, not init a fresh one. Catch it BEFORE + # git init so we don't leave a half-initialized store. if [ -f "$KEY_FILE" ]; then - # Render a runnable clone command when .secrets-store carried a remote - # URL (already sanitized by resolve_store), mirroring check_initialized. local clone_src="" [ -n "${_REMOTE_URL:-}" ] && clone_src="$_REMOTE_URL" die "Found an existing key at $KEY_FILE but no repo at $SECRETS_DIR. -If this is a second machine, don't run 'secrets init' — clone your existing secrets repo instead: +If this is a second machine, don't run 'secrets init' — join your existing vault: - git clone $clone_src $SECRETS_DIR + secrets join --remote $clone_src --key $KEY_FILE Your key file has been left untouched." fi @@ -1268,9 +1285,7 @@ Your key file has been left untouched." # Write .gitignore write_store_gitignore - # Stamp the store format (EGB-703): a fresh store is born v2 — it has no - # v1 blobs, so it is already in v2 shape. The marker is a committed, - # non-secret metadata file (NOT gitignored); the first push stages it. + # Stamp the store format (EGB-703): a fresh store is born v2. printf '2\n' > "$SECRETS_DIR/$STORE_FORMAT_FILE_NAME" # Install pre-commit hook @@ -1282,11 +1297,159 @@ Your key file has been left untouched." info "Done! Your public key is:" echo " $pubkey" + + if [ -n "$remote" ]; then + _init_wire_remote "$remote" + else + echo "" + echo "Next steps:" + echo " 1. Add a remote: secrets init --remote (or: git -C $SECRETS_DIR remote add origin )" + echo " 2. Copy $KEY_FILE to your other machine (AirDrop, scp, USB)" + echo " 3. On the other machine: secrets join --remote --key " + fi + + # Interactive first-add (EGB-671): only when truly interactive AND not --yes. + # Require BOTH stdin and stdout to be ttys — bats/CI capture a command's + # stdout (so `[ -t 1 ]` is false under automation even when stdin is still + # the terminal), which is the reliable "don't prompt" signal. Default is No. + if [ "$assume_yes" != true ] && [ -t 0 ] && [ -t 1 ] && [ -e /dev/tty ]; then + _init_first_add + fi +} + +# EGB-671: wire the encrypted-vault remote and establish an upstream branch. +# Commits the born-v2 store (so .secrets-format etc. exist on the remote) and +# push -u, so a later `secrets push` pulls --ff-only against a real upstream +# instead of dying on a non-existent branch. +_init_wire_remote() { + local remote="$1" + git -C "$SECRETS_DIR" remote add origin "$remote" + ensure_store_protections + _stamp_writer_version + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "Initialize secrets store (format v2)" >/dev/null 2>&1 || true + local br + br=$(git -C "$SECRETS_DIR" symbolic-ref --short HEAD 2>/dev/null || echo main) + if git -C "$SECRETS_DIR" push -u origin "$br" >/dev/null 2>&1; then + info "Wired remote origin=$remote and pushed the initial store (upstream: origin/$br)." + else + info "Added remote origin=$remote, but the initial push failed." + echo " Create the PRIVATE repo first, then: git -C $SECRETS_DIR push -u origin $br" >&2 + fi echo "" - echo "Next steps:" - echo " 1. Add a remote: cd $SECRETS_DIR && git remote add origin " - echo " 2. Copy $KEY_FILE to your other machine (AirDrop, scp, USB)" - echo " 3. Run 'secrets push ' from a project directory" + echo "Next: copy $KEY_FILE to your other machine, then run there:" + echo " secrets join --remote $remote --key " +} + +# EGB-671: interactive "add your first project" post-step. Reads from /dev/tty +# so it never collides with a piped stdin. Default No. Drives the existing +# push path (cmd_push is $PWD-bound) by cd'ing into the chosen project dir. +_init_first_add() { + printf "Add a project's secrets to the vault now? [y/N] " > /dev/tty 2>/dev/null || return 0 + local ans="" + read -r ans < /dev/tty 2>/dev/null || return 0 + case "$ans" in [Yy]*) ;; *) return 0 ;; esac + printf "Path to the project directory: " > /dev/tty 2>/dev/null || return 0 + local dir="" + read -r dir < /dev/tty 2>/dev/null || return 0 + [ -n "$dir" ] || return 0 + case "$dir" in + "~") dir="$HOME" ;; + "~/"*) dir="$HOME/${dir#\~/}" ;; + esac + if [ ! -d "$dir" ]; then + echo "Not a directory: $dir — skipping first-add. Run 'secrets push' from a project later." > /dev/tty 2>/dev/null || true + return 0 + fi + ( cd "$dir" && cmd_push ) || true + return 0 +} + +# EGB-671: second-machine onboarding. Clone the vault, install the key, and +# decrypt-test it BEFORE declaring success — the one thing a hand-copied +# README sequence never did (a mis-copied key fails silently at first pull). +# Security logic (path rails on --key/--store, URL sanitization) lives in the +# audited core, reused — never re-implemented in a standalone install script. +cmd_join() { + check_cmd age + check_cmd git + + local remote="" keyfile="" + while [ $# -gt 0 ]; do + case "$1" in + --remote) [ $# -ge 2 ] || die "--remote requires a URL" + case "$2" in --|-*) die "--remote value looks like a flag: $2" ;; esac + remote="$2"; shift 2 ;; + --remote=*) remote="${1#--remote=}" + [ -n "$remote" ] || die "--remote= requires a value"; shift ;; + --key) [ $# -ge 2 ] || die "--key requires a path" + case "$2" in --|-*) die "--key value looks like a flag: $2" ;; esac + keyfile="$2"; shift 2 ;; + --key=*) keyfile="${1#--key=}" + [ -n "$keyfile" ] || die "--key= requires a value"; shift ;; + -*) die "Unknown join flag: $1. Usage: secrets join --remote --key " ;; + *) die "Unexpected argument to join: $1" ;; + esac + done + + resolve_store + + [ -n "$remote" ] || die "secrets join requires --remote + Usage: secrets join --remote --key " + [ -n "$keyfile" ] || die "secrets join requires --key + This is the age key (key.txt) from your first machine. + Usage: secrets join --remote --key " + if [ -d "$keyfile" ]; then + die "--key must point to the key FILE, not a directory: $keyfile + Did you mean: --key $keyfile/key.txt ?" + fi + [ -e "$keyfile" ] || die "Key file not found: $keyfile + Copy key.txt from your first machine (AirDrop/scp/USB) and pass its path." + [ -r "$keyfile" ] || die "Key file not readable: $keyfile" + + if [ -e "$SECRETS_DIR" ]; then + die "A store already exists at $SECRETS_DIR. + 'secrets join' clones a fresh vault — it won't clobber an existing one. + If you meant to refresh it, run 'secrets pull' instead, or remove $SECRETS_DIR first." + fi + + info "Joining vault: cloning $remote → $SECRETS_DIR" + if ! git clone "$remote" "$SECRETS_DIR" >/dev/null 2>&1; then + rm -rf "$SECRETS_DIR" + die "Failed to clone $remote + Check the URL and that you have access to the repo." + fi + + # Install the key BEFORE anything that decrypts, at mode 600. + cp "$keyfile" "$SECRETS_DIR/key.txt" + chmod 600 "$SECRETS_DIR/key.txt" + KEY_FILE="$SECRETS_DIR/key.txt" + ensure_store_protections + + if ! get_pubkey >/dev/null 2>&1; then + die "The file you passed to --key is not a valid age identity: $keyfile + Your store was cloned to $SECRETS_DIR; replace key.txt with a valid key and run 'secrets pull'." + fi + + # Verify gate: decrypt-test every blob. An EMPTY store returns 0 from + # _verify_all ("nothing to check") — that proves nothing about the key, so + # join must NOT report VERIFIED in that case (EGB-671 / E-S1b). + local blob_count + blob_count=$(find "$SECRETS_DIR" -type f -name '*.age' 2>/dev/null | wc -l | tr -d ' ') + if [ "$blob_count" -eq 0 ]; then + info "Joined $SECRETS_DIR — the vault is empty, so there's nothing to verify yet." + echo "Next: run 'secrets pull' in a project once secrets have been pushed from another machine." + return 0 + fi + if _verify_all >/dev/null 2>&1; then + info "VERIFIED — your key decrypts all $blob_count blob(s). You've joined the vault." + echo "Next: run 'secrets pull' in any project to restore its secrets." + return 0 + fi + die "Your key does NOT decrypt this vault ($blob_count blob(s) failed). + This is almost always the wrong key.txt. The store is at $SECRETS_DIR; + replace key.txt with the correct key and run 'secrets pull', or remove + $SECRETS_DIR and re-run 'secrets join' with the right --key." } # Encrypt env files from a source dir into a project path in the secrets repo. @@ -1498,8 +1661,22 @@ cmd_push() { '.dotenv = ((.dotenv // []) + $add) | .external = ((.external // []) + $ext)' "$manifest" \ | _write_manifest_canonical "$manifest" || die "Failed to update $manifest" else - jq -n --argjson add "$add_json" --argjson ext "$absorbed_json" \ - '{version: '"$MANIFEST_VERSION"', dotenv: $add} | if ($ext | length) > 0 then .external = $ext else . end' \ + # EGB-671 / EGB-677 contract #2: scaffolding the project's FIRST manifest. + # Record an explicit, committed options.autoAdd value. Ask once when + # interactive (read from /dev/tty so a piped stdin never collides); + # otherwise write the tool default (ON) explicitly so the value is + # committed and team-shared rather than left implicit. + local autoadd_commit="true" + # Require BOTH stdin and stdout to be ttys (bats/CI capture stdout, so + # `[ -t 1 ]` is false under automation — never block a scripted push). + if [ -t 0 ] && [ -t 1 ] && [ -e /dev/tty ]; then + printf "Auto-track new env files in this project as you add them? [Y/n] " > /dev/tty 2>/dev/null || true + local _aa="" + read -r _aa < /dev/tty 2>/dev/null || _aa="" + case "$_aa" in [Nn]*) autoadd_commit="false" ;; *) autoadd_commit="true" ;; esac + fi + jq -n --argjson add "$add_json" --argjson ext "$absorbed_json" --argjson autoadd "$autoadd_commit" \ + '{version: '"$MANIFEST_VERSION"', dotenv: $add, options: {autoAdd: $autoadd}} | if ($ext | length) > 0 then .external = $ext else . end' \ | _write_manifest_canonical "$manifest" || die "Failed to write $manifest" fi if [ "$write_adds" = true ]; then @@ -1616,9 +1793,14 @@ cmd_pull() { continue fi case "$rel" in */*) mkdir -p "$target_dir/$(dirname "$rel")" ;; esac - age -d -i "$KEY_FILE" -o "$target_dir/$rel" "$blob" + # EGB-671 (DX-3): a wrong-but-structurally-valid key must NOT fail + # silently. Die loudly on decrypt failure instead of leaving a partial. + if ! age -d -i "$KEY_FILE" -o "$target_dir/$rel" "$blob"; then + die "Failed to decrypt '$rel' with the current key ($KEY_FILE). + Wrong key for this vault? Run 'secrets verify --all' to check the key." + fi if [ ! -s "$target_dir/$rel" ]; then - echo "WARNING: Decrypted file '$rel' is empty (possibly truncated .age blob)" + echo "WARNING: Decrypted file '$rel' is empty (an empty source file, or a truncated .age blob)" fi count=$((count + 1)) done <<< "$declared" @@ -1642,10 +1824,14 @@ cmd_pull() { local name name=$(basename "$f" .age) local outfile="$target_dir/$name" - age -d -i "$KEY_FILE" -o "$outfile" "$f" + # EGB-671 (DX-3): die loudly on decrypt failure (wrong key) — never silent. + if ! age -d -i "$KEY_FILE" -o "$outfile" "$f"; then + die "Failed to decrypt '$name' with the current key ($KEY_FILE). + Wrong key for this vault? Run 'secrets verify --all' to check the key." + fi # Integrity check: verify non-empty if [ ! -s "$outfile" ]; then - echo "WARNING: Decrypted file '$name' is empty (possibly truncated .age blob)" + echo "WARNING: Decrypted file '$name' is empty (an empty source file, or a truncated .age blob)" fi count=$((count + 1)) done @@ -1674,9 +1860,13 @@ pull_project_to_dir() { local name name=$(basename "$f" .age) local outfile="$target_dir/$name" - age -d -i "$KEY_FILE" -o "$outfile" "$f" + # EGB-671 (DX-3): die loudly on decrypt failure (wrong key) — never silent. + if ! age -d -i "$KEY_FILE" -o "$outfile" "$f"; then + die "Failed to decrypt '$name' with the current key ($KEY_FILE). + Wrong key for this vault? Run 'secrets verify --all' to check the key." + fi if [ ! -s "$outfile" ]; then - echo "WARNING: Decrypted file '$name' is empty (possibly truncated .age blob)" + echo "WARNING: Decrypted file '$name' is empty (an empty source file, or a truncated .age blob)" fi count=$((count + 1)) done @@ -2523,6 +2713,10 @@ secrets — encrypted secret file sync between machines Usage: secrets init Initialize the secrets repo and generate an age key + secrets init --remote Init, wire the remote, and push the initial store + secrets join --remote --key + Join an existing vault on a new machine: clone, + install the key, and verify it decrypts the store secrets push [project] Encrypt secret files and push to the secrets repo secrets push --frozen Sync only manifest-declared files (skip auto-add) secrets push --dry-run Show what would be added/synced; change nothing @@ -2679,7 +2873,8 @@ else fi case "${1:-help}" in - init) cmd_init ;; + init) shift; cmd_init "$@" ;; + join) shift; cmd_join "$@" ;; push) if [ "${2:-}" = "-w" ] || [ "${2:-}" = "--workspaces" ]; then cmd_push_workspaces diff --git a/test/join.bats b/test/join.bats new file mode 100644 index 0000000..205570a --- /dev/null +++ b/test/join.bats @@ -0,0 +1,145 @@ +#!/usr/bin/env bats +# EGB-671: `secrets join` (second-machine onboarding) + `secrets init --remote` +# + day-2 silent-decrypt fix. Functional paths only — security-rail tests +# (path traversal on --key/--store, URL injection) are operator-local per +# .ship-policy.json and live in test/run-security.sh. + +load test_helper + +# Push a project to REMOTE_DIR and save the key, then remove the local store +# to simulate a fresh second machine. Leaves: REMOTE_DIR has blobs, +# $TEST_TMPDIR/saved-key.txt is the decrypting key, $SECRETS_DIR is gone. +_machine1_push_then_wipe() { + init_with_remote + cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt" + create_project_dir "joinproj" + "$SECRETS_BIN" push >/dev/null 2>&1 + cd "$HOME" + rm -rf "$SECRETS_DIR" +} + +# Like above but never pushes a project — remote has a store with zero blobs. +_machine1_empty_then_wipe() { + init_with_remote + cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt" + cd "$HOME" + rm -rf "$SECRETS_DIR" +} + +# ─── secrets join ──────────────────────────────────────────────────────── + +@test "join without --remote fails with usage" { + run "$SECRETS_BIN" join + [ "$status" -ne 0 ] + [[ "$output" == *"--remote"* ]] || false +} + +@test "join clones the store, installs the key at 600, verifies, and succeeds" { + _machine1_push_then_wipe + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt" + [ "$status" -eq 0 ] + [[ "$output" == *"VERIFIED"* ]] || false + [ -d "$SECRETS_DIR/.git" ] + [ -f "$SECRETS_DIR/key.txt" ] + # key installed at mode 600 + local perms + perms=$(stat -f '%Lp' "$SECRETS_DIR/key.txt" 2>/dev/null || stat -c '%a' "$SECRETS_DIR/key.txt") + [ "$perms" = "600" ] +} + +@test "join with the wrong key fails loudly and does not report VERIFIED" { + _machine1_push_then_wipe + age-keygen -o "$TEST_TMPDIR/wrong-key.txt" 2>/dev/null + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/wrong-key.txt" + [ "$status" -ne 0 ] + [[ "$output" != *"VERIFIED"* ]] || false +} + +@test "join against an empty store reports nothing-to-verify, NOT VERIFIED" { + _machine1_empty_then_wipe + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt" + [ "$status" -eq 0 ] + [[ "$output" == *"nothing to verify"* ]] || false + [[ "$output" != *"VERIFIED"* ]] || false +} + +@test "join refuses when a store already exists at the target" { + "$SECRETS_BIN" init >/dev/null 2>&1 + cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt" + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt" + [ "$status" -ne 0 ] + [[ "$output" == *"already"* ]] || false +} + +@test "join fails clearly when the key file is missing" { + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/nope.txt" + [ "$status" -ne 0 ] + [[ "$output" == *"key"* ]] || false +} + +@test "join detects a directory passed as --key" { + _machine1_push_then_wipe + run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR" + [ "$status" -ne 0 ] + [[ "$output" == *"key"* ]] || false +} + +# ─── secrets init --remote ──────────────────────────────────────────────── + +@test "init --remote sets origin and establishes an upstream branch" { + run "$SECRETS_BIN" init --remote "$REMOTE_DIR" + [ "$status" -eq 0 ] + run git -C "$SECRETS_DIR" remote get-url origin + [ "$status" -eq 0 ] + [ "$output" = "$REMOTE_DIR" ] + # upstream branch exists on the remote (so a later push won't ff-only die) + run git -C "$SECRETS_DIR" rev-parse --abbrev-ref '@{u}' + [ "$status" -eq 0 ] +} + +@test "init --remote then push does not die on the brand-new remote" { + "$SECRETS_BIN" init --remote "$REMOTE_DIR" >/dev/null 2>&1 + create_project_dir "freshproj" + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + [[ "$output" != *"Fast-forward pull failed"* ]] || false +} + +@test "init with no flags still works (clean primitive)" { + run "$SECRETS_BIN" init + [ "$status" -eq 0 ] + [ -f "$SECRETS_DIR/key.txt" ] +} + +@test "init does not hang on the first-add prompt when stdin is a tty but stdout is captured" { + # Regression: run-security.sh runs bats in a real terminal, so the command's + # stdin stays a tty while bats captures its stdout. The interactive first-add + # prompt must NOT fire in that shape (it gates on stdout being a tty too), + # or the whole suite hangs. Reproduce with a pty via `script`. + command -v script >/dev/null 2>&1 || skip "script (pty) not available" + # macOS/BSD syntax: `script -q `. Skip on other syntaxes. + script -q /dev/null true >/dev/null 2>&1 || skip "unsupported script syntax" + local out="$TEST_TMPDIR/pty-initout" + run timeout 10 script -q /dev/null bash -c "'$SECRETS_BIN' init > '$out' 2>&1" + [ "$status" -ne 124 ] # 124 == timeout == it hung on a prompt + run grep -c "Add a project's secrets" "$out" + [ "$output" = "0" ] +} + +# ─── day-2 silent-decrypt fix ───────────────────────────────────────────── + +@test "pull dies loudly when a blob cannot be decrypted with the current key" { + init_with_remote + create_project_dir "decryptproj" + "$SECRETS_BIN" push >/dev/null 2>&1 + # Swap in a different key so the stored blob no longer decrypts. + # (age-keygen refuses to overwrite, so generate elsewhere then copy.) + age-keygen -o "$TEST_TMPDIR/other-key.txt" 2>/dev/null + cp "$TEST_TMPDIR/other-key.txt" "$SECRETS_DIR/key.txt" + chmod 600 "$SECRETS_DIR/key.txt" + cd "$WORK_DIR/decryptproj" + rm -f .env .env.staging + run "$SECRETS_BIN" pull + [ "$status" -ne 0 ] + [[ "$output" == *"decrypt"* ]] || false +} diff --git a/test/manifest.bats b/test/manifest.bats index eb5c42a..ce16125 100644 --- a/test/manifest.bats +++ b/test/manifest.bats @@ -167,6 +167,17 @@ load test_helper [ "$output" = "2" ] } +@test "bootstrap: first push writes an explicit options.autoAdd value (EGB-677 contract #2)" { + init_with_remote + create_project_dir autoaddproj + # Non-interactive (bats has no tty): the prompt is skipped and the tool + # default (ON) is written explicitly so the value is committed + team-shared. + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + run jq -r '.options.autoAdd' .secrets.json + [ "$output" = "true" ] +} + @test "failed push leaves no bootstrap manifest behind" { init_with_remote mkdir -p "$WORK_DIR/emptyproj" diff --git a/test/secrets.bats b/test/secrets.bats index 3a72910..ee2928f 100644 --- a/test/secrets.bats +++ b/test/secrets.bats @@ -1484,9 +1484,10 @@ gradle_project() { # ─── init second-machine guard + store .gitignore self-heal ──────────── -@test "init with existing key but no repo dies with clone guidance" { - # Second-machine trap: user copies key.txt into ~/.secrets, then runs - # `secrets init` instead of cloning their secrets repo. +@test "init with existing key but no repo dies with join guidance" { + # Second-machine trap (EGB-671): user copies key.txt into ~/.secrets, then + # runs `secrets init` instead of joining their existing vault. The trap now + # points at `secrets join` (the real one-command path), not a manual clone. mkdir -p "$SECRETS_DIR" age-keygen -o "$SECRETS_DIR/key.txt" 2>/dev/null # Guard against a vacuous '' = '' comparison if age-keygen failed @@ -1496,7 +1497,7 @@ gradle_project() { run "$SECRETS_BIN" init [ "$status" -eq 1 ] - [[ "$output" == *"git clone"* ]] || false + [[ "$output" == *"secrets join"* ]] || false # Must not leave a half-initialized store behind [ ! -d "$SECRETS_DIR/.git" ] # Key untouched @@ -1651,7 +1652,7 @@ gradle_project() { [[ "$output" != *"key.txt"* ]] || false } -@test "init guard renders the real clone URL when .secrets-store carries a remote" { +@test "init guard renders the real remote URL in join guidance when .secrets-store carries a remote" { mkdir -p "$HOME/.secrets-work" age-keygen -o "$HOME/.secrets-work/key.txt" 2>/dev/null [ -s "$HOME/.secrets-work/key.txt" ] @@ -1660,7 +1661,7 @@ gradle_project() { run "$SECRETS_BIN" init [ "$status" -eq 1 ] - [[ "$output" == *"git clone git@example.com:me/secrets-work.git"* ]] || false + [[ "$output" == *"secrets join --remote git@example.com:me/secrets-work.git"* ]] || false } # ─── EGB-652: `file` external type (whole-file sync, e.g. Android keystore) ── From 7b041af68b2e13ee4c1281a6be3a4463def14297 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 16:23:50 -0700 Subject: [PATCH 06/31] feat: thin install.sh onboarding bootstrap (EGB-671) Ships in the repo (clone already done). Checks age + jq + git, then PRINTS the PATH line, onboarding next-steps, upgrade one-liner, and key-transfer hint. Never edits shell rc, never runs sudo (prints the command). Exits non-zero with an install hint when a dependency is missing. --- install.sh | 113 ++++++++++++++++++++++++++++++++++++++++++++++ test/install.bats | 72 +++++++++++++++++++++++++++++ 2 files changed, 185 insertions(+) create mode 100755 install.sh create mode 100644 test/install.bats diff --git a/install.sh b/install.sh new file mode 100755 index 0000000..695fc59 --- /dev/null +++ b/install.sh @@ -0,0 +1,113 @@ +#!/usr/bin/env bash +# +# secrets — thin onboarding bootstrap (EGB-671). +# +# This script ships INSIDE the repo: you already cloned the repo to get it, so +# its only jobs are (1) verify the dependencies the tool needs and (2) print the +# exact commands to finish setup. It deliberately does NOT: +# - edit your shell rc files (it prints the PATH line for you to paste) +# - invoke sudo or install packages behind your back (it prints the command) +# - re-implement any of the tool's security logic +# +# This is a security tool whose whole pitch is "verify, don't trust" — so the +# installer holds itself to a higher bar than convenience, not a lower one. +# +# Usage: +# ./install.sh # check deps, print setup + next steps +# ./install.sh --help + +set -euo pipefail + +# Resolve the directory this script lives in (the cloned tool repo). Uses bash +# builtins only so it works under a minimal PATH. +_src="${BASH_SOURCE[0]}" +TOOL_DIR="$(cd "${_src%/*}" 2>/dev/null && pwd)" + +usage() { + cat < + Other machine: secrets join --remote --key +EOF +} + +# Print the install command for a package, using whatever package manager is +# present. For sudo-requiring managers we PRINT the line for you to run — the +# installer never escalates on its own. +install_hint() { + local pkg="$1" + if command -v brew >/dev/null 2>&1; then + echo "brew install $pkg" + elif command -v apt-get >/dev/null 2>&1; then + echo "sudo apt-get install -y $pkg" + elif command -v dnf >/dev/null 2>&1; then + echo "sudo dnf install -y $pkg" + else + echo "install '$pkg' with your system package manager" + fi +} + +case "${1:-}" in + --help|-h) usage; exit 0 ;; + "") ;; + *) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;; +esac + +echo "secrets — bootstrap check (tool dir: $TOOL_DIR)" +echo "" + +# Dependency check. age + jq + git are all load-bearing on the cold-start path: +# jq became required once .secrets.json (manifest) is JSON, so it must be present +# BEFORE the first manifest read. +missing=0 +for dep in git age jq; do + if command -v "$dep" >/dev/null 2>&1; then + echo " ok $dep" + else + echo " MISSING $dep — install it with:" + echo " $(install_hint "$dep")" + missing=1 + fi +done +echo "" + +if [ "$missing" -ne 0 ]; then + echo "Install the missing dependencies above, then re-run ./install.sh." >&2 + exit 1 +fi + +cat < + # then transfer key.txt to your other machines (AirDrop / scp / USB): + # scp :$HOME/.secrets/key.txt ~/.secrets/key.txt + + Other machine (join an existing vault): + secrets join --remote --key + # 'join' clones the vault, installs the key, and VERIFIES it decrypts + # before declaring success — a mis-copied key fails loudly, not silently. + +To update the tool later: + git -C "$TOOL_DIR" pull +EOF diff --git a/test/install.bats b/test/install.bats new file mode 100644 index 0000000..0ce7593 --- /dev/null +++ b/test/install.bats @@ -0,0 +1,72 @@ +#!/usr/bin/env bats +# EGB-671: install.sh thin bootstrap. It ships IN the repo (you clone the repo +# to get it), so its job is: verify deps (age + jq + git), PRINT the PATH line +# and next-step commands — never edit dotfiles, never invoke sudo. Security-rail +# concerns are operator-local (.ship-policy.json); these are functional checks. + +load test_helper + +INSTALL_SH="$(cd "$(dirname "${BATS_TEST_FILENAME}")/.." && pwd)/install.sh" + +@test "install.sh exists and is executable" { + [ -f "$INSTALL_SH" ] + [ -x "$INSTALL_SH" ] +} + +@test "install.sh --help prints usage and exits 0" { + run "$INSTALL_SH" --help + [ "$status" -eq 0 ] + [[ "$output" == *"install.sh"* ]] || false + [[ "$output" == *"join"* ]] || false +} + +@test "install.sh prints the PATH export line for the tool dir (does not edit rc)" { + local tool_dir + tool_dir="$(cd "$(dirname "$INSTALL_SH")" && pwd)" + run "$INSTALL_SH" + [ "$status" -eq 0 ] + [[ "$output" == *"export PATH="* ]] || false + [[ "$output" == *"$tool_dir"* ]] || false + # It must NOT have written to any shell rc in the isolated HOME. + [ ! -f "$HOME/.zshrc" ] + [ ! -f "$HOME/.bashrc" ] +} + +@test "install.sh prints both onboarding next-steps (init --remote and join)" { + run "$INSTALL_SH" + [ "$status" -eq 0 ] + [[ "$output" == *"secrets init --remote"* ]] || false + [[ "$output" == *"secrets join --remote"* ]] || false +} + +@test "install.sh prints the upgrade one-liner" { + run "$INSTALL_SH" + [ "$status" -eq 0 ] + [[ "$output" == *"git -C"* ]] || false + [[ "$output" == *"pull"* ]] || false +} + +@test "install.sh prints a key-transfer hint" { + run "$INSTALL_SH" + [ "$status" -eq 0 ] + [[ "$output" == *"key.txt"* ]] || false +} + +@test "install.sh never invokes sudo (prints it for the user instead)" { + # No executed 'sudo' — any sudo reference must be quoted guidance text. + run grep -nE '^[[:space:]]*sudo ' "$INSTALL_SH" + [ "$status" -ne 0 ] +} + +@test "install.sh reports a missing dependency with an install hint and non-zero exit" { + # Build a minimal PATH that has the tools install.sh needs but NOT jq. + local fake="$TEST_TMPDIR/fakebin" + mkdir -p "$fake" + for t in bash uname env cat grep sed tr dirname command age git printf; do + src="$(command -v "$t" 2>/dev/null || true)" + [ -n "$src" ] && ln -sf "$src" "$fake/$t" 2>/dev/null || true + done + run env PATH="$fake" "$INSTALL_SH" + [ "$status" -ne 0 ] + [[ "$output" == *"jq"* ]] || false +} From 4d975d447df5913a6869a2567c561cfa8db1dbe1 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Mon, 8 Jun 2026 16:23:50 -0700 Subject: [PATCH 07/31] chore: rewrite onboarding docs + bump version (v0.7.3.0) README rewritten to the install.sh + init --remote + join flow; dropped the macOS-only prerequisite (age+jq install hints now cover apt/dnf too). CHANGELOG entry for EGB-671. Co-Authored-By: Claude Opus 4.8 --- CHANGELOG.md | 36 +++++++++++++++ README.md | 122 ++++++++++++++++++++++++++++----------------------- VERSION | 2 +- 3 files changed, 104 insertions(+), 56 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 4a51e6a..23e6122 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,42 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.3.0] - 2026-06-08 + +### Added + +- **Real install / onboarding scripts (EGB-671)** — onboarding a machine is now + (close to) one command, and a mis-copied key fails loudly instead of silently. + - **`secrets join --remote --key `** — second-machine onboarding in + one verb: clones the vault, installs the key at mode 600, and **verifies the + key actually decrypts the store before declaring success**. An empty vault + reports "nothing to verify yet" (it never prints a false `VERIFIED`); a wrong + key fails loudly with the store left in place to fix. All security logic + (store resolution, URL handling, path rails) is reused from the audited core, + not re-implemented in a side script. + - **`secrets init --remote `** — wires the remote and pushes the initial + store so the upstream branch exists, so your first project `push` doesn't trip + the fast-forward-pull guard on a brand-new empty remote. Run interactively, + `init` also offers to add your first project's secrets (default No, skipped + under `--yes` / non-interactive, so it stays a clean primitive for CI). + - **`install.sh`** — thin bootstrap that ships in the repo: checks `age` + `jq` + + `git`, then prints the `PATH` line, the onboarding next-steps, the upgrade + one-liner, and a key-transfer hint. It never edits your shell config and never + runs `sudo` (it prints the command so you stay in control). + - **First-manifest `options.autoAdd` prompt (EGB-677 contract #2)** — the first + `push` that scaffolds a project's manifest now records an explicit, committed + `options.autoAdd` value (asked once when interactive; the default ON, written + explicitly, under automation). + +### Fixed + +- **Day-2 silent decrypt failure** — `secrets pull` now dies loudly when a blob + fails to decrypt with the current key (all three decrypt paths), instead of + emitting a warning and continuing with exit 0. A wrong key can no longer pass + unnoticed after onboarding. +- The `secrets init` second-machine trap now points at `secrets join` (the real + one-command path) instead of a manual `git clone`. + ## [0.7.2.0] - 2026-06-08 ### Added diff --git a/README.md b/README.md index e345d64..4fa3459 100644 --- a/README.md +++ b/README.md @@ -58,83 +58,95 @@ Beyond project files, `secrets` can also sync files that live *outside* the proj ## Prerequisites -- **macOS** (uses Homebrew for installation) -- **git** (already installed on most Macs — type `git --version` to check) -- **age** (the encryption tool — installed in step 1 below) +- **macOS or Linux** +- **git** (`git --version` to check) +- **age** and **jq** — `install.sh` checks for these and prints the exact install command for your platform (Homebrew on macOS, `apt`/`dnf` on Linux) ## Setup -### First machine (one-time setup) +Clone the tool repo, then run `install.sh`. It checks dependencies and prints the +two commands to finish setup. It never edits your shell config and never runs +sudo — it prints the commands so you stay in control. ```bash -# 1. Install the encryption tool -brew install age - -# 2. Download the secrets tool (this repo — contains only the CLI, no secret files) git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets - -# 3. Make the 'secrets' command available everywhere -# Add this line to your shell config file (~/.zshrc on Mac): -export PATH="$HOME/dev/secrets:$PATH" -# Then restart your terminal, or run: -source ~/.zshrc - -# 4. Initialize your encrypted secrets store -# This creates a folder at ~/.secrets/ with your encryption key -secrets init - -# 5. Create a PRIVATE repository on GitHub to store your encrypted secrets -# Go to github.com/new, name it something like 'my-secrets', and make sure -# "Private" is selected. Then connect it: -cd ~/.secrets -git remote add origin git@github.com:/my-secrets.git -git push -u origin main +cd ~/dev/secrets +./install.sh ``` -> **Important:** Step 5 creates a *separate* private repo for your encrypted secrets. This is different from the `secrets` tool repo you cloned in step 2. The tool repo can be public — it contains no secrets. The `~/.secrets/` repo must be private. +`install.sh` prints a `export PATH="$HOME/dev/secrets:$PATH"` line — add it to your +shell config (`~/.zshrc` or `~/.bashrc`) and restart your terminal. Then onboard +this machine with one of the two flows below. -### Additional machines - -On each new machine (your desktop, a teammate's laptop, etc.): +### First machine (new vault) ```bash -# 1. Install prerequisites and the tool (same as steps 1-3 above) -brew install age -git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets -export PATH="$HOME/dev/secrets:$PATH" # add to ~/.zshrc +# 1. Create a PRIVATE repo for your encrypted secrets (github.com/new or a +# Codeberg/GitLab private repo). It holds only ciphertext — never your key. +# Then wire it up and push the store in one command: +secrets init --remote git@github.com:/my-secrets.git -# 2. Clone the encrypted secrets repo -git clone git@github.com:/my-secrets.git ~/.secrets - -# 3. Copy the encryption key from your first machine -# This is the only step that requires direct machine-to-machine transfer. -# Choose one method: -# -# Option A: AirDrop (Mac to Mac) -# On your first machine, right-click ~/.secrets/key.txt → Share → AirDrop -# Save it to ~/.secrets/key.txt on the new machine -# -# Option B: Secure copy over SSH -# scp first-machine:~/.secrets/key.txt ~/.secrets/key.txt -# -# Option C: USB drive -# Copy key.txt to a USB drive, transfer it, delete from USB after - -# 4. Pull your secrets into any project +# 2. (optional) Add a project's secrets. From a project directory: cd ~/myapp -secrets pull +secrets push +# The first push asks once whether to auto-track new env files and records +# your choice in the project's .secrets.json. ``` +`secrets init --remote` generates your key (`~/.secrets/key.txt`), wires the +remote, and pushes the initial store so the upstream branch exists. The private +secrets repo is separate from this tool repo — the tool repo is public and holds +no secrets; the `~/.secrets/` repo must be private. + +> Running `secrets init` interactively (in a terminal) also offers to add your +> first project's secrets right away. Run it with `--yes` (or in any non-tty +> context like CI) to skip that prompt and just create the vault. + +### Other machines (join an existing vault) + +On a second machine, a desktop, or a teammate's laptop: + +```bash +# 1. Clone the tool and run the bootstrap (as in Setup above) +git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets +cd ~/dev/secrets && ./install.sh # add the printed PATH line to your shell config + +# 2. Get key.txt onto this machine (the one manual, out-of-band step): +# AirDrop (Mac→Mac), or +# scp first-machine:~/.secrets/key.txt ~/Downloads/key.txt, or +# a USB drive (delete from the drive afterward) + +# 3. Join the vault in one command: +secrets join --remote git@github.com:/my-secrets.git --key ~/Downloads/key.txt +``` + +`secrets join` clones the vault, installs the key at mode 600, and **verifies the +key actually decrypts the store before declaring success** — a mis-copied key +fails loudly here, not silently on a later `secrets pull`. On success it tells you +to run `secrets pull` in any project. + > **The key file (`~/.secrets/key.txt`) is the only thing that needs to be transferred manually.** It never leaves your machines — it's excluded from git, never uploaded, never transmitted over the internet. Anyone with this file can decrypt all your secrets, so treat it like a password. ### Sharing with teammates To share secrets with a teammate, they need: -1. Access to your private `my-secrets` GitHub repo (add them as a collaborator) -2. A copy of `key.txt` (send it to them directly — AirDrop, USB, or in-person) +1. Access to your private secrets repo (add them as a collaborator) +2. A copy of `key.txt` (send it directly — AirDrop, USB, or in-person) -Everyone on the team uses the same key. When anyone runs `secrets push`, the encrypted files are updated and everyone else can `secrets pull` to get the latest version. +Everyone on the team uses the same key. A teammate joins with +`secrets join --remote --key `. When anyone runs +`secrets push`, the encrypted files update and everyone else runs `secrets pull` +to get the latest. + +### Updating the tool + +```bash +git -C ~/dev/secrets pull +``` + +If your store was last written by a newer client than yours, `secrets` prints a +one-line version-skew nudge — that's your cue to run the command above. ## Usage diff --git a/VERSION b/VERSION index 9872478..934346d 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.2.0 +0.7.3.0 From 1ee096cc3303530dde3265fe7d36d4767c5ff0b6 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Thu, 18 Jun 2026 10:08:13 -0700 Subject: [PATCH 08/31] refactor: dedup external extractor + read guards, warn on legacy-pull nested blobs (EGB-701) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit EGB-677 stage-1 structural cleanups, no behavior change for the v2 happy path: 1. cmd_which reuses _json_external_entries (the push/pull extractor) instead of its own duplicated jq @tsv projection, so `which` applies the same properties->gradle-properties normalization + skip-with-warning rules the sync path does and can't drift from it. 2. The two external-manifest read guards are factored into _json_readable (plain regular file, silent) / _legacy_readable (warn+skip a symlinked .secrets-files), shared by _external_entries_for_push/_pull. 3. The legacy (manifest-less) pull path now warns when nested /.age blobs exist that its non-recursive globs can't see (external/ excluded — pull_external_files handles those), so it never silently under-restores. Tests: +4 in test/manifest.bats (normalized which display, malformed external skipped by which, nested-blob warning fires, external-only no false warning). Full suite green (286/286). Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 6 ++-- secrets | 84 +++++++++++++++++++++++++++++++++++----------- test/manifest.bats | 65 +++++++++++++++++++++++++++++++++++ 3 files changed, 132 insertions(+), 23 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 608040a..bd2954f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,8 +76,8 @@ secrets # CLI script (~2000 lines bash) hooks/pre-commit # Pre-commit hook template test/ secrets.bats # bats-core test suite (140 tests) - manifest.bats # EGB-677 .secrets.json manifest tests (78 tests) - migrate.bats # EGB-703 store-format-v2 migration tests (26 tests) + manifest.bats # EGB-677 .secrets.json manifest tests (83 tests) + migrate.bats # EGB-703 store-format-v2 migration tests (35 tests) test_helper.bash # Shared setup/teardown README.md # User-facing documentation CLAUDE.md # This file @@ -107,7 +107,7 @@ The active store directory is picked by `resolve_store()` using these rules, hig Key design decisions (all driven by /autoplan review): - **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel). -- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv`→`path` or `external`→`subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode). +- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). **EGB-701 cleanups:** (1) the *legacy* (manifest-less) `pull` keeps its non-recursive globs but now **warns** when nested `/.age` blobs exist that those globs can't see (it excludes `external/`, which `pull_external_files` handles) — so a manifest-less pull never silently under-restores; the fix the warning points at is committing a `.secrets.json`. (2) `cmd_which`, push, and pull share one external extractor (`_json_external_entries`), so `which` applies the same `properties`→`gradle-properties` normalization and skip-with-warning rules the sync path does (it shows exactly what will sync, not a stale raw projection). (3) the two external-manifest read guards are factored into `_json_readable` (plain regular file, silent) / `_legacy_readable` (warn-and-skip on a symlinked legacy manifest). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv`→`path` or `external`→`subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode). - **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `.secrets-bak` before each merge. - **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins. - **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped). diff --git a/secrets b/secrets index 219cfbf..2d55aa7 100755 --- a/secrets +++ b/secrets @@ -1038,6 +1038,31 @@ _json_external_entries() { done < <(jq -r '.external // [] | .[] | [.type, .path, ((.keys // []) | join(" "))] | @tsv' "$manifest") } +# EGB-701 item 2: the read guards for the two external-manifest sources, +# factored out of _external_entries_for_push/_pull so they can't drift. +# +# _json_readable — true when a .secrets.json is a safe regular file to read. +# A symlinked manifest is treated as absent and silently ignored: it's the +# project's own committed file, so a symlink there is just skipped (the fatal +# symlink refusal lives in _check_manifest_file, used by the linting paths). +_json_readable() { + [ -f "$1" ] && [ ! -L "$1" ] +} + +# _legacy_readable — true when a legacy .secrets-files is a safe regular file +# to read, warning (and returning false) when it exists but is a symlink: a +# symlinked legacy manifest's target is attacker-influenceable, so never follow +# it. A missing or non-regular file returns false silently. +_legacy_readable() { + local legacy="$1" + [ -e "$legacy" ] || return 1 + if [ -L "$legacy" ]; then + echo "WARNING: $legacy is a symlink; ignoring." >&2 + return 1 + fi + [ -f "$legacy" ] +} + # External tuples for PUSH: .secrets.json entries first, then legacy # .secrets-files entries whose (type, path) the manifest doesn't cover — # the absorb set, which cmd_push folds into the manifest after a @@ -1046,23 +1071,19 @@ _external_entries_for_push() { local root="$1" local json="$root/$SECRETS_JSON_NAME" legacy="$root/$SECRETS_FILES_NAME" local seen="" t p k - if [ -f "$json" ] && [ ! -L "$json" ]; then + if _json_readable "$json"; then while IFS=$'\t' read -r t p k; do [ -n "$t" ] || continue printf '%s\t%s\t%s\n' "$t" "$p" "$k" seen="$seen$t|$p"$'\n' done < <(_json_external_entries "$json") fi - if [ -e "$legacy" ]; then - if [ -L "$legacy" ]; then - echo "WARNING: $legacy is a symlink; ignoring." >&2 - elif [ -f "$legacy" ]; then - while IFS=$'\t' read -r t p k; do - [ -n "$t" ] || continue - case "$seen" in *"$t|$p"$'\n'*) continue ;; esac - printf '%s\t%s\t%s\n' "$t" "$p" "$k" - done < <(_parse_secrets_files_manifest "$legacy") - fi + if _legacy_readable "$legacy"; then + while IFS=$'\t' read -r t p k; do + [ -n "$t" ] || continue + case "$seen" in *"$t|$p"$'\n'*) continue ;; esac + printf '%s\t%s\t%s\n' "$t" "$p" "$k" + done < <(_parse_secrets_files_manifest "$legacy") fi } @@ -1071,19 +1092,18 @@ _external_entries_for_push() { _external_entries_for_pull() { local root="$1" local json="$root/$SECRETS_JSON_NAME" legacy="$root/$SECRETS_FILES_NAME" - if [ -f "$json" ] && [ ! -L "$json" ]; then - if [ -f "$legacy" ] && [ ! -L "$legacy" ]; then + if _json_readable "$json"; then + # A regular (non-symlink) legacy file alongside the manifest is superseded: + # warn but don't read it. _json_readable is the "plain regular file" test — + # exactly the supersede condition (and unlike _legacy_readable it stays + # silent on a symlink, matching the original no-warn-on-symlink behavior). + if _json_readable "$legacy"; then echo "WARNING: $legacy is superseded by $SECRETS_JSON_NAME and was ignored on pull. Run 'secrets push' to absorb it, then delete it." >&2 fi _json_external_entries "$json" return 0 fi - [ -e "$legacy" ] || return 0 - if [ -L "$legacy" ]; then - echo "WARNING: $legacy is a symlink; ignoring." >&2 - return 0 - fi - [ -f "$legacy" ] && _parse_secrets_files_manifest "$legacy" + _legacy_readable "$legacy" && _parse_secrets_files_manifest "$legacy" return 0 } @@ -1838,6 +1858,26 @@ cmd_pull() { info "Decrypted $count file(s) into $target_dir" + # EGB-701 item 3: the globs above are non-recursive, so a nested dotenv blob + # (/.age) written by a manifest-driven push on another + # machine is invisible here — silently restored nothing, counted nothing. + # external/.age blobs are restored by pull_external_files, so exclude + # them. Warn (don't die) so a manifest-less pull never under-restores in + # silence; the fix is a committed .secrets.json, which the recursive + # manifest-driven branch above handles correctly. + local nested + nested=$(find "$SECRETS_DIR/$project" -mindepth 2 -type f -name '*.age' \ + -not -path "$SECRETS_DIR/$project/external/*" 2>/dev/null) + if [ -n "$nested" ]; then + echo "WARNING: this project has nested encrypted files the manifest-less pull can't restore:" >&2 + while IFS= read -r nf; do + [ -n "$nf" ] || continue + local rel="${nf#"$SECRETS_DIR/$project/"}" + echo " ${rel%.age}" >&2 + done <<< "$nested" + echo " Add a $SECRETS_JSON_NAME manifest (run 'secrets push' on a machine that has these files) so they restore." >&2 + fi + # Merge any external files (.secrets-files) declared in this project. pull_external_files "$PWD" "$project" @@ -2307,11 +2347,15 @@ cmd_which() { echo " dotenv $entry [UNSAFE — will be refused]" fi done < <(jq -r '.dotenv // [] | .[]' "$json_manifest") + # EGB-701 item 1: reuse the one external extractor the sync path uses, + # so `which` applies the same normalization + skip-with-warning rules + # push/pull do — `which` shows exactly what will sync, never a stale + # raw projection that drifts from the helper. local etype epath ekeys while IFS=$'\t' read -r etype epath ekeys; do [ -n "$etype" ] || continue echo " $etype $epath $ekeys" - done < <(jq -r '.external // [] | .[] | [.type, .path, ((.keys // []) | join(" "))] | @tsv' "$json_manifest") + done < <(_json_external_entries "$json_manifest") fi # Read back any external-file manifest in cwd (validates the format and diff --git a/test/manifest.bats b/test/manifest.bats index ce16125..f7aeab9 100644 --- a/test/manifest.bats +++ b/test/manifest.bats @@ -642,6 +642,31 @@ m_nojq_path() { [[ "$output" == *"k1"* ]] || false } +# EGB-701 item 1: `which` and the push/pull external extractor share one +# helper (_json_external_entries), so `which` applies the same +# properties→gradle-properties normalization the sync path uses — no drift. +@test "which normalizes a properties external to the gradle-properties token (EGB-701)" { + create_project_dir whichnorm + printf '{"version":2,"dotenv":[".env"],"external":[{"type":"properties","path":"~/.gradle/gradle.properties","keys":["k1"]}]}\n' > .secrets.json + run "$SECRETS_BIN" which + [ "$status" -eq 0 ] + [[ "$output" == *"gradle-properties"* ]] || false +} + +# EGB-701 item 1: a malformed external (a properties entry with no keys) is +# skipped by the sync path; routing `which` through the shared extractor means +# `which` skips+warns it too, so it faithfully shows what actually syncs +# rather than printing an entry push/pull silently drop. +@test "which skips a malformed external entry the sync path would drop (EGB-701)" { + create_project_dir whichmalformed + printf '{"version":2,"dotenv":[".env"],"external":[{"type":"properties","path":"~/.gradle/gradle.properties"}]}\n' > .secrets.json + run "$SECRETS_BIN" which + [ "$status" -eq 0 ] + [[ "$output" == *"has no keys"* ]] || false + # The skipped entry's path must NOT appear in the printed manifest summary. + [[ "$output" != *" gradle-properties ~/.gradle/gradle.properties"* ]] || false +} + # ─── F: ship Step 7 coverage backfill (audit gaps) ───────────────────── @test "which flags an unsafe dotenv entry with the UNSAFE marker" { @@ -739,6 +764,46 @@ m_nojq_path() { [ "$(cat packages/web/.env.development)" = "N=nested" ] } +@test "legacy (manifest-less) pull warns about nested blobs it can't restore (EGB-701)" { + # The legacy pull path globs only top-level *.age/.*.age. A nested dotenv + # blob (/.age) written by a manifest-driven push on another + # machine is invisible to those globs — restored nothing, counted nothing. + # The fix: warn so a manifest-less pull never silently under-restores. + init_with_remote + create_project_dir nestlegacy + mkdir -p packages/web + echo "N=nested" > packages/web/.env.development + "$SECRETS_BIN" add packages/web/.env.development >/dev/null + "$SECRETS_BIN" push >/dev/null 2>&1 + [ -f "$SECRETS_DIR/nestlegacy/packages/web/.env.development.age" ] + # Simulate a machine with no manifest: drop .secrets.json + local files, + # forcing the legacy non-recursive glob branch. + rm -f .secrets.json + rm -rf packages + run "$SECRETS_BIN" pull nestlegacy + [ "$status" -eq 0 ] + # The warning names the nested blob and points at the manifest as the fix. + [[ "$output" == *"packages/web/.env.development"* ]] || false + [[ "$output" == *"$SECRETS_JSON_NAME"* || "$output" == *".secrets.json"* ]] || false + # The legacy path genuinely can't restore it (the warning is the contract). + [ ! -f packages/web/.env.development ] +} + +@test "legacy pull does NOT warn about external/ blobs (handled separately, EGB-701)" { + # external/.age blobs are restored by pull_external_files, not the + # dotenv globs, so they must not trip the nested-blob warning. + init_with_remote + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir extnolwarn + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push >/dev/null 2>&1 + [ -d "$SECRETS_DIR/extnolwarn/external" ] + run "$SECRETS_BIN" pull extnolwarn + [ "$status" -eq 0 ] + [[ "$output" != *"can't restore"* ]] || false + [[ "$output" != *"nested encrypted"* ]] || false +} + @test "list shows a nested manifest blob" { init_with_remote create_project_dir nestlist From 3ece393cc5ada221964976648201216b8174facd Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Thu, 18 Jun 2026 10:08:13 -0700 Subject: [PATCH 09/31] chore: bump version and changelog (v0.7.3.1) Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 28 ++++++++++++++++++++++++++++ VERSION | 2 +- 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 23e6122..475794a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,34 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.3.1] - 2026-06-18 + +### Changed + +- **EGB-677 stage-1 structural cleanups (EGB-701)** — tech-debt dedup with one + new safety warning; no behavior change for the manifest-driven (v2) happy path. + - **`secrets which` now reuses the one external-entry extractor** the push/pull + path uses (`_json_external_entries`) instead of its own duplicated `jq` + projection. So `which` applies the same `properties`→`gradle-properties` + normalization and skips (with a warning) the same malformed external entries + the sync path drops — `which` shows exactly what will sync, not a stale raw + projection that could drift from the real behavior. + - **The two external-manifest read guards are factored into shared helpers** — + `_json_readable` (plain regular file, silent) and `_legacy_readable` (warns + and skips a symlinked `.secrets-files`) — so `_external_entries_for_push` and + `_external_entries_for_pull` can't drift apart. + +### Fixed + +- **Legacy (manifest-less) `pull` no longer silently under-restores (EGB-701)** — + the manifest-less pull path globs only top-level `*.age`/`.*.age`, so a nested + dotenv blob (`/.age`) written by a manifest-driven push on + another machine was invisible: restored nothing, counted nothing, said nothing. + It now **warns** and names each nested blob it can't reach (external blobs are + excluded — `pull_external_files` handles those), pointing at committing a + `.secrets.json` as the fix. The manifest-driven pull already restored nesting + correctly; this only closes the legacy path's blind spot. + ## [0.7.3.0] - 2026-06-08 ### Added diff --git a/VERSION b/VERSION index 934346d..512d674 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.3.0 +0.7.3.1 From 09c4ad54f9be7383dce2d2884e6483a1ef459910 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Thu, 18 Jun 2026 11:01:17 -0700 Subject: [PATCH 10/31] =?UTF-8?q?feat:=20secrets=20upgrade=20verb=20?= =?UTF-8?q?=E2=80=94=20self-update=20+=20re-check=20version=20skew=20(EGB-?= =?UTF-8?q?716)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Pairs the EGB-713 skew WARNING with a fix path. `secrets upgrade` fast-forwards the tool's own git checkout (git -C "$SCRIPT_DIR" pull --ff-only; never merges or rewrites local commits), reports vOLD -> vNEW, then best-effort re-checks the store's recorded writer-version against the new version so the operator sees whether the nudge is cleared. `secrets upgrade --check` reports availability without pulling. Thin and explicit: no auto-update, no background polling (security tool). Directed errors for not-a-checkout / no-upstream / diverged / offline. cmd_upgrade never calls check_initialized (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves. Wired into the dispatcher (upgrade) shift; cmd_upgrade "$@") and cmd_help. Tests: test/upgrade.bats (8) run a relocated script copy in a throwaway git repo with a bare upstream, so the real checkout is never touched. Co-Authored-By: Claude Opus 4.8 (1M context) --- CLAUDE.md | 5 +- README.md | 2 + secrets | 87 +++++++++++++++++++++++++++++++++++ test/upgrade.bats | 115 ++++++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 207 insertions(+), 2 deletions(-) create mode 100644 test/upgrade.bats diff --git a/CLAUDE.md b/CLAUDE.md index bd2954f..9693fde 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,13 +56,13 @@ skips security specialist + red team, and Step 11 skips adversarial review. ## Architecture -Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate. +Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, upgrade. - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable) - Storage: Private git repo at `~/.secrets/` - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `/.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. -- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. +- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves. - Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR//` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`. - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below - Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq` @@ -78,6 +78,7 @@ test/ secrets.bats # bats-core test suite (140 tests) manifest.bats # EGB-677 .secrets.json manifest tests (83 tests) migrate.bats # EGB-703 store-format-v2 migration tests (35 tests) + upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests) test_helper.bash # Shared setup/teardown README.md # User-facing documentation CLAUDE.md # This file diff --git a/README.md b/README.md index 4fa3459..aa4662c 100644 --- a/README.md +++ b/README.md @@ -187,6 +187,8 @@ secrets clear | `secrets migrate [--dry-run]` | Copy-forward this project's encrypted blobs to store format v2 (non-destructive; manifest-free; `--dry-run` previews) | | `secrets migrate --status` | Survey every project's v2 readiness; exits non-zero until the whole store is finalize-ready | | `secrets migrate --finalize` | **Optional GC** — drop the old v1 blobs and mark the store pure v2. Never required: upgraded clients dual-write and read-fall-back, so not finalizing never cuts anyone off | +| `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks | +| `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing | ### Upgrading: do teammates on an older `secrets` get new secrets? diff --git a/secrets b/secrets index 2d55aa7..f567242 100755 --- a/secrets +++ b/secrets @@ -2375,6 +2375,90 @@ cmd_which() { fi } +# `secrets upgrade [--check]` (EGB-716) — self-update the TOOL checkout. +# +# Pairs the EGB-713 skew WARNING with a fix path. Deliberately thin and explicit +# (no auto-update, no background polling — this is a security tool): it only +# fast-forwards the tool's own git checkout, never merges or rewrites local +# commits. --check reports whether an update is available and changes nothing. +# After a real update it best-effort re-checks the store's writer-version skew +# against the NEW on-disk version, so the operator sees whether the EGB-713 +# nudge is now cleared (the new code itself takes effect on the next command). +cmd_upgrade() { + local check_only=0 + while [ $# -gt 0 ]; do + case "$1" in + --check) check_only=1; shift ;; + -*) die "Unknown upgrade flag: $1. Usage: secrets upgrade [--check]" ;; + *) die "Unexpected argument to upgrade: $1. Usage: secrets upgrade [--check]" ;; + esac + done + + check_cmd git + + # Self-update only works on a git checkout of the tool. + if ! git -C "$SCRIPT_DIR" rev-parse --is-inside-work-tree >/dev/null 2>&1; then + die "secrets at $SCRIPT_DIR is not a git checkout, so it can't self-update. + Re-install by cloning the tool repo, e.g.: git clone " + fi + + # Need a tracking branch to compare against / pull from. + local upstream + upstream=$(git -C "$SCRIPT_DIR" rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null || true) + if [ -z "$upstream" ]; then + die "No upstream tracking branch for the secrets checkout at $SCRIPT_DIR. + Set one with: git -C \"$SCRIPT_DIR\" branch --set-upstream-to=origin/main" + fi + + local oldver; oldver=$(_client_version) + + if [ "$check_only" = 1 ]; then + if ! git -C "$SCRIPT_DIR" fetch --quiet 2>/dev/null; then + die "Couldn't reach the tool remote to check for updates (offline?). + Try again when connected, or run: git -C \"$SCRIPT_DIR\" fetch" + fi + local behind; behind=$(git -C "$SCRIPT_DIR" rev-list --count "HEAD..$upstream" 2>/dev/null || echo 0) + if [ "${behind:-0}" -gt 0 ]; then + info "Update available: $behind commit(s) behind $upstream (you're on v$oldver)." + info "Apply it with: secrets upgrade" + else + info "secrets is up to date (v$oldver)." + fi + return 0 + fi + + info "Updating secrets at $SCRIPT_DIR ..." + # Fast-forward only: never merge or rewrite local commits. + if ! git -C "$SCRIPT_DIR" pull --ff-only 2>&1; then + die "Update failed (see git output above). + Likely a local change or a diverged branch in $SCRIPT_DIR. + Inspect with: git -C \"$SCRIPT_DIR\" status" + fi + + local newver; newver=$(_client_version) + if [ "$oldver" = "$newver" ]; then + info "Already up to date (v$newver)." + else + info "Upgraded: v$oldver -> v$newver" + info "The new version takes effect on your next 'secrets' command." + fi + + # Best-effort EGB-713 skew re-check against the NEW version. Silent unless a + # store with a recorded writer-version resolves. + resolve_store 2>/dev/null || true + if [ -n "${SECRETS_DIR:-}" ] && [ -f "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" ]; then + local sv; sv=$(_store_writer_version) + if [ -n "$sv" ]; then + if _version_gt "$sv" "$newver"; then + info "Note: the store was last written by v$sv — still ahead of v$newver. Another machine may run a newer client." + else + info "Your client (v$newver) is now at or ahead of the store's last writer (v$sv)." + fi + fi + fi + return 0 +} + # Decrypt-test one blob with the current key. Plaintext is streamed to # /dev/null and never written to disk (read-only contract). Returns 0 if the # blob decrypts, non-zero otherwise. @@ -2783,6 +2867,8 @@ Usage: secrets which Show the active store, manifest, and external entries secrets where Alias for `which` secrets status Alias for `which` + secrets upgrade Self-update the tool (git pull --ff-only) + recheck skew + secrets upgrade --check Report whether an update is available; change nothing Tracked files: .env, .env.*, .dev.vars @@ -2952,6 +3038,7 @@ case "${1:-help}" in verify) shift; cmd_verify "$@" ;; migrate) shift; cmd_migrate "$@" ;; which|where|status) cmd_which ;; + upgrade) shift; cmd_upgrade "$@" ;; help|--help|-h) cmd_help ;; *) die "Unknown command: $1. Run 'secrets help' for usage." ;; esac diff --git a/test/upgrade.bats b/test/upgrade.bats new file mode 100644 index 0000000..041bacb --- /dev/null +++ b/test/upgrade.bats @@ -0,0 +1,115 @@ +#!/usr/bin/env bats +# EGB-716: `secrets upgrade` verb — self-update (git pull --ff-only) + skew re-check. +# +# These tests never touch the real tool checkout. Each test relocates a COPY of +# the script into a throwaway git repo wired to a bare upstream, so $SCRIPT_DIR +# (computed from BASH_SOURCE) resolves to the fake tool repo and the pull/fetch +# operate there. + +load test_helper + +# Create a fake tool repo at $TOOL (script copy + VERSION), wired to a bare +# upstream at $TOOL_REMOTE, at version $1. cd's into $TOOL (under $HOME so +# resolve_store's walk-up stays bounded and never strays to a real store). +setup_tool_repo() { + TOOL="$TEST_TMPDIR/tool" + TOOL_REMOTE="$TEST_TMPDIR/tool-remote.git" + mkdir -p "$TOOL" + cp "$SECRETS_BIN" "$TOOL/secrets" + echo "$1" > "$TOOL/VERSION" + git -c init.defaultBranch=main init -q "$TOOL" + git -C "$TOOL" add -A + git -C "$TOOL" -c user.email=t@t -c user.name=t commit -qm "v$1" + git -c init.defaultBranch=main init --bare -q "$TOOL_REMOTE" + git -C "$TOOL" remote add origin "$TOOL_REMOTE" + git -C "$TOOL" push -q -u origin HEAD:main + cd "$TOOL" +} + +# Publish a newer VERSION to the upstream (as a different clone would). +advance_tool_remote() { + local clone="$TEST_TMPDIR/tool-pub" + rm -rf "$clone" + git clone -q "$TOOL_REMOTE" "$clone" + echo "$1" > "$clone/VERSION" + git -C "$clone" -c user.email=t@t -c user.name=t commit -qam "v$1" + git -C "$clone" push -q origin HEAD:main + rm -rf "$clone" +} + +@test "upgrade --check reports an available update without changing VERSION (EGB-716)" { + setup_tool_repo 0.1.0.0 + advance_tool_remote 0.2.0.0 + run "$TOOL/secrets" upgrade --check + [ "$status" -eq 0 ] + [[ "$output" == *"Update available"* ]] || false + [[ "$output" == *"0.1.0.0"* ]] || false + # --check must not pull: local VERSION is untouched. + [ "$(cat "$TOOL/VERSION")" = "0.1.0.0" ] +} + +@test "upgrade --check is clean when already current (EGB-716)" { + setup_tool_repo 0.2.0.0 + run "$TOOL/secrets" upgrade --check + [ "$status" -eq 0 ] + [[ "$output" == *"up to date"* ]] || false +} + +@test "upgrade fast-forwards and reports old -> new (EGB-716)" { + setup_tool_repo 0.1.0.0 + advance_tool_remote 0.2.0.0 + run "$TOOL/secrets" upgrade + [ "$status" -eq 0 ] + [[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false + [ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ] +} + +@test "upgrade is a no-op when already at the latest (EGB-716)" { + setup_tool_repo 0.2.0.0 + run "$TOOL/secrets" upgrade + [ "$status" -eq 0 ] + [[ "$output" == *"up to date"* ]] || false + [ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ] +} + +@test "upgrade refuses when the tool dir is not a git checkout (EGB-716)" { + local d="$HOME/plain-tool" + mkdir -p "$d" + cp "$SECRETS_BIN" "$d/secrets" + echo 0.1.0.0 > "$d/VERSION" + cd "$d" + run "$d/secrets" upgrade + [ "$status" -eq 1 ] + [[ "$output" == *"git checkout"* ]] || false +} + +@test "upgrade rejects an unknown flag (EGB-716)" { + setup_tool_repo 0.1.0.0 + run "$TOOL/secrets" upgrade --bogus + [ "$status" -eq 1 ] + [[ "$output" == *"Unknown upgrade flag"* ]] || false +} + +@test "upgrade re-checks store skew and confirms the client caught up (EGB-716)" { + setup_tool_repo 0.1.0.0 + advance_tool_remote 0.9.0.0 + # A store last written by a newer client than our starting version. + git -c init.defaultBranch=main init -q "$SECRETS_DIR" + echo 0.8.0.0 > "$SECRETS_DIR/.secrets-writer-version" + run "$TOOL/secrets" upgrade + [ "$status" -eq 0 ] + [[ "$output" == *"v0.1.0.0 -> v0.9.0.0"* ]] || false + # New client (0.9.0.0) is now ahead of the store's last writer (0.8.0.0). + [[ "$output" == *"at or ahead"* ]] || false +} + +@test "upgrade still notes when the store is ahead of the upgraded client (EGB-716)" { + setup_tool_repo 0.1.0.0 + advance_tool_remote 0.2.0.0 + git -c init.defaultBranch=main init -q "$SECRETS_DIR" + echo 0.9.0.0 > "$SECRETS_DIR/.secrets-writer-version" + run "$TOOL/secrets" upgrade + [ "$status" -eq 0 ] + [[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false + [[ "$output" == *"still ahead"* ]] || false +} From a17ae4448bc4555a0d483cde120c02c127da4bc3 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Thu, 18 Jun 2026 11:01:17 -0700 Subject: [PATCH 11/31] chore: bump version and changelog (v0.7.4.0) Co-Authored-By: Claude Opus 4.8 (1M context) --- CHANGELOG.md | 18 ++++++++++++++++++ VERSION | 2 +- 2 files changed, 19 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 475794a..ce00bf8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,24 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.4.0] - 2026-06-18 + +### Added + +- **`secrets upgrade` verb (EGB-716)** — the fix path paired with the EGB-713 + version-skew *warning*. Until now the warning told you you were behind but not + how to catch up; `secrets upgrade` closes that loop. + - **`secrets upgrade`** — `git -C "$SCRIPT_DIR" pull --ff-only` on the tool's + own checkout (fast-forward only — never merges or rewrites local commits), + reports `vOLD -> vNEW`, then best-effort re-checks the store's recorded + writer-version against the new version so you see whether the EGB-713 nudge + is now cleared (the new code itself takes effect on your next command). + - **`secrets upgrade --check`** — reports whether an update is available + (`git fetch` + compare to upstream) and changes nothing. + - Deliberately thin: no auto-update, no background polling (this is a security + tool). Directed errors for not-a-git-checkout, no upstream, a diverged/dirty + branch, or being offline. + ## [0.7.3.1] - 2026-06-18 ### Changed diff --git a/VERSION b/VERSION index 512d674..584db57 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.3.1 +0.7.4.0 From 9e2a563059eb2a7ed101995633672ded6a635fcd Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Wed, 24 Jun 2026 12:12:14 -0700 Subject: [PATCH 12/31] docs: multi-recipient age encryption design spec (EGB-283) Co-Authored-By: Claude Opus 4.8 (1M context) --- ...4-multi-recipient-age-encryption-design.md | 230 ++++++++++++++++++ 1 file changed, 230 insertions(+) create mode 100644 docs/superpowers/specs/2026-06-24-multi-recipient-age-encryption-design.md diff --git a/docs/superpowers/specs/2026-06-24-multi-recipient-age-encryption-design.md b/docs/superpowers/specs/2026-06-24-multi-recipient-age-encryption-design.md new file mode 100644 index 0000000..2141dea --- /dev/null +++ b/docs/superpowers/specs/2026-06-24-multi-recipient-age-encryption-design.md @@ -0,0 +1,230 @@ +# EGB-283 — Multi-recipient age encryption + +**Date:** 2026-06-24 +**Issue:** [EGB-283](https://linear.app/egbt/issue/EGB-283) — secrets: multi-recipient age encryption (multiple keys per file) +**Status:** Design approved, ready for implementation plan +**Related:** EGB-281 (multi-store), EGB-677/EGB-703 (manifest + store-format v2) + +## Problem + +Today every blob in a store is encrypted to exactly one age public key (`age -r "$pubkey"`, +where `$pubkey` is derived from the store's single `key.txt`). The whole team shares one +private key. EGB-281's multi-store work lets users separate work/personal/client secrets into +different stores, each with its own key — but *within* a single store there is still only one key, +so onboarding/offboarding a teammate means sharing or rotating one secret by hand. + +age natively supports multiple recipients: `age -r KEY1 -r KEY2 -o file.age input` writes one +recipient stanza per key, and any matching identity decrypts. This lets a single store have N +members, each with their own keypair. Adding/removing a teammate becomes a re-encrypt against the +current recipient set — no shared password. + +## Goals + +- A store can encrypt every blob to N recipient public keys. +- Recipient set is managed with first-class commands (`secrets recipients add/rm/list`). +- The recipient set is **singular and consistent per store**: every blob is always readable by + exactly the current set. +- Fully backward compatible: existing single-key stores keep working untouched; the feature is + opt-in and detected by file presence (no store-format-marker bump). +- Decryption path is unchanged (members use their own `key.txt`). + +## Non-goals (YAGNI — explicit scope cuts) + +- **SSH recipients** (`ssh-ed25519` / `ssh-rsa`). Native age X25519 keys cover the team-key use + case; SSH adds a parsing/format axis. Clean future follow-up. +- **Per-file or per-project recipient subsets.** The whole store shares one recipient set. +- **Key discovery / distribution.** Public keys are pasted in out of band, exactly as `key.txt` + is shared today. +- **Merging recipients into a project-level config** (`.secrets.json` / `.secrets-files`). See + "Why recipients are not in the project manifest" below. + +## Design decisions (resolved during brainstorming) + +1. **Storage:** committed `recipients.txt` at the store root, managed via + `secrets recipients add/rm/list` subcommands. +2. **Re-encrypt scope:** `add`/`rm` re-encrypt the **entire store immediately** to the new set in + one commit. The store is always consistent. +3. **Backward compatibility:** absence of `recipients.txt` ⇒ exact current single-key behavior. + First `recipients add` on a legacy store bootstraps the file seeded with the local pubkey plus + the new key. `init` going forward seeds `recipients.txt` with the freshly generated pubkey + (born-multi). +4. **`rekey` semantics:** on a multi-recipient store, `rekey` becomes "re-encrypt all to the + current `recipients.txt` set" (no new keypair). On a legacy store it keeps today's behavior + (generate a new keypair, re-encrypt to it). One shared re-encrypt engine. +5. **Store config shape:** keep `recipients.txt` as its own plain, age-native file (jq-free), + alongside the existing one-line `.secrets-format` marker — matching the repo's + small-single-purpose-plain-file convention. Not folded into a JSON store-config. + +## Why recipients are not in the project manifest + +The tool has two config planes in two different git repos: + +| Plane | Location | Files | Scope | +| ----------- | -------------------------------- | -------------------------------------------------- | --------------------------- | +| **Project** | `$PWD` (the project's own repo) | `.secrets.json` (absorbs legacy `.secrets-files`), `.secrets-store` | *What this project syncs* | +| **Store** | `$SECRETS_DIR` (`~/.secrets`) | `.secrets-format`, **`recipients.txt`** (new) | *Metadata about the encrypted repo* | + +Recipients are **store-scoped** — who can decrypt *this store*, shared by every project in it. +Putting them in a project-level manifest would let each project carry its own copy and **diverge**, +the exact inconsistency the "always re-encrypt the whole store to one set" rule prevents. It also +collides with the deliberate EGB-703 decision that *the store holds no project manifest*. So the +recipient set lives with the store, next to `.secrets-format`. + +## `recipients.txt` format and security rails + +Lives at `$SECRETS_DIR/recipients.txt`, **committed** (public keys are not secret; the store +`.gitignore` only blocks `key.txt` and plaintext env files, so the file is tracked automatically). +age `-R` format: one recipient per line, `# comment` and blank lines allowed. + +``` +# alice (laptop) +age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9qsxxxxxx +# bob +age1lggyhqrw2nlhcxprm67z43rta597azn8gknawjehu9d9dl0jq3yqxxxxxx +``` + +**We do not pass the file path to `age -R`.** A committed file is an injection surface, so the +tool parses it itself into a validated indexed array `RECIPIENT_ARGS=(-r age1… -r age1…)`, +mirroring the conservative posture of `.secrets-store` / `.secrets-files`: + +- Each non-comment, non-blank line (after trim) must match a native age X25519 recipient exactly: + `^age1[0-9a-z]{58}$`. Anything else — SSH recipients, shell metacharacters, whitespace inside the + token, control/ANSI characters, `-r`-injection look-alikes — is **rejected with a clear error**. + No shell expansion, ever. +- A symlinked `recipients.txt` is refused (same rail as the manifests). +- `--name` labels (written as `# ` comment lines above the key) are restricted to + `[A-Za-z0-9 ._-]`; anything else is rejected. This blocks comment-injection into the file. +- The parser is pure bash (bash-3.2 safe). Indexed arrays are fine on bash 3.2; only *associative* + arrays are bash-4. + +Validation is the single source of truth — both the `recipients` subcommands and `_load_recipients` +(below) route through the same validator, so an externally hand-edited malicious file is caught on +the next encrypt, not just at `add` time. + +## Components + +### `_load_recipients()` — populate `RECIPIENT_ARGS` +Called once per command that encrypts. Populates the global indexed array `RECIPIENT_ARGS`: + +- `recipients.txt` present → validated array of every key in the file (error out on any invalid + line; refuse an empty/all-comment file). +- absent (legacy store) → `(-r )`, identical to today's single-recipient + behavior. + +### `_reencrypt_all()` — shared re-encrypt engine +Factored out of today's `cmd_rekey` decrypt→re-encrypt loop: + +1. Decrypt every `*.age` in the store (recursive `find -type f -name '*.age'`, covering nested + manifest dotenv blobs and `external/` blobs) with the local `key.txt` into a tmpdir. The + operator must be a current recipient; a decryption failure aborts with the old state preserved. +2. Re-encrypt each file with `age "${RECIPIENT_ARGS[@]}"` back to its relpath. +3. `ensure_store_protections`, `git add -A`, commit, push (if a remote exists). + +All recipient-changing paths call it: + +| Command | Behavior | +| ------------------------------- | -------------------------------------------------------------------- | +| `recipients add` / `rm` | edit `recipients.txt` → `_load_recipients` → `_reencrypt_all` | +| `rekey` (multi-recipient store) | `_reencrypt_all` to current set, **no new keypair** | +| `rekey` (legacy store) | today's behavior: generate new keypair, set recipients to it, re-encrypt | +| `reencrypt` (new, idempotent) | `_reencrypt_all` — heal/backfill after a manual edit | + +### `secrets recipients` subcommand +- `recipients list` — prints names + keys from `recipients.txt` (read-only, jq-free). On a legacy + store, prints the single derived pubkey with a "single-key (no recipients.txt)" note. +- `recipients add [--name