diff --git a/CHANGELOG.md b/CHANGELOG.md index d7f6ecf..4bee6fb 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,322 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.7.0] - 2026-09-08 + +### Fixed + +- **pnpm monorepos discovered no workspace secrets (EGB-1232)** — both + workspace call sites resolved patterns from `package.json`'s `workspaces` + key only, which pnpm does not use (it declares `packages:` in + `pnpm-workspace.yaml`). `secrets push -w` refused outright; plain `secrets + push` failed *silently* — `_maybe_workspace_env_files` returned 0 the moment + the key was absent, so the auto-discovery that exists to cover push's + root-only scan was inert on every pnpm repo and printed "Nothing new to + add", indistinguishable from a repo that genuinely had nothing. Workspace + patterns now resolve through one shared source that falls back to + `pnpm-workspace.yaml`. +- **yarn's object `workspaces` form was never expanded (EGB-1232)** — the + filter `.workspaces // .workspaces.packages | .[]` short-circuits on yarn's + truthy object, so `.workspaces.packages` was never evaluated and `.[]` + iterated the object's values, yielding the pattern array itself as a single + token. Only npm's array form worked. Now type-aware, handling npm's array, + yarn's object, and absent/null alike. + +### Added + +- **`pnpm-workspace.yaml` support** — the `packages:` block sequence is read + without a YAML dependency: block form only, stopping at the next top-level + key so pnpm 10's `onlyBuiltDependencies:`/`catalog:` cannot leak in as glob + patterns, with quote and inline-comment handling and a symlink refusal. + `package.json` wins when it declares workspaces; `pnpm-workspace.yaml` is the + fallback. jq is now required only when `package.json` is the source, so a + pnpm-only repo resolves workspaces jq-free. +- **Workspace patterns are validated before glob expansion** — no absolute + paths, `..` traversal, shell metacharacters, or whitespace reach the + unquoted expansion; pnpm `!` negations are skipped. Same conservative rail + as `.secrets-store` / `.secrets-files`. +- **A monorepo-shaped root that resolves no workspaces now says so** — if a + `pnpm-workspace.yaml` or `packages/` directory is present but no workspace + packages can be read, `push` warns on stderr and points at `secrets add`, + instead of returning in silence. `secrets push -w`'s error now names + `pnpm-workspace.yaml` when that is the file present, rather than blaming a + `package.json` the repo may not use for workspaces. + +## [0.7.6.0] - 2026-09-08 + +### Added + +- **`secrets sync` — reconcile a diverged store (EGB-1231)** — the store is a + git repo, and once a clone was both ahead and behind its remote the CLI had + no way out: `push` demanded a fast-forward and pointed at `pull`, which + could not fast-forward either, so the advice looped and recovery meant + hand-running git next to a directory of encrypted blobs. `secrets sync` + fetches, stashes uncommitted blob edits, rebases local commits onto the + remote, restores the stash, and then asks before publishing local commits to + the shared store. `--yes` skips the prompt (scripts/CI); `--dry-run` reports + ahead/behind/dirty and what would happen, changing nothing. Deliberately + non-destructive: no merge, no force-push, no `reset --hard`, no `stash + drop`. A rebase conflict aborts, restores the stash, names the conflicting + files, and leaves the store exactly as found. +- **Store state in `secrets which` (EGB-1231)** — a new `remote:` line reports + the store's `ahead N, behind N, N modified` (or `up to date`) against its + upstream, with a `(run: secrets sync)` hint when there is anything to + reconcile. Offline-safe (reports against the last fetch, never reaches the + network) and silent for a local-only store or one with no upstream. + +### Fixed + +- **`secrets pull` no longer fails silently when the store can't sync + (EGB-1230)** — the store sync was `git pull >/dev/null 2>&1` under `set -euo + pipefail`, so a store that could not fast-forward killed the script at that + line with git's exit 128 and *nothing* on stdout or stderr. The user saw a + banner, no restored files, and no reason — indistinguishable from a project + with nothing to pull, and easy to lose entirely in a pipeline. The sync is + now guarded, git's output is captured and surfaced as the diagnosis, and the + error names the store path and points at `secrets sync`. + +### Changed + +- **`secrets pull`'s store sync is now fast-forward only**, matching the push + path. A plain `git pull` could quietly manufacture a merge commit in the + store; divergence is now resolved in exactly one place — `secrets sync`. +- **The push path's dead-end advice** ("Run 'secrets pull' first, then retry + push") now points at `secrets sync` and includes git's own output. + +## [0.7.5.0] - 2026-06-24 + +### Added + +- **Multi-recipient age encryption (EGB-283)** — a store-scoped, committed + `recipients.txt` (age `-R` format, with `# name` comment lines) lets one + store encrypt every blob to N age public keys — one per team member. + `secrets recipients add [--name N]` adds a key and immediately + re-encrypts the whole store; `secrets recipients rm [--yes]` + removes one and re-encrypts; `secrets recipients list` shows the current + set (or a note that the store is still single-key). A new `secrets + reencrypt` command re-encrypts every blob to the current recipients without + changing the set (idempotent heal / backfill after a manual edit). Absence + of `recipients.txt` preserves exact legacy single-key behavior; the first + `recipients add` on a legacy store bootstraps the file seeded with the + local pubkey plus the new key. `init` now seeds `recipients.txt` born-multi + with the freshly generated pubkey. + +### Changed + +- **`secrets rekey` on a multi-recipient store** no longer generates a new + keypair — instead it re-encrypts all blobs to the current `recipients.txt` + set (the shared `_reencrypt_all` engine). On a legacy store (no + `recipients.txt`) `rekey` keeps today's generate-new-keypair behavior. +- **`secrets which`** now prints a `recipients: N (name, …)` line, or + `recipients: single-key (no recipients.txt)` for a legacy store. +- **`secrets verify` / `verify --all`** assert that each blob's age + recipient-stanza count equals the number of entries in `recipients.txt` + (skipped on legacy stores). Exits non-zero on any count mismatch so it can + gate CI or a migration. + +## [0.7.4.0] - 2026-06-18 + +### Added + +- **`secrets upgrade` verb (EGB-716)** — the fix path paired with the EGB-713 + version-skew *warning*. Until now the warning told you you were behind but not + how to catch up; `secrets upgrade` closes that loop. + - **`secrets upgrade`** — `git -C "$SCRIPT_DIR" pull --ff-only` on the tool's + own checkout (fast-forward only — never merges or rewrites local commits), + reports `vOLD -> vNEW`, then best-effort re-checks the store's recorded + writer-version against the new version so you see whether the EGB-713 nudge + is now cleared (the new code itself takes effect on your next command). + - **`secrets upgrade --check`** — reports whether an update is available + (`git fetch` + compare to upstream) and changes nothing. + - Deliberately thin: no auto-update, no background polling (this is a security + tool). Directed errors for not-a-git-checkout, no upstream, a diverged/dirty + branch, or being offline. + +## [0.7.3.1] - 2026-06-18 + +### Changed + +- **EGB-677 stage-1 structural cleanups (EGB-701)** — tech-debt dedup with one + new safety warning; no behavior change for the manifest-driven (v2) happy path. + - **`secrets which` now reuses the one external-entry extractor** the push/pull + path uses (`_json_external_entries`) instead of its own duplicated `jq` + projection. So `which` applies the same `properties`→`gradle-properties` + normalization and skips (with a warning) the same malformed external entries + the sync path drops — `which` shows exactly what will sync, not a stale raw + projection that could drift from the real behavior. + - **The two external-manifest read guards are factored into shared helpers** — + `_json_readable` (plain regular file, silent) and `_legacy_readable` (warns + and skips a symlinked `.secrets-files`) — so `_external_entries_for_push` and + `_external_entries_for_pull` can't drift apart. + +### Fixed + +- **Legacy (manifest-less) `pull` no longer silently under-restores (EGB-701)** — + the manifest-less pull path globs only top-level `*.age`/`.*.age`, so a nested + dotenv blob (`/.age`) written by a manifest-driven push on + another machine was invisible: restored nothing, counted nothing, said nothing. + It now **warns** and names each nested blob it can't reach (external blobs are + excluded — `pull_external_files` handles those), pointing at committing a + `.secrets.json` as the fix. The manifest-driven pull already restored nesting + correctly; this only closes the legacy path's blind spot. + +## [0.7.3.0] - 2026-06-08 + +### Added + +- **Real install / onboarding scripts (EGB-671)** — onboarding a machine is now + (close to) one command, and a mis-copied key fails loudly instead of silently. + - **`secrets join --remote --key `** — second-machine onboarding in + one verb: clones the vault, installs the key at mode 600, and **verifies the + key actually decrypts the store before declaring success**. An empty vault + reports "nothing to verify yet" (it never prints a false `VERIFIED`); a wrong + key fails loudly with the store left in place to fix. All security logic + (store resolution, URL handling, path rails) is reused from the audited core, + not re-implemented in a side script. + - **`secrets init --remote `** — wires the remote and pushes the initial + store so the upstream branch exists, so your first project `push` doesn't trip + the fast-forward-pull guard on a brand-new empty remote. Run interactively, + `init` also offers to add your first project's secrets (default No, skipped + under `--yes` / non-interactive, so it stays a clean primitive for CI). + - **`install.sh`** — thin bootstrap that ships in the repo: checks `age` + `jq` + + `git`, then prints the `PATH` line, the onboarding next-steps, the upgrade + one-liner, and a key-transfer hint. It never edits your shell config and never + runs `sudo` (it prints the command so you stay in control). + - **First-manifest `options.autoAdd` prompt (EGB-677 contract #2)** — the first + `push` that scaffolds a project's manifest now records an explicit, committed + `options.autoAdd` value (asked once when interactive; the default ON, written + explicitly, under automation). + +### Fixed + +- **Day-2 silent decrypt failure** — `secrets pull` now dies loudly when a blob + fails to decrypt with the current key (all three decrypt paths), instead of + emitting a warning and continuing with exit 0. A wrong key can no longer pass + unnoticed after onboarding. +- The `secrets init` second-machine trap now points at `secrets join` (the real + one-command path) instead of a manual `git clone`. + +## [0.7.2.0] - 2026-06-08 + +### Added + +- **`secrets list --json` (EGB-699)** — machine-readable listing for tooling and + CI. Emits a single JSON object on stdout: `{"store", "projects": [{"name", + "entries": [...]}]}`, where each entry self-describes via a `type` + discriminator — `{"type":"dotenv","path":}` or + `{"type":"external","subtype":"properties"|"file","path":}`. Reflects the + same recursive store walk as the human `list` (nested `/.age` + + `external/.age`). jq does the assembly so paths escape correctly; the + human store hint is suppressed so stdout stays pure JSON (notices → stderr). + jq is required only for `--json`. Feeds the EGB-671 install scripts, which need + to enumerate a cloned store programmatically instead of scraping the table. + +## [0.7.1.0] - 2026-06-08 + +### Added + +- **Version-skew nudge (EGB-713)** — the store now records the highest `secrets` + version that has written to it (`.secrets-writer-version`, committed, + monotonic). When you run a command against a store last written by a *newer* + `secrets` than your own, you get a one-line non-fatal stderr nudge to update + your tool; `secrets which` shows the store's `written-by:` version (and flags + when you're behind). Stores written by older builds carry no stamp and stay + silent — no false alarms. The loud counterpart to EGB-712's quiet + forcing function. + +## [0.7.0.0] - 2026-06-08 + +### Changed + +- **Additive store-format v2 (EGB-712)** — upgraded `secrets` clients now read + either external blob suffix (`.properties.age` or the legacy + `.gradle-properties.age`) and **dual-write** a `properties` external whenever a + v1 twin already exists in the store. Existing externals keep working for + teammates on an older `secrets`; only a brand-new `properties` external is + written v2-only (a gentle "upgrade to see it" forcing function). dotenv and + whole-`file` externals are unchanged across formats and always propagate. +- **`secrets migrate --finalize` is now optional GC**, not a required milestone. + Because clients dual-write and read-fall-back, no teammate is ever cut off by + *not* finalizing; finalize only reclaims the duplicate v1 blobs, and stays + deferrable indefinitely. Its safety gates are unchanged. This defuses the + cross-machine "all clients must be v2 before finalize" coordination gate. + +### Added + +- **`secrets migrate --status`** now reports `v2-only` externals per project + (the ones an un-upgraded client cannot read), so you can see the forcing + function's footprint at a glance. + +## [0.6.1.0] - 2026-06-08 + +### Changed + +- **`secrets migrate` copy-forward is now manifest-free (EGB-710)** — the + per-project step enumerates the store's `*.gradle-properties.age` blobs + directly (the same source of truth `--finalize` uses) instead of reading + `.secrets.json`. A legacy `.secrets-files`-only project now migrates cleanly + instead of dead-ending with "No .secrets.json", and a store blob the manifest + no longer declares still gets a v2 twin (so `--finalize` won't refuse it). + Running migrate in a project with no v1 properties blobs is a clean no-op. + +### Added + +- **`secrets migrate --status`** — a read-only survey that walks every project + in the store and reports its v2 readiness (v2-ready / migrated / NEEDS + MIGRATE), then whether the store as a whole is finalize-ready. Exits non-zero + while any v1 blob is un-twinned, so it can gate the path to `--finalize`. + +## [0.6.0.1] - 2026-06-08 + +### Added + +- **`secrets which` now prints the manifest version (EGB-700)** — the manifest + header line shows `version N` alongside the store format, so a single + `secrets which` tells you both the on-disk store format and the `.secrets.json` + schema version at a glance. + +## [0.6.0.0] - 2026-06-07 + +### Added + +- **Self-describing store format (v2) + `secrets migrate` (EGB-703)** — the + store now records its format in a committed `.secrets-format` file, and + `secrets which` prints it (`format: v2`). A fresh `secrets init` creates a + v2 store; existing stores read as v1 until migrated. +- **`secrets migrate`** — copy-forward a project's encrypted blobs to the v2 + layout. It is non-destructive: the old blobs are kept until you finalize, so + a half-migrated store stays fully readable and recoverable. `secrets migrate + --dry-run` previews exactly what would change without writing anything. +- **`secrets migrate --finalize`** — the one destructive step, run once + store-wide. It refuses unless `secrets verify` passes and every blob has its + new-format twin, cuts a `pre-v2-migrate-*` recovery tag first, then drops the + old blobs. It asks for confirmation (or `--yes`) because a machine still on + an older `secrets` will stop seeing migrated external files until it updates. + +### Changed + +- The external `properties` blob is stored as `.properties.age` in a v2 + store (was `.gradle-properties.age`), matching the manifest `type`. + `push`, `pull`, and `verify` pick the right name automatically from the store + format, so v1 and v2 stores both keep working during a migration. + +## [0.5.0.0] - 2026-06-07 + +### Added + +- **`secrets verify` (EGB-698)** — a read-only integrity check. Run it in a + project to cross-check the committed `.secrets.json` against the store both + ways (entries declared but missing from the store, and stored blobs with no + manifest entry) and decrypt-test every blob with your current key. Catches a + partially-synced store, a stale key, or a manifest that has drifted from the + store. Plaintext is streamed to `/dev/null` and never written to disk. +- **`secrets verify --all`** — decrypt-tests every blob in every project in the + store: a fast store-wide integrity sweep. (The store carries no manifests, so + `--all` checks decryptability only, not manifest consistency.) +- Both modes recurse the whole project tree, so nested entries and external + files are covered. `secrets verify` exits non-zero on any problem, so it can + gate CI or a future store migration. + ## [0.4.0.0] - 2026-06-07 ### Added @@ -159,6 +475,6 @@ and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. - 37 → 66 tests. New coverage: store resolution rules and precedence, walk-up boundaries, command-injection prevention, key-file re-derivation across stores, teammate-onboarding error path, monorepo workspace binding, F1–F5 adversarial regressions. -[0.1.1.0]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.1.0 -[0.1.0.1]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.0.1 -[0.1.0.0]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.0.0 +[0.1.1.0]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.1.0 +[0.1.0.1]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.0.1 +[0.1.0.0]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.0.0 diff --git a/CLAUDE.md b/CLAUDE.md index 262818d..a0261b8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -16,7 +16,7 @@ cd ~/my-project && ./secrets pull # Pull + decrypt .env* files ```bash brew install bats-core -bats test/ # runs secrets.bats + manifest.bats +bats test/ # runs secrets.bats + manifest.bats + migrate.bats + upgrade.bats + recipients.bats ./test/run-security.sh # security regression subset + operator sign-off (see below) ``` @@ -56,15 +56,36 @@ skips security specialist + red team, and Step 11 skips adversarial review. ## Architecture -Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey. +Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, sync, upgrade. - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable) - Storage: Private git repo at `~/.secrets/` - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) -- Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). v1 store layout unchanged in stage 1: nested entries land at `/.age`; `properties` blobs keep the legacy `.gradle-properties.age` suffix until the stage-2 store migration. jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. +- Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `/.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. +- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves. +- Store sync + divergence (EGB-1230/EGB-1231): the store is a git repo, so a clone can end up ahead of and behind its remote at once. **EGB-1230:** `cmd_pull`'s sync used to be `git pull >/dev/null 2>&1` under `set -euo pipefail` — a store that couldn't fast-forward killed the script there with git's exit 128 and nothing on either stream (a banner, no files, no reason; invisible in a pipeline). It now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. That sync is **`--ff-only`**, matching the push path — a plain `git pull` could quietly manufacture a merge commit in the store, and divergence is now resolved in exactly one place. **EGB-1231:** `_store_git_state` emits `ahead\tbehind\tdirty` (from `rev-list --left-right --count @{u}...HEAD` plus `status --porcelain`) and `_format_store_state` renders it; `cmd_which` prints a `remote:` line from them — offline-safe (reports against the last fetch), silent with no remote/upstream. `cmd_sync` is the reconcile verb the CLI was missing: fetch → report state → stash (`push -u`) → `rebase @{u}` → restore stash → `ensure_store_protections` (rebased-in history may lack `.gitignore`, and a store missing the `key.txt` line would stage the private key — same reasoning as push) → **confirmation-gated** `git push` of local commits. The gate (`_sync_confirm_push`) reads `/dev/tty` and requires a tty, so it stays CLOSED in scripts/CI rather than publishing to a shared store by default; `--yes` opens it, `--dry-run` reports and returns before any mutation. Non-destructive by construction: no merge, no `--force`, no `reset --hard`, no `stash drop`. A rebase conflict collects the conflicting paths BEFORE `rebase --abort` (the abort clears them), restores the stash, and dies — store byte-identical to how it was found. `_sync_restore_stash` never drops the stash on a failed pop; it tells the operator where their only copy lives. `cmd_sync` does not `_stamp_writer_version`: it replays existing commits rather than authoring content, and the stamp is specified to ride a store-committing `git add -A`. Test suite: `test/sync.bats` (25 tests), including a grep over the `cmd_sync` body asserting the destructive git verbs never appear in it. +- Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR//` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`. - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below -- Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq` +- Workspaces (EGB-1232): `--workspaces` and the plain-push workspace re-scan both resolve patterns through ONE source — `_workspace_patterns()`. It reads `package.json` `.workspaces` via `$WORKSPACES_JQ` (type-aware: handles npm's array AND yarn's object `{packages:[...]}` form) and falls back to `pnpm-workspace.yaml`'s `packages:` block when package.json declares none. **Two defects fixed:** (1) both call sites were package.json-only, so no pnpm monorepo ever resolved a workspace — and `_maybe_workspace_env_files` failed *silently* (`jq -e '.workspaces' ... || return 0`), making auto-discovery inert and `push` print "Nothing new to add", indistinguishable from a repo with nothing new; it bit the same repo twice. (2) the old filter `.workspaces // .workspaces.packages | .[]` short-circuits on yarn's truthy object, iterating the object's values and yielding the pattern ARRAY as a single token. Note a naive reorder does NOT fix it — `.workspaces.packages` errors on an array; hence the `if type == "object"` form. `_pnpm_workspace_packages()` is a deliberate non-parser (block sequence only, stops at the next top-level key so pnpm 10's `onlyBuiltDependencies:`/`catalog:` can't leak in as globs, strips quotes/inline comments, refuses a symlinked file). Patterns are validated by `_valid_workspace_pattern` before they reach the unquoted `for pattern in $patterns` glob expansion (no absolute/`..`/metacharacters/whitespace; pnpm `!` negations skipped) — same posture as `.secrets-store`/`.secrets-files`. `_looks_like_monorepo` + `_workspace_source` turn the old silent return into a warning that names the real file, and `get_workspaces`'s error names `pnpm-workspace.yaml` when that's what's present instead of blaming package.json. jq is required only when package.json is the source. **Scope note:** the workspace re-scan still runs only for projects that already have a `.secrets.json` — push's root-scan-only behavior on a first push is by design (EGB-677 E13), and EGB-1232 is about the fallback that covers it never engaging. Tests: `test/workspaces.bats` (18). - Safety: Pre-commit hook rejects plaintext secret files (`.env`, `.dev.vars`, `gradle.properties`) +- Multi-recipient (EGB-283): a store-scoped, committed `recipients.txt` (age `-R` + format, `# name` comments) lets one store encrypt every blob to N age keys — + one per team member. Managed via `secrets recipients add/rm/list`; absence of + the file ⇒ legacy single-key behavior (recipients = the pubkey derived from + `key.txt`). The file is parsed by us (never `age -R `) into a validated + `RECIPIENT_ARGS` array (native age X25519 only, `age1[0-9a-z]{58}`; SSH + recipients rejected; symlinked file refused) — same conservative posture as + `.secrets-store`/`.secrets-files`. `_load_recipients` populates the array; + every encrypt site routes through it. Any recipient change re-encrypts the + WHOLE store in one commit via the shared `_reencrypt_all` engine (also used by + the new `secrets reencrypt` and by `rekey` on a multi-recipient store, where + rekey re-encrypts to the set with NO new keypair; legacy stores keep rekey's + generate-new-keypair behavior). `init` seeds `recipients.txt` born-multi. + `which` prints `recipients: N`; `verify`/`verify --all` assert each blob's + age recipient-stanza count equals `recipients.txt`'s length. Removal takes + effect going forward — git history stays readable by an old key, so rotate + genuinely-sensitive values. Decryption is unchanged (each member uses their + own `key.txt`). - Portability: must run on system bash 3.2 (macOS) — no associative arrays or bash-4 features ## Project Structure @@ -73,8 +94,13 @@ Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rek secrets # CLI script (~2000 lines bash) hooks/pre-commit # Pre-commit hook template test/ - secrets.bats # bats-core test suite (133 tests) - manifest.bats # EGB-677 .secrets.json manifest tests (60 tests) + secrets.bats # bats-core test suite (140 tests) + manifest.bats # EGB-677 .secrets.json manifest tests (83 tests) + migrate.bats # EGB-703 store-format-v2 migration tests (35 tests) + upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests) + sync.bats # EGB-1230/1231 store sync + divergence reconcile tests (25 tests) + workspaces.bats # EGB-1232 npm/yarn/pnpm workspace discovery tests (18 tests) + recipients.bats # EGB-283 multi-recipient age encryption tests (34 tests) test_helper.bash # Shared setup/teardown README.md # User-facing documentation CLAUDE.md # This file @@ -104,7 +130,7 @@ The active store directory is picked by `resolve_store()` using these rules, hig Key design decisions (all driven by /autoplan review): - **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel). -- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). +- **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). **EGB-701 cleanups:** (1) the *legacy* (manifest-less) `pull` keeps its non-recursive globs but now **warns** when nested `/.age` blobs exist that those globs can't see (it excludes `external/`, which `pull_external_files` handles) — so a manifest-less pull never silently under-restores; the fix the warning points at is committing a `.secrets.json`. (2) `cmd_which`, push, and pull share one external extractor (`_json_external_entries`), so `which` applies the same `properties`→`gradle-properties` normalization and skip-with-warning rules the sync path does (it shows exactly what will sync, not a stale raw projection). (3) the two external-manifest read guards are factored into `_json_readable` (plain regular file, silent) / `_legacy_readable` (warn-and-skip on a symlinked legacy manifest). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv`→`path` or `external`→`subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode). - **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `.secrets-bak` before each merge. - **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins. - **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped). @@ -112,24 +138,43 @@ Key design decisions (all driven by /autoplan review): ## Deploy Configuration -- Platform: NONE (distributed via `git clone` from Codeberg) +- Platform: NONE (distributed via `git clone` from the private Forgejo at `git.dev.egbt.com`) - Production URL: N/A (no live service) - Release model: merge to `main` is the release. Optionally tagged with `v`. - Verification after merge: a fresh `git clone` should produce a working `secrets which` against an isolated `$HOME`. No canary URL. - Staging: none. - Rollback: revert the merge commit on `main` (and delete the tag) to roll back. -## Codeberg operations +## Forge operations (self-hosted Forgejo) -The remote is Codeberg (Forgejo) — `gh`/`glab` do NOT work here. Use `tea` -(login name: `codeberg`, user `egbt`) for forge operations when a skill's +The remote is a private Forgejo instance at `https://git.dev.egbt.com` +(migrated off Codeberg 2026-09-08). `gh`/`glab` do NOT work here. Use `tea` +(login name: `egbt`, user `brian`) for forge operations when a skill's platform detection comes up "unknown": -- PRs: `tea pr create --base main --title ... --description ...` / `tea pr merge ` -- Releases: `tea releases create --tag v --title "v — " --note ...` +**Always pass `--login egbt --repo egbt/secrets` explicitly.** `tea`'s repo +autodetection fails here ("remote repository required"), and this machine also +has a leftover `codeberg` login pointing at the *old* forge +(`https://codeberg.org`) that `tea` will silently fall back to in +non-interactive mode — which would target the wrong server. Confirm with +`tea logins list` if a command errors. + +- PRs: `tea pr create --login egbt --repo egbt/secrets --base main --head --title ... --description ...` / `tea pr merge --login egbt --repo egbt/secrets` +- Releases: `tea releases create --login egbt --repo egbt/secrets --tag v --title "v — " --note ...` (convention: one release per tag, title `v — `) -- Issues/status: `tea issues`, `tea pr list` -- No CI on this repo: the bats suite run locally is the merge gate. +- Issues/status: `tea issues --login egbt --repo egbt/secrets`, `tea pr list --login egbt --repo egbt/secrets` +- **SSH is on port 2222**, not 22 (port 22 is the host's own sshd). Clone/remote + URLs must be `ssh://git@git.dev.egbt.com:2222/egbt/secrets.git`. A bare + `git@git.dev.egbt.com:egbt/secrets.git` will fail with "Permission denied + (publickey)" because it hits the wrong daemon. +- The host resolves to a Tailscale address — the forge is reachable only on the + VPN. Off-net, push/pull/`tea` all fail to connect; that is expected, not a + credentials problem. +- `FORGEJO_URL` and `FORGEJO_TOKEN` (API token for user `brian`) live in + `~/.zshenv` for direct API calls. +- CI: the instance has an Actions runner available, but no workflow is + configured for this repo yet. The bats suite run locally is still the merge + gate. ## Environment variable diff --git a/README.md b/README.md index 770a86d..acfd2b4 100644 --- a/README.md +++ b/README.md @@ -58,83 +58,97 @@ Beyond project files, `secrets` can also sync files that live *outside* the proj ## Prerequisites -- **macOS** (uses Homebrew for installation) -- **git** (already installed on most Macs — type `git --version` to check) -- **age** (the encryption tool — installed in step 1 below) +- **macOS or Linux** +- **git** (`git --version` to check) +- **age** and **jq** — `install.sh` checks for these and prints the exact install command for your platform (Homebrew on macOS, `apt`/`dnf` on Linux) ## Setup -### First machine (one-time setup) +Clone the tool repo, then run `install.sh`. It checks dependencies and prints the +two commands to finish setup. It never edits your shell config and never runs +sudo — it prints the commands so you stay in control. ```bash -# 1. Install the encryption tool -brew install age - -# 2. Download the secrets tool (this repo — contains only the CLI, no secret files) -git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets - -# 3. Make the 'secrets' command available everywhere -# Add this line to your shell config file (~/.zshrc on Mac): -export PATH="$HOME/dev/secrets:$PATH" -# Then restart your terminal, or run: -source ~/.zshrc - -# 4. Initialize your encrypted secrets store -# This creates a folder at ~/.secrets/ with your encryption key -secrets init - -# 5. Create a PRIVATE repository on GitHub to store your encrypted secrets -# Go to github.com/new, name it something like 'my-secrets', and make sure -# "Private" is selected. Then connect it: -cd ~/.secrets -git remote add origin git@github.com:/my-secrets.git -git push -u origin main +git clone https://git.dev.egbt.com/egbt/secrets.git ~/dev/secrets +cd ~/dev/secrets +./install.sh ``` -> **Important:** Step 5 creates a *separate* private repo for your encrypted secrets. This is different from the `secrets` tool repo you cloned in step 2. The tool repo can be public — it contains no secrets. The `~/.secrets/` repo must be private. +`install.sh` prints a `export PATH="$HOME/dev/secrets:$PATH"` line — add it to your +shell config (`~/.zshrc` or `~/.bashrc`) and restart your terminal. Then onboard +this machine with one of the two flows below. -### Additional machines - -On each new machine (your desktop, a teammate's laptop, etc.): +### First machine (new vault) ```bash -# 1. Install prerequisites and the tool (same as steps 1-3 above) -brew install age -git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets -export PATH="$HOME/dev/secrets:$PATH" # add to ~/.zshrc +# 1. Create a PRIVATE repo for your encrypted secrets (github.com/new or a +# GitLab/Forgejo private repo). It holds only ciphertext — never your key. +# Then wire it up and push the store in one command: +secrets init --remote git@github.com:/my-secrets.git -# 2. Clone the encrypted secrets repo -git clone git@github.com:/my-secrets.git ~/.secrets - -# 3. Copy the encryption key from your first machine -# This is the only step that requires direct machine-to-machine transfer. -# Choose one method: -# -# Option A: AirDrop (Mac to Mac) -# On your first machine, right-click ~/.secrets/key.txt → Share → AirDrop -# Save it to ~/.secrets/key.txt on the new machine -# -# Option B: Secure copy over SSH -# scp first-machine:~/.secrets/key.txt ~/.secrets/key.txt -# -# Option C: USB drive -# Copy key.txt to a USB drive, transfer it, delete from USB after - -# 4. Pull your secrets into any project +# 2. (optional) Add a project's secrets. From a project directory: cd ~/myapp -secrets pull +secrets push +# The first push asks once whether to auto-track new env files and records +# your choice in the project's .secrets.json. ``` +`secrets init --remote` generates your key (`~/.secrets/key.txt`), wires the +remote, and pushes the initial store so the upstream branch exists. The private +secrets repo is separate from this tool repo — the tool repo is public and holds +no secrets; the `~/.secrets/` repo must be private. + +> Running `secrets init` interactively (in a terminal) also offers to add your +> first project's secrets right away. Run it with `--yes` (or in any non-tty +> context like CI) to skip that prompt and just create the vault. + +### Other machines (join an existing vault) + +On a second machine, a desktop, or a teammate's laptop: + +```bash +# 1. Clone the tool and run the bootstrap (as in Setup above) +git clone https://git.dev.egbt.com/egbt/secrets.git ~/dev/secrets +cd ~/dev/secrets && ./install.sh # add the printed PATH line to your shell config + +# 2. Get key.txt onto this machine (the one manual, out-of-band step): +# AirDrop (Mac→Mac), or +# scp first-machine:~/.secrets/key.txt ~/Downloads/key.txt, or +# a USB drive (delete from the drive afterward) + +# 3. Join the vault in one command: +secrets join --remote git@github.com:/my-secrets.git --key ~/Downloads/key.txt +``` + +`secrets join` clones the vault, installs the key at mode 600, and **verifies the +key actually decrypts the store before declaring success** — a mis-copied key +fails loudly here, not silently on a later `secrets pull`. On success it tells you +to run `secrets pull` in any project. + > **The key file (`~/.secrets/key.txt`) is the only thing that needs to be transferred manually.** It never leaves your machines — it's excluded from git, never uploaded, never transmitted over the internet. Anyone with this file can decrypt all your secrets, so treat it like a password. ### Sharing with teammates -To share secrets with a teammate, they need: +**Simple approach (shared key):** To share secrets with a teammate, they need: -1. Access to your private `my-secrets` GitHub repo (add them as a collaborator) -2. A copy of `key.txt` (send it to them directly — AirDrop, USB, or in-person) +1. Access to your private secrets repo (add them as a collaborator) +2. A copy of `key.txt` (send it directly — AirDrop, USB, or in-person) -Everyone on the team uses the same key. When anyone runs `secrets push`, the encrypted files are updated and everyone else can `secrets pull` to get the latest version. +Everyone on the team uses the same key. A teammate joins with +`secrets join --remote --key `. When anyone runs +`secrets push`, the encrypted files update and everyone else runs `secrets pull` +to get the latest. + +### Updating the tool + +```bash +git -C ~/dev/secrets pull +``` + +If your store was last written by a newer client than yours, `secrets` prints a +one-line version-skew nudge — that's your cue to run the command above. + +**Per-teammate keys (recommended for teams):** Use `secrets recipients add` so each person keeps their own private key — no key sharing needed. See [Onboarding and offboarding teammates](#onboarding-and-offboarding-teammates) below. ## Usage @@ -167,8 +181,64 @@ secrets clear | `secrets clear` | Delete plaintext secret files from the current directory | | `secrets run ` | Pull secrets, run a command, then clear secrets when it exits | | `secrets list` | Show all projects that have stored secrets | +| `secrets list --json` | Same listing as a machine-readable JSON object (`{store, projects[].entries[]}`, each entry `dotenv`/`external`) for tooling and CI. JSON goes to stdout; notices to stderr | | `secrets rm ` | Delete a project's secrets from the store | -| `secrets rekey` | Generate a new encryption key and re-encrypt everything | +| `secrets rekey` | Generate a new encryption key and re-encrypt everything (single-key store) or re-encrypt to the current recipients without changing keys (multi-recipient store) | +| `secrets verify [project]` | Check the current project's `.secrets.json` against the store (missing/orphaned blobs) and decrypt every blob. `[project]` overrides the store directory name; the manifest is still read from the current directory | +| `secrets verify --all` | Decrypt-test every blob in every project — a store-wide integrity sweep | +| `secrets migrate [--dry-run]` | Copy-forward this project's encrypted blobs to store format v2 (non-destructive; manifest-free; `--dry-run` previews) | +| `secrets migrate --status` | Survey every project's v2 readiness; exits non-zero until the whole store is finalize-ready | +| `secrets migrate --finalize` | **Optional GC** — drop the old v1 blobs and mark the store pure v2. Never required: upgraded clients dual-write and read-fall-back, so not finalizing never cuts anyone off | +| `secrets recipients list` | List the store's recipient public keys (and names if set) | +| `secrets recipients add [--name N]` | Add a recipient key to the store and immediately re-encrypt every blob to the new set | +| `secrets recipients rm [--yes]` | Remove a recipient and re-encrypt the store; `--yes` required when removing your own key | +| `secrets reencrypt` | Re-encrypt every blob to the current recipients (idempotent — useful after a manual edit or partial failure) | +| `secrets sync` | Reconcile a store that has diverged from its remote: stash local blob edits, rebase onto the remote, restore the stash, then offer to publish your local commits. Never merges, force-pushes, or hard-resets | +| `secrets sync --dry-run` | Report the store's ahead/behind/dirty state and what a reconcile would do; changes nothing | +| `secrets sync --yes` | Reconcile and publish local commits without the confirmation prompt (for scripts) | +| `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks | +| `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing | + +### Upgrading: do teammates on an older `secrets` get new secrets? + +Store-format v2 is **additive** — an upgraded client reads either blob suffix and keeps the old (v1) suffix alive for externals that already existed, so you almost never have to coordinate an upgrade: + +| Secret type | Old client gets it? | +|---|---| +| `.env` / `.env.*` / `.dev.vars` | **Yes, always** (blob name is identical across formats) | +| whole-file external (`file`) | **Yes, always** | +| `properties` external that already existed | **Yes** (dual-written so old clients stay fresh) | +| brand-new `properties` external | **No — must upgrade `secrets`** (the gentle forcing function) | + +"Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`. + +And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent. + +### When the store diverges + +The store is a git repo, so two machines pushing at once can leave your clone +both ahead and behind its remote. `secrets push` needs a fast-forward and +`secrets pull` won't silently merge, so both stop and tell you to run: + +```bash +secrets sync +``` + +`sync` fetches, stashes any uncommitted blob edits, rebases your local commits +onto the remote, restores the stash, and then *asks* before publishing your +commits to the shared store (`--yes` skips the prompt; `--dry-run` just +reports). If the rebase conflicts, it aborts, restores your stash, names the +conflicting files, and leaves the store exactly as it found it — nothing in +the path force-pushes, hard-resets, or drops a stash. + +`secrets which` now reports the same state up front, so you can see it coming: + +``` +store: /Users/you/.secrets +source: default +format: v2 +remote: ahead 1, behind 11, 3 modified (run: secrets sync) +``` ### Automatic project detection @@ -355,10 +425,10 @@ git commit -am "switch to personal secrets" ### Monorepo support -For projects with multiple packages (monorepos using `package.json` workspaces), add the `-w` flag to operate on all workspaces at once: +For projects with multiple packages, add the `-w` flag to operate on all workspaces at once: ```bash -cd ~/myapp # has package.json with "workspaces": ["apps/*", "packages/*"] +cd ~/myapp # npm/yarn workspaces, or a pnpm-workspace.yaml secrets push -w # encrypts secrets from root + each workspace secrets pull -w # decrypts into root + each workspace directory secrets clear -w # clears secrets from root + each workspace @@ -375,7 +445,22 @@ Inside `~/.secrets/`, workspace secrets are organized by path: apps/api/.env.age # api workspace ``` -Requires `jq` (`brew install jq`). +**Where workspaces are declared.** All three package managers are supported: + +| Manager | Declaration | +|---|---| +| npm | `package.json` → `"workspaces": ["apps/*"]` | +| yarn | `package.json` → `"workspaces": {"packages": ["apps/*"]}` | +| pnpm | `pnpm-workspace.yaml` → `packages:` block | + +`package.json` wins when it declares any workspaces; `pnpm-workspace.yaml` is +the fallback. Reading `package.json` requires `jq` (`brew install jq`); a +pnpm-only repo needs no jq for workspace discovery. + +If a root *looks* like a monorepo (a `pnpm-workspace.yaml` or a `packages/` +directory) but no workspace packages can be read from it, `secrets push` says +so on stderr rather than silently discovering nothing — that silence +previously made a pnpm repo indistinguishable from one with nothing to sync. ### External files (Gradle properties) @@ -443,6 +528,64 @@ Some external secrets are whole binary files — an Android upload keystore, a c On `secrets push` the file is encrypted into `/external/`. On `secrets pull` it is restored to the same path with mode `600`; if a different version already exists there, it is backed up to `.secrets-bak` first. The same path rules apply (inside `$HOME`, no `..`, no symlinks). Like merged Gradle keys, restored files are permanent plaintext on disk — `secrets clear` does not remove them. +### Onboarding and offboarding teammates + +By default every team member uses the **same** `key.txt` (one shared private key). The multi-recipient feature lets each teammate have their **own** keypair while still sharing one store — so you never hand out a secret key to a new hire, and removing an ex-teammate's access is one command. + +#### Onboarding a teammate + +```bash +# 1. Teammate generates their own keypair on their machine (never shares the private key) +age-keygen -o ~/.secrets/key.txt # writes key.txt; prints the public key + +# 2. Teammate sends you their PUBLIC key (printed by age-keygen, starts with age1…) +# — over Slack, email, whatever. Public keys are not secret. + +# 3. An existing member adds the public key to the store +secrets recipients add age1theirpublickey --name alice +# => Adds alice to recipients.txt, re-encrypts every blob to the full set, pushes. + +# 4. Teammate clones the store repo and drops their key.txt in place +git clone git@github.com:/my-secrets.git ~/.secrets +# (key.txt already generated in step 1 — nothing to copy) + +# 5. Teammate pulls into any project +cd ~/myapp +secrets pull +# => Their key matches one recipient stanza in every blob — it just works. +``` + +Run `secrets recipients list` to confirm who has access: + +``` +alice age1theirpublickey… +you age1yourpublickey… +``` + +#### Offboarding a teammate + +```bash +# Remove the recipient by name (or public key) and re-encrypt the store +secrets recipients rm alice +# => Removes alice from recipients.txt, re-encrypts every blob, pushes. +# Existing blobs are re-encrypted; the removed key can no longer decrypt them. +``` + +> **Important:** git history can't be un-shared. If alice had access during a period when genuinely sensitive values were stored, rotate those values now (update them in the external system and run `secrets push`). The re-encrypt prevents future access; history is permanent. + +#### Managing recipients + +```bash +secrets recipients list # show all recipient keys and names +secrets recipients add age1… # add a key (bootstraps recipients.txt on a legacy store) +secrets recipients add age1… --name bob # attach a human-readable label +secrets recipients rm bob # remove by name +secrets recipients rm age1… # remove by public key +secrets reencrypt # re-encrypt to current recipients (idempotent heal) +``` + +`secrets which` shows a `recipients: N (alice, bob, …)` line so you can always confirm the active set from any project directory. + ## Safety features - **`secrets run` auto-clears** — plaintext files are deleted when the command exits, errors, or is interrupted with Ctrl-C @@ -491,10 +634,12 @@ For complete rotation with no historical exposure, create a fresh `~/.secrets/` **"'jq' is not installed"** — Manifest features need `jq`. The error prints the install command for your platform. Manifest-less projects work without it. +**Not sure the store is intact?** — Run `secrets verify` in a project to check its `.secrets.json` against the store (declared-but-missing blobs and orphaned blobs) and decrypt-test every blob with your current key. Use `secrets verify --all` for a store-wide decrypt sweep across every project. It's read-only — plaintext is streamed to `/dev/null`, never written to disk — and exits non-zero if anything is wrong, so it's safe to run in CI. + ## Development ```bash -# Run the test suite (193 tests across both files) +# Run the test suite (272 tests across four files) brew install bats-core bats test/ diff --git a/VERSION b/VERSION index 9551b0d..d5ba5b5 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.4.0.0 +0.7.7.0 diff --git a/docs/superpowers/plans/2026-06-08-additive-v2-dual-write.md b/docs/superpowers/plans/2026-06-08-additive-v2-dual-write.md new file mode 100644 index 0000000..4b21c0c --- /dev/null +++ b/docs/superpowers/plans/2026-06-08-additive-v2-dual-write.md @@ -0,0 +1,587 @@ +# Additive-v2 Dual-Write Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Make store-format v2 *additive* — upgraded `secrets` clients read either blob suffix and dual-write existing `properties` externals — so the destructive `migrate --finalize` becomes optional GC and the cross-machine coordination gate disappears. + +**Architecture:** Replace the marker-driven single-suffix helper `_external_blob_suffix(type)` with two helpers: a read-resolver that tries `.properties.age` then falls back to `.gradle-properties.age`, and a write-targets helper that writes the v2 suffix always plus the v1 suffix *only when a v1 twin already exists* (dual-write existing externals; brand-new externals are v2-only — the intended forcing function). Reads and writes no longer depend on the `.secrets-format` marker, which keeps its meaning (born-v2 / finalized). Because fresh pushes now write the v2 suffix on any store, the migrate/finalize test fixtures (which relied on push producing a v1 blob) are updated to fabricate an old-client v1 blob. + +**Tech Stack:** Single bash 3.2 script (`secrets`); `age`, `git`, `jq`. Tests: `bats-core` (`test/migrate.bats`, `test/manifest.bats`). Every standalone `[[ ]]` test assertion ends with `|| false` (bash 3.2 ERR-trap gotcha). + +**Spec:** `docs/superpowers/specs/2026-06-08-additive-v2-dual-write-design.md`. **Ticket:** EGB-712. + +--- + +## Background facts (verified against branch `brian/egb-712-...`, post-EGB-710) + +- `_external_blob_suffix()` — `secrets:548-555`. Returns `properties` for `gradle-properties` on a v2 store (`_store_format == 2`), else the type verbatim; `file` always returns `file`. Blob path = `external/..age`. +- Callers of `_external_blob_suffix`: push file write `secrets:655`, push properties write `secrets:684`, pull read `secrets:713`, verify read `secrets:2056`. All four are replaced; then the helper is deleted. +- `_secrets_files_slug(path)` (`secrets:512`) derives the machine-independent slug. dotenv and `file` blobs are byte-identical in v1/v2 (only the `properties` suffix diverges). +- `_store_format()` (`secrets:529`) and the marker stay as-is — set only by `init` (born-v2) and `migrate --finalize`. Push must NOT stamp it (see spec §3). +- Existing test that codifies OLD write behavior: `test/migrate.bats:52` "push on a v1 store still writes .gradle-properties.age (back-compat)" — rewritten in Task 2. +- Tests that fabricate-or-rely-on a v1 properties blob from `make_v1_store; push` and break once push writes v2-only (repaired in Task 3): the migrate copy-forward/idempotent tests, the `--dry-run` rename test, the dotenv+file untouched test, all four finalize tests, the EGB-710 manifest-free / undeclared-twin tests, and the two `--status` tests that need an actual v1 blob. (dotenv-only and already-v2 tests are unaffected.) +- Baseline before this plan: `bats test/` = 242 passing. + +## File structure + +- Modify: `secrets` — delete `_external_blob_suffix` (548-555); add `_resolve_external_blob_read` + `_external_blob_write_targets` in its place; rewire push (651-688), pull (712-713), verify (2052-2068); extend `_migrate_status`; docs in `cmd_help`. +- Modify: `test/migrate.bats` — add `m_fake_v1_blob` helper; add read-fallback + write-rule + status-coverage tests; repair the v1-blob-dependent fixtures. +- Modify: `CLAUDE.md`, `README.md`, `VERSION` (→ `0.7.0.0`), `CHANGELOG.md`. + +--- + +## Task 1: Read-resolver — reads try both suffixes + +**Files:** `secrets` (replace `_external_blob_suffix` with the read-resolver; rewire pull + verify reads), `test/migrate.bats`. + +- [ ] **Step 1: Write the failing test** — a v2 store whose properties blob exists ONLY in the v1 suffix must still pull. + +Add to `test/migrate.bats` (after the format-marker tests, ~line 60): + +```bash +@test "pull reads a v1-suffix properties blob on a v2 store (read-fallback)" { + init_with_remote # born-v2 store (marker=2) + m_gradle_src $'beaconClerkPkTest=pk_test_v1\n' + create_project_dir rffallback + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push rffallback >/dev/null 2>&1 # writes .properties.age on a v2 store + # Simulate an external that exists only in the v1 suffix (an old client wrote it): + local v2blob; v2blob=$(ls "$SECRETS_DIR/rffallback/external/"*.properties.age) + mv "$v2blob" "${v2blob%.properties.age}.gradle-properties.age" + rm -f "$HOME/.gradle/gradle.properties" + "$SECRETS_BIN" pull rffallback >/dev/null 2>&1 + run grep -q 'beaconClerkPkTest=pk_test_v1' "$HOME/.gradle/gradle.properties" + [ "$status" -eq 0 ] +} +``` + +- [ ] **Step 2: Run it, confirm it fails** + +Run: `bats test/migrate.bats -f "read-fallback"` +Expected: FAIL — old pull (`secrets:713`) uses `_external_blob_suffix gradle-properties` = `properties` on a v2 store, looks only for `.properties.age` (which we renamed away), warns "no encrypted data", restores nothing → the grep fails. + +- [ ] **Step 3: Replace `_external_blob_suffix` with the read-resolver.** Replace `secrets:542-555` (the comment block + `_external_blob_suffix()` through its closing `}`) with: + +```bash +# Resolve the on-disk path of an external blob for READING. Tries the v2 suffix +# (.properties.age) first, then falls back to the v1 (.gradle-properties.age) for +# `properties` externals, so an upgraded client finds the blob whichever format +# wrote it (additive v2 — EGB-712). `file` externals share one suffix in both +# formats. Echoes the path of the blob that exists; if neither exists, echoes the +# canonical v2 path so the caller's "no blob" message reads sensibly. Read-only. +_resolve_external_blob_read() { + local project="$1" slug="$2" mtype="$3" + local base="$SECRETS_DIR/$project/external/$slug" + case "$mtype" in + file) + echo "$base.file.age" ;; + properties|gradle-properties) + if [ -f "$base.properties.age" ]; then + echo "$base.properties.age" + elif [ -f "$base.gradle-properties.age" ]; then + echo "$base.gradle-properties.age" + else + echo "$base.properties.age" + fi ;; + *) + echo "$base.$mtype.age" ;; + esac +} +``` + +(The write-targets helper is added in Task 2 — leave a gap; do not reintroduce `_external_blob_suffix`.) + +- [ ] **Step 4: Rewire the pull read.** At `secrets:712-713`, replace: + +```bash + local slug; slug=$(_secrets_files_slug "$mpath") + local blob="$SECRETS_DIR/$project/external/$slug.$(_external_blob_suffix "$mtype").age" +``` + +with: + +```bash + local slug; slug=$(_secrets_files_slug "$mpath") + local blob; blob=$(_resolve_external_blob_read "$project" "$slug" "$mtype") +``` + +- [ ] **Step 5: Rewire the verify read.** At `secrets:2055-2065`, replace: + +```bash + slug=$(_secrets_files_slug "$epath") + erel="external/$slug.$(_external_blob_suffix "$etype").age" + # Account for BOTH the v1 and v2 suffix forms in the orphan set. During the + # migration window (after copy-forward, before --finalize) the v2 twin + # coexists with the v1 blob; neither should read as an orphan whichever + # format the store currently reports. (file's two forms are identical.) + expected="${expected}external/$slug.$etype.age"$'\n' + [ "$etype" = "gradle-properties" ] && expected="${expected}external/$slug.properties.age"$'\n' + eblob="$pdir/$erel" + if [ ! -f "$eblob" ]; then + echo "FINDING: external '$epath' ($etype) is declared but has no blob in the store ($project/$erel missing). Run 'secrets push'." >&2 +``` + +with: + +```bash + slug=$(_secrets_files_slug "$epath") + # Account for BOTH suffix forms in the orphan set — a dual-written `properties` + # external (additive v2 — EGB-712) legitimately has both blobs on disk; neither + # is an orphan. (file's two forms are identical.) + expected="${expected}external/$slug.$etype.age"$'\n' + [ "$etype" = "gradle-properties" ] && expected="${expected}external/$slug.properties.age"$'\n' + eblob=$(_resolve_external_blob_read "$project" "$slug" "$etype") + erel="${eblob#"$pdir"/}" + if [ ! -f "$eblob" ]; then + echo "FINDING: external '$epath' ($etype) is declared but has no blob in the store ($project/$erel missing). Run 'secrets push'." >&2 +``` + +(Note: `_external_blob_suffix` still has two remaining callers in push — push isn't rewired until Task 2, so the script still parses and runs. Those calls keep working because the function is only deleted in Task 2 Step 6, after push is rewired.) + +**IMPORTANT:** do NOT delete `_external_blob_suffix` yet — push (`secrets:655`, `secrets:684`) still calls it until Task 2. Deleting it now breaks push. + +- [ ] **Step 6: Run the read-fallback test + full migrate suite** + +Run: `bats test/migrate.bats` +Expected: the new "read-fallback" test PASSES; all other migrate tests still PASS (push unchanged; pull/verify now use the resolver, which is equivalent to the old behavior whenever the suffix matches the store format). + +- [ ] **Step 7: Commit** + +```bash +git add secrets test/migrate.bats +git commit -m "feat: read-resolver tries both external suffixes (additive v2, EGB-712)" +``` + +--- + +## Task 2: Write-targets — twin rule (dual-write existing, v2-only for new) + +**Files:** `secrets` (add `_external_blob_write_targets`; rewire push file + properties writes; delete `_external_blob_suffix`), `test/migrate.bats`. + +- [ ] **Step 1: Write the failing tests.** Add to `test/migrate.bats`: + +```bash +@test "push writes the v2 suffix for a fresh external even on a v1 store" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir freshv1 + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push freshv1 >/dev/null 2>&1 + run bash -c "ls $SECRETS_DIR/freshv1/external/*.properties.age" + [ "$status" -eq 0 ] # v2 suffix regardless of marker + run bash -c "ls $SECRETS_DIR/freshv1/external/*.gradle-properties.age 2>/dev/null" + [ "$status" -ne 0 ] # no v1 twin for a brand-new external +} + +@test "push dual-writes the v1 twin so old clients stay fresh" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_old\n' + create_project_dir dualwrite + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push dualwrite >/dev/null 2>&1 # v2-only (fresh) + m_fake_v1_twin dualwrite # simulate a pre-existing v1 twin + m_gradle_src $'beaconClerkPkTest=pk_test_new\n' + "$SECRETS_BIN" push dualwrite >/dev/null 2>&1 # twin exists -> dual-write both + # Prove the v1 twin was refreshed: drop the v2 blob, pull, expect the NEW value. + rm -f "$SECRETS_DIR/dualwrite/external/"*.properties.age + rm -f "$HOME/.gradle/gradle.properties" + "$SECRETS_BIN" pull dualwrite >/dev/null 2>&1 + run grep -q 'beaconClerkPkTest=pk_test_new' "$HOME/.gradle/gradle.properties" + [ "$status" -eq 0 ] +} +``` + +- [ ] **Step 2: Add the `m_fake_v1_twin` test helper.** In `test/migrate.bats`, next to `make_v1_store` (~line 12), add: + +```bash +# Simulate an old (v1) client's properties blob: copy the pushed v2 +# .properties.age to its v1 .gradle-properties.age twin. (Current clients never +# write the v1 suffix for a fresh external, so tests fabricate it.) Use `cp` to +# KEEP the v2 blob (dual present); see m_make_v1_only to leave only the v1 blob. +m_fake_v1_twin() { + local proj="$1" v2 + v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age) + cp "$v2" "${v2%.properties.age}.gradle-properties.age" +} +# Like m_fake_v1_twin but renames (leaves ONLY the v1 blob) — for old-client-only +# / copy-forward fixtures. +m_make_v1_only() { + local proj="$1" v2 + v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age) + mv "$v2" "${v2%.properties.age}.gradle-properties.age" +} +``` + +- [ ] **Step 3: Run the new tests, confirm they fail** + +Run: `bats test/migrate.bats -f "fresh external even on a v1 store"` +Expected: FAIL — on a v1 store, the old push (`_external_blob_suffix gradle-properties` = `gradle-properties`) writes `.gradle-properties.age`, so the `*.properties.age` assertion fails. +Run: `bats test/migrate.bats -f "dual-writes the v1 twin"` +Expected: FAIL — old push writes a single suffix; the second push won't refresh the v1 twin. + +- [ ] **Step 4: Add the write-targets helper.** Immediately after `_resolve_external_blob_read` (added in Task 1), insert: + +```bash +# The on-disk path(s) to WRITE for an external blob, one per line. For a +# `properties` external this is the v2 suffix (.properties.age) ALWAYS, plus the +# v1 suffix (.gradle-properties.age) WHEN a v1 twin already exists in the store +# (dual-write keeps old clients fresh; a brand-new external is v2-only — the +# intended forcing function, additive v2 / EGB-712). `file` externals have a +# single suffix in both formats. Independent of the store marker. +_external_blob_write_targets() { + local project="$1" slug="$2" mtype="$3" + local base="$SECRETS_DIR/$project/external/$slug" + case "$mtype" in + file) + echo "$base.file.age" ;; + properties|gradle-properties) + echo "$base.properties.age" + [ -f "$base.gradle-properties.age" ] && echo "$base.gradle-properties.age" ;; + *) + echo "$base.$mtype.age" ;; + esac +} +``` + +- [ ] **Step 5: Rewire the push writes.** At `secrets:651-658` (the `file` branch), replace: + +```bash + if [ "$mtype" = "file" ]; then + # EGB-652: whole-file sync — encrypt the file verbatim (binary-safe). + mkdir -p "$SECRETS_DIR/$project/external" + local fslug; fslug=$(_secrets_files_slug "$mpath") + age -r "$pubkey" -o "$SECRETS_DIR/$project/external/$fslug.$(_external_blob_suffix file).age" "$expanded" + info "Encrypted file $mpath" + pushed=$((pushed + 1)) + continue + fi +``` + +with: + +```bash + if [ "$mtype" = "file" ]; then + # EGB-652: whole-file sync — encrypt the file verbatim (binary-safe). + mkdir -p "$SECRETS_DIR/$project/external" + local fslug; fslug=$(_secrets_files_slug "$mpath") + local wt + while IFS= read -r wt; do + [ -n "$wt" ] || continue + age -r "$pubkey" -o "$wt" "$expanded" + done < <(_external_blob_write_targets "$project" "$fslug" file) + info "Encrypted file $mpath" + pushed=$((pushed + 1)) + continue + fi +``` + +Then at `secrets:682-684` (the properties branch), replace: + +```bash + mkdir -p "$SECRETS_DIR/$project/external" + local slug; slug=$(_secrets_files_slug "$mpath") + age -r "$pubkey" -o "$SECRETS_DIR/$project/external/$slug.$(_external_blob_suffix "$mtype").age" "$tmp" +``` + +with: + +```bash + mkdir -p "$SECRETS_DIR/$project/external" + local slug; slug=$(_secrets_files_slug "$mpath") + local wt + while IFS= read -r wt; do + [ -n "$wt" ] || continue + age -r "$pubkey" -o "$wt" "$tmp" + done < <(_external_blob_write_targets "$project" "$slug" "$mtype") +``` + +- [ ] **Step 6: Delete the now-unused `_external_blob_suffix`.** Confirm zero remaining callers first: + +Run: `grep -n "_external_blob_suffix" secrets` +Expected: no matches (all four call sites rewired). If any remain, rewire them before deleting. Then delete the `_resolve_external_blob_read`-replaced... — it's already gone (replaced in Task 1). Verify the function is absent: `grep -c "_external_blob_suffix()" secrets` → `0`. + +- [ ] **Step 7: Run the two new write tests** + +Run: `bats test/migrate.bats -f "fresh external even on a v1 store"` then `-f "dual-writes the v1 twin"` +Expected: both PASS. + +- [ ] **Step 8: Rewrite the obsolete back-compat test.** Replace the `@test "push on a v1 store still writes .gradle-properties.age (back-compat)"` block (`test/migrate.bats:52-60`) with: + +```bash +@test "push on a v1 store writes the v2 suffix for a fresh external (additive v2)" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir v1push + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push v1push >/dev/null 2>&1 + run bash -c "ls $SECRETS_DIR/v1push/external/*.properties.age" + [ "$status" -eq 0 ] +} +``` + +- [ ] **Step 9: Run the full migrate suite — expect the v1-blob-dependent fixtures to FAIL.** This is expected; Task 3 repairs them. + +Run: `bats test/migrate.bats` +Expected: the read-fallback + two write tests + rewritten back-compat test PASS; several copy-forward/finalize/status tests now FAIL (push no longer writes a `.gradle-properties.age` for them to migrate). Note which fail — Task 3 fixes exactly those. + +- [ ] **Step 10: Commit** (suite intentionally not yet fully green — Task 3 follows immediately) + +```bash +git add secrets test/migrate.bats +git commit -m "feat: twin-rule write targets — dual-write existing, v2-only for new (additive v2, EGB-712)" +``` + +--- + +## Task 3: Repair migrate/finalize/status fixtures + +**Files:** `test/migrate.bats`. No production code changes — this re-greens the suite by fabricating the old-client v1 blobs that push no longer writes. + +The rule for each repair: after the `"$SECRETS_BIN" push ` line, insert a fabrication call: +- Use **`m_make_v1_only `** (rename → only the v1 blob exists) for tests asserting a `*.gradle-properties.age` blob exists / is copy-forwarded (mirrors the pre-EGB-712 state where push produced a v1 blob). +- Use **`m_fake_v1_twin `** (keep both) only where a test needs both suffixes present. + +- [ ] **Step 1: Repair the copy-forward / dry-run / idempotent tests.** In each of these tests, insert `m_make_v1_only ` immediately after the `push ` line: + - `"migrate --dry-run reports the rename and writes nothing"` (proj `dryproj`) + - `"migrate copy-forward creates the v2 twin and keeps the v1 blob (byte-identical)"` (proj `cfproj`) + - `"migrate copy-forward is idempotent"` (proj `idemproj`) + - `"migrate leaves dotenv and file blobs untouched"` (proj `mixproj`) — note this one ALSO pushes a `file` external; `m_make_v1_only` only touches `*.properties.age`, leaving the `.file.age` blob alone (correct). + +Worked example — the copy-forward test becomes: + +```bash +@test "migrate copy-forward creates the v2 twin and keeps the v1 blob (byte-identical)" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir cfproj + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push cfproj >/dev/null 2>&1 + m_make_v1_only cfproj + local old; old=$(ls "$SECRETS_DIR/cfproj/external/"*.gradle-properties.age) + run "$SECRETS_BIN" migrate + [ "$status" -eq 0 ] + local new; new=$(ls "$SECRETS_DIR/cfproj/external/"*.properties.age) + [ -f "$old" ] + [ -f "$new" ] + cmp -s "$old" "$new" +} +``` + +- [ ] **Step 2: Repair the finalize tests.** Insert `m_make_v1_only ` after the `push ` line in: + - `"finalize refuses when verify --all is not green"` (proj `failverify`) — then the existing `migrate` step creates the twin; corrupting `*.properties.age` still trips verify. + - `"finalize refuses an un-twinned v1 blob (project not migrated)"` (proj `untwinned`) — leaves a lone v1 blob, no twin: exactly the un-twinned state the test wants. + - `"finalize green path drops v1, keeps v2, stamps the marker"` (proj `finproj`). + - `"finalize cuts a recovery tag before deleting v1 blobs"` (proj — read it from the test). + +- [ ] **Step 3: Repair the EGB-710 manifest-free / undeclared-twin tests.** Insert `m_make_v1_only ` after the `push ` line in: + - `"migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)"` (proj `nomanifestblob`) — insert BEFORE the `rm -f .secrets.json` line. + - `"migrate twins a store blob even when the manifest no longer declares it"` (proj `staleblob`) — insert before the `.secrets.json` rewrite. + +- [ ] **Step 4: Repair the `--status` tests that need a real v1 blob.** + - `"migrate --status flags a project that needs migrating"` (proj `needsmig`) — insert `m_make_v1_only needsmig` after push, so a lone un-twinned v1 blob exists → NEEDS MIGRATE. + - `"migrate --status reports finalize-ready once every blob is twinned"` (proj `readymig`) — insert `m_make_v1_only readymig` after push and BEFORE the `migrate` step (migrate then creates the twin → finalize-ready). + +- [ ] **Step 5: Run the full migrate suite** + +Run: `bats test/migrate.bats` +Expected: ALL pass. If any copy-forward test still reports "nothing to migrate", its `m_make_v1_only` call is missing or misplaced (must come after push, before migrate). + +- [ ] **Step 6: Run the WHOLE suite** (manifest.bats exercises externals end-to-end and must still be green) + +Run: `bats test/` +Expected: all pass. If a `manifest.bats` external test fails, check it isn't asserting a specific suffix that additive-v2 changed (a fresh push now writes `.properties.age`); update such an assertion the same way (assert `.properties.age`, or use the resolver-agnostic round-trip via pull). + +- [ ] **Step 7: Commit** + +```bash +git add test/migrate.bats +git commit -m "test: fabricate old-client v1 blobs in migrate/finalize/status fixtures (additive v2, EGB-712)" +``` + +--- + +## Task 4: `migrate --status` — dual-write coverage line + +**Files:** `secrets` (`_migrate_status`, `secrets:2190-2227`), `test/migrate.bats`. + +Adds visibility into which `properties` externals are v2-only (old clients can't read them — the forcing function) vs dual-written (old clients still served). + +- [ ] **Step 1: Write the failing test.** Add to `test/migrate.bats`: + +```bash +@test "migrate --status counts v2-only externals (old clients not served)" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir v2onlyext + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push v2onlyext >/dev/null 2>&1 # v2-only (fresh, no v1 twin) + run "$SECRETS_BIN" migrate --status + [ "$status" -eq 0 ] # no v1 blobs -> finalize-ready + [[ "$output" == *"v2-only"* ]] || false # surfaced as v2-only coverage +} +``` + +- [ ] **Step 2: Run it, confirm it fails** + +Run: `bats test/migrate.bats -f "counts v2-only externals"` +Expected: FAIL — `_migrate_status` currently only counts `*.gradle-properties.age`; it never mentions `v2-only`. + +- [ ] **Step 3: Extend `_migrate_status`.** In `_migrate_status` (`secrets:2190`), inside the `for dir` loop, after the existing `while ... done < <(find "$dir" -type f -name '*.gradle-properties.age' ...)` block and before the per-project classification, add a v2-only count, then surface it in the per-project line. Concretely, replace the classification block: + +```bash + if [ "$v1" -eq 0 ]; then + echo " $project: v2-ready (no v1 properties blobs)" + elif [ "$untwinned" -eq 0 ]; then + echo " $project: migrated ($v1 v1 blob(s), all twinned)" + else + echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'" + any_untwinned=1 + fi +``` + +with: + +```bash + # v2-only externals: a .properties.age with no .gradle-properties.age twin — + # old (v1) clients cannot read these (the additive-v2 forcing function). + local v2only=0 pf + while IFS= read -r pf; do + [ -f "$pf" ] || continue + [ -f "${pf%.properties.age}.gradle-properties.age" ] || v2only=$((v2only + 1)) + done < <(find "$dir" -type f -name '*.properties.age' 2>/dev/null) + local v2note="" + [ "$v2only" -gt 0 ] && v2note=" [$v2only v2-only — old clients not served]" + if [ "$v1" -eq 0 ]; then + echo " $project: v2-ready (no v1 properties blobs)$v2note" + elif [ "$untwinned" -eq 0 ]; then + echo " $project: migrated ($v1 v1 blob(s), all twinned)$v2note" + else + echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'$v2note" + any_untwinned=1 + fi +``` + +(`local` inside the loop is bash-3.2-fine — it re-declares per iteration.) + +- [ ] **Step 4: Run the new test + status suite** + +Run: `bats test/migrate.bats -f "status"` +Expected: all status tests PASS, including the new v2-only one. + +- [ ] **Step 5: Run the full suite** + +Run: `bats test/` +Expected: all pass. + +- [ ] **Step 6: Commit** + +```bash +git add secrets test/migrate.bats +git commit -m "feat: migrate --status surfaces v2-only externals (coverage, EGB-712)" +``` + +--- + +## Task 5: Docs, help, version, changelog + +**Files:** `secrets` (`cmd_help`), `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`. + +- [ ] **Step 1: `cmd_help` — reframe finalize as optional.** In `cmd_help()`, replace the finalize line: + +``` + secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify) +``` + +with: + +``` + secrets migrate --finalize Optional GC: drop v1 blobs and mark the store pure v2 +``` + +- [ ] **Step 2: `CLAUDE.md` — additive-v2 paragraph.** In the "Store format" bullet (line ~65), after the migration-chain sentence, add (new sentence, same bullet): + +``` + Additive v2 (EGB-712): upgraded clients read either blob suffix (`_resolve_external_blob_read` tries `.properties.age` then `.gradle-properties.age`) and dual-write a `properties` external only when a v1 twin already exists (`_external_blob_write_targets`) — so existing externals keep old clients fresh, brand-new externals are v2-only (a gentle forcing function), and `migrate --finalize` is now OPTIONAL GC rather than a required, coordination-gated flag-day. dotenv and `file` blobs are identical across formats, so they always propagate to old clients. +``` + +- [ ] **Step 3: `README.md` — propagation table.** Add, near the migrate rows in the command table or in a short "Upgrading / store format" subsection, the propagation-by-secret-type summary (verbatim from spec §6): + +```markdown +**Do teammates on an older `secrets` get new secrets?** + +| Secret type | Old client gets it? | +|---|---| +| `.env` / `.env.*` / `.dev.vars` | **Yes, always** (blob name identical across formats) | +| whole-file external | **Yes, always** | +| `properties` external that already existed | **Yes** (dual-written) | +| brand-new `properties` external | **No — must upgrade `secrets`** (the forcing function) | + +"Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`. +``` + +- [ ] **Step 4: `VERSION`** — set to `0.7.0.0`. + +- [ ] **Step 5: `CHANGELOG.md`** — insert above the top entry: + +```markdown +## [0.7.0.0] - 2026-06-08 + +### Changed + +- **Additive store-format v2 (EGB-712)** — upgraded `secrets` clients now read + either external blob suffix (`.properties.age` or the legacy + `.gradle-properties.age`) and **dual-write** a `properties` external whenever a + v1 twin already exists in the store. Existing externals keep working for + teammates on an older `secrets`; only a brand-new `properties` external is + written v2-only (a gentle "upgrade to see it" forcing function). dotenv and + whole-`file` externals are unchanged across formats and always propagate. +- **`secrets migrate --finalize` is now optional GC**, not a required milestone. + Because clients dual-write and read-fall-back, no teammate is ever cut off by + *not* finalizing; finalize only reclaims the duplicate v1 blobs, and stays + deferrable indefinitely. Its safety gates are unchanged. + +### Added + +- **`secrets migrate --status`** now reports `v2-only` externals per project + (the ones an un-upgraded client cannot read), so you can see the forcing + function's footprint at a glance. +``` + +- [ ] **Step 6: Run the full suite** + +Run: `bats test/` +Expected: all pass (docs don't affect tests). Confirm the count is baseline 242 + net new tests from Tasks 1/2/4 (read-fallback, two write tests, v2-only status) minus the rewritten back-compat test (replaced, not added) = **246**. + +- [ ] **Step 7: Sanity — help renders** + +Run: `./secrets help 2>&1 | grep -- "--finalize"` +Expected: shows the reframed "Optional GC" line. (Help only prints; touches no store.) + +- [ ] **Step 8: Commit** + +```bash +git add secrets CLAUDE.md README.md VERSION CHANGELOG.md +git commit -m "docs: additive-v2 propagation + optional-GC finalize; bump 0.7.0.0 (EGB-712)" +``` + +--- + +## Self-review against the spec + +- **§1 read resolution** → Task 1 (`_resolve_external_blob_read`, wired into pull + verify). Test: read-fallback. +- **§2 write rule / twin rule** → Task 2 (`_external_blob_write_targets`, push wiring). Tests: fresh-→v2-only, existing-twin-→dual-write. +- **§3 marker NOT stamped** → no production change (push never touches the marker); guarded implicitly by Task 3 keeping the `make_v1_store; push; migrate` flow working (store stays markerless after push). The plan deliberately does not add marker-stamping. +- **§4 finalize = optional GC** → unchanged logic; reframed in Task 5 docs/help. Existing finalize tests stay green (Task 3 keeps them green). +- **§5 migrate / --status coverage** → Task 4 (v2-only line); `migrate` copy-forward unchanged (EGB-710). +- **§6 propagation table** → Task 5 README/CLAUDE.md. +- **§7 caveat** → documented in CHANGELOG/CLAUDE.md framing; the read-fallback test exercises the v1-only read path. +- **§8 back-compat matrix** → covered across Task 1 (reads), Task 2 (writes), Task 3 (old-client v1 blobs simulated). +- **Safety invariants** → no path-rail changes; bash 3.2 (no associative arrays; `while read` + `find`); recursive walks unchanged; finalize gating untouched. + +**Type/name consistency:** `_resolve_external_blob_read(project, slug, mtype)` and `_external_blob_write_targets(project, slug, mtype)` use the same arg order everywhere; test helpers `m_fake_v1_twin` (keep both) and `m_make_v1_only` (rename to v1-only) are used consistently per their documented semantics. + +**Placeholder scan:** none — every step carries verbatim code or an exact enumerated edit with the precise insertion point. + +## Operator-local follow-up (not part of this plan) + +Per `.ship-policy.json`, before any PR ask the operator to run `./test/run-security.sh` and complete the SIGNOFF. EGB-712 also needs its one stale AC bullet ("Marker auto-stamps…") corrected to match the §3 decision (no auto-stamp) — a one-line Linear edit. diff --git a/docs/superpowers/plans/2026-06-08-egb-710-migrate-guided-flow.md b/docs/superpowers/plans/2026-06-08-egb-710-migrate-guided-flow.md new file mode 100644 index 0000000..b552428 --- /dev/null +++ b/docs/superpowers/plans/2026-06-08-egb-710-migrate-guided-flow.md @@ -0,0 +1,447 @@ +# EGB-710: `secrets migrate` guided flow (manifest-free copy-forward + `--status` survey) Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Replace the dead-end "No .secrets.json — cd into a project that has a manifest" error in `secrets migrate` with a manifest-free copy-forward that just works, plus a `secrets migrate --status` survey that tells the operator exactly which projects still need migrating. + +**Architecture:** The per-project copy-forward step (`_migrate_project`) currently reads `$PWD/.secrets.json` to find `properties` externals, then looks for their v1 blobs. This makes it die on a legacy `.secrets-files`-only project (the real EGB-710 repro) and — worse — silently skips any store blob the manifest doesn't declare, leaving it un-twinned so `--finalize` later refuses it. The fix: enumerate the **store** (`$SECRETS_DIR//external/*.gradle-properties.age`) directly and copy each to its `.properties.age` twin via suffix-swap — the *exact* logic `_migrate_finalize` already uses. No manifest needed, dotenv/file-only projects become a clean no-op, and migrate twins precisely the blobs finalize will demand twins for. A new `--status` mode walks every project dir and reports per-project readiness + whether the store is finalize-ready. + +**Tech Stack:** Single bash 3.2 script (`secrets`), `age`, `git`, `jq` (unaffected here). Tests: `bats-core` (`test/migrate.bats`). Every standalone `[[ ]]` assertion ends with `|| false` (bash 3.2 ERR-trap gotcha). + +--- + +## Background facts (verified against HEAD 633d19e) + +- `_migrate_project()` lives at `secrets:2135`; it `die`s at `secrets:2142-2148` on missing `$PWD/.secrets.json`, then iterates `_json_external_entries "$manifest"` filtered to `gradle-properties` (`secrets:2154-2174`). +- `_migrate_finalize()` (`secrets:2196`) already enumerates blobs manifest-free: `find "$SECRETS_DIR" -type f -name '*.gradle-properties.age'` and derives the twin as `new="${f%.gradle-properties.age}.properties.age"` (`secrets:2212-2217`). The per-project step will mirror this, scoped to one project dir. +- `cmd_migrate()` flag parser: `secrets:2267-2285`. +- `derive_project_name ""` (`secrets:101`) needs no manifest — it uses the git remote basename or `basename "$PWD"`. +- `info()`/`die()`: `secrets:36-37`. `ensure_store_protections` is called post-write in the existing copy-forward. +- Output-contract strings existing tests depend on (must be preserved): `"would migrate"` (migrate.bats:80), `"0 blob(s) would be copy-forwarded"` (migrate.bats:94), `"1 already present"` (migrate.bats:121, idempotent re-run prints `$already already present`), `"already format v2"` (migrate.bats:139). +- The one test that codifies the OLD dead-end — `"migrate with no manifest in cwd dies with a directed message"` (migrate.bats:126) — is the behavior we are intentionally changing; it gets rewritten in Task 1. +- No test asserts the non-dry-run "nothing to migrate" wording (grep confirmed), so that message is free to change. We keep the substring `no v1 properties blobs` regardless for safety. +- VERSION is `0.6.0.1`; bump to `0.6.1.0` (new subcommand flag + behavior change). `MANIFEST_VERSION` stays `2` (no schema change). + +## File structure + +- Modify: `secrets` — rewrite `_migrate_project` (`secrets:2135-2191`), add `_migrate_status`, extend `cmd_migrate` flag parsing + dispatch (`secrets:2267-2285`), update `cmd_help` migrate lines (`secrets:2308-2309`). +- Modify: `test/migrate.bats` — rewrite the dead-end test; add manifest-free, finalize-consistency, and `--status` tests. +- Modify: `CLAUDE.md` — update the migrate paragraph (Architecture → Store format) to note manifest-free copy-forward + `--status`. +- Modify: `README.md` — migrate usage/help. +- Modify: `VERSION` → `0.6.1.0`; `CHANGELOG.md` — new entry. + +--- + +## Task 1: Manifest-free per-project copy-forward + +**Files:** +- Modify: `secrets` — `_migrate_project()` (`secrets:2135-2191`) +- Test: `test/migrate.bats` + +- [ ] **Step 1: Write the failing test — migrate works with no `.secrets.json` (the EGB-710 repro)** + +Add to `test/migrate.bats` (after the existing copy-forward tests, ~line 124): + +```bash +@test "migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)" { + # The EGB-710 repro: a legacy project has a v1 properties blob in the store + # but no .secrets.json (it predates the manifest). migrate must NOT dead-end. + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir nomanifestblob + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push nomanifestblob >/dev/null 2>&1 + rm -f .secrets.json # simulate a pre-manifest project + run "$SECRETS_BIN" migrate + [ "$status" -eq 0 ] + run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.properties.age" + [ "$status" -eq 0 ] # v2 twin written despite no manifest + run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.gradle-properties.age" + [ "$status" -eq 0 ] # v1 kept (non-destructive) +} +``` + +- [ ] **Step 2: Run it to confirm it fails** + +Run: `bats test/migrate.bats -f "manifest-free"` +Expected: FAIL — current code `die`s with "No .secrets.json in …" (status 1), so the first `[ "$status" -eq 0 ]` fails. + +- [ ] **Step 3: Rewrite `_migrate_project` to enumerate the store, not the manifest** + +Replace the body of `_migrate_project()` (`secrets:2135-2191`, from `_migrate_project() {` through its closing `}`) with: + +```bash +_migrate_project() { + local dry_run="$1" + if [ "$(_store_format)" = "2" ]; then + info "Store is already format v2 — nothing to migrate." + return 0 + fi + local project; project=$(derive_project_name "") + local pdir="$SECRETS_DIR/$project" + + # Source of truth for the copy-forward is the STORE, not a project manifest. + # Every v1 properties blob is a `*.gradle-properties.age` file whose v2 twin + # is the same name with the `.properties.age` suffix (the only on-disk change + # v2 makes). Enumerating the store — exactly as `_migrate_finalize` does — + # means migrate twins precisely the blobs finalize will demand twins for, with + # no manifest dependency. This is why a legacy `.secrets-files`-only project + # (no `.secrets.json` yet) migrates cleanly instead of dead-ending, and why a + # store blob the manifest no longer declares still gets a twin. + local moved=0 already=0 would=0 v1count=0 f new + while IFS= read -r f; do + [ -f "$f" ] || continue + v1count=$((v1count + 1)) + new="${f%.gradle-properties.age}.properties.age" + if [ -f "$new" ]; then + already=$((already + 1)) + continue + fi + if [ "$dry_run" = true ]; then + echo "would migrate: ${f#"$SECRETS_DIR"/} -> $(basename "$new")" + would=$((would + 1)) + else + cp "$f" "$new" + moved=$((moved + 1)) + fi + done < <(find "$pdir/external" -type f -name '*.gradle-properties.age' 2>/dev/null) + + if [ "$dry_run" = true ]; then + echo "migrate --dry-run: $would blob(s) would be copy-forwarded for '$project' (writes nothing); $already already present. v1 blobs are kept until 'secrets migrate --finalize'." + return 0 + fi + if [ "$moved" -eq 0 ] && [ "$already" -eq 0 ]; then + info "Nothing to migrate for '$project' — no v1 properties blobs in the store (already v2-shaped). If you expected one, run 'secrets push' first, then re-run 'secrets migrate'." + return 0 + fi + ensure_store_protections + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "migrate: copy-forward v2 twins for $project" >/dev/null 2>&1 || true + # Push the twins so a --finalize on another machine sees them (finalize + # refuses any v1 blob without a twin). Mirrors push/rekey's push behavior. + git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 && git -C "$SECRETS_DIR" push >/dev/null 2>&1 || true + info "Copy-forward for '$project': $moved new v2 twin(s), $already already present. v1 blobs kept (non-destructive). Run 'secrets migrate --finalize' once every project is migrated and every machine is upgraded." +} +``` + +Notes for the implementer: +- This removes the only in-script caller of `_check_manifest_file`/`_json_external_entries` *inside migrate*; both remain defined and used by push/pull/verify, so do **not** delete them. +- `${f#"$SECRETS_DIR"/}` keeps the dry-run line's store-relative form (matches the old `$project/external/...` style closely enough; the test only checks the substring `would migrate`). +- The `moved==0 && already==0` branch keeps the substring `no v1 properties blobs`. The idempotent re-run path (`moved==0, already>0`) falls through to the final `info` printing `$already already present`, preserving the `1 already present` contract. + +- [ ] **Step 4: Run the new test + the full migrate suite** + +Run: `bats test/migrate.bats` +Expected: the new "manifest-free" test PASSES; existing dry-run/copy-forward/idempotent/finalize tests still PASS; the "migrate with no manifest in cwd dies" test (migrate.bats:126) now FAILS (we fix it in Step 5). + +- [ ] **Step 5: Update the test that codified the old dead-end** + +Replace the test at `test/migrate.bats:126-132` (`"migrate with no manifest in cwd dies with a directed message"`) with: + +```bash +@test "migrate in a project with no manifest and no store blobs is a clean no-op" { + make_v1_store + local dir="$WORK_DIR/nomanifest"; mkdir -p "$dir"; cd "$dir" + run "$SECRETS_BIN" migrate + [ "$status" -eq 0 ] + [[ "$output" == *"no v1 properties blobs"* ]] || false +} +``` + +- [ ] **Step 6: Run the full suite to confirm green** + +Run: `bats test/migrate.bats` +Expected: all PASS. + +- [ ] **Step 7: Commit** + +```bash +git add secrets test/migrate.bats +git commit -m "fix: migrate copy-forward is manifest-free, no dead-end on legacy projects (EGB-710)" +``` + +--- + +## Task 2: Finalize-consistency regression test (store blob not in manifest) + +**Files:** +- Test: `test/migrate.bats` + +This proves the latent-bug fix: the old manifest-driven migrate skipped store blobs the manifest didn't declare, leaving them un-twinned so `--finalize` refused them. The rewrite twins them. + +- [ ] **Step 1: Write the test** + +Add to `test/migrate.bats`: + +```bash +@test "migrate twins a store blob even when the manifest no longer declares it" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir staleblob + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push staleblob >/dev/null 2>&1 + # The blob is now in the store. Drop the external from the project's manifest + # entirely (and remove the legacy file) so NO manifest declares it. + printf '{"version":2,"dotenv":[".env",".env.staging"]}\n' > .secrets.json + rm -f .secrets-files + run "$SECRETS_BIN" migrate + [ "$status" -eq 0 ] + run bash -c "ls $SECRETS_DIR/staleblob/external/*.properties.age" + [ "$status" -eq 0 ] # twinned despite not being declared anywhere +} +``` + +- [ ] **Step 2: Run it** + +Run: `bats test/migrate.bats -f "no longer declares"` +Expected: PASS (the Task 1 rewrite already makes this green — this test guards against regressing back to manifest-driven enumeration). + +- [ ] **Step 3: Commit** + +```bash +git add test/migrate.bats +git commit -m "test: migrate twins undeclared store blobs (finalize-consistency, EGB-710)" +``` + +--- + +## Task 3: `secrets migrate --status` survey + +**Files:** +- Modify: `secrets` — add `_migrate_status()`; extend `cmd_migrate` (`secrets:2267-2285`) +- Test: `test/migrate.bats` + +- [ ] **Step 1: Write the failing tests** + +Add to `test/migrate.bats`: + +```bash +@test "migrate --status flags a project that needs migrating" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir needsmig + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push needsmig >/dev/null 2>&1 # v1 blob, no twin yet + run "$SECRETS_BIN" migrate --status + [ "$status" -ne 0 ] # not finalize-ready + [[ "$output" == *"needsmig"* ]] || false + [[ "$output" == *"NEEDS MIGRATE"* ]] || false + [[ "$output" == *"Not finalize-ready"* ]] || false +} + +@test "migrate --status reports finalize-ready once every blob is twinned" { + make_v1_store + m_gradle_src $'beaconClerkPkTest=pk_test_abc\n' + create_project_dir readymig + printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files + "$SECRETS_BIN" push readymig >/dev/null 2>&1 + "$SECRETS_BIN" migrate >/dev/null 2>&1 # create the twin + run "$SECRETS_BIN" migrate --status + [ "$status" -eq 0 ] + [[ "$output" == *"Finalize-ready"* ]] || false +} + +@test "migrate --status on an already-v2 store says nothing to do" { + init_with_remote + create_project_dir v2status + run "$SECRETS_BIN" migrate --status + [ "$status" -eq 0 ] + [[ "$output" == *"v2"* ]] || false +} +``` + +- [ ] **Step 2: Run them to confirm they fail** + +Run: `bats test/migrate.bats -f "status"` +Expected: FAIL — `--status` is an unknown flag today (`die "Unknown migrate flag: --status…"`, status 1), so the assertions fail. + +- [ ] **Step 3: Add `_migrate_status` (place it just before `_migrate_finalize`, ~`secrets:2195`)** + +```bash +# Read-only survey: walk every project dir in the store and report each one's +# v2-readiness from the blobs on disk (no manifest, no decryption). Exits +# non-zero when any v1 properties blob lacks a v2 twin (i.e. the store is not +# yet finalize-ready) so it can gate scripting, mirroring `verify`'s posture. +_migrate_status() { + if [ "$(_store_format)" = "2" ]; then + info "Store format: v2 (finalized) — nothing to migrate." + return 0 + fi + echo "Store format: v1 (not finalized). Per-project migration status:" + local any_untwinned=0 dir project v1 untwinned f new + for dir in "$SECRETS_DIR"/*/; do + [ -d "$dir" ] || continue + project=$(basename "$dir") + case "$project" in .*) continue ;; esac + v1=0; untwinned=0 + while IFS= read -r f; do + [ -f "$f" ] || continue + v1=$((v1 + 1)) + new="${f%.gradle-properties.age}.properties.age" + [ -f "$new" ] || untwinned=$((untwinned + 1)) + done < <(find "$dir" -type f -name '*.gradle-properties.age' 2>/dev/null) + if [ "$v1" -eq 0 ]; then + echo " $project: v2-ready (no v1 properties blobs)" + elif [ "$untwinned" -eq 0 ]; then + echo " $project: migrated ($v1 v1 blob(s), all twinned)" + else + echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'" + any_untwinned=1 + fi + done + echo + if [ "$any_untwinned" -eq 1 ]; then + echo "Not finalize-ready: migrate the projects marked NEEDS MIGRATE, then run 'secrets migrate --finalize'." + return 1 + fi + echo "Finalize-ready: every v1 properties blob has a v2 twin. Run 'secrets migrate --finalize' once every machine is upgraded." + return 0 +} +``` + +- [ ] **Step 4: Wire `--status` into `cmd_migrate`** + +In `cmd_migrate()` (`secrets:2267`), add the `status` local and parse + dispatch. Replace: + +```bash + local dry_run=false finalize=false force=false + while [ $# -gt 0 ]; do + case "$1" in + --dry-run) dry_run=true; shift ;; + --finalize) finalize=true; shift ;; + --yes) force=true; shift ;; + -*) die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --finalize] [--yes]" ;; + *) die "migrate takes no project argument. Run it from inside a project (copy-forward) or use --finalize (store-wide)." ;; + esac + done + if [ "$finalize" = true ]; then + _migrate_finalize "$force" + else + _migrate_project "$dry_run" + fi +``` + +with: + +```bash + local dry_run=false finalize=false force=false status=false + while [ $# -gt 0 ]; do + case "$1" in + --dry-run) dry_run=true; shift ;; + --finalize) finalize=true; shift ;; + --status) status=true; shift ;; + --yes) force=true; shift ;; + -*) die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --status | --finalize] [--yes]" ;; + *) die "migrate takes no project argument. Run it from inside a project (copy-forward), or use --status / --finalize (store-wide)." ;; + esac + done + if [ "$status" = true ]; then + _migrate_status + elif [ "$finalize" = true ]; then + _migrate_finalize "$force" + else + _migrate_project "$dry_run" + fi +``` + +- [ ] **Step 5: Run the status tests + full suite** + +Run: `bats test/migrate.bats` +Expected: all PASS. + +- [ ] **Step 6: Commit** + +```bash +git add secrets test/migrate.bats +git commit -m "feat: secrets migrate --status surveys per-project v2 readiness (EGB-710)" +``` + +--- + +## Task 4: Docs, help text, version, changelog + +**Files:** +- Modify: `secrets` — `cmd_help` (`secrets:2308-2309`) +- Modify: `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md` + +- [ ] **Step 1: Update `cmd_help` migrate lines** + +In `cmd_help()` replace the two migrate lines (`secrets:2308-2309`): + +``` + secrets migrate [--dry-run] Copy-forward this project's blobs to store format v2 + secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify) +``` + +with: + +``` + secrets migrate [--dry-run] Copy-forward this project's v1 blobs to store format v2 + secrets migrate --status Survey every project's v2 readiness (finalize gate) + secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify) +``` + +- [ ] **Step 2: Update `CLAUDE.md` migrate paragraph** + +In the "Store format (EGB-677 stage 2 / EGB-703)" bullet, update the migration description: copy-forward is now **manifest-free** — `secrets migrate` enumerates the project's `*.gradle-properties.age` store blobs directly (same source of truth as `--finalize`), so a legacy `.secrets-files`-only project migrates without a `.secrets.json` and no store blob is left un-twinned. Add `secrets migrate --status` to the workflow line as the read-only survey that reports per-project readiness and gates `--finalize`. Reference EGB-710. + +- [ ] **Step 3: Update `README.md`** + +Find the migrate section/help block and add the `--status` line and the manifest-free note, mirroring the help text. + +- [ ] **Step 4: Bump `VERSION`** + +Set `VERSION` to `0.6.1.0`. + +- [ ] **Step 5: Add `CHANGELOG.md` entry** + +Insert above `## [0.6.0.1]`: + +```markdown +## [0.6.1.0] - 2026-06-08 + +### Changed + +- **`secrets migrate` copy-forward is now manifest-free (EGB-710)** — the + per-project step enumerates the store's `*.gradle-properties.age` blobs + directly (the same source of truth `--finalize` uses) instead of reading + `.secrets.json`. A legacy `.secrets-files`-only project now migrates cleanly + instead of dead-ending with "No .secrets.json", and a store blob the manifest + no longer declares still gets a v2 twin (so `--finalize` won't refuse it). + Running migrate in a project with no v1 properties blobs is a clean no-op. + +### Added + +- **`secrets migrate --status`** — a read-only survey that walks every project + in the store and reports its v2 readiness (v2-ready / migrated / NEEDS + MIGRATE), then whether the store as a whole is finalize-ready. Exits non-zero + while any v1 blob is un-twinned, so it can gate the path to `--finalize`. +``` + +- [ ] **Step 6: Run the full bats suite (all files)** + +Run: `bats test/` +Expected: all PASS (secrets.bats + manifest.bats + migrate.bats). Confirm the count went up by the tests added here. + +- [ ] **Step 7: Commit** + +```bash +git add secrets CLAUDE.md README.md VERSION CHANGELOG.md +git commit -m "docs: migrate --status + manifest-free copy-forward; bump 0.6.1.0 (EGB-710)" +``` + +--- + +## Self-review against the EGB-710 spec + +- **AC: dotenv-only project exits 0 "already v2-shaped"** → Task 1 (`moved==0 && already==0` branch, substring `no v1 properties blobs`). Test: "no manifest and no store blobs is a clean no-op" (Task 1 Step 5). +- **AC: `.secrets-files`-only project with v1 blobs migrates instead of the generic error** → the manifest-free rewrite makes it *just migrate* (better than guiding to push first). Test: "manifest-free" (Task 1 Step 1). Note the rewrite supersedes the ticket's "offer to run absorb then migrate" branch — migrate no longer needs the manifest, so there's nothing to prompt for; the only destructive step (`--finalize`) keeps its existing `/dev/tty` confirmation. +- **AC: interactive prompts read `/dev/tty`, degrade non-interactively, bash 3.2** → no new prompt is introduced (copy-forward is non-destructive); `--finalize`'s existing `/dev/tty` confirm is untouched. `--status` is pure read-only output. All new code is bash 3.2 (no associative arrays, `[[ ]]` only in tests with `|| false`). +- **AC: no behavior change to copy-forward/finalize safety gates** → finalize untouched; copy-forward stays non-destructive (v1 kept, commit + push twins). Verified by the unchanged finalize tests (migrate.bats:168-214). +- **AC: bats coverage per branch** → Tasks 1-3 add: manifest-free migrate, no-op no-blobs, undeclared-blob twinning, `--status` needs-migrate / finalize-ready / already-v2. +- **Stretch: `--status` survey** → Task 3, delivered. +- **Sequencing: independent of the EGB-709 gate** → confirmed; this only touches migrate ergonomics and helps operators *reach* all-v2. EGB-703 safety posture (recovery tag, verify-green gate, twin-before-drop) is preserved. + +## Operator-local follow-up (not part of this plan) + +This repo's `.ship-policy.json` opts out of AI security/red-team passes; before any PR, ask the operator to run `./test/run-security.sh` and complete the SIGNOFF. After landing, the operator can re-run their real-store migration for `onefinalmessage` et al. (`secrets migrate` per project → `secrets migrate --finalize`), which is the path to clearing the EGB-709 v2 gate. diff --git a/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md b/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md new file mode 100644 index 0000000..61d1ce5 --- /dev/null +++ b/docs/superpowers/plans/2026-06-08-egb-713-version-skew-nudge.md @@ -0,0 +1,300 @@ +# EGB-713: Version-skew nudge Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans / subagent-driven-development. Steps use `- [ ]`. + +**Goal:** Warn (non-fatally) when the active store was last written by a newer `secrets` version than the running client, so a behind user is told to update — the loud counterpart to EGB-712's quiet forcing function. + +**Architecture:** Stamp the store with the highest writer `VERSION` seen (`$SECRETS_DIR/.secrets-writer-version`, committed, monotonic) on every store-committing write. On any store command, compare that stamp to the client's own `VERSION` (read from `$SCRIPT_DIR/VERSION`); if the stamp is newer, print a one-time stderr nudge. Legacy stores with no stamp are silent. + +**Tech Stack:** bash 3.2 (`secrets`); bats. Spec/idea: ticket **EGB-713**. + +## Background (verified against `main`, post-EGB-712) +- `SCRIPT_DIR` already defined at `secrets:21` (`$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)`). The repo-root `VERSION` file lives next to the script. +- Store-committing write sites (each does `git -C "$SECRETS_DIR" add -A`): `commit_and_push_secrets()` `secrets:1265` (push), rekey `secrets:1822`, migrate copy-forward `secrets:2215`, finalize no-v1 `secrets:2305`, finalize drop `secrets:2340`. Plus `cmd_init` (born store) and `cmd_rm`. +- `ensure_store_protections()` (`secrets:1144`) is shared with pull (read) — do NOT stamp there. +- `check_initialized()` (`secrets:54`) early-returns when `$SECRETS_DIR/.git` exists and is called by push/pull/list/rm/rekey/verify/migrate — the natural warning hook. +- `cmd_which()` (`secrets:1945`) prints `format: v$(_store_format)` — add the writer-version line here. +- `.gitignore` only ignores `key.txt`, so `.secrets-writer-version` commits normally. +- VERSION currently `0.7.0.0` → bump to `0.7.1.0`. +- Baseline: `bats test/` = 246 passing. New tests land in a new file `test/version.bats`. + +--- + +## Task 1: Version helpers + comparator (TDD) + +**Files:** `secrets`, `test/version.bats` (new). + +- [ ] Step 1: Create `test/version.bats` testing the comparator via a tiny harness that sources the script's functions is awkward (the script runs main). Instead test through observable behavior in later tasks; for the comparator, add a hidden debug subcommand is overkill. Use this approach: test `_version_gt` indirectly by exporting it is not possible. So test the comparator by adding the helpers and a **`secrets __vercmp `** internal is overkill too. Decision: test the comparator's *effect* in Task 3 (warning) and Task 2 (stamp monotonicity), which exercise it end-to-end. For Task 1, write the helpers and verify with a one-off `bash -c` sourcing guard. + +Add to `test/version.bats`: +```bash +load test_helper + +# Exercises the comparator through a bash subshell that defines the same logic +# the script uses, guarding the numeric (not lexical) ordering contract. +@test "version comparator orders 0.7.0.0 < 0.10.0.0 numerically" { + run bash -c ' + _version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done; return 1 + } + _version_gt 0.10.0.0 0.7.0.0 && echo "10gt7" + _version_gt 0.7.0.0 0.10.0.0 || echo "7not_gt_10" + _version_gt 0.7.0.0 0.7.0.0 || echo "equal_not_gt" + ' + [ "$status" -eq 0 ] + [[ "$output" == *"10gt7"* ]] || false + [[ "$output" == *"7not_gt_10"* ]] || false + [[ "$output" == *"equal_not_gt"* ]] || false +} +``` + +- [ ] Step 2: Run `bats test/version.bats` → PASS (pins the contract the script must match). + +- [ ] Step 3: Add the helpers to `secrets` (near `_store_format`, after `SCRIPT_DIR`/version constants — place after the `MANIFEST_VERSION=2` area or near `_store_format`): +```bash +# The running client's own version, read from the VERSION file shipped beside +# the script. Empty/"0.0.0.0" if absent (e.g. an odd install) — treated as +# "unknown/oldest" so a missing VERSION never triggers a spurious nudge. +_client_version() { + local v="" + [ -f "$SCRIPT_DIR/VERSION" ] && v=$(head -1 "$SCRIPT_DIR/VERSION" 2>/dev/null | tr -d '\r\n[:space:]') + printf '%s' "${v:-0.0.0.0}" +} + +# Numeric four-field (MAJOR.MINOR.PATCH.MICRO) compare. Returns 0 iff $1 > $2. +# Per-field numeric (so 0.10.0.0 > 0.7.0.0); missing/garbage fields → 0. +_version_gt() { + local a="$1" b="$2" i ai bi; local -a af bf + IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b" + for i in 0 1 2 3; do + ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0 + bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0 + if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi + if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi + done + return 1 +} + +WRITER_VERSION_FILE_NAME=".secrets-writer-version" +# Highest client version recorded as having written to the store (empty if the +# store predates this feature — "minus the initial builds", silent by design). +_store_writer_version() { + local f="$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + [ -f "$f" ] && head -1 "$f" 2>/dev/null | tr -d '\r\n[:space:]' +} +``` + +- [ ] Step 4: `bash -n secrets` parses; `bats test/` still 246 + 1 (the comparator test) = 247. +- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: version helpers + numeric comparator (EGB-713)"` + +--- + +## Task 2: Stamp the writer-version on write (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add tests: +```bash +@test "push stamps the store writer-version with the client version" { + init_with_remote + create_project_dir wvstamp + "$SECRETS_BIN" push wvstamp >/dev/null 2>&1 + [ -f "$SECRETS_DIR/.secrets-writer-version" ] + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "$(cat "$(dirname "$SECRETS_BIN")/VERSION")" ] +} + +@test "writer-version stamp is monotonic (a push never lowers a higher stamp)" { + init_with_remote + create_project_dir wvmono + printf '9.9.9.9\n' > "$SECRETS_DIR/.secrets-writer-version" + "$SECRETS_BIN" push wvmono >/dev/null 2>&1 + run cat "$SECRETS_DIR/.secrets-writer-version" + [ "$output" = "9.9.9.9" ] # not lowered to the client's version +} + +@test "writer-version stamp is committed, not gitignored" { + init_with_remote + create_project_dir wvcommit + "$SECRETS_BIN" push wvcommit >/dev/null 2>&1 + run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-writer-version" + [ "$status" -eq 0 ] +} +``` + +- [ ] Step 2: Run `bats test/version.bats -f "stamp"` → FAIL (no stamping yet). + +- [ ] Step 3: Add the stamp helper (after `_store_writer_version`): +```bash +# Raise the store's recorded writer-version to the client's version (monotonic; +# never lowers it). Called right before each store-committing `git add -A` so +# the stamp rides the same commit. Read paths (pull) never call this. +_stamp_writer_version() { + local cur cli + cur=$(_store_writer_version) + cli=$(_client_version) + if [ -z "$cur" ] || _version_gt "$cli" "$cur"; then + printf '%s\n' "$cli" > "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" + fi +} +``` + +- [ ] Step 4: Call `_stamp_writer_version` immediately before each store-committing `git -C "$SECRETS_DIR" add -A`: + - `secrets:1265` (in `commit_and_push_secrets`, before `git add -A`) + - `secrets:1822` (rekey) + - `secrets:2215` (migrate copy-forward) + - `secrets:2305` (finalize, no-v1 path) + - `secrets:2340` (finalize, drop path) + Also in `cmd_init`, after the store repo is created and before its first commit (so a born store records its version), and in `cmd_rm` before its commit. + Each insertion is the single line ` _stamp_writer_version` at the matching indentation directly above the `git ... add -A` (or before the `git ... commit` where there's no add -A, e.g. rm/init — there, stamp then ensure it's staged via the existing add/commit). + +- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 250 (247 + 3). +- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: stamp store writer-version on write, monotonic (EGB-713)"` + +--- + +## Task 3: Skew warning on command (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add tests: +```bash +@test "a store written by a newer version warns on a command (non-fatal)" { + init_with_remote + create_project_dir skewwarn + "$SECRETS_BIN" push skewwarn >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] # non-fatal + [[ "$output" == *"newer"* || "$output" == *"update"* ]] || false +} + +@test "a store at the same/older version is silent" { + init_with_remote + create_project_dir noskew + "$SECRETS_BIN" push noskew >/dev/null 2>&1 # stamp == client version + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"update your secrets"* ]] || false +} + +@test "a store with no writer-version marker is silent (legacy store)" { + init_with_remote + create_project_dir legacynostamp + "$SECRETS_BIN" push legacynostamp >/dev/null 2>&1 + rm -f "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" list + [ "$status" -eq 0 ] + [[ "$output" != *"update your secrets"* ]] || false +} +``` + +- [ ] Step 2: Run `bats test/version.bats -f "skew\|silent\|legacy"` → the "newer" test FAILS (no warning yet). + +- [ ] Step 3: Add the skew check (after `_stamp_writer_version`): +```bash +# Warn ONCE per invocation if the store was last written by a newer client than +# us. Non-fatal (read/write paths keep their exit codes). Silent when the store +# carries no writer-version (legacy) or is same/older than us. +_VERSION_SKEW_WARNED=0 +_check_store_version_skew() { + [ "$_VERSION_SKEW_WARNED" = 1 ] && return 0 + local sv cv + sv=$(_store_writer_version) + [ -n "$sv" ] || return 0 + cv=$(_client_version) + if _version_gt "$sv" "$cv"; then + _VERSION_SKEW_WARNED=1 + echo "NOTE: this store was last written by secrets v$sv; you're on v$cv." >&2 + echo " Update your secrets tool: git -C \"$SCRIPT_DIR\" pull" >&2 + fi + return 0 +} +``` + +- [ ] Step 4: Hook it into `check_initialized` — change `secrets:55-57`: +```bash + if [ -d "$SECRETS_DIR/.git" ]; then + return + fi +``` +to: +```bash + if [ -d "$SECRETS_DIR/.git" ]; then + _check_store_version_skew + return + fi +``` + +- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 253. +- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: warn on store version skew (once per invocation, EGB-713)"` + +--- + +## Task 4: `secrets which` surfaces the writer-version (TDD) + +**Files:** `secrets`, `test/version.bats`. + +- [ ] Step 1: Add test: +```bash +@test "which prints the store writer-version and a behind note" { + init_with_remote + create_project_dir whichwv + "$SECRETS_BIN" push whichwv >/dev/null 2>&1 + printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version" + run "$SECRETS_BIN" which + [ "$status" -eq 0 ] + [[ "$output" == *"written-by: v99.0.0.0"* ]] || false + [[ "$output" == *"behind"* || "$output" == *"update"* ]] || false +} +``` + +- [ ] Step 2: Run → FAIL (which doesn't print written-by). + +- [ ] Step 3: In `cmd_which`, after the `echo "format: v$(_store_format)"` line (`secrets:1951`), add: +```bash + local _wv; _wv=$(_store_writer_version) + if [ -n "$_wv" ]; then + local _cv; _cv=$(_client_version) + if _version_gt "$_wv" "$_cv"; then + echo "written-by: v$_wv (you're on v$_cv — behind; run: git -C \"$SCRIPT_DIR\" pull)" + else + echo "written-by: v$_wv" + fi + fi +``` +Note: `cmd_which` calls `resolve_store` but may not call `check_initialized`, so this also avoids double-printing the skew NOTE; the `which` line is the dedicated surface. + +- [ ] Step 4: `bats test/version.bats` → pass. `bats test/` → 254. +- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: secrets which shows store writer-version + behind note (EGB-713)"` + +--- + +## Task 5: Docs + version bump + +**Files:** `secrets` (cmd_help unchanged unless adding a note), `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`. + +- [ ] Step 1: `CLAUDE.md` — add to the "Store format" bullet a sentence on the writer-version: a committed `.secrets-writer-version` records the highest client `VERSION` that has written (monotonic, stamped on store-committing writes); commands warn once (stderr, non-fatal) when the store's stamp exceeds the running client, and `secrets which` shows `written-by: vN`. Legacy stores (no marker) are silent. (EGB-713.) +- [ ] Step 2: `README.md` — under the upgrading section, note that an out-of-date `secrets` prints a one-line "update" nudge when it touches a store newer than itself. +- [ ] Step 3: `VERSION` → `0.7.1.0`. +- [ ] Step 4: `CHANGELOG.md` — new `## [0.7.1.0] - 2026-06-08` with an Added entry for the version-skew nudge + `secrets which` writer-version line. +- [ ] Step 5: `bats test/` → all green (254). `./secrets which` against a scratch store renders (covered by tests). +- [ ] Step 6: Commit: `git add secrets CLAUDE.md README.md VERSION CHANGELOG.md && git commit -m "docs: version-skew nudge + writer-version; bump 0.7.1.0 (EGB-713)"` + +--- + +## Self-review vs ticket AC +- "Newer stamp → nudge; same/older → silent" → Task 3. +- "No marker → silent (legacy)" → Task 3 + `_store_writer_version` empty. +- "Monotonic, committed" → Task 2. +- "Numeric comparator (0.7.0.0 < 0.10.0.0)" → Task 1. +- "Non-fatal, never changes read/pull exit codes" → Task 3 (`_check_store_version_skew` always `return 0`). +- "`which` surfaces it" → Task 4. "Warn once per invocation" → `_VERSION_SKEW_WARNED` guard. +- bash 3.2: `read -a`, `local -a`, `10#`, parameter strips — all 3.2-safe. diff --git a/docs/superpowers/plans/2026-06-24-multi-recipient-age-encryption.md b/docs/superpowers/plans/2026-06-24-multi-recipient-age-encryption.md new file mode 100644 index 0000000..54a38d8 --- /dev/null +++ b/docs/superpowers/plans/2026-06-24-multi-recipient-age-encryption.md @@ -0,0 +1,1207 @@ +# Multi-recipient age encryption Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Let a single `secrets` store encrypt every blob to N age recipient public keys (a per-store team-key set) instead of one shared key, managed via `secrets recipients add/rm/list`, with full backward compatibility for existing single-key stores. + +**Architecture:** A committed, store-scoped `recipients.txt` (age `-R` format) holds the recipient set. A new `_load_recipients` parses + validates it into a global `RECIPIENT_ARGS=(-r k1 -r k2 …)` array that every encrypt site uses; absence of the file means legacy single-key behavior (`-r $(get_pubkey)`). A shared `_reencrypt_all` engine (factored from today's `rekey`) decrypts the whole store with the local key and re-encrypts to the current set, and is called by `recipients add/rm`, the new `reencrypt`, and multi-recipient `rekey`. Decryption is unchanged — each member uses their own `key.txt`. + +**Tech Stack:** Single POSIX-ish bash script (`secrets`), system bash 3.2 compatible (indexed arrays OK, NO associative arrays). `age` / `age-keygen` for crypto. `git` for the store. `bats-core` for tests. `jq` is NOT introduced anywhere in this feature (recipients.txt is plain text, keeping store ops jq-free). + +## Global Constraints + +- **bash 3.2 only:** no associative arrays, no bash-4 features. Indexed arrays (`RECIPIENT_ARGS=()`, `arr+=(x)`, `"${arr[@]}"`) are fine. +- **bats `[[ ]]` gotcha:** every standalone `[[ … ]]` assertion in a test MUST end with `|| false`. Single-bracket `[ ]` is unaffected. +- **age recipient format accepted:** native age X25519 only — `age1` + exactly 58 chars of `[0-9a-z]`. SSH recipients are out of scope (reject them). This regex/charset is also the injection rail: it cannot contain shell metacharacters, whitespace, or extra flags. +- **`recipients.txt` is committed, NOT gitignored** (public keys are not secret). The store `.gitignore` only blocks `key.txt` and plaintext env files, so the file is tracked automatically — do not add it to `.gitignore`. +- **Security-review policy (`.ship-policy.json`, CLAUDE.md):** adversarial fixtures in this plan are ordinary bats regression tests, NOT AI red-team passes. Do NOT run `./test/run-security.sh` on the user's behalf. Before ship, the human operator runs it and signs off. +- **Re-encrypt invariant:** any change to the recipient set re-encrypts the WHOLE store in one commit. `RECIPIENT_ARGS` is always populated by `_load_recipients` before any `age "${RECIPIENT_ARGS[@]}"` call (never reference the array empty under `set -u`). +- **Commit cadence:** one commit per task (TDD: test → impl → green → commit). + +## File map + +- `secrets` — all code changes (helpers, `recipients`/`reencrypt` commands, encrypt-site refactor, `init`/`which`/`verify`/`rekey` edits, dispatch + help). +- `test/recipients.bats` — NEW suite for all multi-recipient behavior + security fixtures. +- `CLAUDE.md`, `README.md` — docs + test counts. + +## Conventions referenced + +- Constants like `SECRETS_FILES_NAME=".secrets-files"` live ~line 360; `KEY_FILE` is set both as a global default (~line 20) and re-set inside `resolve_store` (~line 301). Mirror this for `RECIPIENTS_FILE`. +- Existing encrypt sites (all `age -r "$pubkey" -o …`): `push_dir_to_project` (~1207), `cmd_push` inline (~1357), `push_external_files` (~655 and ~684), `cmd_rekey` (~1778). `get_pubkey` (~98) derives the pubkey from `key.txt`. +- Tests run via `run "$SECRETS_BIN" ` with isolated `$HOME` and `$SECRETS_DIR`; helpers `init_with_remote`, `create_project_dir` live in `test/test_helper.bash`. + +--- + +### Task 1: Recipient core (`RECIPIENTS_FILE`, validation, `_load_recipients`) wired into the push encrypt path + +**Files:** +- Modify: `secrets` (constants ~line 19-21; `resolve_store` ~301; new helpers after `get_pubkey` ~99; encrypt sites ~655, ~684, ~1207, ~1357; `cmd_push` ~1268; `cmd_push_workspaces` ~1428; `push_dir_to_project`/`push_external_files` signatures) +- Test: `test/recipients.bats` (new) + +**Interfaces:** +- Produces: global `RECIPIENT_ARGS` (indexed array of `-r ` pairs); `RECIPIENTS_FILE` / `RECIPIENTS_FILE_NAME`; `_validate_age_recipient ` (0 = valid age1 key); `_load_recipients` (populates `RECIPIENT_ARGS`, dies on bad/symlinked/empty file). +- Consumes: `get_pubkey`, `SECRETS_DIR`, `KEY_FILE`. + +- [ ] **Step 1: Write failing tests** in new `test/recipients.bats`: + +```bash +#!/usr/bin/env bats +load test_helper + +# A throwaway second identity for "another teammate". +make_second_identity() { + age-keygen -o "$TEST_TMPDIR/bob.txt" 2>/dev/null + BOB_PUB=$(age-keygen -y "$TEST_TMPDIR/bob.txt") +} + +@test "push without recipients.txt stays single-key (legacy behavior)" { + init_with_remote + create_project_dir myproj + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + # No recipients.txt was created by push. + [ ! -e "$SECRETS_DIR/recipients.txt" ] + # Blob decrypts with the store's own key. + run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] +} + +@test "push with a hand-written recipients.txt encrypts to every listed key" { + init_with_remote + make_second_identity + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '# self\n%s\n# bob\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt" + create_project_dir myproj + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + # Bob (a recipient) can decrypt the pushed blob with HIS key. + run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] + # And the store key still can too. + run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] +} + +@test "push refuses a recipients.txt with an invalid key" { + init_with_remote + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '%s\nnot-an-age-key\n' "$STORE_PUB" > "$SECRETS_DIR/recipients.txt" + create_project_dir myproj + run "$SECRETS_BIN" push + [ "$status" -ne 0 ] + [[ "$output" == *"Invalid recipient"* ]] || false +} + +@test "push refuses a symlinked recipients.txt" { + init_with_remote + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '%s\n' "$STORE_PUB" > "$TEST_TMPDIR/elsewhere.txt" + ln -s "$TEST_TMPDIR/elsewhere.txt" "$SECRETS_DIR/recipients.txt" + create_project_dir myproj + run "$SECRETS_BIN" push + [ "$status" -ne 0 ] + [[ "$output" == *"symlink"* ]] || false +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `bats test/recipients.bats` +Expected: FAIL (recipients.txt is ignored today; multi-recipient blob won't decrypt with bob's key; invalid/symlink cases don't error). + +- [ ] **Step 3: Add the constant + `RECIPIENTS_FILE` plumbing** + +Near `KEY_FILE="$SECRETS_DIR/key.txt"` (~line 20), add the name constant just above it and the path just below: + +```bash +RECIPIENTS_FILE_NAME="recipients.txt" +KEY_FILE="$SECRETS_DIR/key.txt" +RECIPIENTS_FILE="$SECRETS_DIR/$RECIPIENTS_FILE_NAME" +``` + +Inside `resolve_store`, right after the line that re-sets `KEY_FILE="$SECRETS_DIR/key.txt"` (~line 301), add: + +```bash + RECIPIENTS_FILE="$SECRETS_DIR/$RECIPIENTS_FILE_NAME" +``` + +- [ ] **Step 4: Add `_validate_age_recipient` and `_load_recipients`** immediately after `get_pubkey` (~line 99): + +```bash +# A native age X25519 recipient: "age1" + exactly 58 chars of [0-9a-z]. +# This is also the injection rail — it cannot hold shell metacharacters, +# whitespace, control chars, or extra flags. SSH recipients are intentionally +# unsupported (EGB-283 scope cut). +_validate_age_recipient() { + case "$1" in + age1*) : ;; + *) return 1 ;; + esac + local body="${1#age1}" + [ "${#body}" -eq 58 ] || return 1 + case "$body" in + *[!0-9a-z]*) return 1 ;; + esac + return 0 +} + +# Populate the global RECIPIENT_ARGS array with one "-r " per store +# recipient. recipients.txt present -> validated keys from the file (the store +# is multi-recipient). Absent -> the single pubkey derived from key.txt (legacy +# single-key store, exactly today's behavior). We parse the file ourselves +# (never `age -R `) because it is committed = an injection surface; every +# line is validated and the file is refused if symlinked. Dies on any problem. +RECIPIENT_ARGS=() +_load_recipients() { + RECIPIENT_ARGS=() + if [ ! -e "$RECIPIENTS_FILE" ]; then + RECIPIENT_ARGS=(-r "$(get_pubkey)") + return 0 + fi + if [ -L "$RECIPIENTS_FILE" ]; then + die "Refusing to read symlinked $RECIPIENTS_FILE_NAME (security)." + fi + local line trimmed n=0 + while IFS= read -r line || [ -n "$line" ]; do + trimmed="${line#"${line%%[![:space:]]*}"}" # lstrip + trimmed="${trimmed%"${trimmed##*[![:space:]]}"}" # rstrip + [ -z "$trimmed" ] && continue + case "$trimmed" in '#'*) continue ;; esac + if ! _validate_age_recipient "$trimmed"; then + die "Invalid recipient in $RECIPIENTS_FILE_NAME: '$trimmed' (expected a native age key: age1...)." + fi + RECIPIENT_ARGS+=(-r "$trimmed") + n=$((n + 1)) + done < "$RECIPIENTS_FILE" + if [ "$n" -eq 0 ]; then + die "$RECIPIENTS_FILE_NAME has no recipients — a store must have at least one. Run 'secrets recipients add '." + fi +} +``` + +- [ ] **Step 5: Route every encrypt site through `RECIPIENT_ARGS`** + +In `push_dir_to_project`, change the encrypt line (~1207): +```bash + age "${RECIPIENT_ARGS[@]}" -o "$SECRETS_DIR/$project/${name}.age" "$f" +``` +and delete its now-unused `local pubkey="$3"` line (~1192). + +In `cmd_push`, change the inline encrypt (~1357): +```bash + age "${RECIPIENT_ARGS[@]}" -o "$SECRETS_DIR/$project/${rel}.age" "$PWD/$rel" +``` + +In `push_external_files`, change both encrypt lines (~655 and ~684) to `age "${RECIPIENT_ARGS[@]}" -o …` (keep the rest of each line identical) and delete its `local pubkey="$3"` from the signature line `local root="$1" project="$2" pubkey="$3"` → `local root="$1" project="$2"`. + +- [ ] **Step 6: Load recipients in the push commands and drop the old `pubkey` threading** + +In `cmd_push` (~1268-1269) replace: +```bash + local pubkey + pubkey=$(get_pubkey) +``` +with: +```bash + _load_recipients +``` +and change the external call (~1365) `push_external_files "$PWD" "$project"` (drop `"$pubkey"`). + +In `cmd_push_workspaces` (~1428) replace the `pubkey=$(get_pubkey)` pair with `_load_recipients`, and drop the `"$pubkey"` argument from the `push_dir_to_project …` (~1432, ~1443) and `push_external_files …` (~1450) calls. + +- [ ] **Step 7: Run the tests** + +Run: `bats test/recipients.bats` +Expected: PASS (4 tests). + +- [ ] **Step 8: Run the full suite to confirm no regression** + +Run: `bats test/` +Expected: PASS (all existing tests still green — legacy push/pull unchanged). + +- [ ] **Step 9: Commit** + +```bash +git add secrets test/recipients.bats +git commit -m "feat: multi-recipient encrypt core + recipients.txt (EGB-283)" +``` + +--- + +### Task 2: `secrets recipients list` + +**Files:** +- Modify: `secrets` (new `_recipients_dump`, `cmd_recipients`, `_recipients_list`; dispatch + nothing in help yet) +- Test: `test/recipients.bats` + +**Interfaces:** +- Produces: `_recipients_dump` (emits `\t` per recipient, name = nearest preceding `# ` comment or empty); `cmd_recipients …` (routes `list`); `_recipients_list`. +- Consumes: `_load_recipients`, `RECIPIENTS_FILE`, `get_pubkey`. + +- [ ] **Step 1: Write failing tests** + +```bash +@test "recipients list on a legacy store shows the single derived key" { + init_with_remote + run "$SECRETS_BIN" recipients list + [ "$status" -eq 0 ] + [[ "$output" == *"single-key"* ]] || false + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + [[ "$output" == *"$STORE_PUB"* ]] || false +} + +@test "recipients list shows names and keys from recipients.txt" { + init_with_remote + make_second_identity + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '# alice\n%s\n# bob\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt" + run "$SECRETS_BIN" recipients list + [ "$status" -eq 0 ] + [[ "$output" == *"recipients: 2"* ]] || false + [[ "$output" == *"alice"* ]] || false + [[ "$output" == *"bob"* ]] || false +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `bats test/recipients.bats -f "recipients list"` +Expected: FAIL ("Unknown command: recipients"). + +- [ ] **Step 3: Add `_recipients_dump`** (place after `_load_recipients`): + +```bash +# Emit "\t" for each recipient in recipients.txt. is the most +# recent preceding "# " comment, or empty. Read-only; no validation +# (callers that need rails call _load_recipients separately). +_recipients_dump() { + [ -e "$RECIPIENTS_FILE" ] || return 0 + local line trimmed name="" + while IFS= read -r line || [ -n "$line" ]; do + trimmed="${line#"${line%%[![:space:]]*}"}" + trimmed="${trimmed%"${trimmed##*[![:space:]]}"}" + [ -z "$trimmed" ] && continue + case "$trimmed" in + '#'*) + name="${trimmed#\#}" + name="${name#"${name%%[![:space:]]*}"}" + ;; + *) + printf '%s\t%s\n' "$trimmed" "$name" + name="" + ;; + esac + done < "$RECIPIENTS_FILE" +} +``` + +- [ ] **Step 4: Add `cmd_recipients` + `_recipients_list`** (place near `cmd_which`): + +```bash +cmd_recipients() { + resolve_store + local sub="${1:-list}" + [ $# -gt 0 ] && shift + case "$sub" in + list) _recipients_list ;; + *) die "Unknown recipients subcommand: '$sub'. Usage: secrets recipients [list]" ;; + esac +} + +_recipients_list() { + check_initialized + if [ ! -e "$RECIPIENTS_FILE" ]; then + check_key + echo "recipients: single-key (no $RECIPIENTS_FILE_NAME)" + echo " $(get_pubkey)" + return 0 + fi + _load_recipients # validates the file (dies on bad key / symlink) + local count=0 k n + while IFS=$'\t' read -r k n; do count=$((count + 1)); done < <(_recipients_dump) + echo "recipients: $count (from $RECIPIENTS_FILE_NAME)" + while IFS=$'\t' read -r k n; do + if [ -n "$n" ]; then echo " $k ($n)"; else echo " $k"; fi + done < <(_recipients_dump) +} +``` + +- [ ] **Step 5: Wire dispatch.** In the `case "${1:-help}"` block, add above `which|where|status`: +```bash + recipients) shift; cmd_recipients "$@" ;; +``` + +- [ ] **Step 6: Run the tests** + +Run: `bats test/recipients.bats -f "recipients list"` +Expected: PASS. + +- [ ] **Step 7: Commit** + +```bash +git add secrets test/recipients.bats +git commit -m "feat: secrets recipients list (EGB-283)" +``` + +--- + +### Task 3: Shared `_reencrypt_all` engine + `secrets reencrypt` + dual `rekey` + +**Files:** +- Modify: `secrets` (new `_reencrypt_all`, `cmd_reencrypt`; rewrite `cmd_rekey` head to branch; dispatch) +- Test: `test/recipients.bats` + +**Interfaces:** +- Produces: `_reencrypt_all ` (decrypt whole store with `KEY_FILE`, re-encrypt to `RECIPIENT_ARGS`, commit + push; aborts with store intact on decrypt failure; no-op on empty store); `cmd_reencrypt`. +- Consumes: `_load_recipients`, `RECIPIENT_ARGS`, `KEY_FILE`, `ensure_store_protections`. + +- [ ] **Step 1: Write failing tests** + +```bash +@test "reencrypt re-encrypts existing blobs to a newly added recipient line" { + init_with_remote + create_project_dir myproj + run "$SECRETS_BIN" push # single-key blob (project name = "myproj"; blob at $SECRETS_DIR/myproj/.env.age) + [ "$status" -eq 0 ] + make_second_identity + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '%s\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt" + # Bob cannot read the old single-key blob yet. + run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -ne 0 ] + run "$SECRETS_BIN" reencrypt + [ "$status" -eq 0 ] + # Now he can. + run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] +} + +@test "rekey on a multi-recipient store keeps recipients and the same key" { + init_with_remote + create_project_dir myproj + run "$SECRETS_BIN" push + before=$(cat "$SECRETS_DIR/key.txt") + make_second_identity + STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt") + printf '%s\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt" + run "$SECRETS_BIN" rekey + [ "$status" -eq 0 ] + # No new keypair was generated. + [ "$(cat "$SECRETS_DIR/key.txt")" = "$before" ] + # Both recipients can decrypt. + run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] +} + +@test "rekey on a legacy store still rotates to a new key (unchanged)" { + init_with_remote + create_project_dir myproj + run "$SECRETS_BIN" push + before=$(cat "$SECRETS_DIR/key.txt") + run "$SECRETS_BIN" rekey + [ "$status" -eq 0 ] + [ "$(cat "$SECRETS_DIR/key.txt")" != "$before" ] + run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age" + [ "$status" -eq 0 ] +} +``` + +- [ ] **Step 2: Run to verify they fail** + +Run: `bats test/recipients.bats -f "reencrypt|rekey on"` +Expected: FAIL ("Unknown command: reencrypt"; multi rekey generates a new key today). + +- [ ] **Step 3: Add `_reencrypt_all`** (place just before `cmd_rekey`): + +```bash +# Decrypt every blob in the store with the local key and re-encrypt each to the +# currently-loaded RECIPIENT_ARGS, then commit + push. The caller MUST have run +# _load_recipients (or set RECIPIENT_ARGS) and check_key first. Aborts with the +# store untouched on any decrypt failure (you must be a current recipient). +# Shared by recipients add/rm, reencrypt, and multi-recipient rekey. +_reencrypt_all() { + local commit_msg="$1" + local tmpdir + tmpdir=$(mktemp -d) + trap 'rm -rf "${tmpdir:-}"' EXIT INT TERM + + info "Decrypting all blobs with your key..." + local file_count=0 dir project f rel dest + for dir in "$SECRETS_DIR"/*/; do + [ -d "$dir" ] || continue + project=$(basename "$dir") + case "$project" in .*) continue ;; esac + mkdir -p "$tmpdir/$project" + while IFS= read -r f; do + [ -f "$f" ] || continue + rel=${f#"$dir"}; rel=${rel%.age} + dest="$tmpdir/$project/$rel" + mkdir -p "$(dirname "$dest")" + if ! age -d -i "$KEY_FILE" -o "$dest" "$f"; then + die "Decryption failed for $project/$rel (are you a current recipient?). Aborted; store unchanged." + fi + file_count=$((file_count + 1)) + done < <(find "$dir" -type f -name '*.age') + done + + if [ "$file_count" -eq 0 ]; then + rm -rf "$tmpdir"; trap - EXIT INT TERM + info "No encrypted blobs in the store — nothing to re-encrypt." + return 0 + fi + + local rc=$(( ${#RECIPIENT_ARGS[@]} / 2 )) + info "Re-encrypting $file_count blob(s) to $rc recipient(s)..." + for dir in "$tmpdir"/*/; do + [ -d "$dir" ] || continue + project=$(basename "$dir") + mkdir -p "$SECRETS_DIR/$project" + while IFS= read -r f; do + [ -f "$f" ] || continue + rel=${f#"$dir"} + mkdir -p "$(dirname "$SECRETS_DIR/$project/$rel")" + age "${RECIPIENT_ARGS[@]}" -o "$SECRETS_DIR/$project/${rel}.age" "$f" + done < <(find "$dir" -type f) + done + + ensure_store_protections + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "$commit_msg" >/dev/null + if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then + git -C "$SECRETS_DIR" push >/dev/null 2>&1 + info "Pushed re-encrypted secrets to remote" + else + info "Committed re-encrypted secrets locally (no remote configured)" + fi + rm -rf "$tmpdir"; trap - EXIT INT TERM +} + +cmd_reencrypt() { + check_cmd age + check_cmd git + resolve_store + check_initialized + check_key + _load_recipients + _reencrypt_all "reencrypt: re-encrypt all to current recipients" +} +``` + +- [ ] **Step 4: Branch `cmd_rekey`.** Replace the head of `cmd_rekey` — from its `check_cmd age` line down to and including the `info "Decrypting all files with current key..."` line — with the block below. **Leave the rest of the existing legacy body (temp dir, decrypt loop, keygen, re-encrypt loop, commit/push) exactly as-is** below this insertion: + +```bash +cmd_rekey() { + check_cmd age + check_cmd git + resolve_store + check_initialized + check_key + + # EGB-283: on a multi-recipient store, rekey means "re-encrypt every blob to + # the current recipients.txt set" — NOT a new keypair (rotating an identity is + # the member's own age-keygen + recipients rm/add). Legacy stores (no + # recipients.txt) keep the original generate-new-keypair behavior below. + if [ -e "$RECIPIENTS_FILE" ]; then + _load_recipients + info "Multi-recipient store — re-encrypting to $RECIPIENTS_FILE_NAME (no new key generated)." + _reencrypt_all "rekey: re-encrypt all to current recipients" + return 0 + fi + + # ── Legacy single-key rotation (unchanged) ── + info "Decrypting all files with current key..." +``` + +- [ ] **Step 5: Wire dispatch.** Add near `rekey)`: +```bash + reencrypt) cmd_reencrypt ;; +``` + +- [ ] **Step 6: Run the tests** + +Run: `bats test/recipients.bats -f "reencrypt|rekey"` +Expected: PASS (3 tests). + +- [ ] **Step 7: Run the full suite** (the legacy rekey tests in `secrets.bats` must still pass) + +Run: `bats test/` +Expected: PASS. + +- [ ] **Step 8: Commit** + +```bash +git add secrets test/recipients.bats +git commit -m "feat: shared _reencrypt_all + reencrypt cmd + dual rekey (EGB-283)" +``` + +--- + +### Task 4: `secrets recipients add` + +**Files:** +- Modify: `secrets` (`_recipients_add`, `_validate_recipient_name`; extend `cmd_recipients` case) +- Test: `test/recipients.bats` + +**Interfaces:** +- Produces: `_recipients_add [--name