Compare commits

...
Sign in to create a new pull request.

100 commits

Author SHA1 Message Date
Brian Majewski
d984e2abc7 docs: repoint forge references from Codeberg to git.dev.egbt.com
The origin moved to the private Forgejo instance at git.dev.egbt.com
(egbt/secrets). Git history, all 16 branches, 13 tags, and all 13 releases
were migrated with identical SHAs and release bodies.

- CLAUDE.md: replace the "Codeberg operations" section with "Forge
  operations", switching tea invocations to `--login egbt --repo
  egbt/secrets`. Records three things that are easy to get wrong:
  Forgejo's SSH listens on port 2222 (port 22 is the host's own sshd, so
  a bare git@host:path fails with "Permission denied (publickey)"); the
  host is Tailscale-only, so off-VPN failures are connectivity, not
  credentials; and the stale `codeberg` tea login is still configured and
  can be silently fallen back to in non-interactive mode.
- CLAUDE.md: note the instance has an Actions runner available, though no
  workflow is configured yet — local bats remains the merge gate.
- README.md: clone URLs now point at the new host.
- CHANGELOG.md: release links now point at the new host.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB
2026-09-08 16:00:54 -07:00
EGBT Technologies
c83935a92f Merge pull request 'v0.7.7.0 fix: discover workspaces in pnpm and yarn monorepos (EGB-1232)' (#16) from brian/egb-1232-pnpm-workspace-discovery into main 2026-09-09 00:26:05 +02:00
Brian Majewski
c09ac38b16 fix: discover workspaces in pnpm and yarn monorepos (EGB-1232)
Workspace patterns came from package.json's `workspaces` key only, at both
call sites. pnpm declares them in pnpm-workspace.yaml instead, so no pnpm
monorepo ever resolved a workspace: `push -w` refused outright, and plain
`push` failed silently — _maybe_workspace_env_files returned 0 the moment
the key was absent, leaving the auto-discovery that covers push's root-only
scan inert and printing "Nothing new to add", indistinguishable from a repo
with genuinely nothing new. That silence cost two sessions on the same repo.

Both call sites now resolve through one shared source, _workspace_patterns:
package.json when it declares any, else pnpm-workspace.yaml's `packages:`
block. The YAML read is deliberately not a parser — block sequence only,
stopping at the next top-level key so pnpm 10's onlyBuiltDependencies:/
catalog: cannot leak in as globs, with quote/comment handling and a symlink
refusal.

Also fixes yarn's object form. `.workspaces // .workspaces.packages | .[]`
short-circuits on the truthy object, iterating its values and yielding the
pattern array as one token; only npm's array form ever worked. Note the
obvious reorder is NOT the fix — `.workspaces.packages` errors on an array —
so the filter is type-aware.

Patterns are validated before reaching the unquoted glob expansion (no
absolute paths, `..`, metacharacters, or whitespace; pnpm `!` negations
skipped), matching the .secrets-store/.secrets-files posture. jq is now
required only when package.json is the source. A monorepo-shaped root that
resolves nothing warns and points at `secrets add` instead of returning in
silence, and `-w`'s error names pnpm-workspace.yaml when that is the file
present.

Scope note: the workspace re-scan still runs only for projects that already
have a .secrets.json — push's root-scan-only first push is by design
(EGB-677 E13), and this bug is the fallback covering it never engaging.

test/workspaces.bats: 18 new tests. Full suite 371/371 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB
2026-09-08 15:22:04 -07:00
EGBT Technologies
b3a727c8fb Merge pull request 'v0.7.6.0 fix: loud store sync failures + secrets sync reconcile (EGB-1230, EGB-1231)' (#15) from brian/egb-1230-1231-store-sync into main 2026-09-08 23:32:25 +02:00
Brian Majewski
f1cf9d9e7e docs: correct Codeberg tea invocation in CLAUDE.md
`tea` autodetection fails in this repo, and the machine carries a second
login (`egbt`) pointing at a different forge that tea silently falls back
to in non-interactive mode. Document the explicit
`--login codeberg --repo egbt/secrets` form for every verb.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB
2026-09-08 14:31:16 -07:00
Brian Majewski
dd6a025fa0 fix: surface store sync failures + add secrets sync (EGB-1230, EGB-1231)
EGB-1230: `cmd_pull` synced the store with `git pull >/dev/null 2>&1` under
`set -euo pipefail`. A store that couldn't fast-forward killed the script at
that line with git's exit 128 and nothing on stdout or stderr — a banner, no
restored files, no reason, and invisible in a pipeline. The sync now routes
through `_store_sync_pull`, which guards the pull, captures git's output as
the diagnosis, and dies naming the store path and `secrets sync`. It is now
`--ff-only` to match the push path, so a plain pull can no longer manufacture
a merge commit in the store.

EGB-1231: once a store diverged there was no way out — push demanded a
fast-forward and pointed at pull, which couldn't fast-forward either, so the
advice looped and recovery meant hand-running git next to encrypted blobs.
Adds `secrets sync`: fetch, stash, rebase onto the remote, restore the stash,
then a confirmation-gated push of local commits (`--yes` to skip the prompt,
`--dry-run` to report only). Non-destructive by construction — no merge, no
force-push, no `reset --hard`, no `stash drop`; a rebase conflict names the
conflicting files and leaves the store exactly as found. `secrets which` gains
a `remote:` line reporting ahead/behind/dirty, and push's dead-end message now
points at `sync`.

test/sync.bats: 25 new tests. Full suite 353/353 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB
2026-09-08 14:20:20 -07:00
EGBT Technologies
2558ea3c23 Merge pull request 'v0.7.5.0 feat: multi-recipient age encryption (recipients/reencrypt, EGB-283)' (#14) from brian/egb-283-secrets-multi-recipient-age-encryption-multiple-keys-per into main 2026-06-24 23:55:29 +02:00
Brian Majewski
e29024bd63 Merge origin/main into EGB-283 (multi-recipient age encryption)
Reconcile the multi-recipient branch (cut from v0.6.1.0) with origin/main
at v0.7.4.0. The two feature lines are largely orthogonal; the one real
integration point is the external-blob encrypt path:

- EGB-712 added an additive-v2 dual-write loop (_external_blob_write_targets,
  writing v2 + any v1 twin). EGB-283 routes every encrypt site through
  RECIPIENT_ARGS for N-recipient encryption. Resolution keeps the dual-write
  loop but encrypts each target to the full recipient set
  (age "${RECIPIENT_ARGS[@]}" per write target), so dual-write and
  multi-recipient compose. cmd_push loads recipients before both external
  push sites; legacy single-key rekey keeps its fresh-keypair pubkey path.

Version: 0.6.2.0 + 0.7.4.0 -> 0.7.5.0. Docs (CLAUDE.md/README/CHANGELOG)
merged to carry both feature sets; subcommand list now includes
recipients/reencrypt and upgrade.

Tests: full `bats test/` green except 6 pre-existing host-environment
failures (4 chmod-600 restore assertions + 2 jq-PATH-shadow tests, all
macOS-authored), none touching merged code. recipients.bats 34/34 pass;
external/dual-write area passes except the same mode-600 host artifacts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 14:52:32 -07:00
Brian Majewski
17772dfec6 fix: validate-before-mutate in recipients add/rm + verify messaging + doc/UX polish (EGB-283)
- Fix 1 (IMPORTANT): _recipients_rm and _recipients_add now call _load_recipients
  BEFORE any mutation. A hand-corrupted recipients.txt dies at validation, leaving
  the file untouched — prevents inconsistent state where the file is changed but
  blobs are not re-encrypted. On a legacy store (no recipients.txt), _load_recipients
  succeeds via the derived-pubkey path so the bootstrap path still works.

- Fix 2 (MINOR): Guard _check_blob_recipient_count behind a successful decrypt in
  both _verify_all and _verify_project — an undecryptable blob no longer produces a
  spurious "encrypted to 0 recipients" finding. Reword _verify_all summary to
  "failed (decrypt or recipient-count)" since both failure modes now increment the
  counter.

- Fix 3 (MINOR): Correct README offboarding comment from "New blobs are no longer
  readable" (contradicts the re-encrypt of EVERY blob) to "Existing blobs are
  re-encrypted; the removed key can no longer decrypt them."

- Fix 4 (MINOR): cmd_reencrypt prints an advisory when no recipients.txt exists
  (single-key store), so the operator knows they can add teammates.

- Fix 5 (MINOR): Test coverage for ambiguous-name rm refusing to remove when
  multiple recipients share a --name label.

Tests: 5 new tests in test/recipients.bats (34 total, all pass). Full suite
276 tests: 5 known pre-existing failures (3 mode-600/stat, 2 jq-PATH), none new.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:37:56 -07:00
Brian Majewski
f7576a3eae docs: multi-recipient age encryption (recipients/reencrypt) + bump 0.6.2.0 (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:25:33 -07:00
Brian Majewski
b09f94c92f test: pin security fixtures to the validation rail (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:21:33 -07:00
Brian Majewski
28f44f043c test: recipients.txt security regression fixtures + dangling-symlink fix (EGB-283)
Swap _load_recipients check order so [ -L ] (symlink) runs before [ ! -e ]
(missing), closing the gap where a dangling symlink bypassed the refusal
and silently fell back to legacy single-key mode.

Add 6 SECURITY-tagged fixtures to test/recipients.bats: dangling-symlink
refused (the ordering gap), shell-metachar injection (no execution), extra-
age-flag-looking line, control/ANSI chars, embedded whitespace via add, and
symlinked file refused on add. All 6 pass immediately after the ordering fix;
the existing rails (_validate_age_recipient, -L checks) were already tight
enough that only the production swap was needed.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:17:28 -07:00
Brian Majewski
e4cd524483 feat: verify asserts blob recipient-count matches recipients.txt (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:10:53 -07:00
Brian Majewski
ae52b8c077 test: restore bootstrap-path + which-legacy coverage (EGB-283)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 13:06:57 -07:00
Brian Majewski
50b73083f1 feat: init born-multi recipients.txt + which recipients line (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 13:03:20 -07:00
Brian Majewski
cad5b66f77 feat: secrets recipients rm with lockout guards (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:56:50 -07:00
Brian Majewski
4ba0234bd7 fix: harden recipients add --name validation (EGB-283)
Require a non-empty argument after --name (dies if it is the last token)
and reject whitespace-only labels that would write a blank comment line.
Two regression tests added to test/recipients.bats.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:53:27 -07:00
Brian Majewski
c262661030 feat: secrets recipients add (EGB-283)
Add `secrets recipients add <age1...> [--name <label>]`: validates the
age key and optional display name, bootstraps recipients.txt with the
local pubkey on a legacy store (keeping the operator as a recipient),
rejects duplicates, appends the new key (with optional name comment),
then re-encrypts the entire store to the updated recipient list.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:50:00 -07:00
Brian Majewski
a10f390519 feat: shared _reencrypt_all + reencrypt cmd + dual rekey (EGB-283)
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:44:44 -07:00
Brian Majewski
649aa78063 feat: secrets recipients list (EGB-283) 2026-06-24 12:39:52 -07:00
Brian Majewski
14cf86fba9 docs: plan fixups from Task 1 review (blob path, dangling-symlink fix to Task 8)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:37:55 -07:00
Brian Majewski
0f9de1c2fd feat: multi-recipient encrypt core + recipients.txt (EGB-283)
- Add RECIPIENTS_FILE_NAME / RECIPIENTS_FILE constants; update resolve_store to re-derive RECIPIENTS_FILE after store resolution.
- Add _validate_age_recipient (native age1 X25519 key format check, injection rail).
- Add RECIPIENT_ARGS global array and _load_recipients (absent → single pubkey legacy path; present → parse+validate recipients.txt, refuse symlink, die on bad/empty).
- Rewire all 5 push encrypt sites (push_dir_to_project, cmd_push inline, push_external_files ×2, cmd_push_workspaces) to use RECIPIENT_ARGS; drop pubkey threading from push_dir_to_project and push_external_files signatures.
- New test/recipients.bats (4 tests): legacy single-key, multi-recipient decrypt, invalid key rejection, symlink rejection.
- Fix test/test_helper.bash: set GIT_AUTHOR/COMMITTER env vars so git commit works with isolated $HOME.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-24 12:33:05 -07:00
Brian Majewski
3a7eea5529 docs: multi-recipient age encryption implementation plan (EGB-283)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:20:32 -07:00
Brian Majewski
9e2a563059 docs: multi-recipient age encryption design spec (EGB-283)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-24 12:12:14 -07:00
EGBT Technologies
6b98ee8208 Merge pull request 'v0.7.4.0 feat: secrets upgrade verb — self-update + re-check version skew (EGB-716)' (#13) from brian/egb-716-secrets-secrets-upgrade-verb-self-update-re-check-version into main 2026-06-18 21:16:21 +02:00
Brian Majewski
a17ae4448b chore: bump version and changelog (v0.7.4.0)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 11:01:17 -07:00
Brian Majewski
09c4ad54f9 feat: secrets upgrade verb — self-update + re-check version skew (EGB-716)
Pairs the EGB-713 skew WARNING with a fix path. `secrets upgrade` fast-forwards
the tool's own git checkout (git -C "$SCRIPT_DIR" pull --ff-only; never merges or
rewrites local commits), reports vOLD -> vNEW, then best-effort re-checks the
store's recorded writer-version against the new version so the operator sees
whether the nudge is cleared. `secrets upgrade --check` reports availability
without pulling. Thin and explicit: no auto-update, no background polling
(security tool). Directed errors for not-a-checkout / no-upstream / diverged /
offline. cmd_upgrade never calls check_initialized (it's about the tool, not
the store); the skew re-check is silent unless a store with a writer-version
resolves.

Wired into the dispatcher (upgrade) shift; cmd_upgrade "$@") and cmd_help.
Tests: test/upgrade.bats (8) run a relocated script copy in a throwaway git
repo with a bare upstream, so the real checkout is never touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 11:01:17 -07:00
EGBT Technologies
33a5bb0758 Merge pull request 'v0.7.3.1 refactor: external-extractor + read-guard dedup, legacy-pull nested-blob warning (EGB-701)' (#12) from brian/egb-701-secrets-stage-1-structural-cleanups-dedup-external-entry into main 2026-06-18 19:40:18 +02:00
Brian Majewski
3ece393cc5 chore: bump version and changelog (v0.7.3.1)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 10:08:13 -07:00
Brian Majewski
1ee096cc33 refactor: dedup external extractor + read guards, warn on legacy-pull nested blobs (EGB-701)
EGB-677 stage-1 structural cleanups, no behavior change for the v2 happy path:

1. cmd_which reuses _json_external_entries (the push/pull extractor) instead of
   its own duplicated jq @tsv projection, so `which` applies the same
   properties->gradle-properties normalization + skip-with-warning rules the
   sync path does and can't drift from it.
2. The two external-manifest read guards are factored into _json_readable
   (plain regular file, silent) / _legacy_readable (warn+skip a symlinked
   .secrets-files), shared by _external_entries_for_push/_pull.
3. The legacy (manifest-less) pull path now warns when nested <project>/<relpath>.age
   blobs exist that its non-recursive globs can't see (external/ excluded —
   pull_external_files handles those), so it never silently under-restores.

Tests: +4 in test/manifest.bats (normalized which display, malformed external
skipped by which, nested-blob warning fires, external-only no false warning).
Full suite green (286/286).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-18 10:08:13 -07:00
EGBT Technologies
57f1280262 Merge pull request 'v0.7.3.0 feat: secrets join + verified onboarding scripts (EGB-671)' (#11) from brian/egb-671-create-real-install-scripts into main 2026-06-09 01:30:08 +02:00
Brian Majewski
4d975d447d chore: rewrite onboarding docs + bump version (v0.7.3.0)
README rewritten to the install.sh + init --remote + join flow; dropped the
macOS-only prerequisite (age+jq install hints now cover apt/dnf too). CHANGELOG
entry for EGB-671.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 16:23:50 -07:00
Brian Majewski
7b041af68b feat: thin install.sh onboarding bootstrap (EGB-671)
Ships in the repo (clone already done). Checks age + jq + git, then PRINTS the
PATH line, onboarding next-steps, upgrade one-liner, and key-transfer hint.
Never edits shell rc, never runs sudo (prints the command). Exits non-zero with
an install hint when a dependency is missing.
2026-06-08 16:23:50 -07:00
Brian Majewski
6319313ee4 feat: secrets join + init --remote + verified onboarding (EGB-671)
Add second-machine onboarding as a first-class verb rather than a manual
clone + key-copy sequence:

- secrets join --remote <url> --key <path>: clone the vault, install the key
  at mode 600, then decrypt-test it before declaring success. An empty vault
  reports "nothing to verify yet" (never a false VERIFIED); a wrong key fails
  loudly. Reuses the audited core (resolve_store, get_pubkey, _verify_all) —
  no security logic re-implemented.
- secrets init --remote <url>: wire the remote and push the initial store so
  the upstream branch exists (fixes the commit_and_push_secrets pull --ff-only
  die against a brand-new empty remote). init also offers an interactive
  first-add of a project (default No; skipped under --yes / non-interactive).
- cmd_push first-manifest scaffold writes an explicit committed options.autoAdd
  value, asked once when interactive (EGB-677 contract #2).
- secrets pull now dies loudly when a blob fails to decrypt (all three decrypt
  paths) instead of warning and exiting 0 — a wrong key can't pass silently.
- Interactive prompts gate on stdin AND stdout being ttys, so bats/CI never hang.
- Dispatcher routes init/join args correctly; second-machine trap points at join.

Tests: 20 new (join, autoAdd, pty-no-hang regression); 2 trap tests updated.
2026-06-08 16:23:50 -07:00
EGBT Technologies
ec538d7ef0 Merge pull request 'v0.7.2.0 feat: secrets list --json machine-readable output (EGB-699)' (#10) from brian/egb-699-secrets-list-json-machine-readable-output into main 2026-06-08 23:00:47 +02:00
Brian Majewski
446256caf1 feat: secrets list --json machine-readable output (EGB-699)
Add a --json flag to `secrets list` that emits a structured object for
tooling/CI instead of the human table — feeds the EGB-671 install scripts,
which need to enumerate a cloned store programmatically.

Contract: {"store", "projects":[{"name","entries":[...]}]}, each entry
self-describing via a type discriminator — {type:dotenv,path} or
{type:external,subtype:properties|file,path}. cmd_list_json mirrors the same
recursive store walk as the human list (nested <project>/<relpath>.age +
external/<slug>.age); jq assembles the JSON so paths escape correctly and
stdout stays pure JSON (the non-default-store hint is suppressed; jq is a
hard dep only in --json mode).

Tests: 7 new bats cases (dotenv, nested relpath, external properties + file
subtypes, empty store, pure-stdout-under-notice, store path). Full suite
261 pass / 0 fail.

VERSION 0.7.1.0 -> 0.7.2.0; CHANGELOG/README/CLAUDE.md updated.
2026-06-08 13:56:45 -07:00
EGBT Technologies
b8fe20f9bf Merge pull request 'v0.7.1.0 feat: version-skew nudge (EGB-713)' (#9) from brian/egb-713-secrets-warn-when-the-store-was-written-by-a-newer-client into main 2026-06-08 21:11:31 +02:00
Brian Majewski
50476e19fd docs: version-skew nudge + writer-version; bump 0.7.1.0 (EGB-713) 2026-06-08 12:07:51 -07:00
Brian Majewski
5461418c5d feat: version-skew nudge — stamp store writer-version, warn when behind (EGB-713) 2026-06-08 12:07:50 -07:00
Brian Majewski
367b70cba1 docs: EGB-713 version-skew nudge plan 2026-06-08 11:56:47 -07:00
EGBT Technologies
40635f9e1f Merge pull request 'v0.7.0.0 feat: additive-v2 dual-write — optional finalize (EGB-712)' (#8) from brian/egb-712-secrets-additive-v2-dual-write-defuse-the-finalize into main 2026-06-08 20:01:47 +02:00
Brian Majewski
9b9af2f30c docs: additive-v2 propagation + optional-GC finalize; bump 0.7.0.0 (EGB-712) 2026-06-08 10:52:34 -07:00
Brian Majewski
040782cad0 feat: secrets migrate --status surfaces v2-only externals (coverage, EGB-712) 2026-06-08 10:44:34 -07:00
Brian Majewski
69ab9636e9 test: fabricate old-client v1 blobs in migrate/finalize/status fixtures (additive v2, EGB-712) 2026-06-08 10:38:18 -07:00
Brian Majewski
2f36fe1898 feat: twin-rule write targets — dual-write existing, v2-only for new (additive v2, EGB-712) 2026-06-08 10:31:00 -07:00
Brian Majewski
2866e5f4b1 feat: read-resolver tries both external suffixes (additive v2, EGB-712) 2026-06-08 10:08:41 -07:00
Brian Majewski
ee4ea413ef docs: additive-v2 implementation plan + spec §3 fix (no marker auto-stamp) 2026-06-08 09:58:23 -07:00
Brian Majewski
54575af61c docs: additive-v2 dual-write design spec (defuse the finalize gate) 2026-06-08 09:45:10 -07:00
EGBT Technologies
836b418a12 Merge pull request 'v0.6.1.0 feat: secrets migrate guided flow (manifest-free + --status, EGB-710)' (#7) from brian/egb-710-secrets-make-migrate-a-guidedinteractive-flow-not-a-dead-end into main 2026-06-08 18:06:18 +02:00
Brian Majewski
3ac2a86a94 docs: fix stale manifest-dependency comments in migrate (EGB-710) 2026-06-08 08:26:16 -07:00
Brian Majewski
5729d84e75 docs: migrate --status + manifest-free copy-forward; bump 0.6.1.0 (EGB-710) 2026-06-08 08:01:44 -07:00
Brian Majewski
469b08203b feat: secrets migrate --status surveys per-project v2 readiness (EGB-710) 2026-06-08 07:50:32 -07:00
Brian Majewski
e75627deff test: migrate twins undeclared store blobs (finalize-consistency, EGB-710) 2026-06-08 07:18:22 -07:00
Brian Majewski
c121982dcd fix: migrate copy-forward is manifest-free, no dead-end on legacy projects (EGB-710) 2026-06-08 07:08:08 -07:00
Brian Majewski
679ddbc1b7 docs: EGB-710 implementation plan (migrate guided flow) 2026-06-08 06:46:50 -07:00
EGBT Technologies
633d19e5f7 Merge pull request 'v0.6.0.1 feat: secrets which prints manifest version (EGB-700)' (#6) from brian/egb-700-which-manifest-version into main 2026-06-08 15:23:18 +02:00
Brian Majewski
2549832f0e docs: sync test counts and which manifest-version note (EGB-700)
- README: test suite total 236 -> 237
- CLAUDE.md: manifest.bats 77 -> 78 tests
- CLAUDE.md: note `secrets which` now prints the .secrets.json schema
  version in its manifest header line alongside `format: vN`

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 06:21:26 -07:00
Brian Majewski
6a84846e64 chore: bump version and changelog (v0.6.0.1)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-08 06:19:57 -07:00
Brian Majewski
94ee6ec9a2 feat: secrets which prints the manifest version (EGB-700)
Completes EGB-700 — the store-format line (format: vN) shipped in v0.6.0.0;
this adds the manifest schema version to the `which` manifest header
(version N, validated == MANIFEST_VERSION by _check_manifest_file). Now a
single `secrets which` surfaces both the store format and the manifest
version for the dual-format debugging window. 1 bats test. Suite 237/237.
2026-06-08 06:02:11 -07:00
EGBT Technologies
eaa08e8561 Merge pull request 'v0.6.0.0 feat: store-format-v2 migration (EGB-703)' (#5) from brian/egb-703-store-format-v2-migration into main 2026-06-08 02:00:34 +02:00
Brian Majewski
c4da47f320 docs: correct test counts for migrate.bats (236 total)
README.md: test suite count 210 → 236, "both files" → "three files".
CLAUDE.md: testing command lists migrate.bats; Project Structure tree
adds migrate.bats (EGB-703 store-format-v2 migration, 26 tests).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 16:57:53 -07:00
Brian Majewski
cefa06280e chore: bump version and changelog (v0.6.0.0)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 16:56:27 -07:00
Brian Majewski
c10e89fb51 fix: pre-landing review for store-format-v2 (verify-window orphan, copy-forward push, --force, comment)
Pre-landing review (1 critical, 4 informational):
- CRITICAL: per-project `secrets verify` during the migration window flagged
  the freshly-written v2 twin as a spurious orphan and exited 1 (store still
  reads v1, so verify's expected set only held the .gradle-properties.age name).
  Breaks the documented migrate->verify->finalize workflow and CI. Fix: verify's
  orphan set now accounts for BOTH suffix forms of a properties external, so the
  twin is never a false orphan mid-migration. Regression test added.
- Copy-forward now pushes the twins (mirrors push/rekey) so a --finalize on
  another machine sees them; previously twins were local-only until finalize,
  a multi-machine footgun.
- Dropped the undocumented `--force` alias (keep `--yes`).
- Clarified the EGB-700 comment (which-format line, folded into EGB-703).

Deferred to EGB-701: the two finalize find-walks over *.gradle-properties.age
could collapse to one pass. Full suite 236/236.
2026-06-07 16:47:17 -07:00
Brian Majewski
5521d978a3 test: coverage for migrate gap branches (dotenv-only finalize, positional-arg die, already-v2 finalize, strict marker parse)
Coverage audit found 4 untested branches (all single-test fills): finalize on
a v1 dotenv/file-only store (v1count==0 stamp path — a real user scenario),
the migrate positional-arg die, finalize on an already-v2 store, and the
_store_format strict-parse defense (garbage marker reads as v1). Full suite 235/235.
2026-06-07 16:35:26 -07:00
Brian Majewski
2192b5a2df fix: tighten _store_format parse + correct finalize crash-orphan comment (EGB-703 review)
Self-review of the data-safety paths (verdict SHIP-SAFE) flagged two
non-blocking nits, both fixed:
- _store_format used `tr -dc '0-9'` which read garbage like "v2"/"x2x" as v2.
  Tightened to a strict exact match (modulo line endings) so only "2" reads as
  v2; anything else falls back to v1, the safe default.
- The stamp-before-delete comment claimed a re-run "cleans" crash-orphaned v1
  blobs; it doesn't (finalize early-returns once the store is v2). Corrected to
  note the orphans are harmless and `verify` flags them for manual removal.
2026-06-07 16:18:16 -07:00
Brian Majewski
e2ad661da5 feat: store-format-v2 self-describing migration (EGB-703, folds in EGB-700)
Stage 2 of EGB-677. Makes the store self-describing and unifies the legacy
external `properties` blob suffix, via a copy-forward migration that never
destroys data until an explicit, gated finalize.

Scope decision (see eureka): the EGB-677 CEO plan's "flatten dotenv blobs to
basename" was dropped as LOSSY — it discards the restore relpath that makes
the store self-describing and adds basename collisions. Engineering analysis
(4 parallel design agents) showed the store is already relpath-self-describing;
the only real v1→v2 delta is the `properties` suffix. This implements the
minimal, safe v2 that achieves the epic's self-describing goal.

What's added:
- `.secrets-format` marker (committed, one line `2`). Absence ⇒ v1 (every
  pre-EGB-703 store). `_store_format()` reads it; `init` stamps fresh stores
  born-v2. `secrets which` prints `format: vN` (EGB-700 folded in).
- `_external_blob_suffix(type)` — single source of truth for the external
  suffix (v2: gradle-properties → properties; file unchanged). push/pull/verify
  all route through it, so v1 and v2 stores never disagree on blob location.
- `secrets migrate` — per-project copy-forward (writes `.properties.age` twins
  beside v1 blobs; idempotent; needs the project manifest), `--dry-run`
  (reports old→new, writes nothing), `--finalize` (store-wide, the only
  destructive step: gates on `verify --all` green + every v1 blob twinned,
  cuts a `pre-v2-migrate-<sha>` recovery tag, stamps the marker, then drops v1
  blobs; refuses without `--yes`/operator confirmation).

rekey and verify --all stay format-agnostic (recursive find walk) — no change.

21 new bats tests (test/migrate.bats): marker/born-v2, format-aware suffix,
v1 back-compat, dry-run, copy-forward idempotency, no-manifest die, finalize
gates (verify-not-green refusal, untwinned refusal, recovery tag, confirmation),
and full v1→window→finalize round-trip. Updated 4 existing tests for the
born-v2 suffix. Full suite 231/231, bash 3.2 clean.
2026-06-07 16:07:31 -07:00
EGBT Technologies
144ff3692b Merge pull request 'v0.5.0.0 feat: secrets verify integrity check (EGB-698)' (#4) from brian/egb-698-secrets-verify-command into main 2026-06-08 00:34:02 +02:00
Brian Majewski
e33bc272d2 docs: surface secrets verify in troubleshooting, refresh test counts (v0.5.0.0)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 15:31:30 -07:00
Brian Majewski
d170fb03db chore: bump version and changelog (v0.5.0.0)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 15:29:01 -07:00
Brian Majewski
414c02b902 fix: pre-landing review fixes for verify (test assertions, ekeys discard, double-report, docs)
Pre-landing review (0 critical, all informational) auto-fixes:
- Tighten external-corrupt test to assert the decrypt-fail message, not any
  external finding (was *"external"*, now *"does not decrypt"*).
- Pin the verified-count in the nested+external happy-path test so a silent
  under-count (exit 0 while skipping a blob) is caught.
- Account for an unsafe dotenv entry in `expected` so a matching stray blob
  isn't double-reported as both unsafe and orphan.
- Discard the unused external `keys` read field (read -r etype epath _).
- Document the optional [project] positional in the README verify row.

Deferred to EGB-701 (stage-2 dedup): the external blob-path literal and the
find-walk overlap with cmd_rekey/cmd_list. Full suite 210/210.
2026-06-07 15:22:58 -07:00
Brian Majewski
33aad4f89a test: coverage for verify gap branches (external decrypt-fail, unsafe path, unknown flag, malformed manifest, empty --all)
Coverage audit found 5 untested branches in cmd_verify (all single-test
fills, no logic defects): external blob decrypt-failure (only the missing
case was covered), the rail-skip finding for an unsafe dotenv path in the
manifest, the unknown-flag die, a malformed manifest through the verify
entry point, and the empty-store 'verify --all' no-op. Full suite 210/210.
2026-06-07 15:14:54 -07:00
Brian Majewski
52528f2e06 feat: secrets verify — manifest↔store consistency + decrypt integrity (EGB-698)
Read-only integrity check, the safety net for the stage-2 store migration.

Default mode (current project) cross-checks $PWD/.secrets.json against the
store both ways — declared-but-missing blobs and orphaned blobs (no manifest
entry) — and decrypt-tests every dotenv + external blob with the current key,
streaming plaintext to /dev/null so nothing is ever written to disk. `verify
--all` decrypt-tests every blob in every project (integrity only; the store
carries no manifests, so consistency can't be checked store-wide). Both
recurse the whole project tree (find -type f), the same walk rekey/list use,
so nested manifest blobs are covered. Exits non-zero on any finding so it can
gate `migrate --finalize` and CI.

12 bats tests (clean, nested+external, missing blob, decrypt failure, orphan,
missing external, no-manifest die, symlink refusal, --all clean/corrupt/orphan,
nested decrypt failure). Full suite 205/205. bash 3.2 clean.
2026-06-07 14:55:42 -07:00
EGBT Technologies
fb71b956da Merge pull request 'v0.4.0.0 feat: .secrets.json manifest-driven sync (EGB-677 stage 1)' (#3) from brian/egb-677-unify-storage-model-manifest-driven-sync-store-format-v2 into main 2026-06-07 22:57:21 +02:00
Brian Majewski
a35bff8d4b docs: correct manifest.bats and total test counts for v0.4.0.0
CLAUDE.md: manifest.bats 58 -> 60 tests (actual @test count).
README.md: total 191 -> 193 tests across both files.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 13:53:05 -07:00
Brian Majewski
30c203cafd chore: bump version and changelog (v0.4.0.0)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-07 13:50:51 -07:00
Brian Majewski
588f290dcc chore: security review policy + operator runner, doc updates (EGB-677 stage 1)
Add .ship-policy.json (opts out AI adversarial/red-team/security-specialist
review; requires local operator sign-off) and test/run-security.sh (the
operator-local security regression subset). Document the policy in CLAUDE.md
and README, fix stale test counts (manifest.bats 41->58, total 174->191), and
update the storage-recursion note to reflect rekey/list now walking the full
project tree.
2026-06-07 13:12:39 -07:00
Brian Majewski
c6ea724ddb fix: rekey and list recurse into nested manifest blobs (EGB-677 stage 1)
Pre-landing review (testing + checklist specialists, reproduced) caught a
data-loss bug: cmd_rekey's decrypt/re-encrypt globs were non-recursive and
only special-cased external/. Nested manifest dotenv blobs
(<project>/<relpath>.age, new this branch) were never visited, so after a key
rotation they stayed encrypted under the discarded old key = permanently
undecryptable. cmd_list had the same blind spot (cosmetic: nested entries
invisible in listings).

Both now walk the entire project tree with `find -type f` (bash 3.2 safe,
includes dotfiles natively), unifying top-level / nested / external blobs into
one recursive pass and dropping the now-redundant external/ special-casing.

Regression tests: nested-blob rekey round-trip (survives rotation) + list
shows nested entry. Full suite 193/193.
2026-06-07 13:12:27 -07:00
Brian Majewski
5489f89446 test: coverage backfill — which UNSAFE marker, malformed-manifest add, dry-run declared list, frozen absorb suppression, file-type absorb round-trip, external dedup 2026-06-07 10:17:07 -07:00
Brian Majewski
ecc4f23494 test: coverage for stage-1 gap paths (json rail warns, symlink legacy, directed errors, dry-run/which branches) 2026-06-07 09:40:43 -07:00
Brian Majewski
89e851278b feat: jq gating, platform-aware install hints, stage-1 docs (EGB-677 stage 1)
- jq required only when .secrets.json exists or is being written;
  manifest-less projects run jq-free (features skipped with a notice) —
  clone-and-run for v1 users survives (no-jq fixture excludes /usr/bin,
  macOS ships jq there now)
- check_cmd: platform-aware hints (brew/apt-get/dnf/generic) instead of
  hardcoded brew — correct guidance on Linux/CI
- cmd_help: add command, push flags, manifest section with example
- README: manifest section, external files rewritten around
  .secrets.json (legacy .secrets-files documented as absorbed),
  troubleshooting entries, command table, test instructions
- CLAUDE.md: manifest architecture notes, bash-3.2 '[[ ]] || false'
  testing convention, project structure refresh
2026-06-07 08:58:16 -07:00
Brian Majewski
0049584d9b feat: manifest-driven pull — nested restore, restore-time rail, empty no-op (EGB-677 stage 1)
- pull with .secrets.json restores exactly the declared entries (nested
  paths get mkdir -p); stray store blobs are not restored
- dotenv rail re-runs at restore time: unsafe entries warn+skip (pull
  never dies on one bad entry), missing blobs warn with a directed hint
- empty manifest = warn no-op instead of a confusing 'not found' death
- manifest-less projects keep the legacy glob pull verbatim
2026-06-07 08:49:02 -07:00
Brian Majewski
d7e1400487 feat: external entries via .secrets.json + legacy absorb + properties rail (EGB-677 stage 1)
- .secrets.json external[] drives push/pull: type 'properties' (alias of
  gradle-properties; blob suffix stays legacy-compatible in stage 1) and
  type 'file'; same charset rails as the legacy parser
- push absorbs uncovered .secrets-files entries into the manifest
  (idempotent, gradle-properties → properties) with a delete hint
- pull: manifest wins entirely; a coexisting .secrets-files warns as
  superseded instead of being silently ignored
- basename rail generalized: properties targets must end '.properties'
  (was exact 'gradle.properties') — rc files/gitconfig still blocked;
  EGB-531 wrong-basename test updated for the sanctioned change
2026-06-07 08:44:02 -07:00
Brian Majewski
884da0965c feat: manifest-aware push — generator auto-add, autoAdd toggle, --frozen/--dry-run (EGB-677 stage 1)
- push syncs FROM the manifest; v1 store layout unchanged (nested
  entries land at <project>/<relpath>.age, same shape -w always used)
- discovery (root globs + quiet package.json workspace re-scan when a
  manifest exists) feeds the manifest as a generator; new files auto-add
  with ==> notice + undo guidance
- options.autoAdd committed toggle (default ON when absent); explicit
  false warns on undeclared files instead of enrolling them
- push --frozen: declared-only for one invocation; push --dry-run:
  reports would-add/would-sync, touches nothing
- bootstrap ordering: manifest written only after >=1 blob encrypts
- declared-but-missing warns and continues; unsafe manifest path dies
- jq // falsy gotcha: explicit autoAdd:false compared directly
2026-06-07 08:32:17 -07:00
Brian Majewski
18018dbd3b feat: .secrets.json manifest core — add command, rails, canonical form (EGB-677 stage 1)
- secrets add <path>: explicit manifest writer, idempotent, atomic write
- _validate_dotenv_rel_path: project-relative confinement rail (no .. /
  absolute / shell metas; @ allowed for npm-scoped workspace dirs)
- _check_manifest_file: refuses symlinks, malformed JSON (jq error with
  file named), unsupported schema versions (directed upgrade error)
- canonical serialization: jq --sort-keys + sorted/deduped dotenv —
  add order produces byte-identical manifests
- which: validates + summarizes the manifest (doubles as linter)
- jq required only when a manifest exists/is written
2026-06-07 08:24:35 -07:00
Brian Majewski
6dbc4e0d01 test: make [[ ]] assertions effective under bash 3.2 (EGB-677 precursor)
bats on macOS runs under system bash 3.2, where a failing [[ ]] compound
command mid-test does NOT trip the ERR trap — every mid-test
[[ "$output" == *...* ]] assertion in the suite was decorative.
Append '|| false' to all 123 standalone [[ ]] assertion lines so failures
become plain-command failures, which bats catches. Full suite still green
(133/133) — no latent false positives were hiding.
2026-06-07 08:21:29 -07:00
Brian Majewski
a3c71f293b Merge branch 'feat/external-file-type' (v0.3.0.0) 2026-06-07 06:44:40 -07:00
Brian Majewski
a314cb1fb1 docs: CLAUDE.md — record the 'file' external type (EGB-652) 2026-06-07 06:44:22 -07:00
Brian Majewski
e0474f3ce9 feat: 'file' external type — whole-file sync for .secrets-files (EGB-652)
Built for the Beacon Android upload keystore: binary files outside the
project root can now ride the encrypted store.

- manifest: 'file <path>' (no keys; keys present = rejected loudly)
- push: encrypts the file verbatim (age is binary-safe)
- pull: restores next to target (atomic same-fs mv), mode 600, TOCTOU
  symlink recheck, divergent existing target backed up to .secrets-bak
- _validate_external_target_path parameterized by type (basename
  restriction stays gradle-properties-only; $HOME/../symlink rails apply
  to both)
- README/help/CLAUDE docs + 7 bats tests (133/133 pass)
2026-06-07 06:43:45 -07:00
EGBT Technologies
ab45c4c94b Merge pull request 'docs: Codeberg/tea operations guide' (#2) from docs/codeberg-tea-guide into main 2026-06-05 19:59:02 +02:00
Brian Majewski
783be5cd96 docs: add Codeberg/tea operations guide to CLAUDE.md 2026-06-05 10:58:31 -07:00
Brian Majewski
6ab22c2b95 Merge branch 'fix/init-second-machine-guard' (v0.2.1.0)
Second-machine init guard, store-protection self-heal (gitignore + hook +
key untrack, post-pull ordering), and the rekey dotfile data-loss fix.
2026-06-05 10:49:54 -07:00
Brian Majewski
ba4c53a160 docs: update README for v0.2.1.0
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 10:29:07 -07:00
Brian Majewski
8d9e9a3bfc chore: bump version and changelog (v0.2.1.0)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 10:26:07 -07:00
Brian Majewski
2a7afc34dd fix: pre-landing review fixes (heal ordering, key untrack, content-aware gitignore)
Red-team + specialist findings from /ship pre-landing review:

- CRITICAL: ensure_store_protections ran BEFORE `git pull --ff-only` in
  commit_and_push_secrets; a pull bringing history without .gitignore
  left the post-pull window unprotected and `git add -A` committed AND
  pushed key.txt (reproduced end-to-end). Heal now runs after the pull,
  immediately before staging.
- CRITICAL: .gitignore can't untrack an already-tracked key.txt. Heal
  now defensively `git rm --cached`s a tracked key with a history-scrub
  warning.
- Content-aware heal: a present .gitignore missing the key.txt line is
  rewritten, not skipped (security specialist).
- Init guard now renders the real sanitized clone URL from
  .secrets-store when available, mirroring EGB-282's check_initialized.
- Tests: positive heal-message assertions, non-empty key guard in the
  init test, 4 new regression tests (126 total).
2026-06-05 10:14:46 -07:00
Brian Majewski
5865c40d77 test: coverage for store protections self-heal (hook on push/rekey, no-op, gitignore content) 2026-06-05 09:52:51 -07:00
Brian Majewski
53350853d9 Merge remote-tracking branch 'origin/main' into fix/init-second-machine-guard 2026-06-05 09:41:27 -07:00
Brian Majewski
6db4f2b217 chore: add gstack skill routing rules to CLAUDE.md 2026-06-05 09:40:39 -07:00
Brian Majewski
25cd1feb43 fix: init second-machine guard, store .gitignore self-heal, rekey dotfile loss
Three fixes, found from a real Ubuntu second-machine setup failure:

1. cmd_init: if key.txt exists but the store has no .git, die BEFORE
   git init with directed guidance to clone the existing secrets repo
   instead. Previously init ran git init, then age-keygen died on the
   existing key, leaving a half-initialized store (.git but no
   .gitignore or hook).

2. ensure_store_protections: push/pull/rekey now restore a missing
   store .gitignore (and pre-commit hook) before any `git add -A`.
   Without the .gitignore, add -A would commit key.txt to the remote.

3. cmd_rekey: the re-encrypt loop used a bare "$dir"* glob, which never
   matches dotfiles — .env blobs were decrypted to the tmpdir but never
   re-encrypted, leaving them on the OLD key (undecryptable) while the
   new key overwrote key.txt. Glob now matches the decrypt loop
   ("$dir"* "$dir".*). Exposed by the self-heal test: with nothing
   staged, the empty commit failed mid-rekey.

Tests: 5 new bats tests (118 total) — init guard, .gitignore self-heal
on push/pull/rekey, and a rekey round-trip that survives key rotation.
2026-06-05 09:38:16 -07:00
Brian Majewski
8b2fc4d980 docs: update CLAUDE.md deploy note to Codeberg
The Codeberg migration commit updated README/CHANGELOG but missed the
"distributed via git clone from GitHub" line in CLAUDE.md.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-01 18:31:04 -07:00
EGBT Technologies
e947cfde19 Merge pull request 'brian/egb-531-secrets-support-gradleproperties-files-not-just-dotenv' (#1) from brian/egb-531-secrets-support-gradleproperties-files-not-just-dotenv into main
Reviewed-on: https://codeberg.org/egbt/secrets/pulls/1
2026-05-26 23:59:38 +02:00
25 changed files with 9956 additions and 365 deletions

15
.ship-policy.json Normal file
View file

@ -0,0 +1,15 @@
{
"version": 1,
"ai_review": {
"adversarial": false,
"red_team": false,
"security_specialist": false,
"reason": "This repo ships defensive security regression tests with attack-payload fixtures (shell injection, path traversal, symlink rails). AI-driven adversarial, red-team, and security-specialist review triggers Anthropic Usage Policy blocks (real-time cyber safeguards on hosted models). Do not dispatch those subagents or prompts for this repository."
},
"manual_security_signoff": {
"script": "./test/run-security.sh",
"signoff_file": ".gstack/security-signoff.json",
"required_before_ship": true,
"instructions": "A human operator runs the script locally, reviews output, and types SIGNOFF. Agents must not run the security suite or substitute AI review for sign-off."
}
}

View file

@ -5,6 +5,395 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme.
## [0.7.7.0] - 2026-09-08
### Fixed
- **pnpm monorepos discovered no workspace secrets (EGB-1232)** — both
workspace call sites resolved patterns from `package.json`'s `workspaces`
key only, which pnpm does not use (it declares `packages:` in
`pnpm-workspace.yaml`). `secrets push -w` refused outright; plain `secrets
push` failed *silently* — `_maybe_workspace_env_files` returned 0 the moment
the key was absent, so the auto-discovery that exists to cover push's
root-only scan was inert on every pnpm repo and printed "Nothing new to
add", indistinguishable from a repo that genuinely had nothing. Workspace
patterns now resolve through one shared source that falls back to
`pnpm-workspace.yaml`.
- **yarn's object `workspaces` form was never expanded (EGB-1232)** — the
filter `.workspaces // .workspaces.packages | .[]` short-circuits on yarn's
truthy object, so `.workspaces.packages` was never evaluated and `.[]`
iterated the object's values, yielding the pattern array itself as a single
token. Only npm's array form worked. Now type-aware, handling npm's array,
yarn's object, and absent/null alike.
### Added
- **`pnpm-workspace.yaml` support** — the `packages:` block sequence is read
without a YAML dependency: block form only, stopping at the next top-level
key so pnpm 10's `onlyBuiltDependencies:`/`catalog:` cannot leak in as glob
patterns, with quote and inline-comment handling and a symlink refusal.
`package.json` wins when it declares workspaces; `pnpm-workspace.yaml` is the
fallback. jq is now required only when `package.json` is the source, so a
pnpm-only repo resolves workspaces jq-free.
- **Workspace patterns are validated before glob expansion** — no absolute
paths, `..` traversal, shell metacharacters, or whitespace reach the
unquoted expansion; pnpm `!` negations are skipped. Same conservative rail
as `.secrets-store` / `.secrets-files`.
- **A monorepo-shaped root that resolves no workspaces now says so** — if a
`pnpm-workspace.yaml` or `packages/` directory is present but no workspace
packages can be read, `push` warns on stderr and points at `secrets add`,
instead of returning in silence. `secrets push -w`'s error now names
`pnpm-workspace.yaml` when that is the file present, rather than blaming a
`package.json` the repo may not use for workspaces.
## [0.7.6.0] - 2026-09-08
### Added
- **`secrets sync` — reconcile a diverged store (EGB-1231)** — the store is a
git repo, and once a clone was both ahead and behind its remote the CLI had
no way out: `push` demanded a fast-forward and pointed at `pull`, which
could not fast-forward either, so the advice looped and recovery meant
hand-running git next to a directory of encrypted blobs. `secrets sync`
fetches, stashes uncommitted blob edits, rebases local commits onto the
remote, restores the stash, and then asks before publishing local commits to
the shared store. `--yes` skips the prompt (scripts/CI); `--dry-run` reports
ahead/behind/dirty and what would happen, changing nothing. Deliberately
non-destructive: no merge, no force-push, no `reset --hard`, no `stash
drop`. A rebase conflict aborts, restores the stash, names the conflicting
files, and leaves the store exactly as found.
- **Store state in `secrets which` (EGB-1231)** — a new `remote:` line reports
the store's `ahead N, behind N, N modified` (or `up to date`) against its
upstream, with a `(run: secrets sync)` hint when there is anything to
reconcile. Offline-safe (reports against the last fetch, never reaches the
network) and silent for a local-only store or one with no upstream.
### Fixed
- **`secrets pull` no longer fails silently when the store can't sync
(EGB-1230)** — the store sync was `git pull >/dev/null 2>&1` under `set -euo
pipefail`, so a store that could not fast-forward killed the script at that
line with git's exit 128 and *nothing* on stdout or stderr. The user saw a
banner, no restored files, and no reason — indistinguishable from a project
with nothing to pull, and easy to lose entirely in a pipeline. The sync is
now guarded, git's output is captured and surfaced as the diagnosis, and the
error names the store path and points at `secrets sync`.
### Changed
- **`secrets pull`'s store sync is now fast-forward only**, matching the push
path. A plain `git pull` could quietly manufacture a merge commit in the
store; divergence is now resolved in exactly one place — `secrets sync`.
- **The push path's dead-end advice** ("Run 'secrets pull' first, then retry
push") now points at `secrets sync` and includes git's own output.
## [0.7.5.0] - 2026-06-24
### Added
- **Multi-recipient age encryption (EGB-283)** — a store-scoped, committed
`recipients.txt` (age `-R` format, with `# name` comment lines) lets one
store encrypt every blob to N age public keys — one per team member.
`secrets recipients add <age1…> [--name N]` adds a key and immediately
re-encrypts the whole store; `secrets recipients rm <key|name> [--yes]`
removes one and re-encrypts; `secrets recipients list` shows the current
set (or a note that the store is still single-key). A new `secrets
reencrypt` command re-encrypts every blob to the current recipients without
changing the set (idempotent heal / backfill after a manual edit). Absence
of `recipients.txt` preserves exact legacy single-key behavior; the first
`recipients add` on a legacy store bootstraps the file seeded with the
local pubkey plus the new key. `init` now seeds `recipients.txt` born-multi
with the freshly generated pubkey.
### Changed
- **`secrets rekey` on a multi-recipient store** no longer generates a new
keypair — instead it re-encrypts all blobs to the current `recipients.txt`
set (the shared `_reencrypt_all` engine). On a legacy store (no
`recipients.txt`) `rekey` keeps today's generate-new-keypair behavior.
- **`secrets which`** now prints a `recipients: N (name, …)` line, or
`recipients: single-key (no recipients.txt)` for a legacy store.
- **`secrets verify` / `verify --all`** assert that each blob's age
recipient-stanza count equals the number of entries in `recipients.txt`
(skipped on legacy stores). Exits non-zero on any count mismatch so it can
gate CI or a migration.
## [0.7.4.0] - 2026-06-18
### Added
- **`secrets upgrade` verb (EGB-716)** — the fix path paired with the EGB-713
version-skew *warning*. Until now the warning told you you were behind but not
how to catch up; `secrets upgrade` closes that loop.
- **`secrets upgrade`** — `git -C "$SCRIPT_DIR" pull --ff-only` on the tool's
own checkout (fast-forward only — never merges or rewrites local commits),
reports `vOLD -> vNEW`, then best-effort re-checks the store's recorded
writer-version against the new version so you see whether the EGB-713 nudge
is now cleared (the new code itself takes effect on your next command).
- **`secrets upgrade --check`** — reports whether an update is available
(`git fetch` + compare to upstream) and changes nothing.
- Deliberately thin: no auto-update, no background polling (this is a security
tool). Directed errors for not-a-git-checkout, no upstream, a diverged/dirty
branch, or being offline.
## [0.7.3.1] - 2026-06-18
### Changed
- **EGB-677 stage-1 structural cleanups (EGB-701)** — tech-debt dedup with one
new safety warning; no behavior change for the manifest-driven (v2) happy path.
- **`secrets which` now reuses the one external-entry extractor** the push/pull
path uses (`_json_external_entries`) instead of its own duplicated `jq`
projection. So `which` applies the same `properties``gradle-properties`
normalization and skips (with a warning) the same malformed external entries
the sync path drops — `which` shows exactly what will sync, not a stale raw
projection that could drift from the real behavior.
- **The two external-manifest read guards are factored into shared helpers**
`_json_readable` (plain regular file, silent) and `_legacy_readable` (warns
and skips a symlinked `.secrets-files`) — so `_external_entries_for_push` and
`_external_entries_for_pull` can't drift apart.
### Fixed
- **Legacy (manifest-less) `pull` no longer silently under-restores (EGB-701)**
the manifest-less pull path globs only top-level `*.age`/`.*.age`, so a nested
dotenv blob (`<project>/<relpath>.age`) written by a manifest-driven push on
another machine was invisible: restored nothing, counted nothing, said nothing.
It now **warns** and names each nested blob it can't reach (external blobs are
excluded — `pull_external_files` handles those), pointing at committing a
`.secrets.json` as the fix. The manifest-driven pull already restored nesting
correctly; this only closes the legacy path's blind spot.
## [0.7.3.0] - 2026-06-08
### Added
- **Real install / onboarding scripts (EGB-671)** — onboarding a machine is now
(close to) one command, and a mis-copied key fails loudly instead of silently.
- **`secrets join --remote <url> --key <path>`** — second-machine onboarding in
one verb: clones the vault, installs the key at mode 600, and **verifies the
key actually decrypts the store before declaring success**. An empty vault
reports "nothing to verify yet" (it never prints a false `VERIFIED`); a wrong
key fails loudly with the store left in place to fix. All security logic
(store resolution, URL handling, path rails) is reused from the audited core,
not re-implemented in a side script.
- **`secrets init --remote <url>`** — wires the remote and pushes the initial
store so the upstream branch exists, so your first project `push` doesn't trip
the fast-forward-pull guard on a brand-new empty remote. Run interactively,
`init` also offers to add your first project's secrets (default No, skipped
under `--yes` / non-interactive, so it stays a clean primitive for CI).
- **`install.sh`** — thin bootstrap that ships in the repo: checks `age` + `jq`
+ `git`, then prints the `PATH` line, the onboarding next-steps, the upgrade
one-liner, and a key-transfer hint. It never edits your shell config and never
runs `sudo` (it prints the command so you stay in control).
- **First-manifest `options.autoAdd` prompt (EGB-677 contract #2)** — the first
`push` that scaffolds a project's manifest now records an explicit, committed
`options.autoAdd` value (asked once when interactive; the default ON, written
explicitly, under automation).
### Fixed
- **Day-2 silent decrypt failure**`secrets pull` now dies loudly when a blob
fails to decrypt with the current key (all three decrypt paths), instead of
emitting a warning and continuing with exit 0. A wrong key can no longer pass
unnoticed after onboarding.
- The `secrets init` second-machine trap now points at `secrets join` (the real
one-command path) instead of a manual `git clone`.
## [0.7.2.0] - 2026-06-08
### Added
- **`secrets list --json` (EGB-699)** — machine-readable listing for tooling and
CI. Emits a single JSON object on stdout: `{"store", "projects": [{"name",
"entries": [...]}]}`, where each entry self-describes via a `type`
discriminator — `{"type":"dotenv","path":<relpath>}` or
`{"type":"external","subtype":"properties"|"file","path":<slug>}`. Reflects the
same recursive store walk as the human `list` (nested `<project>/<relpath>.age`
+ `external/<slug>.age`). jq does the assembly so paths escape correctly; the
human store hint is suppressed so stdout stays pure JSON (notices → stderr).
jq is required only for `--json`. Feeds the EGB-671 install scripts, which need
to enumerate a cloned store programmatically instead of scraping the table.
## [0.7.1.0] - 2026-06-08
### Added
- **Version-skew nudge (EGB-713)** — the store now records the highest `secrets`
version that has written to it (`.secrets-writer-version`, committed,
monotonic). When you run a command against a store last written by a *newer*
`secrets` than your own, you get a one-line non-fatal stderr nudge to update
your tool; `secrets which` shows the store's `written-by:` version (and flags
when you're behind). Stores written by older builds carry no stamp and stay
silent — no false alarms. The loud counterpart to EGB-712's quiet
forcing function.
## [0.7.0.0] - 2026-06-08
### Changed
- **Additive store-format v2 (EGB-712)** — upgraded `secrets` clients now read
either external blob suffix (`.properties.age` or the legacy
`.gradle-properties.age`) and **dual-write** a `properties` external whenever a
v1 twin already exists in the store. Existing externals keep working for
teammates on an older `secrets`; only a brand-new `properties` external is
written v2-only (a gentle "upgrade to see it" forcing function). dotenv and
whole-`file` externals are unchanged across formats and always propagate.
- **`secrets migrate --finalize` is now optional GC**, not a required milestone.
Because clients dual-write and read-fall-back, no teammate is ever cut off by
*not* finalizing; finalize only reclaims the duplicate v1 blobs, and stays
deferrable indefinitely. Its safety gates are unchanged. This defuses the
cross-machine "all clients must be v2 before finalize" coordination gate.
### Added
- **`secrets migrate --status`** now reports `v2-only` externals per project
(the ones an un-upgraded client cannot read), so you can see the forcing
function's footprint at a glance.
## [0.6.1.0] - 2026-06-08
### Changed
- **`secrets migrate` copy-forward is now manifest-free (EGB-710)** — the
per-project step enumerates the store's `*.gradle-properties.age` blobs
directly (the same source of truth `--finalize` uses) instead of reading
`.secrets.json`. A legacy `.secrets-files`-only project now migrates cleanly
instead of dead-ending with "No .secrets.json", and a store blob the manifest
no longer declares still gets a v2 twin (so `--finalize` won't refuse it).
Running migrate in a project with no v1 properties blobs is a clean no-op.
### Added
- **`secrets migrate --status`** — a read-only survey that walks every project
in the store and reports its v2 readiness (v2-ready / migrated / NEEDS
MIGRATE), then whether the store as a whole is finalize-ready. Exits non-zero
while any v1 blob is un-twinned, so it can gate the path to `--finalize`.
## [0.6.0.1] - 2026-06-08
### Added
- **`secrets which` now prints the manifest version (EGB-700)** — the manifest
header line shows `version N` alongside the store format, so a single
`secrets which` tells you both the on-disk store format and the `.secrets.json`
schema version at a glance.
## [0.6.0.0] - 2026-06-07
### Added
- **Self-describing store format (v2) + `secrets migrate` (EGB-703)** — the
store now records its format in a committed `.secrets-format` file, and
`secrets which` prints it (`format: v2`). A fresh `secrets init` creates a
v2 store; existing stores read as v1 until migrated.
- **`secrets migrate`** — copy-forward a project's encrypted blobs to the v2
layout. It is non-destructive: the old blobs are kept until you finalize, so
a half-migrated store stays fully readable and recoverable. `secrets migrate
--dry-run` previews exactly what would change without writing anything.
- **`secrets migrate --finalize`** — the one destructive step, run once
store-wide. It refuses unless `secrets verify` passes and every blob has its
new-format twin, cuts a `pre-v2-migrate-*` recovery tag first, then drops the
old blobs. It asks for confirmation (or `--yes`) because a machine still on
an older `secrets` will stop seeing migrated external files until it updates.
### Changed
- The external `properties` blob is stored as `<name>.properties.age` in a v2
store (was `<name>.gradle-properties.age`), matching the manifest `type`.
`push`, `pull`, and `verify` pick the right name automatically from the store
format, so v1 and v2 stores both keep working during a migration.
## [0.5.0.0] - 2026-06-07
### Added
- **`secrets verify` (EGB-698)** — a read-only integrity check. Run it in a
project to cross-check the committed `.secrets.json` against the store both
ways (entries declared but missing from the store, and stored blobs with no
manifest entry) and decrypt-test every blob with your current key. Catches a
partially-synced store, a stale key, or a manifest that has drifted from the
store. Plaintext is streamed to `/dev/null` and never written to disk.
- **`secrets verify --all`** — decrypt-tests every blob in every project in the
store: a fast store-wide integrity sweep. (The store carries no manifests, so
`--all` checks decryptability only, not manifest consistency.)
- Both modes recurse the whole project tree, so nested entries and external
files are covered. `secrets verify` exits non-zero on any problem, so it can
gate CI or a future store migration.
## [0.4.0.0] - 2026-06-07
### Added
- **`.secrets.json` manifest (EGB-677 stage 1)** — a committed, project-root
manifest is now the source of truth for what syncs. List the env files you
want under `dotenv[]` (project-relative, nested paths and `@`-scoped
workspaces allowed; `..`, absolute, and symlink paths are rejected) and
out-of-project files under `external[]` (`properties` or `file`). The
manifest is shared across machines, so a teammate who clones the project
sees exactly what to pull.
- **`secrets add <path>`** — declare an env file in the manifest without
pushing. Bootstraps `.secrets.json` on first use, dedupes, and writes a
stable canonical form.
- **Auto-add on push**`secrets push` discovers new `.env*` / `.dev.vars`
files and adds them to the manifest (prints what it added and reminds you to
commit). Gated by `options.autoAdd` in the manifest (default on);
`push --frozen` syncs only declared files, and `push --dry-run` previews
what would change without writing anything.
- **Manifest-driven pull** — restores every declared file, recreating nested
directories as needed, with the same path-safety rail applied at restore
time so a malicious committed manifest can't write outside the project. An
empty manifest is a safe no-op.
- **Legacy `.secrets-files` absorb** — an existing `.secrets-files` is folded
into `.secrets.json` on first push (gradle-properties entries become
`properties`); on pull the legacy file is superseded with a warning.
- **Platform-aware install hints** — missing-dependency errors now print the
right install command for your platform (brew / apt-get / dnf).
### Changed
- `jq` is required only when a manifest is present or being written;
manifest-less projects keep working without `jq` (manifest features are
skipped with a notice).
### Fixed
- **Key rotation no longer orphans nested or external blobs.** `secrets rekey`
and `secrets list` now walk the entire project tree, so nested manifest
entries (`<project>/<relpath>.age`) and `external/` blobs are re-encrypted
and listed correctly. Previously a rekey could leave nested blobs encrypted
under the discarded old key, making them permanently undecryptable.
- Test assertions now fail correctly under system bash 3.2 (standalone
`[[ ]]` checks no longer pass silently).
## [0.3.0.0] - 2026-06-07
### Added
- **`file` external type (EGB-652)** — `.secrets-files` can now sync whole
files outside the project root (binary-safe; built for the Beacon Android
upload keystore): `file ~/keystores/beacon-upload.keystore`. Push encrypts
the file verbatim into `<project>/external/`; pull restores it with mode
600, backing up a divergent existing target to `<name>.secrets-bak`. Same
path safety rails as `gradle-properties` (inside `$HOME`, no `..`, no
symlinks) minus the basename restriction. 7 new bats tests.
## [0.2.1.0] - 2026-06-05
### Fixed
- **`secrets rekey` no longer bricks dotenv stores.** The re-encrypt loop used a bare `"$dir"*` glob, which never matches dotfiles — so `.env` blobs were decrypted to the temp dir but never re-encrypted, leaving them on the **old** key while the new key overwrote `key.txt`. After a rotation, every dotenv file in the store was undecryptable. The glob now mirrors the decrypt loop (`"$dir"* "$dir".*`), and a round-trip test (push → rekey → pull) pins it. If you ran `rekey` on an earlier version and `pull` now fails with `no identity matched any of the recipients`, your blobs are on a pre-rotation key — recover with an old `key.txt` from another machine.
- **`secrets init` on a second machine now fails helpfully instead of half-initializing.** Copying `key.txt` into `~/.secrets` and then running `init` (instead of cloning your secrets repo) used to run `git init`, crash on the existing key, and leave a store with no `.gitignore` — a state where a later `push` would commit the private key. The guard now fires *before* `git init`, leaves the key untouched, and prints the exact `git clone` command to run — using the real remote URL when your `.secrets-store` file declares one.
### Security
- **The private key can no longer be committed by a store missing its `.gitignore`.** `push`, `pull`, and `rekey` now self-heal store protections immediately before any `git add -A`: a missing *or corrupted* `.gitignore` (one without the `key.txt` line) is rewritten, and the pre-commit hook is reinstalled if absent. The heal runs *after* the fast-forward pull, closing a window where remote history without a `.gitignore` could strip protection mid-push.
- **An already-tracked `key.txt` is now untracked automatically.** `.gitignore` can't untrack a file that was committed in the past; the heal now removes a tracked key from the index with a warning that history may need scrubbing and the key may warrant rotation.
### Changed
- Project `CLAUDE.md` gained agent skill-routing guidance and an updated test-suite count (126 bats tests, up from 113).
## [0.2.0.0] - 2026-05-26 ## [0.2.0.0] - 2026-05-26
### Added ### Added
@ -86,6 +475,6 @@ and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme.
- 37 → 66 tests. New coverage: store resolution rules and precedence, walk-up boundaries, command-injection prevention, key-file re-derivation across stores, teammate-onboarding error path, monorepo workspace binding, F1F5 adversarial regressions. - 37 → 66 tests. New coverage: store resolution rules and precedence, walk-up boundaries, command-injection prevention, key-file re-derivation across stores, teammate-onboarding error path, monorepo workspace binding, F1F5 adversarial regressions.
[0.1.1.0]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.1.0 [0.1.1.0]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.1.0
[0.1.0.1]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.0.1 [0.1.0.1]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.0.1
[0.1.0.0]: https://codeberg.org/egbt/secrets/releases/tag/v0.1.0.0 [0.1.0.0]: https://git.dev.egbt.com/egbt/secrets/releases/tag/v0.1.0.0

131
CLAUDE.md
View file

@ -16,28 +16,91 @@ cd ~/my-project && ./secrets pull # Pull + decrypt .env* files
```bash ```bash
brew install bats-core brew install bats-core
bats test/secrets.bats bats test/ # runs secrets.bats + manifest.bats + migrate.bats + upgrade.bats + recipients.bats
./test/run-security.sh # security regression subset + operator sign-off (see below)
``` ```
**bash 3.2 assertion gotcha:** bats runs under system bash 3.2, where a
failing `[[ ]]` mid-test does NOT fail the test (the ERR trap skips `[[`
compound commands). Every standalone `[[ ... ]]` assertion MUST end with
`|| false`. Single-bracket `[ ]` assertions are unaffected.
## Security review policy (mandatory for agents)
This repo contains **defensive** security regression tests with deliberate
attack-payload fixtures (shell injection strings, symlink escape cases, path
rails). Hosted models (including Opus) block AI-driven "think like an attacker"
/ red-team / security-specialist review of that material under Anthropic Usage
Policy (real-time cyber safeguards).
**Read `.ship-policy.json` before any `/ship`, `/review`, or `/autoplan` work.**
When that file is present (it always is in this repo):
1. **Never** dispatch red-team subagents, security specialist subagents, or
adversarial review prompts ("think like an attacker", "find security holes",
"chaos engineer", exploit hunting) against this codebase.
2. **Never** run `./test/run-security.sh` or filter bats to SECURITY/adversarial
tests on behalf of the user — that is operator-local work only.
3. **Before ship/PR:** ask the human operator to run `./test/run-security.sh`
locally and complete the SIGNOFF prompt. Record lives at
`.gstack/security-signoff.json` (gitignored). Proceed with ship only after
they confirm sign-off for the current `HEAD` (or note in the PR that sign-off
is pending).
4. Normal `./ship` steps may continue (tests via `bats test/`, eng review,
changelog, PR) — only the AI security/red-team/adversarial passes are opted out.
Gstack `/ship` reads the same policy via `gstack-ship-policy` (sources
`.ship-policy.json` at repo root): Step 9 blocks when manual sign-off is missing,
skips security specialist + red team, and Step 11 skips adversarial review.
## Architecture ## Architecture
Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey. Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, sync, upgrade.
- Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable) - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable)
- Storage: Private git repo at `~/.secrets/` - Storage: Private git repo at `~/.secrets/`
- Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`)
- External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`) — merged, not overwritten (EGB-531, see below) - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `<project>/<relpath>.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints.
- Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq` - Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age``.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at <path>, version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-<sha>` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves.
- Store sync + divergence (EGB-1230/EGB-1231): the store is a git repo, so a clone can end up ahead of and behind its remote at once. **EGB-1230:** `cmd_pull`'s sync used to be `git pull >/dev/null 2>&1` under `set -euo pipefail` — a store that couldn't fast-forward killed the script there with git's exit 128 and nothing on either stream (a banner, no files, no reason; invisible in a pipeline). It now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. That sync is **`--ff-only`**, matching the push path — a plain `git pull` could quietly manufacture a merge commit in the store, and divergence is now resolved in exactly one place. **EGB-1231:** `_store_git_state` emits `ahead\tbehind\tdirty` (from `rev-list --left-right --count @{u}...HEAD` plus `status --porcelain`) and `_format_store_state` renders it; `cmd_which` prints a `remote:` line from them — offline-safe (reports against the last fetch), silent with no remote/upstream. `cmd_sync` is the reconcile verb the CLI was missing: fetch → report state → stash (`push -u`) → `rebase @{u}` → restore stash → `ensure_store_protections` (rebased-in history may lack `.gitignore`, and a store missing the `key.txt` line would stage the private key — same reasoning as push) → **confirmation-gated** `git push` of local commits. The gate (`_sync_confirm_push`) reads `/dev/tty` and requires a tty, so it stays CLOSED in scripts/CI rather than publishing to a shared store by default; `--yes` opens it, `--dry-run` reports and returns before any mutation. Non-destructive by construction: no merge, no `--force`, no `reset --hard`, no `stash drop`. A rebase conflict collects the conflicting paths BEFORE `rebase --abort` (the abort clears them), restores the stash, and dies — store byte-identical to how it was found. `_sync_restore_stash` never drops the stash on a failed pop; it tells the operator where their only copy lives. `cmd_sync` does not `_stamp_writer_version`: it replays existing commits rather than authoring content, and the stamp is specified to ride a store-committing `git add -A`. Test suite: `test/sync.bats` (25 tests), including a grep over the `cmd_sync` body asserting the destructive git verbs never appear in it.
- Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR/<project>/` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`.
- External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below
- Workspaces (EGB-1232): `--workspaces` and the plain-push workspace re-scan both resolve patterns through ONE source — `_workspace_patterns()`. It reads `package.json` `.workspaces` via `$WORKSPACES_JQ` (type-aware: handles npm's array AND yarn's object `{packages:[...]}` form) and falls back to `pnpm-workspace.yaml`'s `packages:` block when package.json declares none. **Two defects fixed:** (1) both call sites were package.json-only, so no pnpm monorepo ever resolved a workspace — and `_maybe_workspace_env_files` failed *silently* (`jq -e '.workspaces' ... || return 0`), making auto-discovery inert and `push` print "Nothing new to add", indistinguishable from a repo with nothing new; it bit the same repo twice. (2) the old filter `.workspaces // .workspaces.packages | .[]` short-circuits on yarn's truthy object, iterating the object's values and yielding the pattern ARRAY as a single token. Note a naive reorder does NOT fix it — `.workspaces.packages` errors on an array; hence the `if type == "object"` form. `_pnpm_workspace_packages()` is a deliberate non-parser (block sequence only, stops at the next top-level key so pnpm 10's `onlyBuiltDependencies:`/`catalog:` can't leak in as globs, strips quotes/inline comments, refuses a symlinked file). Patterns are validated by `_valid_workspace_pattern` before they reach the unquoted `for pattern in $patterns` glob expansion (no absolute/`..`/metacharacters/whitespace; pnpm `!` negations skipped) — same posture as `.secrets-store`/`.secrets-files`. `_looks_like_monorepo` + `_workspace_source` turn the old silent return into a warning that names the real file, and `get_workspaces`'s error names `pnpm-workspace.yaml` when that's what's present instead of blaming package.json. jq is required only when package.json is the source. **Scope note:** the workspace re-scan still runs only for projects that already have a `.secrets.json` — push's root-scan-only behavior on a first push is by design (EGB-677 E13), and EGB-1232 is about the fallback that covers it never engaging. Tests: `test/workspaces.bats` (18).
- Safety: Pre-commit hook rejects plaintext secret files (`.env`, `.dev.vars`, `gradle.properties`) - Safety: Pre-commit hook rejects plaintext secret files (`.env`, `.dev.vars`, `gradle.properties`)
- Multi-recipient (EGB-283): a store-scoped, committed `recipients.txt` (age `-R`
format, `# name` comments) lets one store encrypt every blob to N age keys —
one per team member. Managed via `secrets recipients add/rm/list`; absence of
the file ⇒ legacy single-key behavior (recipients = the pubkey derived from
`key.txt`). The file is parsed by us (never `age -R <path>`) into a validated
`RECIPIENT_ARGS` array (native age X25519 only, `age1[0-9a-z]{58}`; SSH
recipients rejected; symlinked file refused) — same conservative posture as
`.secrets-store`/`.secrets-files`. `_load_recipients` populates the array;
every encrypt site routes through it. Any recipient change re-encrypts the
WHOLE store in one commit via the shared `_reencrypt_all` engine (also used by
the new `secrets reencrypt` and by `rekey` on a multi-recipient store, where
rekey re-encrypts to the set with NO new keypair; legacy stores keep rekey's
generate-new-keypair behavior). `init` seeds `recipients.txt` born-multi.
`which` prints `recipients: N`; `verify`/`verify --all` assert each blob's
age recipient-stanza count equals `recipients.txt`'s length. Removal takes
effect going forward — git history stays readable by an old key, so rotate
genuinely-sensitive values. Decryption is unchanged (each member uses their
own `key.txt`).
- Portability: must run on system bash 3.2 (macOS) — no associative arrays or bash-4 features - Portability: must run on system bash 3.2 (macOS) — no associative arrays or bash-4 features
## Project Structure ## Project Structure
``` ```
secrets # CLI script (~600 lines bash) secrets # CLI script (~2000 lines bash)
hooks/pre-commit # Pre-commit hook template hooks/pre-commit # Pre-commit hook template
test/ test/
secrets.bats # bats-core test suite (113 tests) secrets.bats # bats-core test suite (140 tests)
manifest.bats # EGB-677 .secrets.json manifest tests (83 tests)
migrate.bats # EGB-703 store-format-v2 migration tests (35 tests)
upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests)
sync.bats # EGB-1230/1231 store sync + divergence reconcile tests (25 tests)
workspaces.bats # EGB-1232 npm/yarn/pnpm workspace discovery tests (18 tests)
recipients.bats # EGB-283 multi-recipient age encryption tests (34 tests)
test_helper.bash # Shared setup/teardown test_helper.bash # Shared setup/teardown
README.md # User-facing documentation README.md # User-facing documentation
CLAUDE.md # This file CLAUDE.md # This file
@ -62,12 +125,12 @@ The active store directory is picked by `resolve_store()` using these rules, hig
## External files (.secrets-files) — EGB-531 ## External files (.secrets-files) — EGB-531
`.secrets-files` is a committed, project-root manifest declaring keys to sync from files **outside** the project (motivating case: `~/.gradle/gradle.properties`, which Android Studio GUI builds read but terminal env vars can't reach). One entry per line: `<type> <path> <key>...`. Only type `gradle-properties` is supported; the type token leaves room for future types **without** a plugin-dispatch framework (build the concrete case — a deliberate scope cut). `.secrets-files` is a committed, project-root manifest declaring keys to sync from files **outside** the project (motivating case: `~/.gradle/gradle.properties`, which Android Studio GUI builds read but terminal env vars can't reach). One entry per line: `<type> <path> <key>...`. Two types: `gradle-properties` (named-key merge) and `file` (EGB-652 — whole-file verbatim sync, binary-safe, built for the Beacon Android upload keystore; no keys, restored at mode 600 with a `.secrets-bak` backup of a divergent existing target, basename restriction waived but all other path rails apply). Still no plugin-dispatch framework — each type is a concrete `case` branch (deliberate scope cut).
Key design decisions (all driven by /autoplan review): Key design decisions (all driven by /autoplan review):
- **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel). - **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel).
- **Storage:** blobs live in `$SECRETS_DIR/<project>/external/<slug>.gradle-properties.age`. The `external/` subdir keeps them out of the existing non-recursive `*.age` / `.*.age` globs (pull, list, rekey), so the old dotenv path can never decrypt a blob into cwd. `cmd_rekey` and `cmd_list` recurse into `external/` explicitly (rekey MUST, or the blob is orphaned after rotation = data loss). `<slug>` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). - **Storage:** blobs live in `$SECRETS_DIR/<project>/external/<slug>.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/<slug>.age` and nested manifest dotenv blobs (`<project>/<relpath>.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). **EGB-701 cleanups:** (1) the *legacy* (manifest-less) `pull` keeps its non-recursive globs but now **warns** when nested `<project>/<relpath>.age` blobs exist that those globs can't see (it excludes `external/`, which `pull_external_files` handles) — so a manifest-less pull never silently under-restores; the fix the warning points at is committing a `.secrets.json`. (2) `cmd_which`, push, and pull share one external extractor (`_json_external_entries`), so `which` applies the same `properties``gradle-properties` normalization and skip-with-warning rules the sync path does (it shows exactly what will sync, not a stale raw projection). (3) the two external-manifest read guards are factored into `_json_readable` (plain regular file, silent) / `_legacy_readable` (warn-and-skip on a symlinked legacy manifest). `<slug>` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv``path` or `external``subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode).
- **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `<target>.secrets-bak` before each merge. - **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `<target>.secrets-bak` before each merge.
- **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins. - **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins.
- **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped). - **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped).
@ -75,13 +138,63 @@ Key design decisions (all driven by /autoplan review):
## Deploy Configuration ## Deploy Configuration
- Platform: NONE (distributed via `git clone` from GitHub) - Platform: NONE (distributed via `git clone` from the private Forgejo at `git.dev.egbt.com`)
- Production URL: N/A (no live service) - Production URL: N/A (no live service)
- Release model: merge to `main` is the release. Optionally tagged with `v<X.Y.Z.W>`. - Release model: merge to `main` is the release. Optionally tagged with `v<X.Y.Z.W>`.
- Verification after merge: a fresh `git clone` should produce a working `secrets which` against an isolated `$HOME`. No canary URL. - Verification after merge: a fresh `git clone` should produce a working `secrets which` against an isolated `$HOME`. No canary URL.
- Staging: none. - Staging: none.
- Rollback: revert the merge commit on `main` (and delete the tag) to roll back. - Rollback: revert the merge commit on `main` (and delete the tag) to roll back.
## Forge operations (self-hosted Forgejo)
The remote is a private Forgejo instance at `https://git.dev.egbt.com`
(migrated off Codeberg 2026-09-08). `gh`/`glab` do NOT work here. Use `tea`
(login name: `egbt`, user `brian`) for forge operations when a skill's
platform detection comes up "unknown":
**Always pass `--login egbt --repo egbt/secrets` explicitly.** `tea`'s repo
autodetection fails here ("remote repository required"), and this machine also
has a leftover `codeberg` login pointing at the *old* forge
(`https://codeberg.org`) that `tea` will silently fall back to in
non-interactive mode — which would target the wrong server. Confirm with
`tea logins list` if a command errors.
- PRs: `tea pr create --login egbt --repo egbt/secrets --base main --head <branch> --title ... --description ...` / `tea pr merge <index> --login egbt --repo egbt/secrets`
- Releases: `tea releases create --login egbt --repo egbt/secrets --tag v<X.Y.Z.W> --title "v<X.Y.Z.W> — <YYYY-MM-DD>" --note ...`
(convention: one release per tag, title `v<X.Y.Z.W> — <date>`)
- Issues/status: `tea issues --login egbt --repo egbt/secrets`, `tea pr list --login egbt --repo egbt/secrets`
- **SSH is on port 2222**, not 22 (port 22 is the host's own sshd). Clone/remote
URLs must be `ssh://git@git.dev.egbt.com:2222/egbt/secrets.git`. A bare
`git@git.dev.egbt.com:egbt/secrets.git` will fail with "Permission denied
(publickey)" because it hits the wrong daemon.
- The host resolves to a Tailscale address — the forge is reachable only on the
VPN. Off-net, push/pull/`tea` all fail to connect; that is expected, not a
credentials problem.
- `FORGEJO_URL` and `FORGEJO_TOKEN` (API token for user `brian`) live in
`~/.zshenv` for direct API calls.
- CI: the instance has an Actions runner available, but no workflow is
configured for this repo yet. The bats suite run locally is still the merge
gate.
## Environment variable ## Environment variable
`SECRETS_DIR` overrides the default `~/.secrets` location (useful for testing). Per-project bindings via `.secrets-store` file beat this env var; use `--store <dir>` for one-shot overrides that beat everything. `SECRETS_DIR` overrides the default `~/.secrets` location (useful for testing). Per-project bindings via `.secrets-store` file beat this env var; use `--store <dir>` for one-shot overrides that beat everything.
## Skill routing
When the user's request matches an available skill, invoke it via the Skill tool. When in doubt, invoke the skill.
Key routing rules:
- Product ideas/brainstorming → invoke /office-hours
- Strategy/scope → invoke /plan-ceo-review
- Architecture → invoke /plan-eng-review
- Design system/plan review → invoke /design-consultation or /plan-design-review
- Full review pipeline → invoke /autoplan
- Bugs/errors → invoke /investigate
- QA/testing site behavior → invoke /qa or /qa-only
- Code review/diff check → invoke /review
- Visual polish → invoke /design-review
- Ship/deploy/PR → invoke /ship or /land-and-deploy (after reading `.ship-policy.json`; no AI adversarial/red-team/security-specialist review in this repo)
- Save progress → invoke /context-save
- Resume context → invoke /context-restore
- Author a backlog-ready spec/issue → invoke /spec

364
README.md
View file

@ -54,87 +54,101 @@ flowchart TD
Files like `.envrc` (direnv) and `.environment-*` are intentionally **not** tracked. Files like `.envrc` (direnv) and `.environment-*` are intentionally **not** tracked.
Beyond project files, `secrets` can also sync designated keys from files that live *outside* the project — like `~/.gradle/gradle.properties` — merging them in without clobbering unrelated keys. See [External files (Gradle properties)](#external-files-gradle-properties). Beyond project files, `secrets` can also sync files that live *outside* the project — designated keys from `~/.gradle/gradle.properties` (merged without clobbering unrelated keys), or whole files like an Android upload keystore. See [External files (Gradle properties)](#external-files-gradle-properties).
## Prerequisites ## Prerequisites
- **macOS** (uses Homebrew for installation) - **macOS or Linux**
- **git** (already installed on most Macs — type `git --version` to check) - **git** (`git --version` to check)
- **age** (the encryption tool — installed in step 1 below) - **age** and **jq**`install.sh` checks for these and prints the exact install command for your platform (Homebrew on macOS, `apt`/`dnf` on Linux)
## Setup ## Setup
### First machine (one-time setup) Clone the tool repo, then run `install.sh`. It checks dependencies and prints the
two commands to finish setup. It never edits your shell config and never runs
sudo — it prints the commands so you stay in control.
```bash ```bash
# 1. Install the encryption tool git clone https://git.dev.egbt.com/egbt/secrets.git ~/dev/secrets
brew install age cd ~/dev/secrets
./install.sh
# 2. Download the secrets tool (this repo — contains only the CLI, no secret files)
git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets
# 3. Make the 'secrets' command available everywhere
# Add this line to your shell config file (~/.zshrc on Mac):
export PATH="$HOME/dev/secrets:$PATH"
# Then restart your terminal, or run:
source ~/.zshrc
# 4. Initialize your encrypted secrets store
# This creates a folder at ~/.secrets/ with your encryption key
secrets init
# 5. Create a PRIVATE repository on GitHub to store your encrypted secrets
# Go to github.com/new, name it something like 'my-secrets', and make sure
# "Private" is selected. Then connect it:
cd ~/.secrets
git remote add origin git@github.com:<you>/my-secrets.git
git push -u origin main
``` ```
> **Important:** Step 5 creates a *separate* private repo for your encrypted secrets. This is different from the `secrets` tool repo you cloned in step 2. The tool repo can be public — it contains no secrets. The `~/.secrets/` repo must be private. `install.sh` prints a `export PATH="$HOME/dev/secrets:$PATH"` line — add it to your
shell config (`~/.zshrc` or `~/.bashrc`) and restart your terminal. Then onboard
this machine with one of the two flows below.
### Additional machines ### First machine (new vault)
On each new machine (your desktop, a teammate's laptop, etc.):
```bash ```bash
# 1. Install prerequisites and the tool (same as steps 1-3 above) # 1. Create a PRIVATE repo for your encrypted secrets (github.com/new or a
brew install age # GitLab/Forgejo private repo). It holds only ciphertext — never your key.
git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets # Then wire it up and push the store in one command:
export PATH="$HOME/dev/secrets:$PATH" # add to ~/.zshrc secrets init --remote git@github.com:<you>/my-secrets.git
# 2. Clone the encrypted secrets repo # 2. (optional) Add a project's secrets. From a project directory:
git clone git@github.com:<you>/my-secrets.git ~/.secrets
# 3. Copy the encryption key from your first machine
# This is the only step that requires direct machine-to-machine transfer.
# Choose one method:
#
# Option A: AirDrop (Mac to Mac)
# On your first machine, right-click ~/.secrets/key.txt → Share → AirDrop
# Save it to ~/.secrets/key.txt on the new machine
#
# Option B: Secure copy over SSH
# scp first-machine:~/.secrets/key.txt ~/.secrets/key.txt
#
# Option C: USB drive
# Copy key.txt to a USB drive, transfer it, delete from USB after
# 4. Pull your secrets into any project
cd ~/myapp cd ~/myapp
secrets pull secrets push
# The first push asks once whether to auto-track new env files and records
# your choice in the project's .secrets.json.
``` ```
`secrets init --remote` generates your key (`~/.secrets/key.txt`), wires the
remote, and pushes the initial store so the upstream branch exists. The private
secrets repo is separate from this tool repo — the tool repo is public and holds
no secrets; the `~/.secrets/` repo must be private.
> Running `secrets init` interactively (in a terminal) also offers to add your
> first project's secrets right away. Run it with `--yes` (or in any non-tty
> context like CI) to skip that prompt and just create the vault.
### Other machines (join an existing vault)
On a second machine, a desktop, or a teammate's laptop:
```bash
# 1. Clone the tool and run the bootstrap (as in Setup above)
git clone https://git.dev.egbt.com/egbt/secrets.git ~/dev/secrets
cd ~/dev/secrets && ./install.sh # add the printed PATH line to your shell config
# 2. Get key.txt onto this machine (the one manual, out-of-band step):
# AirDrop (Mac→Mac), or
# scp first-machine:~/.secrets/key.txt ~/Downloads/key.txt, or
# a USB drive (delete from the drive afterward)
# 3. Join the vault in one command:
secrets join --remote git@github.com:<you>/my-secrets.git --key ~/Downloads/key.txt
```
`secrets join` clones the vault, installs the key at mode 600, and **verifies the
key actually decrypts the store before declaring success** — a mis-copied key
fails loudly here, not silently on a later `secrets pull`. On success it tells you
to run `secrets pull` in any project.
> **The key file (`~/.secrets/key.txt`) is the only thing that needs to be transferred manually.** It never leaves your machines — it's excluded from git, never uploaded, never transmitted over the internet. Anyone with this file can decrypt all your secrets, so treat it like a password. > **The key file (`~/.secrets/key.txt`) is the only thing that needs to be transferred manually.** It never leaves your machines — it's excluded from git, never uploaded, never transmitted over the internet. Anyone with this file can decrypt all your secrets, so treat it like a password.
### Sharing with teammates ### Sharing with teammates
To share secrets with a teammate, they need: **Simple approach (shared key):** To share secrets with a teammate, they need:
1. Access to your private `my-secrets` GitHub repo (add them as a collaborator) 1. Access to your private secrets repo (add them as a collaborator)
2. A copy of `key.txt` (send it to them directly — AirDrop, USB, or in-person) 2. A copy of `key.txt` (send it directly — AirDrop, USB, or in-person)
Everyone on the team uses the same key. When anyone runs `secrets push`, the encrypted files are updated and everyone else can `secrets pull` to get the latest version. Everyone on the team uses the same key. A teammate joins with
`secrets join --remote <repo-url> --key <path-to-key.txt>`. When anyone runs
`secrets push`, the encrypted files update and everyone else runs `secrets pull`
to get the latest.
### Updating the tool
```bash
git -C ~/dev/secrets pull
```
If your store was last written by a newer client than yours, `secrets` prints a
one-line version-skew nudge — that's your cue to run the command above.
**Per-teammate keys (recommended for teams):** Use `secrets recipients add` so each person keeps their own private key — no key sharing needed. See [Onboarding and offboarding teammates](#onboarding-and-offboarding-teammates) below.
## Usage ## Usage
@ -160,12 +174,71 @@ secrets clear
|---------|-------------| |---------|-------------|
| `secrets init` | Create the `~/.secrets/` repo and generate an encryption key | | `secrets init` | Create the `~/.secrets/` repo and generate an encryption key |
| `secrets push` | Encrypt secret files in the current directory and upload them | | `secrets push` | Encrypt secret files in the current directory and upload them |
| `secrets push --frozen` | Sync only what `.secrets.json` declares (skip auto-add) |
| `secrets push --dry-run` | Show what would be added/synced without changing anything |
| `secrets pull` | Download and decrypt secret files into the current directory | | `secrets pull` | Download and decrypt secret files into the current directory |
| `secrets add <path>` | Declare a project-relative file in `.secrets.json` |
| `secrets clear` | Delete plaintext secret files from the current directory | | `secrets clear` | Delete plaintext secret files from the current directory |
| `secrets run <command>` | Pull secrets, run a command, then clear secrets when it exits | | `secrets run <command>` | Pull secrets, run a command, then clear secrets when it exits |
| `secrets list` | Show all projects that have stored secrets | | `secrets list` | Show all projects that have stored secrets |
| `secrets list --json` | Same listing as a machine-readable JSON object (`{store, projects[].entries[]}`, each entry `dotenv`/`external`) for tooling and CI. JSON goes to stdout; notices to stderr |
| `secrets rm <project>` | Delete a project's secrets from the store | | `secrets rm <project>` | Delete a project's secrets from the store |
| `secrets rekey` | Generate a new encryption key and re-encrypt everything | | `secrets rekey` | Generate a new encryption key and re-encrypt everything (single-key store) or re-encrypt to the current recipients without changing keys (multi-recipient store) |
| `secrets verify [project]` | Check the current project's `.secrets.json` against the store (missing/orphaned blobs) and decrypt every blob. `[project]` overrides the store directory name; the manifest is still read from the current directory |
| `secrets verify --all` | Decrypt-test every blob in every project — a store-wide integrity sweep |
| `secrets migrate [--dry-run]` | Copy-forward this project's encrypted blobs to store format v2 (non-destructive; manifest-free; `--dry-run` previews) |
| `secrets migrate --status` | Survey every project's v2 readiness; exits non-zero until the whole store is finalize-ready |
| `secrets migrate --finalize` | **Optional GC** — drop the old v1 blobs and mark the store pure v2. Never required: upgraded clients dual-write and read-fall-back, so not finalizing never cuts anyone off |
| `secrets recipients list` | List the store's recipient public keys (and names if set) |
| `secrets recipients add <age1…> [--name N]` | Add a recipient key to the store and immediately re-encrypt every blob to the new set |
| `secrets recipients rm <key\|name> [--yes]` | Remove a recipient and re-encrypt the store; `--yes` required when removing your own key |
| `secrets reencrypt` | Re-encrypt every blob to the current recipients (idempotent — useful after a manual edit or partial failure) |
| `secrets sync` | Reconcile a store that has diverged from its remote: stash local blob edits, rebase onto the remote, restore the stash, then offer to publish your local commits. Never merges, force-pushes, or hard-resets |
| `secrets sync --dry-run` | Report the store's ahead/behind/dirty state and what a reconcile would do; changes nothing |
| `secrets sync --yes` | Reconcile and publish local commits without the confirmation prompt (for scripts) |
| `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks |
| `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing |
### Upgrading: do teammates on an older `secrets` get new secrets?
Store-format v2 is **additive** — an upgraded client reads either blob suffix and keeps the old (v1) suffix alive for externals that already existed, so you almost never have to coordinate an upgrade:
| Secret type | Old client gets it? |
|---|---|
| `.env` / `.env.*` / `.dev.vars` | **Yes, always** (blob name is identical across formats) |
| whole-file external (`file`) | **Yes, always** |
| `properties` external that already existed | **Yes** (dual-written so old clients stay fresh) |
| brand-new `properties` external | **No — must upgrade `secrets`** (the gentle forcing function) |
"Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`.
And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent.
### When the store diverges
The store is a git repo, so two machines pushing at once can leave your clone
both ahead and behind its remote. `secrets push` needs a fast-forward and
`secrets pull` won't silently merge, so both stop and tell you to run:
```bash
secrets sync
```
`sync` fetches, stashes any uncommitted blob edits, rebases your local commits
onto the remote, restores the stash, and then *asks* before publishing your
commits to the shared store (`--yes` skips the prompt; `--dry-run` just
reports). If the rebase conflicts, it aborts, restores your stash, names the
conflicting files, and leaves the store exactly as it found it — nothing in
the path force-pushes, hard-resets, or drops a stash.
`secrets which` now reports the same state up front, so you can see it coming:
```
store: /Users/you/.secrets
source: default
format: v2
remote: ahead 1, behind 11, 3 modified (run: secrets sync)
```
### Automatic project detection ### Automatic project detection
@ -176,6 +249,32 @@ When you run `secrets push` or `secrets pull` without specifying a project name,
You can also specify a name explicitly: `secrets push myapp`. You can also specify a name explicitly: `secrets push myapp`.
### The manifest (`.secrets.json`)
Every project gets a committed `.secrets.json` at its root declaring exactly what syncs — the single source of truth `push` and `pull` operate from (requires `jq`):
```json
{
"version": 2,
"options": { "autoAdd": true },
"dotenv": [".env", ".env.staging", "packages/web/.env.development"],
"external": [
{ "type": "properties", "path": "~/.gradle/gradle.properties",
"keys": ["beaconClerkPkTest"] },
{ "type": "file", "path": "~/keystores/upload.keystore" }
]
}
```
You rarely write it by hand:
- **Auto-add (default):** `secrets push` discovers conventional files (`.env`, `.env.*`, `.dev.vars` — plus `package.json` workspace dirs once a manifest exists) and adds them to the manifest with an `==>` notice. Commit the manifest so other machines pick it up.
- **Explicit mode:** set `"options": {"autoAdd": false}` (a committed, team-shared setting) and `push` only syncs declared entries, warning about undeclared files. `secrets add <path>` is then the only manifest writer. Per-invocation: `push --frozen` (declared-only once) and `push --dry-run` (preview).
- `dotenv` paths are project-relative — nested monorepo paths like `packages/@acme/web/.env` are welcome; `..`, absolute paths, and symlinked manifests are refused.
- On the other machine, `secrets pull` restores exactly what the committed manifest declares, creating nested directories as needed.
Projects without a manifest keep working exactly as before (and work without `jq`); the first `push` bootstraps one for you.
### secrets run ### secrets run
`secrets run` is a **pull → run → clear** pipeline: it runs `secrets pull` to decrypt the latest files into your project, executes your command, then runs `secrets clear` when that command finishes. Plaintext `.env` / `.dev.vars` files exist only while your command is running. `secrets run` is a **pull → run → clear** pipeline: it runs `secrets pull` to decrypt the latest files into your project, executes your command, then runs `secrets clear` when that command finishes. Plaintext `.env` / `.dev.vars` files exist only while your command is running.
@ -326,10 +425,10 @@ git commit -am "switch to personal secrets"
### Monorepo support ### Monorepo support
For projects with multiple packages (monorepos using `package.json` workspaces), add the `-w` flag to operate on all workspaces at once: For projects with multiple packages, add the `-w` flag to operate on all workspaces at once:
```bash ```bash
cd ~/myapp # has package.json with "workspaces": ["apps/*", "packages/*"] cd ~/myapp # npm/yarn workspaces, or a pnpm-workspace.yaml
secrets push -w # encrypts secrets from root + each workspace secrets push -w # encrypts secrets from root + each workspace
secrets pull -w # decrypts into root + each workspace directory secrets pull -w # decrypts into root + each workspace directory
secrets clear -w # clears secrets from root + each workspace secrets clear -w # clears secrets from root + each workspace
@ -346,31 +445,61 @@ Inside `~/.secrets/`, workspace secrets are organized by path:
apps/api/.env.age # api workspace apps/api/.env.age # api workspace
``` ```
Requires `jq` (`brew install jq`). **Where workspaces are declared.** All three package managers are supported:
| Manager | Declaration |
|---|---|
| npm | `package.json``"workspaces": ["apps/*"]` |
| yarn | `package.json``"workspaces": {"packages": ["apps/*"]}` |
| pnpm | `pnpm-workspace.yaml``packages:` block |
`package.json` wins when it declares any workspaces; `pnpm-workspace.yaml` is
the fallback. Reading `package.json` requires `jq` (`brew install jq`); a
pnpm-only repo needs no jq for workspace discovery.
If a root *looks* like a monorepo (a `pnpm-workspace.yaml` or a `packages/`
directory) but no workspace packages can be read from it, `secrets push` says
so on stderr rather than silently discovering nothing — that silence
previously made a pnpm repo indistinguishable from one with nothing to sync.
### External files (Gradle properties) ### External files (Gradle properties)
Some credentials don't live in your project at all. Android builds, for example, read keys from `~/.gradle/gradle.properties` — a global file, outside any project, shared by every Gradle project on your machine (the project's own `gradle.properties` is git-tracked, so it's the wrong home for secrets). `secrets` can sync specific keys from such a file without touching the unrelated keys around them. Some credentials don't live in your project at all. Android builds, for example, read keys from `~/.gradle/gradle.properties` — a global file, outside any project, shared by every Gradle project on your machine (the project's own `gradle.properties` is git-tracked, so it's the wrong home for secrets). `secrets` can sync specific keys from such a file without touching the unrelated keys around them.
You declare what to sync in a committed `.secrets-files` manifest at your project root, one entry per line: You declare what to sync in the `external` array of your committed `.secrets.json`:
``` ```json
# <type> <path> <keys...> {
gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest beaconClerkPkLive "version": 2,
"external": [
{ "type": "properties", "path": "~/.gradle/gradle.properties",
"keys": ["beaconClerkPkTest", "beaconClerkPkLive"] }
]
}
``` ```
- **type**`gradle-properties` (the only supported type today). - **type**`properties` (sync named keys from a Java-properties-style file) or `file` (sync the whole file — see below).
- **path** — absolute or `~/`-relative. The basename must be `gradle.properties` and must resolve inside `$HOME`. - **path** — absolute or `~/`-relative; must resolve inside `$HOME`. For `properties` the basename must end in `.properties`.
- **keys** — the property names to sync. Only these keys are read on push and merged on pull; everything else in the file is left alone. - **keys** — the property names to sync (`properties` only). Only these keys are read on push and merged on pull; everything else in the file is left alone. `file` entries take no keys.
> **Legacy `.secrets-files`:** older projects declared these entries in a line-based `.secrets-files`. It still parses, and the next `secrets push` absorbs its entries into `.secrets.json` (type `gradle-properties` becomes `properties`) — after that the legacy file is superseded and can be deleted.
#### Syncing to a second machine #### Syncing to a second machine
On the machine that already has the keys set: On the machine that already has the keys set, add the entry to `.secrets.json` (create the file if the project doesn't have one yet):
```bash ```bash
cd ~/myapp cd ~/myapp
echo "gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest beaconClerkPkLive" > .secrets-files cat > .secrets.json <<'EOF'
git add .secrets-files && git commit -m "sync gradle Clerk keys" {
"version": 2,
"external": [
{ "type": "properties", "path": "~/.gradle/gradle.properties",
"keys": ["beaconClerkPkTest", "beaconClerkPkLive"] }
]
}
EOF
git add .secrets.json && git commit -m "sync gradle Clerk keys"
secrets push secrets push
# ==> Extracted 2 key(s) from ~/.gradle/gradle.properties # ==> Extracted 2 key(s) from ~/.gradle/gradle.properties
``` ```
@ -389,6 +518,74 @@ secrets pull
> **Note:** unlike `.env` files, merged Gradle keys are written as **permanent plaintext** into the target file — `secrets clear` does **not** remove them. This is appropriate for publishable / low-secrecy values (like Clerk publishable keys, `pk_*`). For high-value secrets that should never sit on disk, use `secrets run` with a `.env` instead. > **Note:** unlike `.env` files, merged Gradle keys are written as **permanent plaintext** into the target file — `secrets clear` does **not** remove them. This is appropriate for publishable / low-secrecy values (like Clerk publishable keys, `pk_*`). For high-value secrets that should never sit on disk, use `secrets run` with a `.env` instead.
#### Whole files (`file` type)
Some external secrets are whole binary files — an Android upload keystore, a certificate. The `file` type syncs the file verbatim (binary-safe, encrypted with age like everything else):
```json
{ "type": "file", "path": "~/keystores/beacon-upload.keystore" }
```
On `secrets push` the file is encrypted into `<project>/external/`. On `secrets pull` it is restored to the same path with mode `600`; if a different version already exists there, it is backed up to `<name>.secrets-bak` first. The same path rules apply (inside `$HOME`, no `..`, no symlinks). Like merged Gradle keys, restored files are permanent plaintext on disk — `secrets clear` does not remove them.
### Onboarding and offboarding teammates
By default every team member uses the **same** `key.txt` (one shared private key). The multi-recipient feature lets each teammate have their **own** keypair while still sharing one store — so you never hand out a secret key to a new hire, and removing an ex-teammate's access is one command.
#### Onboarding a teammate
```bash
# 1. Teammate generates their own keypair on their machine (never shares the private key)
age-keygen -o ~/.secrets/key.txt # writes key.txt; prints the public key
# 2. Teammate sends you their PUBLIC key (printed by age-keygen, starts with age1…)
# — over Slack, email, whatever. Public keys are not secret.
# 3. An existing member adds the public key to the store
secrets recipients add age1theirpublickey --name alice
# => Adds alice to recipients.txt, re-encrypts every blob to the full set, pushes.
# 4. Teammate clones the store repo and drops their key.txt in place
git clone git@github.com:<you>/my-secrets.git ~/.secrets
# (key.txt already generated in step 1 — nothing to copy)
# 5. Teammate pulls into any project
cd ~/myapp
secrets pull
# => Their key matches one recipient stanza in every blob — it just works.
```
Run `secrets recipients list` to confirm who has access:
```
alice age1theirpublickey…
you age1yourpublickey…
```
#### Offboarding a teammate
```bash
# Remove the recipient by name (or public key) and re-encrypt the store
secrets recipients rm alice
# => Removes alice from recipients.txt, re-encrypts every blob, pushes.
# Existing blobs are re-encrypted; the removed key can no longer decrypt them.
```
> **Important:** git history can't be un-shared. If alice had access during a period when genuinely sensitive values were stored, rotate those values now (update them in the external system and run `secrets push`). The re-encrypt prevents future access; history is permanent.
#### Managing recipients
```bash
secrets recipients list # show all recipient keys and names
secrets recipients add age1… # add a key (bootstraps recipients.txt on a legacy store)
secrets recipients add age1… --name bob # attach a human-readable label
secrets recipients rm bob # remove by name
secrets recipients rm age1… # remove by public key
secrets reencrypt # re-encrypt to current recipients (idempotent heal)
```
`secrets which` shows a `recipients: N (alice, bob, …)` line so you can always confirm the active set from any project directory.
## Safety features ## Safety features
- **`secrets run` auto-clears** — plaintext files are deleted when the command exits, errors, or is interrupted with Ctrl-C - **`secrets run` auto-clears** — plaintext files are deleted when the command exits, errors, or is interrupted with Ctrl-C
@ -404,13 +601,15 @@ If you suspect your key has been compromised, or a teammate leaves the team:
secrets rekey secrets rekey
``` ```
This generates a new key and re-encrypts all secrets. After rekeying: This generates a new key and re-encrypts all secrets (including `.env` and other dotfiles). After rekeying:
1. Copy the new `~/.secrets/key.txt` to every machine and teammate 1. Copy the new `~/.secrets/key.txt` to every machine and teammate
2. Old encrypted files remain in git history (encrypted with the old key, which should be discarded) 2. Old encrypted files remain in git history (encrypted with the old key, which should be discarded)
For complete rotation with no historical exposure, create a fresh `~/.secrets/` repo. For complete rotation with no historical exposure, create a fresh `~/.secrets/` repo.
> **Recovering from a broken rekey (pre-0.2.1.0):** Older versions of `rekey` skipped dotfiles (`.env`, `.dev.vars`) when re-encrypting, leaving their blobs on the *old* key while `key.txt` was replaced. If `secrets pull` now fails with `no identity matched any of the recipients`, those blobs are still encrypted to a key you no longer have. Restore the **old** `key.txt` from another machine that hasn't rekeyed, `secrets pull` to recover the plaintext, then `secrets rekey` again on 0.2.1.0 or later.
## Environment variables ## Environment variables
| Variable | Default | Purpose | | Variable | Default | Purpose |
@ -423,16 +622,31 @@ For complete rotation with no historical exposure, create a fresh `~/.secrets/`
**"Not initialized"** — Run `secrets init` to create the `~/.secrets/` directory. **"Not initialized"** — Run `secrets init` to create the `~/.secrets/` directory.
**"Found an existing key ... but no repo"** — You copied `key.txt` into `~/.secrets` and then ran `secrets init`. On a second machine you should *clone* your existing secrets repo, not re-initialize it (`init` is only for the very first machine). The error prints the exact `git clone` command to run — copy-paste it, or see [Additional machines](#additional-machines). When your project's `.secrets-store` file declares a remote URL, the command is filled in with the real URL.
**"No secret files found"** — You're in a directory that doesn't have `.env`, `.env.*`, or `.dev.vars` files. Make sure you're in the right project directory. **"No secret files found"** — You're in a directory that doesn't have `.env`, `.env.*`, or `.dev.vars` files. Make sure you're in the right project directory.
**"Project not found"** — The project name doesn't match anything in `~/.secrets/`. Run `secrets list` to see what's stored. The name is usually derived from your directory name or git remote. **"Project not found"** — The project name doesn't match anything in `~/.secrets/`. Run `secrets list` to see what's stored. The name is usually derived from your directory name or git remote.
**"Fast-forward pull failed"** — Someone else pushed secrets while you had local changes. Run `secrets pull` first, then retry your push. **"Fast-forward pull failed"** — Someone else pushed secrets while you had local changes. Run `secrets pull` first, then retry your push.
**".secrets.json: invalid JSON" / "manifest version N is not supported"** — The committed manifest is malformed or written by a newer `secrets`. The error names the file; fix the syntax, or update the tool (`git pull` in the tool's clone).
**"'jq' is not installed"** — Manifest features need `jq`. The error prints the install command for your platform. Manifest-less projects work without it.
**Not sure the store is intact?** — Run `secrets verify` in a project to check its `.secrets.json` against the store (declared-but-missing blobs and orphaned blobs) and decrypt-test every blob with your current key. Use `secrets verify --all` for a store-wide decrypt sweep across every project. It's read-only — plaintext is streamed to `/dev/null`, never written to disk — and exits non-zero if anything is wrong, so it's safe to run in CI.
## Development ## Development
```bash ```bash
# Run the test suite (113 tests) # Run the test suite (272 tests across four files)
brew install bats-core brew install bats-core
bats test/secrets.bats bats test/
# Security regression subset — operator-local only (attack-payload fixtures).
# Required before ship; records sign-off in .gstack/security-signoff.json.
./test/run-security.sh
``` ```
Hosted AI agents must not run the security script or perform red-team/adversarial
review on this repo — see `.ship-policy.json` and `CLAUDE.md`.

View file

@ -1 +1 @@
0.2.0.0 0.7.7.0

View file

@ -0,0 +1,587 @@
# Additive-v2 Dual-Write Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Make store-format v2 *additive* — upgraded `secrets` clients read either blob suffix and dual-write existing `properties` externals — so the destructive `migrate --finalize` becomes optional GC and the cross-machine coordination gate disappears.
**Architecture:** Replace the marker-driven single-suffix helper `_external_blob_suffix(type)` with two helpers: a read-resolver that tries `.properties.age` then falls back to `.gradle-properties.age`, and a write-targets helper that writes the v2 suffix always plus the v1 suffix *only when a v1 twin already exists* (dual-write existing externals; brand-new externals are v2-only — the intended forcing function). Reads and writes no longer depend on the `.secrets-format` marker, which keeps its meaning (born-v2 / finalized). Because fresh pushes now write the v2 suffix on any store, the migrate/finalize test fixtures (which relied on push producing a v1 blob) are updated to fabricate an old-client v1 blob.
**Tech Stack:** Single bash 3.2 script (`secrets`); `age`, `git`, `jq`. Tests: `bats-core` (`test/migrate.bats`, `test/manifest.bats`). Every standalone `[[ ]]` test assertion ends with `|| false` (bash 3.2 ERR-trap gotcha).
**Spec:** `docs/superpowers/specs/2026-06-08-additive-v2-dual-write-design.md`. **Ticket:** EGB-712.
---
## Background facts (verified against branch `brian/egb-712-...`, post-EGB-710)
- `_external_blob_suffix()``secrets:548-555`. Returns `properties` for `gradle-properties` on a v2 store (`_store_format == 2`), else the type verbatim; `file` always returns `file`. Blob path = `external/<slug>.<suffix>.age`.
- Callers of `_external_blob_suffix`: push file write `secrets:655`, push properties write `secrets:684`, pull read `secrets:713`, verify read `secrets:2056`. All four are replaced; then the helper is deleted.
- `_secrets_files_slug(path)` (`secrets:512`) derives the machine-independent slug. dotenv and `file` blobs are byte-identical in v1/v2 (only the `properties` suffix diverges).
- `_store_format()` (`secrets:529`) and the marker stay as-is — set only by `init` (born-v2) and `migrate --finalize`. Push must NOT stamp it (see spec §3).
- Existing test that codifies OLD write behavior: `test/migrate.bats:52` "push on a v1 store still writes .gradle-properties.age (back-compat)" — rewritten in Task 2.
- Tests that fabricate-or-rely-on a v1 properties blob from `make_v1_store; push` and break once push writes v2-only (repaired in Task 3): the migrate copy-forward/idempotent tests, the `--dry-run` rename test, the dotenv+file untouched test, all four finalize tests, the EGB-710 manifest-free / undeclared-twin tests, and the two `--status` tests that need an actual v1 blob. (dotenv-only and already-v2 tests are unaffected.)
- Baseline before this plan: `bats test/` = 242 passing.
## File structure
- Modify: `secrets` — delete `_external_blob_suffix` (548-555); add `_resolve_external_blob_read` + `_external_blob_write_targets` in its place; rewire push (651-688), pull (712-713), verify (2052-2068); extend `_migrate_status`; docs in `cmd_help`.
- Modify: `test/migrate.bats` — add `m_fake_v1_blob` helper; add read-fallback + write-rule + status-coverage tests; repair the v1-blob-dependent fixtures.
- Modify: `CLAUDE.md`, `README.md`, `VERSION` (→ `0.7.0.0`), `CHANGELOG.md`.
---
## Task 1: Read-resolver — reads try both suffixes
**Files:** `secrets` (replace `_external_blob_suffix` with the read-resolver; rewire pull + verify reads), `test/migrate.bats`.
- [ ] **Step 1: Write the failing test** — a v2 store whose properties blob exists ONLY in the v1 suffix must still pull.
Add to `test/migrate.bats` (after the format-marker tests, ~line 60):
```bash
@test "pull reads a v1-suffix properties blob on a v2 store (read-fallback)" {
init_with_remote # born-v2 store (marker=2)
m_gradle_src $'beaconClerkPkTest=pk_test_v1\n'
create_project_dir rffallback
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push rffallback >/dev/null 2>&1 # writes .properties.age on a v2 store
# Simulate an external that exists only in the v1 suffix (an old client wrote it):
local v2blob; v2blob=$(ls "$SECRETS_DIR/rffallback/external/"*.properties.age)
mv "$v2blob" "${v2blob%.properties.age}.gradle-properties.age"
rm -f "$HOME/.gradle/gradle.properties"
"$SECRETS_BIN" pull rffallback >/dev/null 2>&1
run grep -q 'beaconClerkPkTest=pk_test_v1' "$HOME/.gradle/gradle.properties"
[ "$status" -eq 0 ]
}
```
- [ ] **Step 2: Run it, confirm it fails**
Run: `bats test/migrate.bats -f "read-fallback"`
Expected: FAIL — old pull (`secrets:713`) uses `_external_blob_suffix gradle-properties` = `properties` on a v2 store, looks only for `.properties.age` (which we renamed away), warns "no encrypted data", restores nothing → the grep fails.
- [ ] **Step 3: Replace `_external_blob_suffix` with the read-resolver.** Replace `secrets:542-555` (the comment block + `_external_blob_suffix()` through its closing `}`) with:
```bash
# Resolve the on-disk path of an external blob for READING. Tries the v2 suffix
# (.properties.age) first, then falls back to the v1 (.gradle-properties.age) for
# `properties` externals, so an upgraded client finds the blob whichever format
# wrote it (additive v2 — EGB-712). `file` externals share one suffix in both
# formats. Echoes the path of the blob that exists; if neither exists, echoes the
# canonical v2 path so the caller's "no blob" message reads sensibly. Read-only.
_resolve_external_blob_read() {
local project="$1" slug="$2" mtype="$3"
local base="$SECRETS_DIR/$project/external/$slug"
case "$mtype" in
file)
echo "$base.file.age" ;;
properties|gradle-properties)
if [ -f "$base.properties.age" ]; then
echo "$base.properties.age"
elif [ -f "$base.gradle-properties.age" ]; then
echo "$base.gradle-properties.age"
else
echo "$base.properties.age"
fi ;;
*)
echo "$base.$mtype.age" ;;
esac
}
```
(The write-targets helper is added in Task 2 — leave a gap; do not reintroduce `_external_blob_suffix`.)
- [ ] **Step 4: Rewire the pull read.** At `secrets:712-713`, replace:
```bash
local slug; slug=$(_secrets_files_slug "$mpath")
local blob="$SECRETS_DIR/$project/external/$slug.$(_external_blob_suffix "$mtype").age"
```
with:
```bash
local slug; slug=$(_secrets_files_slug "$mpath")
local blob; blob=$(_resolve_external_blob_read "$project" "$slug" "$mtype")
```
- [ ] **Step 5: Rewire the verify read.** At `secrets:2055-2065`, replace:
```bash
slug=$(_secrets_files_slug "$epath")
erel="external/$slug.$(_external_blob_suffix "$etype").age"
# Account for BOTH the v1 and v2 suffix forms in the orphan set. During the
# migration window (after copy-forward, before --finalize) the v2 twin
# coexists with the v1 blob; neither should read as an orphan whichever
# format the store currently reports. (file's two forms are identical.)
expected="${expected}external/$slug.$etype.age"$'\n'
[ "$etype" = "gradle-properties" ] && expected="${expected}external/$slug.properties.age"$'\n'
eblob="$pdir/$erel"
if [ ! -f "$eblob" ]; then
echo "FINDING: external '$epath' ($etype) is declared but has no blob in the store ($project/$erel missing). Run 'secrets push'." >&2
```
with:
```bash
slug=$(_secrets_files_slug "$epath")
# Account for BOTH suffix forms in the orphan set — a dual-written `properties`
# external (additive v2 — EGB-712) legitimately has both blobs on disk; neither
# is an orphan. (file's two forms are identical.)
expected="${expected}external/$slug.$etype.age"$'\n'
[ "$etype" = "gradle-properties" ] && expected="${expected}external/$slug.properties.age"$'\n'
eblob=$(_resolve_external_blob_read "$project" "$slug" "$etype")
erel="${eblob#"$pdir"/}"
if [ ! -f "$eblob" ]; then
echo "FINDING: external '$epath' ($etype) is declared but has no blob in the store ($project/$erel missing). Run 'secrets push'." >&2
```
(Note: `_external_blob_suffix` still has two remaining callers in push — push isn't rewired until Task 2, so the script still parses and runs. Those calls keep working because the function is only deleted in Task 2 Step 6, after push is rewired.)
**IMPORTANT:** do NOT delete `_external_blob_suffix` yet — push (`secrets:655`, `secrets:684`) still calls it until Task 2. Deleting it now breaks push.
- [ ] **Step 6: Run the read-fallback test + full migrate suite**
Run: `bats test/migrate.bats`
Expected: the new "read-fallback" test PASSES; all other migrate tests still PASS (push unchanged; pull/verify now use the resolver, which is equivalent to the old behavior whenever the suffix matches the store format).
- [ ] **Step 7: Commit**
```bash
git add secrets test/migrate.bats
git commit -m "feat: read-resolver tries both external suffixes (additive v2, EGB-712)"
```
---
## Task 2: Write-targets — twin rule (dual-write existing, v2-only for new)
**Files:** `secrets` (add `_external_blob_write_targets`; rewire push file + properties writes; delete `_external_blob_suffix`), `test/migrate.bats`.
- [ ] **Step 1: Write the failing tests.** Add to `test/migrate.bats`:
```bash
@test "push writes the v2 suffix for a fresh external even on a v1 store" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir freshv1
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push freshv1 >/dev/null 2>&1
run bash -c "ls $SECRETS_DIR/freshv1/external/*.properties.age"
[ "$status" -eq 0 ] # v2 suffix regardless of marker
run bash -c "ls $SECRETS_DIR/freshv1/external/*.gradle-properties.age 2>/dev/null"
[ "$status" -ne 0 ] # no v1 twin for a brand-new external
}
@test "push dual-writes the v1 twin so old clients stay fresh" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_old\n'
create_project_dir dualwrite
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push dualwrite >/dev/null 2>&1 # v2-only (fresh)
m_fake_v1_twin dualwrite # simulate a pre-existing v1 twin
m_gradle_src $'beaconClerkPkTest=pk_test_new\n'
"$SECRETS_BIN" push dualwrite >/dev/null 2>&1 # twin exists -> dual-write both
# Prove the v1 twin was refreshed: drop the v2 blob, pull, expect the NEW value.
rm -f "$SECRETS_DIR/dualwrite/external/"*.properties.age
rm -f "$HOME/.gradle/gradle.properties"
"$SECRETS_BIN" pull dualwrite >/dev/null 2>&1
run grep -q 'beaconClerkPkTest=pk_test_new' "$HOME/.gradle/gradle.properties"
[ "$status" -eq 0 ]
}
```
- [ ] **Step 2: Add the `m_fake_v1_twin` test helper.** In `test/migrate.bats`, next to `make_v1_store` (~line 12), add:
```bash
# Simulate an old (v1) client's properties blob: copy the pushed v2
# .properties.age to its v1 .gradle-properties.age twin. (Current clients never
# write the v1 suffix for a fresh external, so tests fabricate it.) Use `cp` to
# KEEP the v2 blob (dual present); see m_make_v1_only to leave only the v1 blob.
m_fake_v1_twin() {
local proj="$1" v2
v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age)
cp "$v2" "${v2%.properties.age}.gradle-properties.age"
}
# Like m_fake_v1_twin but renames (leaves ONLY the v1 blob) — for old-client-only
# / copy-forward fixtures.
m_make_v1_only() {
local proj="$1" v2
v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age)
mv "$v2" "${v2%.properties.age}.gradle-properties.age"
}
```
- [ ] **Step 3: Run the new tests, confirm they fail**
Run: `bats test/migrate.bats -f "fresh external even on a v1 store"`
Expected: FAIL — on a v1 store, the old push (`_external_blob_suffix gradle-properties` = `gradle-properties`) writes `.gradle-properties.age`, so the `*.properties.age` assertion fails.
Run: `bats test/migrate.bats -f "dual-writes the v1 twin"`
Expected: FAIL — old push writes a single suffix; the second push won't refresh the v1 twin.
- [ ] **Step 4: Add the write-targets helper.** Immediately after `_resolve_external_blob_read` (added in Task 1), insert:
```bash
# The on-disk path(s) to WRITE for an external blob, one per line. For a
# `properties` external this is the v2 suffix (.properties.age) ALWAYS, plus the
# v1 suffix (.gradle-properties.age) WHEN a v1 twin already exists in the store
# (dual-write keeps old clients fresh; a brand-new external is v2-only — the
# intended forcing function, additive v2 / EGB-712). `file` externals have a
# single suffix in both formats. Independent of the store marker.
_external_blob_write_targets() {
local project="$1" slug="$2" mtype="$3"
local base="$SECRETS_DIR/$project/external/$slug"
case "$mtype" in
file)
echo "$base.file.age" ;;
properties|gradle-properties)
echo "$base.properties.age"
[ -f "$base.gradle-properties.age" ] && echo "$base.gradle-properties.age" ;;
*)
echo "$base.$mtype.age" ;;
esac
}
```
- [ ] **Step 5: Rewire the push writes.** At `secrets:651-658` (the `file` branch), replace:
```bash
if [ "$mtype" = "file" ]; then
# EGB-652: whole-file sync — encrypt the file verbatim (binary-safe).
mkdir -p "$SECRETS_DIR/$project/external"
local fslug; fslug=$(_secrets_files_slug "$mpath")
age -r "$pubkey" -o "$SECRETS_DIR/$project/external/$fslug.$(_external_blob_suffix file).age" "$expanded"
info "Encrypted file $mpath"
pushed=$((pushed + 1))
continue
fi
```
with:
```bash
if [ "$mtype" = "file" ]; then
# EGB-652: whole-file sync — encrypt the file verbatim (binary-safe).
mkdir -p "$SECRETS_DIR/$project/external"
local fslug; fslug=$(_secrets_files_slug "$mpath")
local wt
while IFS= read -r wt; do
[ -n "$wt" ] || continue
age -r "$pubkey" -o "$wt" "$expanded"
done < <(_external_blob_write_targets "$project" "$fslug" file)
info "Encrypted file $mpath"
pushed=$((pushed + 1))
continue
fi
```
Then at `secrets:682-684` (the properties branch), replace:
```bash
mkdir -p "$SECRETS_DIR/$project/external"
local slug; slug=$(_secrets_files_slug "$mpath")
age -r "$pubkey" -o "$SECRETS_DIR/$project/external/$slug.$(_external_blob_suffix "$mtype").age" "$tmp"
```
with:
```bash
mkdir -p "$SECRETS_DIR/$project/external"
local slug; slug=$(_secrets_files_slug "$mpath")
local wt
while IFS= read -r wt; do
[ -n "$wt" ] || continue
age -r "$pubkey" -o "$wt" "$tmp"
done < <(_external_blob_write_targets "$project" "$slug" "$mtype")
```
- [ ] **Step 6: Delete the now-unused `_external_blob_suffix`.** Confirm zero remaining callers first:
Run: `grep -n "_external_blob_suffix" secrets`
Expected: no matches (all four call sites rewired). If any remain, rewire them before deleting. Then delete the `_resolve_external_blob_read`-replaced... — it's already gone (replaced in Task 1). Verify the function is absent: `grep -c "_external_blob_suffix()" secrets``0`.
- [ ] **Step 7: Run the two new write tests**
Run: `bats test/migrate.bats -f "fresh external even on a v1 store"` then `-f "dual-writes the v1 twin"`
Expected: both PASS.
- [ ] **Step 8: Rewrite the obsolete back-compat test.** Replace the `@test "push on a v1 store still writes .gradle-properties.age (back-compat)"` block (`test/migrate.bats:52-60`) with:
```bash
@test "push on a v1 store writes the v2 suffix for a fresh external (additive v2)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir v1push
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push v1push >/dev/null 2>&1
run bash -c "ls $SECRETS_DIR/v1push/external/*.properties.age"
[ "$status" -eq 0 ]
}
```
- [ ] **Step 9: Run the full migrate suite — expect the v1-blob-dependent fixtures to FAIL.** This is expected; Task 3 repairs them.
Run: `bats test/migrate.bats`
Expected: the read-fallback + two write tests + rewritten back-compat test PASS; several copy-forward/finalize/status tests now FAIL (push no longer writes a `.gradle-properties.age` for them to migrate). Note which fail — Task 3 fixes exactly those.
- [ ] **Step 10: Commit** (suite intentionally not yet fully green — Task 3 follows immediately)
```bash
git add secrets test/migrate.bats
git commit -m "feat: twin-rule write targets — dual-write existing, v2-only for new (additive v2, EGB-712)"
```
---
## Task 3: Repair migrate/finalize/status fixtures
**Files:** `test/migrate.bats`. No production code changes — this re-greens the suite by fabricating the old-client v1 blobs that push no longer writes.
The rule for each repair: after the `"$SECRETS_BIN" push <proj>` line, insert a fabrication call:
- Use **`m_make_v1_only <proj>`** (rename → only the v1 blob exists) for tests asserting a `*.gradle-properties.age` blob exists / is copy-forwarded (mirrors the pre-EGB-712 state where push produced a v1 blob).
- Use **`m_fake_v1_twin <proj>`** (keep both) only where a test needs both suffixes present.
- [ ] **Step 1: Repair the copy-forward / dry-run / idempotent tests.** In each of these tests, insert `m_make_v1_only <proj>` immediately after the `push <proj>` line:
- `"migrate --dry-run reports the rename and writes nothing"` (proj `dryproj`)
- `"migrate copy-forward creates the v2 twin and keeps the v1 blob (byte-identical)"` (proj `cfproj`)
- `"migrate copy-forward is idempotent"` (proj `idemproj`)
- `"migrate leaves dotenv and file blobs untouched"` (proj `mixproj`) — note this one ALSO pushes a `file` external; `m_make_v1_only` only touches `*.properties.age`, leaving the `.file.age` blob alone (correct).
Worked example — the copy-forward test becomes:
```bash
@test "migrate copy-forward creates the v2 twin and keeps the v1 blob (byte-identical)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir cfproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push cfproj >/dev/null 2>&1
m_make_v1_only cfproj
local old; old=$(ls "$SECRETS_DIR/cfproj/external/"*.gradle-properties.age)
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
local new; new=$(ls "$SECRETS_DIR/cfproj/external/"*.properties.age)
[ -f "$old" ]
[ -f "$new" ]
cmp -s "$old" "$new"
}
```
- [ ] **Step 2: Repair the finalize tests.** Insert `m_make_v1_only <proj>` after the `push <proj>` line in:
- `"finalize refuses when verify --all is not green"` (proj `failverify`) — then the existing `migrate` step creates the twin; corrupting `*.properties.age` still trips verify.
- `"finalize refuses an un-twinned v1 blob (project not migrated)"` (proj `untwinned`) — leaves a lone v1 blob, no twin: exactly the un-twinned state the test wants.
- `"finalize green path drops v1, keeps v2, stamps the marker"` (proj `finproj`).
- `"finalize cuts a recovery tag before deleting v1 blobs"` (proj — read it from the test).
- [ ] **Step 3: Repair the EGB-710 manifest-free / undeclared-twin tests.** Insert `m_make_v1_only <proj>` after the `push <proj>` line in:
- `"migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)"` (proj `nomanifestblob`) — insert BEFORE the `rm -f .secrets.json` line.
- `"migrate twins a store blob even when the manifest no longer declares it"` (proj `staleblob`) — insert before the `.secrets.json` rewrite.
- [ ] **Step 4: Repair the `--status` tests that need a real v1 blob.**
- `"migrate --status flags a project that needs migrating"` (proj `needsmig`) — insert `m_make_v1_only needsmig` after push, so a lone un-twinned v1 blob exists → NEEDS MIGRATE.
- `"migrate --status reports finalize-ready once every blob is twinned"` (proj `readymig`) — insert `m_make_v1_only readymig` after push and BEFORE the `migrate` step (migrate then creates the twin → finalize-ready).
- [ ] **Step 5: Run the full migrate suite**
Run: `bats test/migrate.bats`
Expected: ALL pass. If any copy-forward test still reports "nothing to migrate", its `m_make_v1_only` call is missing or misplaced (must come after push, before migrate).
- [ ] **Step 6: Run the WHOLE suite** (manifest.bats exercises externals end-to-end and must still be green)
Run: `bats test/`
Expected: all pass. If a `manifest.bats` external test fails, check it isn't asserting a specific suffix that additive-v2 changed (a fresh push now writes `.properties.age`); update such an assertion the same way (assert `.properties.age`, or use the resolver-agnostic round-trip via pull).
- [ ] **Step 7: Commit**
```bash
git add test/migrate.bats
git commit -m "test: fabricate old-client v1 blobs in migrate/finalize/status fixtures (additive v2, EGB-712)"
```
---
## Task 4: `migrate --status` — dual-write coverage line
**Files:** `secrets` (`_migrate_status`, `secrets:2190-2227`), `test/migrate.bats`.
Adds visibility into which `properties` externals are v2-only (old clients can't read them — the forcing function) vs dual-written (old clients still served).
- [ ] **Step 1: Write the failing test.** Add to `test/migrate.bats`:
```bash
@test "migrate --status counts v2-only externals (old clients not served)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir v2onlyext
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push v2onlyext >/dev/null 2>&1 # v2-only (fresh, no v1 twin)
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ] # no v1 blobs -> finalize-ready
[[ "$output" == *"v2-only"* ]] || false # surfaced as v2-only coverage
}
```
- [ ] **Step 2: Run it, confirm it fails**
Run: `bats test/migrate.bats -f "counts v2-only externals"`
Expected: FAIL — `_migrate_status` currently only counts `*.gradle-properties.age`; it never mentions `v2-only`.
- [ ] **Step 3: Extend `_migrate_status`.** In `_migrate_status` (`secrets:2190`), inside the `for dir` loop, after the existing `while ... done < <(find "$dir" -type f -name '*.gradle-properties.age' ...)` block and before the per-project classification, add a v2-only count, then surface it in the per-project line. Concretely, replace the classification block:
```bash
if [ "$v1" -eq 0 ]; then
echo " $project: v2-ready (no v1 properties blobs)"
elif [ "$untwinned" -eq 0 ]; then
echo " $project: migrated ($v1 v1 blob(s), all twinned)"
else
echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'"
any_untwinned=1
fi
```
with:
```bash
# v2-only externals: a .properties.age with no .gradle-properties.age twin —
# old (v1) clients cannot read these (the additive-v2 forcing function).
local v2only=0 pf
while IFS= read -r pf; do
[ -f "$pf" ] || continue
[ -f "${pf%.properties.age}.gradle-properties.age" ] || v2only=$((v2only + 1))
done < <(find "$dir" -type f -name '*.properties.age' 2>/dev/null)
local v2note=""
[ "$v2only" -gt 0 ] && v2note=" [$v2only v2-only — old clients not served]"
if [ "$v1" -eq 0 ]; then
echo " $project: v2-ready (no v1 properties blobs)$v2note"
elif [ "$untwinned" -eq 0 ]; then
echo " $project: migrated ($v1 v1 blob(s), all twinned)$v2note"
else
echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'$v2note"
any_untwinned=1
fi
```
(`local` inside the loop is bash-3.2-fine — it re-declares per iteration.)
- [ ] **Step 4: Run the new test + status suite**
Run: `bats test/migrate.bats -f "status"`
Expected: all status tests PASS, including the new v2-only one.
- [ ] **Step 5: Run the full suite**
Run: `bats test/`
Expected: all pass.
- [ ] **Step 6: Commit**
```bash
git add secrets test/migrate.bats
git commit -m "feat: migrate --status surfaces v2-only externals (coverage, EGB-712)"
```
---
## Task 5: Docs, help, version, changelog
**Files:** `secrets` (`cmd_help`), `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`.
- [ ] **Step 1: `cmd_help` — reframe finalize as optional.** In `cmd_help()`, replace the finalize line:
```
secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify)
```
with:
```
secrets migrate --finalize Optional GC: drop v1 blobs and mark the store pure v2
```
- [ ] **Step 2: `CLAUDE.md` — additive-v2 paragraph.** In the "Store format" bullet (line ~65), after the migration-chain sentence, add (new sentence, same bullet):
```
Additive v2 (EGB-712): upgraded clients read either blob suffix (`_resolve_external_blob_read` tries `.properties.age` then `.gradle-properties.age`) and dual-write a `properties` external only when a v1 twin already exists (`_external_blob_write_targets`) — so existing externals keep old clients fresh, brand-new externals are v2-only (a gentle forcing function), and `migrate --finalize` is now OPTIONAL GC rather than a required, coordination-gated flag-day. dotenv and `file` blobs are identical across formats, so they always propagate to old clients.
```
- [ ] **Step 3: `README.md` — propagation table.** Add, near the migrate rows in the command table or in a short "Upgrading / store format" subsection, the propagation-by-secret-type summary (verbatim from spec §6):
```markdown
**Do teammates on an older `secrets` get new secrets?**
| Secret type | Old client gets it? |
|---|---|
| `.env` / `.env.*` / `.dev.vars` | **Yes, always** (blob name identical across formats) |
| whole-file external | **Yes, always** |
| `properties` external that already existed | **Yes** (dual-written) |
| brand-new `properties` external | **No — must upgrade `secrets`** (the forcing function) |
"Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`.
```
- [ ] **Step 4: `VERSION`** — set to `0.7.0.0`.
- [ ] **Step 5: `CHANGELOG.md`** — insert above the top entry:
```markdown
## [0.7.0.0] - 2026-06-08
### Changed
- **Additive store-format v2 (EGB-712)** — upgraded `secrets` clients now read
either external blob suffix (`.properties.age` or the legacy
`.gradle-properties.age`) and **dual-write** a `properties` external whenever a
v1 twin already exists in the store. Existing externals keep working for
teammates on an older `secrets`; only a brand-new `properties` external is
written v2-only (a gentle "upgrade to see it" forcing function). dotenv and
whole-`file` externals are unchanged across formats and always propagate.
- **`secrets migrate --finalize` is now optional GC**, not a required milestone.
Because clients dual-write and read-fall-back, no teammate is ever cut off by
*not* finalizing; finalize only reclaims the duplicate v1 blobs, and stays
deferrable indefinitely. Its safety gates are unchanged.
### Added
- **`secrets migrate --status`** now reports `v2-only` externals per project
(the ones an un-upgraded client cannot read), so you can see the forcing
function's footprint at a glance.
```
- [ ] **Step 6: Run the full suite**
Run: `bats test/`
Expected: all pass (docs don't affect tests). Confirm the count is baseline 242 + net new tests from Tasks 1/2/4 (read-fallback, two write tests, v2-only status) minus the rewritten back-compat test (replaced, not added) = **246**.
- [ ] **Step 7: Sanity — help renders**
Run: `./secrets help 2>&1 | grep -- "--finalize"`
Expected: shows the reframed "Optional GC" line. (Help only prints; touches no store.)
- [ ] **Step 8: Commit**
```bash
git add secrets CLAUDE.md README.md VERSION CHANGELOG.md
git commit -m "docs: additive-v2 propagation + optional-GC finalize; bump 0.7.0.0 (EGB-712)"
```
---
## Self-review against the spec
- **§1 read resolution** → Task 1 (`_resolve_external_blob_read`, wired into pull + verify). Test: read-fallback.
- **§2 write rule / twin rule** → Task 2 (`_external_blob_write_targets`, push wiring). Tests: fresh-→v2-only, existing-twin-→dual-write.
- **§3 marker NOT stamped** → no production change (push never touches the marker); guarded implicitly by Task 3 keeping the `make_v1_store; push; migrate` flow working (store stays markerless after push). The plan deliberately does not add marker-stamping.
- **§4 finalize = optional GC** → unchanged logic; reframed in Task 5 docs/help. Existing finalize tests stay green (Task 3 keeps them green).
- **§5 migrate / --status coverage** → Task 4 (v2-only line); `migrate` copy-forward unchanged (EGB-710).
- **§6 propagation table** → Task 5 README/CLAUDE.md.
- **§7 caveat** → documented in CHANGELOG/CLAUDE.md framing; the read-fallback test exercises the v1-only read path.
- **§8 back-compat matrix** → covered across Task 1 (reads), Task 2 (writes), Task 3 (old-client v1 blobs simulated).
- **Safety invariants** → no path-rail changes; bash 3.2 (no associative arrays; `while read` + `find`); recursive walks unchanged; finalize gating untouched.
**Type/name consistency:** `_resolve_external_blob_read(project, slug, mtype)` and `_external_blob_write_targets(project, slug, mtype)` use the same arg order everywhere; test helpers `m_fake_v1_twin` (keep both) and `m_make_v1_only` (rename to v1-only) are used consistently per their documented semantics.
**Placeholder scan:** none — every step carries verbatim code or an exact enumerated edit with the precise insertion point.
## Operator-local follow-up (not part of this plan)
Per `.ship-policy.json`, before any PR ask the operator to run `./test/run-security.sh` and complete the SIGNOFF. EGB-712 also needs its one stale AC bullet ("Marker auto-stamps…") corrected to match the §3 decision (no auto-stamp) — a one-line Linear edit.

View file

@ -0,0 +1,447 @@
# EGB-710: `secrets migrate` guided flow (manifest-free copy-forward + `--status` survey) Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** Replace the dead-end "No .secrets.json — cd into a project that has a manifest" error in `secrets migrate` with a manifest-free copy-forward that just works, plus a `secrets migrate --status` survey that tells the operator exactly which projects still need migrating.
**Architecture:** The per-project copy-forward step (`_migrate_project`) currently reads `$PWD/.secrets.json` to find `properties` externals, then looks for their v1 blobs. This makes it die on a legacy `.secrets-files`-only project (the real EGB-710 repro) and — worse — silently skips any store blob the manifest doesn't declare, leaving it un-twinned so `--finalize` later refuses it. The fix: enumerate the **store** (`$SECRETS_DIR/<project>/external/*.gradle-properties.age`) directly and copy each to its `.properties.age` twin via suffix-swap — the *exact* logic `_migrate_finalize` already uses. No manifest needed, dotenv/file-only projects become a clean no-op, and migrate twins precisely the blobs finalize will demand twins for. A new `--status` mode walks every project dir and reports per-project readiness + whether the store is finalize-ready.
**Tech Stack:** Single bash 3.2 script (`secrets`), `age`, `git`, `jq` (unaffected here). Tests: `bats-core` (`test/migrate.bats`). Every standalone `[[ ]]` assertion ends with `|| false` (bash 3.2 ERR-trap gotcha).
---
## Background facts (verified against HEAD 633d19e)
- `_migrate_project()` lives at `secrets:2135`; it `die`s at `secrets:2142-2148` on missing `$PWD/.secrets.json`, then iterates `_json_external_entries "$manifest"` filtered to `gradle-properties` (`secrets:2154-2174`).
- `_migrate_finalize()` (`secrets:2196`) already enumerates blobs manifest-free: `find "$SECRETS_DIR" -type f -name '*.gradle-properties.age'` and derives the twin as `new="${f%.gradle-properties.age}.properties.age"` (`secrets:2212-2217`). The per-project step will mirror this, scoped to one project dir.
- `cmd_migrate()` flag parser: `secrets:2267-2285`.
- `derive_project_name ""` (`secrets:101`) needs no manifest — it uses the git remote basename or `basename "$PWD"`.
- `info()`/`die()`: `secrets:36-37`. `ensure_store_protections` is called post-write in the existing copy-forward.
- Output-contract strings existing tests depend on (must be preserved): `"would migrate"` (migrate.bats:80), `"0 blob(s) would be copy-forwarded"` (migrate.bats:94), `"1 already present"` (migrate.bats:121, idempotent re-run prints `$already already present`), `"already format v2"` (migrate.bats:139).
- The one test that codifies the OLD dead-end — `"migrate with no manifest in cwd dies with a directed message"` (migrate.bats:126) — is the behavior we are intentionally changing; it gets rewritten in Task 1.
- No test asserts the non-dry-run "nothing to migrate" wording (grep confirmed), so that message is free to change. We keep the substring `no v1 properties blobs` regardless for safety.
- VERSION is `0.6.0.1`; bump to `0.6.1.0` (new subcommand flag + behavior change). `MANIFEST_VERSION` stays `2` (no schema change).
## File structure
- Modify: `secrets` — rewrite `_migrate_project` (`secrets:2135-2191`), add `_migrate_status`, extend `cmd_migrate` flag parsing + dispatch (`secrets:2267-2285`), update `cmd_help` migrate lines (`secrets:2308-2309`).
- Modify: `test/migrate.bats` — rewrite the dead-end test; add manifest-free, finalize-consistency, and `--status` tests.
- Modify: `CLAUDE.md` — update the migrate paragraph (Architecture → Store format) to note manifest-free copy-forward + `--status`.
- Modify: `README.md` — migrate usage/help.
- Modify: `VERSION``0.6.1.0`; `CHANGELOG.md` — new entry.
---
## Task 1: Manifest-free per-project copy-forward
**Files:**
- Modify: `secrets``_migrate_project()` (`secrets:2135-2191`)
- Test: `test/migrate.bats`
- [ ] **Step 1: Write the failing test — migrate works with no `.secrets.json` (the EGB-710 repro)**
Add to `test/migrate.bats` (after the existing copy-forward tests, ~line 124):
```bash
@test "migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)" {
# The EGB-710 repro: a legacy project has a v1 properties blob in the store
# but no .secrets.json (it predates the manifest). migrate must NOT dead-end.
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir nomanifestblob
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push nomanifestblob >/dev/null 2>&1
rm -f .secrets.json # simulate a pre-manifest project
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.properties.age"
[ "$status" -eq 0 ] # v2 twin written despite no manifest
run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.gradle-properties.age"
[ "$status" -eq 0 ] # v1 kept (non-destructive)
}
```
- [ ] **Step 2: Run it to confirm it fails**
Run: `bats test/migrate.bats -f "manifest-free"`
Expected: FAIL — current code `die`s with "No .secrets.json in …" (status 1), so the first `[ "$status" -eq 0 ]` fails.
- [ ] **Step 3: Rewrite `_migrate_project` to enumerate the store, not the manifest**
Replace the body of `_migrate_project()` (`secrets:2135-2191`, from `_migrate_project() {` through its closing `}`) with:
```bash
_migrate_project() {
local dry_run="$1"
if [ "$(_store_format)" = "2" ]; then
info "Store is already format v2 — nothing to migrate."
return 0
fi
local project; project=$(derive_project_name "")
local pdir="$SECRETS_DIR/$project"
# Source of truth for the copy-forward is the STORE, not a project manifest.
# Every v1 properties blob is a `*.gradle-properties.age` file whose v2 twin
# is the same name with the `.properties.age` suffix (the only on-disk change
# v2 makes). Enumerating the store — exactly as `_migrate_finalize` does —
# means migrate twins precisely the blobs finalize will demand twins for, with
# no manifest dependency. This is why a legacy `.secrets-files`-only project
# (no `.secrets.json` yet) migrates cleanly instead of dead-ending, and why a
# store blob the manifest no longer declares still gets a twin.
local moved=0 already=0 would=0 v1count=0 f new
while IFS= read -r f; do
[ -f "$f" ] || continue
v1count=$((v1count + 1))
new="${f%.gradle-properties.age}.properties.age"
if [ -f "$new" ]; then
already=$((already + 1))
continue
fi
if [ "$dry_run" = true ]; then
echo "would migrate: ${f#"$SECRETS_DIR"/} -> $(basename "$new")"
would=$((would + 1))
else
cp "$f" "$new"
moved=$((moved + 1))
fi
done < <(find "$pdir/external" -type f -name '*.gradle-properties.age' 2>/dev/null)
if [ "$dry_run" = true ]; then
echo "migrate --dry-run: $would blob(s) would be copy-forwarded for '$project' (writes nothing); $already already present. v1 blobs are kept until 'secrets migrate --finalize'."
return 0
fi
if [ "$moved" -eq 0 ] && [ "$already" -eq 0 ]; then
info "Nothing to migrate for '$project' — no v1 properties blobs in the store (already v2-shaped). If you expected one, run 'secrets push' first, then re-run 'secrets migrate'."
return 0
fi
ensure_store_protections
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "migrate: copy-forward v2 twins for $project" >/dev/null 2>&1 || true
# Push the twins so a --finalize on another machine sees them (finalize
# refuses any v1 blob without a twin). Mirrors push/rekey's push behavior.
git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 && git -C "$SECRETS_DIR" push >/dev/null 2>&1 || true
info "Copy-forward for '$project': $moved new v2 twin(s), $already already present. v1 blobs kept (non-destructive). Run 'secrets migrate --finalize' once every project is migrated and every machine is upgraded."
}
```
Notes for the implementer:
- This removes the only in-script caller of `_check_manifest_file`/`_json_external_entries` *inside migrate*; both remain defined and used by push/pull/verify, so do **not** delete them.
- `${f#"$SECRETS_DIR"/}` keeps the dry-run line's store-relative form (matches the old `$project/external/...` style closely enough; the test only checks the substring `would migrate`).
- The `moved==0 && already==0` branch keeps the substring `no v1 properties blobs`. The idempotent re-run path (`moved==0, already>0`) falls through to the final `info` printing `$already already present`, preserving the `1 already present` contract.
- [ ] **Step 4: Run the new test + the full migrate suite**
Run: `bats test/migrate.bats`
Expected: the new "manifest-free" test PASSES; existing dry-run/copy-forward/idempotent/finalize tests still PASS; the "migrate with no manifest in cwd dies" test (migrate.bats:126) now FAILS (we fix it in Step 5).
- [ ] **Step 5: Update the test that codified the old dead-end**
Replace the test at `test/migrate.bats:126-132` (`"migrate with no manifest in cwd dies with a directed message"`) with:
```bash
@test "migrate in a project with no manifest and no store blobs is a clean no-op" {
make_v1_store
local dir="$WORK_DIR/nomanifest"; mkdir -p "$dir"; cd "$dir"
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
[[ "$output" == *"no v1 properties blobs"* ]] || false
}
```
- [ ] **Step 6: Run the full suite to confirm green**
Run: `bats test/migrate.bats`
Expected: all PASS.
- [ ] **Step 7: Commit**
```bash
git add secrets test/migrate.bats
git commit -m "fix: migrate copy-forward is manifest-free, no dead-end on legacy projects (EGB-710)"
```
---
## Task 2: Finalize-consistency regression test (store blob not in manifest)
**Files:**
- Test: `test/migrate.bats`
This proves the latent-bug fix: the old manifest-driven migrate skipped store blobs the manifest didn't declare, leaving them un-twinned so `--finalize` refused them. The rewrite twins them.
- [ ] **Step 1: Write the test**
Add to `test/migrate.bats`:
```bash
@test "migrate twins a store blob even when the manifest no longer declares it" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir staleblob
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push staleblob >/dev/null 2>&1
# The blob is now in the store. Drop the external from the project's manifest
# entirely (and remove the legacy file) so NO manifest declares it.
printf '{"version":2,"dotenv":[".env",".env.staging"]}\n' > .secrets.json
rm -f .secrets-files
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/staleblob/external/*.properties.age"
[ "$status" -eq 0 ] # twinned despite not being declared anywhere
}
```
- [ ] **Step 2: Run it**
Run: `bats test/migrate.bats -f "no longer declares"`
Expected: PASS (the Task 1 rewrite already makes this green — this test guards against regressing back to manifest-driven enumeration).
- [ ] **Step 3: Commit**
```bash
git add test/migrate.bats
git commit -m "test: migrate twins undeclared store blobs (finalize-consistency, EGB-710)"
```
---
## Task 3: `secrets migrate --status` survey
**Files:**
- Modify: `secrets` — add `_migrate_status()`; extend `cmd_migrate` (`secrets:2267-2285`)
- Test: `test/migrate.bats`
- [ ] **Step 1: Write the failing tests**
Add to `test/migrate.bats`:
```bash
@test "migrate --status flags a project that needs migrating" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir needsmig
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push needsmig >/dev/null 2>&1 # v1 blob, no twin yet
run "$SECRETS_BIN" migrate --status
[ "$status" -ne 0 ] # not finalize-ready
[[ "$output" == *"needsmig"* ]] || false
[[ "$output" == *"NEEDS MIGRATE"* ]] || false
[[ "$output" == *"Not finalize-ready"* ]] || false
}
@test "migrate --status reports finalize-ready once every blob is twinned" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir readymig
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push readymig >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1 # create the twin
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ]
[[ "$output" == *"Finalize-ready"* ]] || false
}
@test "migrate --status on an already-v2 store says nothing to do" {
init_with_remote
create_project_dir v2status
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ]
[[ "$output" == *"v2"* ]] || false
}
```
- [ ] **Step 2: Run them to confirm they fail**
Run: `bats test/migrate.bats -f "status"`
Expected: FAIL — `--status` is an unknown flag today (`die "Unknown migrate flag: --status…"`, status 1), so the assertions fail.
- [ ] **Step 3: Add `_migrate_status` (place it just before `_migrate_finalize`, ~`secrets:2195`)**
```bash
# Read-only survey: walk every project dir in the store and report each one's
# v2-readiness from the blobs on disk (no manifest, no decryption). Exits
# non-zero when any v1 properties blob lacks a v2 twin (i.e. the store is not
# yet finalize-ready) so it can gate scripting, mirroring `verify`'s posture.
_migrate_status() {
if [ "$(_store_format)" = "2" ]; then
info "Store format: v2 (finalized) — nothing to migrate."
return 0
fi
echo "Store format: v1 (not finalized). Per-project migration status:"
local any_untwinned=0 dir project v1 untwinned f new
for dir in "$SECRETS_DIR"/*/; do
[ -d "$dir" ] || continue
project=$(basename "$dir")
case "$project" in .*) continue ;; esac
v1=0; untwinned=0
while IFS= read -r f; do
[ -f "$f" ] || continue
v1=$((v1 + 1))
new="${f%.gradle-properties.age}.properties.age"
[ -f "$new" ] || untwinned=$((untwinned + 1))
done < <(find "$dir" -type f -name '*.gradle-properties.age' 2>/dev/null)
if [ "$v1" -eq 0 ]; then
echo " $project: v2-ready (no v1 properties blobs)"
elif [ "$untwinned" -eq 0 ]; then
echo " $project: migrated ($v1 v1 blob(s), all twinned)"
else
echo " $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'"
any_untwinned=1
fi
done
echo
if [ "$any_untwinned" -eq 1 ]; then
echo "Not finalize-ready: migrate the projects marked NEEDS MIGRATE, then run 'secrets migrate --finalize'."
return 1
fi
echo "Finalize-ready: every v1 properties blob has a v2 twin. Run 'secrets migrate --finalize' once every machine is upgraded."
return 0
}
```
- [ ] **Step 4: Wire `--status` into `cmd_migrate`**
In `cmd_migrate()` (`secrets:2267`), add the `status` local and parse + dispatch. Replace:
```bash
local dry_run=false finalize=false force=false
while [ $# -gt 0 ]; do
case "$1" in
--dry-run) dry_run=true; shift ;;
--finalize) finalize=true; shift ;;
--yes) force=true; shift ;;
-*) die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --finalize] [--yes]" ;;
*) die "migrate takes no project argument. Run it from inside a project (copy-forward) or use --finalize (store-wide)." ;;
esac
done
if [ "$finalize" = true ]; then
_migrate_finalize "$force"
else
_migrate_project "$dry_run"
fi
```
with:
```bash
local dry_run=false finalize=false force=false status=false
while [ $# -gt 0 ]; do
case "$1" in
--dry-run) dry_run=true; shift ;;
--finalize) finalize=true; shift ;;
--status) status=true; shift ;;
--yes) force=true; shift ;;
-*) die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --status | --finalize] [--yes]" ;;
*) die "migrate takes no project argument. Run it from inside a project (copy-forward), or use --status / --finalize (store-wide)." ;;
esac
done
if [ "$status" = true ]; then
_migrate_status
elif [ "$finalize" = true ]; then
_migrate_finalize "$force"
else
_migrate_project "$dry_run"
fi
```
- [ ] **Step 5: Run the status tests + full suite**
Run: `bats test/migrate.bats`
Expected: all PASS.
- [ ] **Step 6: Commit**
```bash
git add secrets test/migrate.bats
git commit -m "feat: secrets migrate --status surveys per-project v2 readiness (EGB-710)"
```
---
## Task 4: Docs, help text, version, changelog
**Files:**
- Modify: `secrets``cmd_help` (`secrets:2308-2309`)
- Modify: `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`
- [ ] **Step 1: Update `cmd_help` migrate lines**
In `cmd_help()` replace the two migrate lines (`secrets:2308-2309`):
```
secrets migrate [--dry-run] Copy-forward this project's blobs to store format v2
secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify)
```
with:
```
secrets migrate [--dry-run] Copy-forward this project's v1 blobs to store format v2
secrets migrate --status Survey every project's v2 readiness (finalize gate)
secrets migrate --finalize Drop v1 blobs and mark the store v2 (after verify)
```
- [ ] **Step 2: Update `CLAUDE.md` migrate paragraph**
In the "Store format (EGB-677 stage 2 / EGB-703)" bullet, update the migration description: copy-forward is now **manifest-free**`secrets migrate` enumerates the project's `*.gradle-properties.age` store blobs directly (same source of truth as `--finalize`), so a legacy `.secrets-files`-only project migrates without a `.secrets.json` and no store blob is left un-twinned. Add `secrets migrate --status` to the workflow line as the read-only survey that reports per-project readiness and gates `--finalize`. Reference EGB-710.
- [ ] **Step 3: Update `README.md`**
Find the migrate section/help block and add the `--status` line and the manifest-free note, mirroring the help text.
- [ ] **Step 4: Bump `VERSION`**
Set `VERSION` to `0.6.1.0`.
- [ ] **Step 5: Add `CHANGELOG.md` entry**
Insert above `## [0.6.0.1]`:
```markdown
## [0.6.1.0] - 2026-06-08
### Changed
- **`secrets migrate` copy-forward is now manifest-free (EGB-710)** — the
per-project step enumerates the store's `*.gradle-properties.age` blobs
directly (the same source of truth `--finalize` uses) instead of reading
`.secrets.json`. A legacy `.secrets-files`-only project now migrates cleanly
instead of dead-ending with "No .secrets.json", and a store blob the manifest
no longer declares still gets a v2 twin (so `--finalize` won't refuse it).
Running migrate in a project with no v1 properties blobs is a clean no-op.
### Added
- **`secrets migrate --status`** — a read-only survey that walks every project
in the store and reports its v2 readiness (v2-ready / migrated / NEEDS
MIGRATE), then whether the store as a whole is finalize-ready. Exits non-zero
while any v1 blob is un-twinned, so it can gate the path to `--finalize`.
```
- [ ] **Step 6: Run the full bats suite (all files)**
Run: `bats test/`
Expected: all PASS (secrets.bats + manifest.bats + migrate.bats). Confirm the count went up by the tests added here.
- [ ] **Step 7: Commit**
```bash
git add secrets CLAUDE.md README.md VERSION CHANGELOG.md
git commit -m "docs: migrate --status + manifest-free copy-forward; bump 0.6.1.0 (EGB-710)"
```
---
## Self-review against the EGB-710 spec
- **AC: dotenv-only project exits 0 "already v2-shaped"** → Task 1 (`moved==0 && already==0` branch, substring `no v1 properties blobs`). Test: "no manifest and no store blobs is a clean no-op" (Task 1 Step 5).
- **AC: `.secrets-files`-only project with v1 blobs migrates instead of the generic error** → the manifest-free rewrite makes it *just migrate* (better than guiding to push first). Test: "manifest-free" (Task 1 Step 1). Note the rewrite supersedes the ticket's "offer to run absorb then migrate" branch — migrate no longer needs the manifest, so there's nothing to prompt for; the only destructive step (`--finalize`) keeps its existing `/dev/tty` confirmation.
- **AC: interactive prompts read `/dev/tty`, degrade non-interactively, bash 3.2** → no new prompt is introduced (copy-forward is non-destructive); `--finalize`'s existing `/dev/tty` confirm is untouched. `--status` is pure read-only output. All new code is bash 3.2 (no associative arrays, `[[ ]]` only in tests with `|| false`).
- **AC: no behavior change to copy-forward/finalize safety gates** → finalize untouched; copy-forward stays non-destructive (v1 kept, commit + push twins). Verified by the unchanged finalize tests (migrate.bats:168-214).
- **AC: bats coverage per branch** → Tasks 1-3 add: manifest-free migrate, no-op no-blobs, undeclared-blob twinning, `--status` needs-migrate / finalize-ready / already-v2.
- **Stretch: `--status` survey** → Task 3, delivered.
- **Sequencing: independent of the EGB-709 gate** → confirmed; this only touches migrate ergonomics and helps operators *reach* all-v2. EGB-703 safety posture (recovery tag, verify-green gate, twin-before-drop) is preserved.
## Operator-local follow-up (not part of this plan)
This repo's `.ship-policy.json` opts out of AI security/red-team passes; before any PR, ask the operator to run `./test/run-security.sh` and complete the SIGNOFF. After landing, the operator can re-run their real-store migration for `onefinalmessage` et al. (`secrets migrate` per project → `secrets migrate --finalize`), which is the path to clearing the EGB-709 v2 gate.

View file

@ -0,0 +1,300 @@
# EGB-713: Version-skew nudge Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: superpowers:executing-plans / subagent-driven-development. Steps use `- [ ]`.
**Goal:** Warn (non-fatally) when the active store was last written by a newer `secrets` version than the running client, so a behind user is told to update — the loud counterpart to EGB-712's quiet forcing function.
**Architecture:** Stamp the store with the highest writer `VERSION` seen (`$SECRETS_DIR/.secrets-writer-version`, committed, monotonic) on every store-committing write. On any store command, compare that stamp to the client's own `VERSION` (read from `$SCRIPT_DIR/VERSION`); if the stamp is newer, print a one-time stderr nudge. Legacy stores with no stamp are silent.
**Tech Stack:** bash 3.2 (`secrets`); bats. Spec/idea: ticket **EGB-713**.
## Background (verified against `main`, post-EGB-712)
- `SCRIPT_DIR` already defined at `secrets:21` (`$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)`). The repo-root `VERSION` file lives next to the script.
- Store-committing write sites (each does `git -C "$SECRETS_DIR" add -A`): `commit_and_push_secrets()` `secrets:1265` (push), rekey `secrets:1822`, migrate copy-forward `secrets:2215`, finalize no-v1 `secrets:2305`, finalize drop `secrets:2340`. Plus `cmd_init` (born store) and `cmd_rm`.
- `ensure_store_protections()` (`secrets:1144`) is shared with pull (read) — do NOT stamp there.
- `check_initialized()` (`secrets:54`) early-returns when `$SECRETS_DIR/.git` exists and is called by push/pull/list/rm/rekey/verify/migrate — the natural warning hook.
- `cmd_which()` (`secrets:1945`) prints `format: v$(_store_format)` — add the writer-version line here.
- `.gitignore` only ignores `key.txt`, so `.secrets-writer-version` commits normally.
- VERSION currently `0.7.0.0` → bump to `0.7.1.0`.
- Baseline: `bats test/` = 246 passing. New tests land in a new file `test/version.bats`.
---
## Task 1: Version helpers + comparator (TDD)
**Files:** `secrets`, `test/version.bats` (new).
- [ ] Step 1: Create `test/version.bats` testing the comparator via a tiny harness that sources the script's functions is awkward (the script runs main). Instead test through observable behavior in later tasks; for the comparator, add a hidden debug subcommand is overkill. Use this approach: test `_version_gt` indirectly by exporting it is not possible. So test the comparator by adding the helpers and a **`secrets __vercmp <a> <b>`** internal is overkill too. Decision: test the comparator's *effect* in Task 3 (warning) and Task 2 (stamp monotonicity), which exercise it end-to-end. For Task 1, write the helpers and verify with a one-off `bash -c` sourcing guard.
Add to `test/version.bats`:
```bash
load test_helper
# Exercises the comparator through a bash subshell that defines the same logic
# the script uses, guarding the numeric (not lexical) ordering contract.
@test "version comparator orders 0.7.0.0 < 0.10.0.0 numerically" {
run bash -c '
_version_gt() {
local a="$1" b="$2" i ai bi; local -a af bf
IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b"
for i in 0 1 2 3; do
ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0
bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0
if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi
if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi
done; return 1
}
_version_gt 0.10.0.0 0.7.0.0 && echo "10gt7"
_version_gt 0.7.0.0 0.10.0.0 || echo "7not_gt_10"
_version_gt 0.7.0.0 0.7.0.0 || echo "equal_not_gt"
'
[ "$status" -eq 0 ]
[[ "$output" == *"10gt7"* ]] || false
[[ "$output" == *"7not_gt_10"* ]] || false
[[ "$output" == *"equal_not_gt"* ]] || false
}
```
- [ ] Step 2: Run `bats test/version.bats` → PASS (pins the contract the script must match).
- [ ] Step 3: Add the helpers to `secrets` (near `_store_format`, after `SCRIPT_DIR`/version constants — place after the `MANIFEST_VERSION=2` area or near `_store_format`):
```bash
# The running client's own version, read from the VERSION file shipped beside
# the script. Empty/"0.0.0.0" if absent (e.g. an odd install) — treated as
# "unknown/oldest" so a missing VERSION never triggers a spurious nudge.
_client_version() {
local v=""
[ -f "$SCRIPT_DIR/VERSION" ] && v=$(head -1 "$SCRIPT_DIR/VERSION" 2>/dev/null | tr -d '\r\n[:space:]')
printf '%s' "${v:-0.0.0.0}"
}
# Numeric four-field (MAJOR.MINOR.PATCH.MICRO) compare. Returns 0 iff $1 > $2.
# Per-field numeric (so 0.10.0.0 > 0.7.0.0); missing/garbage fields → 0.
_version_gt() {
local a="$1" b="$2" i ai bi; local -a af bf
IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b"
for i in 0 1 2 3; do
ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0
bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0
if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi
if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi
done
return 1
}
WRITER_VERSION_FILE_NAME=".secrets-writer-version"
# Highest client version recorded as having written to the store (empty if the
# store predates this feature — "minus the initial builds", silent by design).
_store_writer_version() {
local f="$SECRETS_DIR/$WRITER_VERSION_FILE_NAME"
[ -f "$f" ] && head -1 "$f" 2>/dev/null | tr -d '\r\n[:space:]'
}
```
- [ ] Step 4: `bash -n secrets` parses; `bats test/` still 246 + 1 (the comparator test) = 247.
- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: version helpers + numeric comparator (EGB-713)"`
---
## Task 2: Stamp the writer-version on write (TDD)
**Files:** `secrets`, `test/version.bats`.
- [ ] Step 1: Add tests:
```bash
@test "push stamps the store writer-version with the client version" {
init_with_remote
create_project_dir wvstamp
"$SECRETS_BIN" push wvstamp >/dev/null 2>&1
[ -f "$SECRETS_DIR/.secrets-writer-version" ]
run cat "$SECRETS_DIR/.secrets-writer-version"
[ "$output" = "$(cat "$(dirname "$SECRETS_BIN")/VERSION")" ]
}
@test "writer-version stamp is monotonic (a push never lowers a higher stamp)" {
init_with_remote
create_project_dir wvmono
printf '9.9.9.9\n' > "$SECRETS_DIR/.secrets-writer-version"
"$SECRETS_BIN" push wvmono >/dev/null 2>&1
run cat "$SECRETS_DIR/.secrets-writer-version"
[ "$output" = "9.9.9.9" ] # not lowered to the client's version
}
@test "writer-version stamp is committed, not gitignored" {
init_with_remote
create_project_dir wvcommit
"$SECRETS_BIN" push wvcommit >/dev/null 2>&1
run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-writer-version"
[ "$status" -eq 0 ]
}
```
- [ ] Step 2: Run `bats test/version.bats -f "stamp"` → FAIL (no stamping yet).
- [ ] Step 3: Add the stamp helper (after `_store_writer_version`):
```bash
# Raise the store's recorded writer-version to the client's version (monotonic;
# never lowers it). Called right before each store-committing `git add -A` so
# the stamp rides the same commit. Read paths (pull) never call this.
_stamp_writer_version() {
local cur cli
cur=$(_store_writer_version)
cli=$(_client_version)
if [ -z "$cur" ] || _version_gt "$cli" "$cur"; then
printf '%s\n' "$cli" > "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME"
fi
}
```
- [ ] Step 4: Call `_stamp_writer_version` immediately before each store-committing `git -C "$SECRETS_DIR" add -A`:
- `secrets:1265` (in `commit_and_push_secrets`, before `git add -A`)
- `secrets:1822` (rekey)
- `secrets:2215` (migrate copy-forward)
- `secrets:2305` (finalize, no-v1 path)
- `secrets:2340` (finalize, drop path)
Also in `cmd_init`, after the store repo is created and before its first commit (so a born store records its version), and in `cmd_rm` before its commit.
Each insertion is the single line ` _stamp_writer_version` at the matching indentation directly above the `git ... add -A` (or before the `git ... commit` where there's no add -A, e.g. rm/init — there, stamp then ensure it's staged via the existing add/commit).
- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 250 (247 + 3).
- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: stamp store writer-version on write, monotonic (EGB-713)"`
---
## Task 3: Skew warning on command (TDD)
**Files:** `secrets`, `test/version.bats`.
- [ ] Step 1: Add tests:
```bash
@test "a store written by a newer version warns on a command (non-fatal)" {
init_with_remote
create_project_dir skewwarn
"$SECRETS_BIN" push skewwarn >/dev/null 2>&1
printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" list
[ "$status" -eq 0 ] # non-fatal
[[ "$output" == *"newer"* || "$output" == *"update"* ]] || false
}
@test "a store at the same/older version is silent" {
init_with_remote
create_project_dir noskew
"$SECRETS_BIN" push noskew >/dev/null 2>&1 # stamp == client version
run "$SECRETS_BIN" list
[ "$status" -eq 0 ]
[[ "$output" != *"update your secrets"* ]] || false
}
@test "a store with no writer-version marker is silent (legacy store)" {
init_with_remote
create_project_dir legacynostamp
"$SECRETS_BIN" push legacynostamp >/dev/null 2>&1
rm -f "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" list
[ "$status" -eq 0 ]
[[ "$output" != *"update your secrets"* ]] || false
}
```
- [ ] Step 2: Run `bats test/version.bats -f "skew\|silent\|legacy"` → the "newer" test FAILS (no warning yet).
- [ ] Step 3: Add the skew check (after `_stamp_writer_version`):
```bash
# Warn ONCE per invocation if the store was last written by a newer client than
# us. Non-fatal (read/write paths keep their exit codes). Silent when the store
# carries no writer-version (legacy) or is same/older than us.
_VERSION_SKEW_WARNED=0
_check_store_version_skew() {
[ "$_VERSION_SKEW_WARNED" = 1 ] && return 0
local sv cv
sv=$(_store_writer_version)
[ -n "$sv" ] || return 0
cv=$(_client_version)
if _version_gt "$sv" "$cv"; then
_VERSION_SKEW_WARNED=1
echo "NOTE: this store was last written by secrets v$sv; you're on v$cv." >&2
echo " Update your secrets tool: git -C \"$SCRIPT_DIR\" pull" >&2
fi
return 0
}
```
- [ ] Step 4: Hook it into `check_initialized` — change `secrets:55-57`:
```bash
if [ -d "$SECRETS_DIR/.git" ]; then
return
fi
```
to:
```bash
if [ -d "$SECRETS_DIR/.git" ]; then
_check_store_version_skew
return
fi
```
- [ ] Step 5: `bats test/version.bats` → all pass. `bats test/` → 253.
- [ ] Step 6: Commit: `git add secrets test/version.bats && git commit -m "feat: warn on store version skew (once per invocation, EGB-713)"`
---
## Task 4: `secrets which` surfaces the writer-version (TDD)
**Files:** `secrets`, `test/version.bats`.
- [ ] Step 1: Add test:
```bash
@test "which prints the store writer-version and a behind note" {
init_with_remote
create_project_dir whichwv
"$SECRETS_BIN" push whichwv >/dev/null 2>&1
printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"written-by: v99.0.0.0"* ]] || false
[[ "$output" == *"behind"* || "$output" == *"update"* ]] || false
}
```
- [ ] Step 2: Run → FAIL (which doesn't print written-by).
- [ ] Step 3: In `cmd_which`, after the `echo "format: v$(_store_format)"` line (`secrets:1951`), add:
```bash
local _wv; _wv=$(_store_writer_version)
if [ -n "$_wv" ]; then
local _cv; _cv=$(_client_version)
if _version_gt "$_wv" "$_cv"; then
echo "written-by: v$_wv (you're on v$_cv — behind; run: git -C \"$SCRIPT_DIR\" pull)"
else
echo "written-by: v$_wv"
fi
fi
```
Note: `cmd_which` calls `resolve_store` but may not call `check_initialized`, so this also avoids double-printing the skew NOTE; the `which` line is the dedicated surface.
- [ ] Step 4: `bats test/version.bats` → pass. `bats test/` → 254.
- [ ] Step 5: Commit: `git add secrets test/version.bats && git commit -m "feat: secrets which shows store writer-version + behind note (EGB-713)"`
---
## Task 5: Docs + version bump
**Files:** `secrets` (cmd_help unchanged unless adding a note), `CLAUDE.md`, `README.md`, `VERSION`, `CHANGELOG.md`.
- [ ] Step 1: `CLAUDE.md` — add to the "Store format" bullet a sentence on the writer-version: a committed `.secrets-writer-version` records the highest client `VERSION` that has written (monotonic, stamped on store-committing writes); commands warn once (stderr, non-fatal) when the store's stamp exceeds the running client, and `secrets which` shows `written-by: vN`. Legacy stores (no marker) are silent. (EGB-713.)
- [ ] Step 2: `README.md` — under the upgrading section, note that an out-of-date `secrets` prints a one-line "update" nudge when it touches a store newer than itself.
- [ ] Step 3: `VERSION``0.7.1.0`.
- [ ] Step 4: `CHANGELOG.md` — new `## [0.7.1.0] - 2026-06-08` with an Added entry for the version-skew nudge + `secrets which` writer-version line.
- [ ] Step 5: `bats test/` → all green (254). `./secrets which` against a scratch store renders (covered by tests).
- [ ] Step 6: Commit: `git add secrets CLAUDE.md README.md VERSION CHANGELOG.md && git commit -m "docs: version-skew nudge + writer-version; bump 0.7.1.0 (EGB-713)"`
---
## Self-review vs ticket AC
- "Newer stamp → nudge; same/older → silent" → Task 3.
- "No marker → silent (legacy)" → Task 3 + `_store_writer_version` empty.
- "Monotonic, committed" → Task 2.
- "Numeric comparator (0.7.0.0 < 0.10.0.0)" Task 1.
- "Non-fatal, never changes read/pull exit codes" → Task 3 (`_check_store_version_skew` always `return 0`).
- "`which` surfaces it" → Task 4. "Warn once per invocation" → `_VERSION_SKEW_WARNED` guard.
- bash 3.2: `read -a`, `local -a`, `10#`, parameter strips — all 3.2-safe.

File diff suppressed because it is too large Load diff

View file

@ -0,0 +1,216 @@
# Additive v2 — automatic dual-write (no flag-day finalize)
**Status:** Design approved (2026-06-08), pending implementation plan.
**Supersedes the framing of:** EGB-703 ("finalize is the destructive milestone you must reach") and EGB-709 ("delete v1 paths once every client is v2").
**Relates to:** EGB-677 (storage-model unification), EGB-710 (manifest-free migrate).
## Problem
Store-format v2 (EGB-703) renamed the `properties` external blob suffix
(`.gradle-properties.age``.properties.age`) and made the transition a
three-step migration ending in a **destructive** `secrets migrate --finalize`
that drops the v1 blobs. Finalize is gated on "every machine and store in use is
v2" — a coordination requirement that's trivial for a solo dev but **uncertain
for a distributed team of 23+**: there is no reliable "all-clear" signal, and
finalizing early silently cuts an un-upgraded client off from `properties`
externals (it looks for `.gradle-properties.age`, which is gone).
The hard gate exists **only** because finalize permanently drops the v1 blobs.
Remove the obligation to drop, and there is nothing to coordinate.
## Decision
Adopt **Strategy B — Additive v2**: "becoming v2" stops being a destructive
milestone and becomes a property an upgraded client maintains **automatically**.
Reads try both suffixes; writes keep alive whatever old clients already knew.
There is no required `migrate` ceremony and no required `finalize` — **upgrading
the `secrets` tool *is* the migration.** `finalize` survives only as optional,
indefinitely-deferrable garbage collection.
Direction chosen over the alternatives: "park it / never finalize" (leaves the
two-format wart in place, resolves nothing) and "abandon the suffix rename"
(reverts shipped EGB-703 behavior, keeps the ugly `gradle-properties` suffix
forever). Additive v2 is the only option that reaches a clean v2 end-state
*without* a coordinated flag-day.
## Design
### 1. Read resolution — suffix-agnostic
Any upgraded client resolving a `properties` external blob looks for the v2
suffix `.properties.age` first, then falls back to the v1 `.gradle-properties.age`.
An upgraded client therefore **never fails to find a blob** regardless of which
suffix is on disk. Old (pre-0.6.0.0) clients still read v1-only — shipped code
cannot be changed.
Code: today's `_external_blob_suffix(type)` (a single suffix string) is replaced
by a resolver that returns the **path of the blob that exists** for a given
`(project, slug, type)`, trying v2 then v1. `file` externals are unchanged in
both formats, so the resolver is a no-op identity for them (single suffix
`.file.age`).
### 2. Write rule — the self-managing twin rule
On push of a `properties` external, the client checks the store for an existing
v1 twin (`<slug>.gradle-properties.age`):
- **v1 twin exists** (the external was first pushed by an old client) →
**dual-write both suffixes.** Old clients stay fresh forever; nobody is cut off.
- **No v1 twin exists** (brand-new external, first pushed by an upgraded client) →
**write v2-only (`.properties.age`).** Old clients cannot see it → the gentle,
*intended* forcing function. It only ever bites on genuinely new externals,
never on anything that previously worked.
No stored state is required: the presence/absence of the v1 twin **is** the
signal. Mental model: *"keep alive what old clients already knew; new things are
v2-only."*
### 3. The marker — unchanged meaning, NOT auto-stamped
Additive-v2 read/write behavior does **not** depend on the `.secrets-format`
marker at all (reads try both suffixes; writes follow the twin rule). So the
marker is left exactly as it is today: set only by `init` (born-v2, a fresh pure-v2
store) or `migrate --finalize` (a store GC'd to pure v2). A transitional store
that upgraded clients are dual-writing stays **markerless (v1)** — which is
*accurate*: it has not been finalized to pure v2, and v1 twins still exist.
Push deliberately does **not** auto-stamp the marker. (An earlier draft proposed
auto-stamping on first push; that was dropped during planning because it would
make a v1 store read as v2 the moment anyone pushed — turning every `migrate`
into a no-op and contradicting the "v1 until finalized" model the whole
migration relies on. The marker's only purposes — `secrets which` display and
gating `migrate`/`finalize` — are better served by it continuing to mean
"finalized/pure v2.")
`secrets which` continues to report `format: vN` from the marker.
### 4. `finalize` → optional GC, never required
`secrets migrate --finalize` remains the *only* operation that stops dual-writing
and drops the v1 twins. It is pure space reclamation (kilobytes), still
coordination-gated **if** you choose to run it, but never obligatory and
deferrable forever. Its existing safety posture is unchanged (recovery tag,
`verify --all` green gate, twin-before-drop, `--yes`/operator confirmation). This
is what defuses the gate: the destructive step still exists but is now optional
housekeeping, not a release blocker.
### 5. `migrate` / `migrate --status` → normalize + inspect
- `secrets migrate` (the EGB-710 manifest-free copy-forward) stays as an optional
"backfill v2 twins for existing v1-only externals" command — useful right
before a `finalize`. With read-fallback (§1) it is no longer required for
correctness, only for tidiness.
- `secrets migrate --status` becomes the **coverage survey**: per external, which
suffixes exist, and whether old clients are still being served (i.e. whether a
v1 twin is still present and being dual-written). This is the operator's
dashboard for "is anyone still relying on v1?" before an optional `finalize`.
### 6. Propagation semantics (→ user docs verbatim)
"Has not migrated" means **old client** (`secrets` < 0.6.0.0), not "hasn't run
`migrate`". `migrate` is a per-store op done once by anyone; what protects a
teammate is their **client version**. A read-only teammate needs only the tool
`git pull` (binary ≥ 0.6.0.0), not to run `migrate` themselves. "Upgrade your
secrets" (git pull the tool and/or migrate the store) is the correct umbrella —
and for a solo dev the two are one motion.
Does a teammate on an old client get a secret User 1 just added?
| Secret type | Blob name v1 vs v2 | Old client gets the new secret? |
|---|---|---|
| **dotenv** (`.env`, `.env.*`, `.dev.vars`) | identical (`.env.age`) | **Yes, always.** No forcing function possible — the blob name never changed. |
| **`file` external** (keystore, etc.) | identical (`.file.age`) | **Yes, always.** |
| **`properties` external** — existing (has a v1 twin) | dual-written | **Yes.** Old client reads the maintained `.gradle-properties.age`. |
| **`properties` external** — brand-new (no v1 twin) | v2-only | **No → must upgrade.** The forcing function; rare, and never breaks anything that previously worked. |
Net: a teammate on an old client keeps getting **all** everyday `.env` updates
indefinitely (a good safety property — no one silently misses everyday secrets),
and only hits a wall on a genuinely new `properties`-style external.
### 7. Known caveat (documented, not engineered around)
Dual-write keeps v1 *readers* fresh, but a v1 *writer* writes only
`.gradle-properties.age`. A v2 reader (reading `.properties.age` first) could
therefore read stale data until that external is re-pushed by an upgraded client.
For the normal shape — one writer per external, who upgrades first — it never
bites. This matches today's reality and is documented rather than solved
(solving it would require timestamp/newest-wins arbitration across two encrypted
blobs — YAGNI for v1).
### 8. Backward-compatibility matrix
| Actor | dotenv / `file` | `properties` (existing twin) | `properties` (new, v2-only) |
|---|---|---|---|
| Upgraded client reads | ✓ | ✓ (resolver finds either) | ✓ |
| Upgraded client writes | unchanged | dual-writes both | writes v2-only |
| Old client reads | ✓ | ✓ (reads maintained v1 twin) | ✗ (forcing function) |
| Old client writes | unchanged | writes v1 twin only (see §7) | n/a (can't create v2) |
## Code-level surface (for the implementation plan)
- **`_external_blob_suffix(type)`** → split into:
- `_resolve_external_blob_read(project, slug, type)` — returns the path of the
blob that exists, trying `.properties.age` then `.gradle-properties.age` for
`properties`; identity for `file`. Used by `pull_external_files`, `verify`,
and any read path.
- `_external_blob_write_targets(project, slug, type)` — returns the suffix
path(s) to write: for `properties`, both suffixes when a v1 twin already
exists, else v2-only; single path for `file`.
- **`push_external_files`** — write to every path from
`_external_blob_write_targets` (was a single `age -o`). Push does **not** stamp
the marker (see §3).
- **`pull_external_files` / `cmd_verify`** — resolve blobs via
`_resolve_external_blob_read` (was the single-suffix lookup).
- **`_migrate_finalize`** — unchanged logic; doc/help reframed as optional GC.
- **`_migrate_project` / `_migrate_status`** — retained (EGB-710); `--status`
extended to report dual-write coverage per external.
- **Docs:** CLAUDE.md "Store format" bullet, README, `cmd_help`, CHANGELOG,
VERSION bump (minor — new write semantics).
## Safety / constraints (unchanged invariants)
- bash 3.2 portable; every store walk stays recursive (`find -type f`).
- Path-validation rails (`_validate_external_target_path`, slug derivation,
symlink/`..` refusal) untouched.
- `finalize`'s destructive gating (recovery tag, verify-green, twin-before-drop,
`--yes`) untouched.
- The store still carries no manifest; `.secrets.json` remains per-project.
## Test plan (bats, outline)
1. Read-fallback: a v2 client resolves a `properties` external that exists only
as `.gradle-properties.age` (no twin) — pull succeeds.
2. Twin rule — existing twin → dual-write: push an external that has a v1 twin;
assert **both** suffixes are written and an old-client read path (v1 suffix)
sees the fresh value.
3. Twin rule — new external → v2-only: push a brand-new `properties` external;
assert **only** `.properties.age` is written (no v1 twin created) — the
forcing function.
4. Marker NOT stamped on push: a `make_v1_store` + push leaves `.secrets-format`
absent (store stays v1/transitional); the existing `make_v1_store; push;
migrate` flow is unaffected.
5. dotenv/`file` unaffected: a dotenv and a `file` external round-trip identically
regardless of store marker.
6. `migrate --status` coverage: reports which externals are dual-written vs
v2-only.
7. `finalize` still green: existing finalize gates and drop behavior unchanged.
8. Caveat is observable (optional): a v1-suffix-only update is read by the v2
client via fallback (documents the one-writer assumption).
## Ripple to the roadmap
- **EGB-703**: "finalize is the destructive milestone you must reach" → "finalize
is optional GC." Update the ticket/notes.
- **EGB-709** (collapse v1 paths): no longer gates on "all clients v2." Becomes
"delete the dual-write/transitional code **if/when** every store is GC'd to
pure v2" — much later, low stakes. Add a note to EGB-709.
- A new ticket should track this work (additive-v2 dual-write).
## Decided knobs (no longer open)
- Migration ceremony: **automatic** (upgraded client dual-writes on push; no
required `migrate`). `migrate`/`--status` remain manual/inspection tools.
- Forcing function: **kept**, scoped to brand-new `properties` externals via the
twin rule (existing twins always dual-written).
- `finalize`: **kept** as optional GC (not removed), so a fully-upgraded store
can still be reclaimed to pure v2.

View file

@ -0,0 +1,230 @@
# EGB-283 — Multi-recipient age encryption
**Date:** 2026-06-24
**Issue:** [EGB-283](https://linear.app/egbt/issue/EGB-283) — secrets: multi-recipient age encryption (multiple keys per file)
**Status:** Design approved, ready for implementation plan
**Related:** EGB-281 (multi-store), EGB-677/EGB-703 (manifest + store-format v2)
## Problem
Today every blob in a store is encrypted to exactly one age public key (`age -r "$pubkey"`,
where `$pubkey` is derived from the store's single `key.txt`). The whole team shares one
private key. EGB-281's multi-store work lets users separate work/personal/client secrets into
different stores, each with its own key — but *within* a single store there is still only one key,
so onboarding/offboarding a teammate means sharing or rotating one secret by hand.
age natively supports multiple recipients: `age -r KEY1 -r KEY2 -o file.age input` writes one
recipient stanza per key, and any matching identity decrypts. This lets a single store have N
members, each with their own keypair. Adding/removing a teammate becomes a re-encrypt against the
current recipient set — no shared password.
## Goals
- A store can encrypt every blob to N recipient public keys.
- Recipient set is managed with first-class commands (`secrets recipients add/rm/list`).
- The recipient set is **singular and consistent per store**: every blob is always readable by
exactly the current set.
- Fully backward compatible: existing single-key stores keep working untouched; the feature is
opt-in and detected by file presence (no store-format-marker bump).
- Decryption path is unchanged (members use their own `key.txt`).
## Non-goals (YAGNI — explicit scope cuts)
- **SSH recipients** (`ssh-ed25519` / `ssh-rsa`). Native age X25519 keys cover the team-key use
case; SSH adds a parsing/format axis. Clean future follow-up.
- **Per-file or per-project recipient subsets.** The whole store shares one recipient set.
- **Key discovery / distribution.** Public keys are pasted in out of band, exactly as `key.txt`
is shared today.
- **Merging recipients into a project-level config** (`.secrets.json` / `.secrets-files`). See
"Why recipients are not in the project manifest" below.
## Design decisions (resolved during brainstorming)
1. **Storage:** committed `recipients.txt` at the store root, managed via
`secrets recipients add/rm/list` subcommands.
2. **Re-encrypt scope:** `add`/`rm` re-encrypt the **entire store immediately** to the new set in
one commit. The store is always consistent.
3. **Backward compatibility:** absence of `recipients.txt` ⇒ exact current single-key behavior.
First `recipients add` on a legacy store bootstraps the file seeded with the local pubkey plus
the new key. `init` going forward seeds `recipients.txt` with the freshly generated pubkey
(born-multi).
4. **`rekey` semantics:** on a multi-recipient store, `rekey` becomes "re-encrypt all to the
current `recipients.txt` set" (no new keypair). On a legacy store it keeps today's behavior
(generate a new keypair, re-encrypt to it). One shared re-encrypt engine.
5. **Store config shape:** keep `recipients.txt` as its own plain, age-native file (jq-free),
alongside the existing one-line `.secrets-format` marker — matching the repo's
small-single-purpose-plain-file convention. Not folded into a JSON store-config.
## Why recipients are not in the project manifest
The tool has two config planes in two different git repos:
| Plane | Location | Files | Scope |
| ----------- | -------------------------------- | -------------------------------------------------- | --------------------------- |
| **Project** | `$PWD` (the project's own repo) | `.secrets.json` (absorbs legacy `.secrets-files`), `.secrets-store` | *What this project syncs* |
| **Store** | `$SECRETS_DIR` (`~/.secrets`) | `.secrets-format`, **`recipients.txt`** (new) | *Metadata about the encrypted repo* |
Recipients are **store-scoped** — who can decrypt *this store*, shared by every project in it.
Putting them in a project-level manifest would let each project carry its own copy and **diverge**,
the exact inconsistency the "always re-encrypt the whole store to one set" rule prevents. It also
collides with the deliberate EGB-703 decision that *the store holds no project manifest*. So the
recipient set lives with the store, next to `.secrets-format`.
## `recipients.txt` format and security rails
Lives at `$SECRETS_DIR/recipients.txt`, **committed** (public keys are not secret; the store
`.gitignore` only blocks `key.txt` and plaintext env files, so the file is tracked automatically).
age `-R` format: one recipient per line, `# comment` and blank lines allowed.
```
# alice (laptop)
age1ql3z7hjy54pw3hyww5ayyfg7zqgvc7w3j2elw8zmrj2kg5sfn9qsxxxxxx
# bob
age1lggyhqrw2nlhcxprm67z43rta597azn8gknawjehu9d9dl0jq3yqxxxxxx
```
**We do not pass the file path to `age -R`.** A committed file is an injection surface, so the
tool parses it itself into a validated indexed array `RECIPIENT_ARGS=(-r age1… -r age1…)`,
mirroring the conservative posture of `.secrets-store` / `.secrets-files`:
- Each non-comment, non-blank line (after trim) must match a native age X25519 recipient exactly:
`^age1[0-9a-z]{58}$`. Anything else — SSH recipients, shell metacharacters, whitespace inside the
token, control/ANSI characters, `-r`-injection look-alikes — is **rejected with a clear error**.
No shell expansion, ever.
- A symlinked `recipients.txt` is refused (same rail as the manifests).
- `--name` labels (written as `# <name>` comment lines above the key) are restricted to
`[A-Za-z0-9 ._-]`; anything else is rejected. This blocks comment-injection into the file.
- The parser is pure bash (bash-3.2 safe). Indexed arrays are fine on bash 3.2; only *associative*
arrays are bash-4.
Validation is the single source of truth — both the `recipients` subcommands and `_load_recipients`
(below) route through the same validator, so an externally hand-edited malicious file is caught on
the next encrypt, not just at `add` time.
## Components
### `_load_recipients()` — populate `RECIPIENT_ARGS`
Called once per command that encrypts. Populates the global indexed array `RECIPIENT_ARGS`:
- `recipients.txt` present → validated array of every key in the file (error out on any invalid
line; refuse an empty/all-comment file).
- absent (legacy store) → `(-r <derived-local-pubkey>)`, identical to today's single-recipient
behavior.
### `_reencrypt_all()` — shared re-encrypt engine
Factored out of today's `cmd_rekey` decrypt→re-encrypt loop:
1. Decrypt every `*.age` in the store (recursive `find -type f -name '*.age'`, covering nested
manifest dotenv blobs and `external/` blobs) with the local `key.txt` into a tmpdir. The
operator must be a current recipient; a decryption failure aborts with the old state preserved.
2. Re-encrypt each file with `age "${RECIPIENT_ARGS[@]}"` back to its relpath.
3. `ensure_store_protections`, `git add -A`, commit, push (if a remote exists).
All recipient-changing paths call it:
| Command | Behavior |
| ------------------------------- | -------------------------------------------------------------------- |
| `recipients add` / `rm` | edit `recipients.txt``_load_recipients``_reencrypt_all` |
| `rekey` (multi-recipient store) | `_reencrypt_all` to current set, **no new keypair** |
| `rekey` (legacy store) | today's behavior: generate new keypair, set recipients to it, re-encrypt |
| `reencrypt` (new, idempotent) | `_reencrypt_all` — heal/backfill after a manual edit |
### `secrets recipients` subcommand
- `recipients list` — prints names + keys from `recipients.txt` (read-only, jq-free). On a legacy
store, prints the single derived pubkey with a "single-key (no recipients.txt)" note.
- `recipients add <age1…> [--name <label>]` — validates the key, refuses duplicates, appends
(with the optional `# <label>` comment), bootstraps the file with `{local pubkey, new key}` if
the store is still legacy, then `_reencrypt_all`.
- `recipients rm <age1…|name>` — removes the matching entry, then `_reencrypt_all`. **Guards:**
refuses to remove the last recipient; removing *your own* key (which would lock you out of future
pulls) requires `--yes`. Removal takes effect going forward — git history can't be un-shared
(documented, same caveat as today's `rekey`).
### Encryption call-site changes
Every existing `age -r "$pubkey" -o …` site becomes `age "${RECIPIENT_ARGS[@]}" -o …`:
- `push_dir_to_project` (dotenv) and the nested manifest dotenv path
- `push_external_files` (external `properties` and `file` blobs)
- `cmd_init` (seeds the store; born-multi)
- `cmd_rekey` re-encrypt loop (now `_reencrypt_all`)
Functions that currently take `pubkey` as a positional arg are updated to rely on the
`RECIPIENT_ARGS` global populated by `_load_recipients` at command entry, avoiding array-passing
gymnastics on bash 3.2. **Decryption paths (`pull`, `verify`) are unchanged** — `age -d -i
"$KEY_FILE"` already tries the identity against all recipient stanzas.
### `init` / `which` / `verify` integration
- **`init`** writes `recipients.txt` seeded with the freshly generated pubkey (born-multi), staged
like `.secrets-format`.
- **`which`** prints a `recipients: N (alice, bob, …)` line, or `recipients: single-key (no
recipients.txt)` for a legacy store.
- **`verify`** already decrypt-tests with the local key — works as-is for a member. **Added cheap
invariant:** count the `-> X25519` recipient stanzas in each blob header and assert it equals the
number of entries in `recipients.txt`. age exposes no way to list *which* recipients a file
targets (X25519 stanzas are ephemeral), so a count check is the strongest machine-checkable
invariant; full-identity consistency relies on the always-re-encrypt-all rule. The check is
skipped on legacy stores (no `recipients.txt`).
## Data flow
**Onboarding a teammate**
1. Teammate runs `age-keygen` locally, sends their **public** key out of band.
2. An existing member: `secrets recipients add age1theirpub --name them` → store re-encrypts to
`{existing…, them}` in one commit, pushed.
3. Teammate clones the store repo, drops their own `key.txt` in place, and `secrets pull` works —
their key matches one stanza in every blob.
**Offboarding**
1. `secrets recipients rm them` → store re-encrypts to the remaining set, pushed. New blobs are no
longer readable by the removed key. (Historical git revisions remain readable by their old key —
rotate any still-sensitive secret values, same as today.)
## Error handling
- Invalid/duplicate key on `add` → reject before any re-encrypt; store untouched.
- Invalid line discovered by `_load_recipients` during any encrypt → abort the command with a clear
pointer to the offending line; nothing written.
- `_reencrypt_all` decryption failure (operator not a current recipient, or corrupt blob) → abort,
old store state preserved (mirrors today's `rekey` safety).
- `rm` last recipient → refused. `rm` own key → requires `--yes`.
- Empty/all-comment `recipients.txt` → treated as an error (a store with zero recipients can encrypt
nothing).
## Testing
New `test/recipients.bats` suite plus additions to existing suites:
- **Happy path:** `init` born-multi; `add` bootstraps a legacy store; `add`/`rm` round-trip; a blob
encrypted to 3 keys decrypts with each of the 3 identities; `list` output; `rekey` on a
multi-recipient store keeps the set and generates no new key; `reencrypt` is idempotent.
- **Backward compat:** a legacy store (no `recipients.txt`) still pushes/pulls/rekeys exactly as
before; an old single-key client decrypts a recipients.txt-store where its key is a recipient.
- **`verify`:** stanza-count invariant passes on a healthy multi-recipient store and flags a blob
whose recipient count drifted.
- **Security regression fixtures** (`recipients.txt` is a new committed attack surface): non-age /
malformed keys, shell metacharacters, control/ANSI characters, `-r`-injection look-alikes,
symlinked `recipients.txt`, bad `--name`, remove-last-recipient, remove-self-without-`--yes`.
Per the repo security-review policy (`.ship-policy.json`, CLAUDE.md): these adversarial fixtures are
written as **ordinary bats regression tests**, not AI red-team/adversarial-review passes. Before any
ship/PR, the human operator runs `./test/run-security.sh` and completes the SIGNOFF prompt;
`./test/run-security.sh` is operator-local and is **not** run on the user's behalf.
## Backward compatibility / migration
No store-format-marker bump. Multi-recipient is purely additive and detected by the **presence** of
`recipients.txt`, mirroring the `.secrets-format` "absence implies v1" pattern. Existing stores keep
working with zero action; a store becomes multi-recipient the first time `recipients add` (or `init`
on a fresh store) writes `recipients.txt`.
## Documentation
- `CLAUDE.md` Architecture section: add a multi-recipient bullet (store-scoped `recipients.txt`,
`recipients` subcommand, shared `_reencrypt_all` engine, `rekey` dual semantics, security rails).
- `README.md`: onboarding/offboarding a teammate; `recipients add/rm/list`.
- Test counts in `CLAUDE.md` Project Structure updated.
## Open questions
None blocking. (SSH-recipient support and a JSON store-config remain possible future follow-ups,
explicitly out of scope here.)

113
install.sh Executable file
View file

@ -0,0 +1,113 @@
#!/usr/bin/env bash
#
# secrets — thin onboarding bootstrap (EGB-671).
#
# This script ships INSIDE the repo: you already cloned the repo to get it, so
# its only jobs are (1) verify the dependencies the tool needs and (2) print the
# exact commands to finish setup. It deliberately does NOT:
# - edit your shell rc files (it prints the PATH line for you to paste)
# - invoke sudo or install packages behind your back (it prints the command)
# - re-implement any of the tool's security logic
#
# This is a security tool whose whole pitch is "verify, don't trust" — so the
# installer holds itself to a higher bar than convenience, not a lower one.
#
# Usage:
# ./install.sh # check deps, print setup + next steps
# ./install.sh --help
set -euo pipefail
# Resolve the directory this script lives in (the cloned tool repo). Uses bash
# builtins only so it works under a minimal PATH.
_src="${BASH_SOURCE[0]}"
TOOL_DIR="$(cd "${_src%/*}" 2>/dev/null && pwd)"
usage() {
cat <<EOF
install.sh — finish setting up the 'secrets' tool.
Run this once after cloning the repo. It verifies dependencies (age, jq, git)
and prints the commands to put 'secrets' on your PATH and onboard a machine.
Usage:
./install.sh Check dependencies and print setup + next steps
./install.sh --help Show this help
It never edits your shell config and never runs sudo — it prints the exact
commands so you stay in control (this is a secrets tool, after all).
Onboarding after setup:
First machine: secrets init --remote <your-private-repo-url>
Other machine: secrets join --remote <your-private-repo-url> --key <key.txt>
EOF
}
# Print the install command for a package, using whatever package manager is
# present. For sudo-requiring managers we PRINT the line for you to run — the
# installer never escalates on its own.
install_hint() {
local pkg="$1"
if command -v brew >/dev/null 2>&1; then
echo "brew install $pkg"
elif command -v apt-get >/dev/null 2>&1; then
echo "sudo apt-get install -y $pkg"
elif command -v dnf >/dev/null 2>&1; then
echo "sudo dnf install -y $pkg"
else
echo "install '$pkg' with your system package manager"
fi
}
case "${1:-}" in
--help|-h) usage; exit 0 ;;
"") ;;
*) echo "Unknown option: $1" >&2; usage >&2; exit 2 ;;
esac
echo "secrets — bootstrap check (tool dir: $TOOL_DIR)"
echo ""
# Dependency check. age + jq + git are all load-bearing on the cold-start path:
# jq became required once .secrets.json (manifest) is JSON, so it must be present
# BEFORE the first manifest read.
missing=0
for dep in git age jq; do
if command -v "$dep" >/dev/null 2>&1; then
echo " ok $dep"
else
echo " MISSING $dep — install it with:"
echo " $(install_hint "$dep")"
missing=1
fi
done
echo ""
if [ "$missing" -ne 0 ]; then
echo "Install the missing dependencies above, then re-run ./install.sh." >&2
exit 1
fi
cat <<EOF
All dependencies present. Two steps to finish:
1) Put 'secrets' on your PATH. Add this line to your shell config
(~/.zshrc or ~/.bashrc), then restart your terminal:
export PATH="$TOOL_DIR:\$PATH"
2) Onboard this machine:
First machine (new vault):
secrets init --remote <your-private-repo-url>
# then transfer key.txt to your other machines (AirDrop / scp / USB):
# scp <this-host>:$HOME/.secrets/key.txt ~/.secrets/key.txt
Other machine (join an existing vault):
secrets join --remote <your-private-repo-url> --key <path-to-key.txt>
# 'join' clones the vault, installs the key, and VERIFIES it decrypts
# before declaring success — a mis-copied key fails loudly, not silently.
To update the tool later:
git -C "$TOOL_DIR" pull
EOF

2658
secrets

File diff suppressed because it is too large Load diff

72
test/install.bats Normal file
View file

@ -0,0 +1,72 @@
#!/usr/bin/env bats
# EGB-671: install.sh thin bootstrap. It ships IN the repo (you clone the repo
# to get it), so its job is: verify deps (age + jq + git), PRINT the PATH line
# and next-step commands — never edit dotfiles, never invoke sudo. Security-rail
# concerns are operator-local (.ship-policy.json); these are functional checks.
load test_helper
INSTALL_SH="$(cd "$(dirname "${BATS_TEST_FILENAME}")/.." && pwd)/install.sh"
@test "install.sh exists and is executable" {
[ -f "$INSTALL_SH" ]
[ -x "$INSTALL_SH" ]
}
@test "install.sh --help prints usage and exits 0" {
run "$INSTALL_SH" --help
[ "$status" -eq 0 ]
[[ "$output" == *"install.sh"* ]] || false
[[ "$output" == *"join"* ]] || false
}
@test "install.sh prints the PATH export line for the tool dir (does not edit rc)" {
local tool_dir
tool_dir="$(cd "$(dirname "$INSTALL_SH")" && pwd)"
run "$INSTALL_SH"
[ "$status" -eq 0 ]
[[ "$output" == *"export PATH="* ]] || false
[[ "$output" == *"$tool_dir"* ]] || false
# It must NOT have written to any shell rc in the isolated HOME.
[ ! -f "$HOME/.zshrc" ]
[ ! -f "$HOME/.bashrc" ]
}
@test "install.sh prints both onboarding next-steps (init --remote and join)" {
run "$INSTALL_SH"
[ "$status" -eq 0 ]
[[ "$output" == *"secrets init --remote"* ]] || false
[[ "$output" == *"secrets join --remote"* ]] || false
}
@test "install.sh prints the upgrade one-liner" {
run "$INSTALL_SH"
[ "$status" -eq 0 ]
[[ "$output" == *"git -C"* ]] || false
[[ "$output" == *"pull"* ]] || false
}
@test "install.sh prints a key-transfer hint" {
run "$INSTALL_SH"
[ "$status" -eq 0 ]
[[ "$output" == *"key.txt"* ]] || false
}
@test "install.sh never invokes sudo (prints it for the user instead)" {
# No executed 'sudo' — any sudo reference must be quoted guidance text.
run grep -nE '^[[:space:]]*sudo ' "$INSTALL_SH"
[ "$status" -ne 0 ]
}
@test "install.sh reports a missing dependency with an install hint and non-zero exit" {
# Build a minimal PATH that has the tools install.sh needs but NOT jq.
local fake="$TEST_TMPDIR/fakebin"
mkdir -p "$fake"
for t in bash uname env cat grep sed tr dirname command age git printf; do
src="$(command -v "$t" 2>/dev/null || true)"
[ -n "$src" ] && ln -sf "$src" "$fake/$t" 2>/dev/null || true
done
run env PATH="$fake" "$INSTALL_SH"
[ "$status" -ne 0 ]
[[ "$output" == *"jq"* ]] || false
}

145
test/join.bats Normal file
View file

@ -0,0 +1,145 @@
#!/usr/bin/env bats
# EGB-671: `secrets join` (second-machine onboarding) + `secrets init --remote`
# + day-2 silent-decrypt fix. Functional paths only — security-rail tests
# (path traversal on --key/--store, URL injection) are operator-local per
# .ship-policy.json and live in test/run-security.sh.
load test_helper
# Push a project to REMOTE_DIR and save the key, then remove the local store
# to simulate a fresh second machine. Leaves: REMOTE_DIR has blobs,
# $TEST_TMPDIR/saved-key.txt is the decrypting key, $SECRETS_DIR is gone.
_machine1_push_then_wipe() {
init_with_remote
cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt"
create_project_dir "joinproj"
"$SECRETS_BIN" push >/dev/null 2>&1
cd "$HOME"
rm -rf "$SECRETS_DIR"
}
# Like above but never pushes a project — remote has a store with zero blobs.
_machine1_empty_then_wipe() {
init_with_remote
cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt"
cd "$HOME"
rm -rf "$SECRETS_DIR"
}
# ─── secrets join ────────────────────────────────────────────────────────
@test "join without --remote fails with usage" {
run "$SECRETS_BIN" join
[ "$status" -ne 0 ]
[[ "$output" == *"--remote"* ]] || false
}
@test "join clones the store, installs the key at 600, verifies, and succeeds" {
_machine1_push_then_wipe
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt"
[ "$status" -eq 0 ]
[[ "$output" == *"VERIFIED"* ]] || false
[ -d "$SECRETS_DIR/.git" ]
[ -f "$SECRETS_DIR/key.txt" ]
# key installed at mode 600
local perms
perms=$(stat -f '%Lp' "$SECRETS_DIR/key.txt" 2>/dev/null || stat -c '%a' "$SECRETS_DIR/key.txt")
[ "$perms" = "600" ]
}
@test "join with the wrong key fails loudly and does not report VERIFIED" {
_machine1_push_then_wipe
age-keygen -o "$TEST_TMPDIR/wrong-key.txt" 2>/dev/null
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/wrong-key.txt"
[ "$status" -ne 0 ]
[[ "$output" != *"VERIFIED"* ]] || false
}
@test "join against an empty store reports nothing-to-verify, NOT VERIFIED" {
_machine1_empty_then_wipe
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt"
[ "$status" -eq 0 ]
[[ "$output" == *"nothing to verify"* ]] || false
[[ "$output" != *"VERIFIED"* ]] || false
}
@test "join refuses when a store already exists at the target" {
"$SECRETS_BIN" init >/dev/null 2>&1
cp "$SECRETS_DIR/key.txt" "$TEST_TMPDIR/saved-key.txt"
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/saved-key.txt"
[ "$status" -ne 0 ]
[[ "$output" == *"already"* ]] || false
}
@test "join fails clearly when the key file is missing" {
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR/nope.txt"
[ "$status" -ne 0 ]
[[ "$output" == *"key"* ]] || false
}
@test "join detects a directory passed as --key" {
_machine1_push_then_wipe
run "$SECRETS_BIN" join --remote "$REMOTE_DIR" --key "$TEST_TMPDIR"
[ "$status" -ne 0 ]
[[ "$output" == *"key"* ]] || false
}
# ─── secrets init --remote ────────────────────────────────────────────────
@test "init --remote sets origin and establishes an upstream branch" {
run "$SECRETS_BIN" init --remote "$REMOTE_DIR"
[ "$status" -eq 0 ]
run git -C "$SECRETS_DIR" remote get-url origin
[ "$status" -eq 0 ]
[ "$output" = "$REMOTE_DIR" ]
# upstream branch exists on the remote (so a later push won't ff-only die)
run git -C "$SECRETS_DIR" rev-parse --abbrev-ref '@{u}'
[ "$status" -eq 0 ]
}
@test "init --remote then push does not die on the brand-new remote" {
"$SECRETS_BIN" init --remote "$REMOTE_DIR" >/dev/null 2>&1
create_project_dir "freshproj"
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
[[ "$output" != *"Fast-forward pull failed"* ]] || false
}
@test "init with no flags still works (clean primitive)" {
run "$SECRETS_BIN" init
[ "$status" -eq 0 ]
[ -f "$SECRETS_DIR/key.txt" ]
}
@test "init does not hang on the first-add prompt when stdin is a tty but stdout is captured" {
# Regression: run-security.sh runs bats in a real terminal, so the command's
# stdin stays a tty while bats captures its stdout. The interactive first-add
# prompt must NOT fire in that shape (it gates on stdout being a tty too),
# or the whole suite hangs. Reproduce with a pty via `script`.
command -v script >/dev/null 2>&1 || skip "script (pty) not available"
# macOS/BSD syntax: `script -q <file> <cmd...>`. Skip on other syntaxes.
script -q /dev/null true >/dev/null 2>&1 || skip "unsupported script syntax"
local out="$TEST_TMPDIR/pty-initout"
run timeout 10 script -q /dev/null bash -c "'$SECRETS_BIN' init > '$out' 2>&1"
[ "$status" -ne 124 ] # 124 == timeout == it hung on a prompt
run grep -c "Add a project's secrets" "$out"
[ "$output" = "0" ]
}
# ─── day-2 silent-decrypt fix ─────────────────────────────────────────────
@test "pull dies loudly when a blob cannot be decrypted with the current key" {
init_with_remote
create_project_dir "decryptproj"
"$SECRETS_BIN" push >/dev/null 2>&1
# Swap in a different key so the stored blob no longer decrypts.
# (age-keygen refuses to overwrite, so generate elsewhere then copy.)
age-keygen -o "$TEST_TMPDIR/other-key.txt" 2>/dev/null
cp "$TEST_TMPDIR/other-key.txt" "$SECRETS_DIR/key.txt"
chmod 600 "$SECRETS_DIR/key.txt"
cd "$WORK_DIR/decryptproj"
rm -f .env .env.staging
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
[[ "$output" == *"decrypt"* ]] || false
}

1000
test/manifest.bats Normal file

File diff suppressed because it is too large Load diff

468
test/migrate.bats Normal file
View file

@ -0,0 +1,468 @@
#!/usr/bin/env bats
# EGB-703 store-format-v2: marker, format-aware suffix, migrate (dry-run /
# copy-forward / finalize). bash 3.2: every standalone [[ ]] ends with || false.
load test_helper
# A v1 (legacy) store: born-v2 init, then strip the marker so it reads as v1
# and pushes write the legacy .gradle-properties.age suffix.
make_v1_store() {
init_with_remote
rm -f "$SECRETS_DIR/.secrets-format"
}
# Simulate an old (v1) client's properties blob: copy the pushed v2
# .properties.age to its v1 .gradle-properties.age twin (KEEPS both present).
m_fake_v1_twin() {
local proj="$1" v2
v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age)
cp "$v2" "${v2%.properties.age}.gradle-properties.age"
}
# Like m_fake_v1_twin but RENAMES (leaves ONLY the v1 blob) — for copy-forward fixtures.
m_make_v1_only() {
local proj="$1" v2
v2=$(ls "$SECRETS_DIR/$proj/external/"*.properties.age)
mv "$v2" "${v2%.properties.age}.gradle-properties.age"
}
m_gradle_src() { mkdir -p "$HOME/.gradle"; printf '%s' "$1" > "$HOME/.gradle/gradle.properties"; }
m_file_src() { mkdir -p "$HOME/keystores"; printf 'KS\x00\x01\x02\xffDATA\n' > "$HOME/keystores/upload.keystore"; }
# ─── Format marker + format-aware suffix (increment 1) ────────────────
@test "init stamps the store format marker as v2 (born-v2)" {
init_with_remote
[ -f "$SECRETS_DIR/.secrets-format" ]
[ "$(cat "$SECRETS_DIR/.secrets-format")" = "2" ]
}
@test "which prints format v2 for a born-v2 store" {
init_with_remote
create_project_dir whichv2
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"format: v2"* ]] || false
}
@test "which prints format v1 for a markerless (legacy) store" {
make_v1_store
create_project_dir whichv1
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"format: v1"* ]] || false
}
@test "push on a born-v2 store writes the properties blob as .properties.age" {
init_with_remote
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir v2push
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push v2push >/dev/null 2>&1
run bash -c "ls $SECRETS_DIR/v2push/external/*.properties.age"
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/v2push/external/*.gradle-properties.age 2>/dev/null"
[ "$status" -ne 0 ]
}
@test "push on a v1 store writes the v2 suffix for a fresh external (additive v2)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir v1push
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push v1push >/dev/null 2>&1
run bash -c "ls $SECRETS_DIR/v1push/external/*.properties.age"
[ "$status" -eq 0 ]
}
@test "push writes the v2 suffix for a fresh external even on a v1 store" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir freshv1
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push freshv1 >/dev/null 2>&1
run bash -c "ls $SECRETS_DIR/freshv1/external/*.properties.age"
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/freshv1/external/*.gradle-properties.age 2>/dev/null"
[ "$status" -ne 0 ]
}
@test "push dual-writes the v1 twin so old clients stay fresh" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_old\n'
create_project_dir dualwrite
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push dualwrite >/dev/null 2>&1
m_fake_v1_twin dualwrite
m_gradle_src $'beaconClerkPkTest=pk_test_new\n'
"$SECRETS_BIN" push dualwrite >/dev/null 2>&1
rm -f "$SECRETS_DIR/dualwrite/external/"*.properties.age
rm -f "$HOME/.gradle/gradle.properties"
"$SECRETS_BIN" pull dualwrite >/dev/null 2>&1
run grep -q 'beaconClerkPkTest=pk_test_new' "$HOME/.gradle/gradle.properties"
[ "$status" -eq 0 ]
}
@test "the format marker is committed, not gitignored" {
init_with_remote
create_project_dir markercommit
"$SECRETS_BIN" push markercommit >/dev/null 2>&1
run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-format"
[ "$status" -eq 0 ]
}
@test "pull reads a v1-suffix properties blob on a v2 store (read-fallback)" {
init_with_remote # born-v2 store (marker=2)
m_gradle_src $'beaconClerkPkTest=pk_test_v1\n'
create_project_dir rffallback
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push rffallback >/dev/null 2>&1 # writes .properties.age on a v2 store
# Simulate an external that exists only in the v1 suffix (an old client wrote it):
local v2blob; v2blob=$(ls "$SECRETS_DIR/rffallback/external/"*.properties.age)
mv "$v2blob" "${v2blob%.properties.age}.gradle-properties.age"
rm -f "$HOME/.gradle/gradle.properties"
"$SECRETS_BIN" pull rffallback >/dev/null 2>&1
run grep -q 'beaconClerkPkTest=pk_test_v1' "$HOME/.gradle/gradle.properties"
[ "$status" -eq 0 ]
}
# ─── migrate --dry-run / copy-forward (increment 2) ───────────────────
@test "migrate --dry-run reports the rename and writes nothing" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir dryproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push dryproj >/dev/null 2>&1
m_make_v1_only dryproj
run "$SECRETS_BIN" migrate --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"would migrate"* ]] || false
# nothing written
run bash -c "ls $SECRETS_DIR/dryproj/external/*.properties.age 2>/dev/null"
[ "$status" -ne 0 ]
# marker still absent (store still v1)
[ ! -f "$SECRETS_DIR/.secrets-format" ]
}
@test "migrate --dry-run on a dotenv-only project reports nothing to migrate" {
make_v1_store
create_project_dir dotenvonly
"$SECRETS_BIN" push dotenvonly >/dev/null 2>&1
run "$SECRETS_BIN" migrate --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"0 blob(s) would be copy-forwarded"* ]] || false
}
@test "migrate copy-forward creates the v2 twin and keeps the v1 blob (byte-identical)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir cfproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push cfproj >/dev/null 2>&1
m_make_v1_only cfproj
local old; old=$(ls "$SECRETS_DIR/cfproj/external/"*.gradle-properties.age)
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
local new; new=$(ls "$SECRETS_DIR/cfproj/external/"*.properties.age)
[ -f "$old" ] # v1 kept (non-destructive)
[ -f "$new" ] # v2 twin written
cmp -s "$old" "$new" # byte-identical ciphertext copy
}
@test "migrate copy-forward is idempotent" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir idemproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push idemproj >/dev/null 2>&1
m_make_v1_only idemproj
"$SECRETS_BIN" migrate >/dev/null 2>&1
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
[[ "$output" == *"1 already present"* ]] || false
run bash -c "ls $SECRETS_DIR/idemproj/external/*.properties.age | wc -l | tr -d ' '"
[ "$output" = "1" ]
}
@test "migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)" {
# The EGB-710 repro: a legacy project has a v1 properties blob in the store
# but no .secrets.json (it predates the manifest). migrate must NOT dead-end.
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir nomanifestblob
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push nomanifestblob >/dev/null 2>&1
m_make_v1_only nomanifestblob
rm -f .secrets.json # simulate a pre-manifest project
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.properties.age"
[ "$status" -eq 0 ] # v2 twin written despite no manifest
run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.gradle-properties.age"
[ "$status" -eq 0 ] # v1 kept (non-destructive)
}
@test "migrate twins a store blob even when the manifest no longer declares it" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir staleblob
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push staleblob >/dev/null 2>&1
m_make_v1_only staleblob
# The blob is now in the store. Drop the external from the project's manifest
# entirely (and remove the legacy file) so NO manifest declares it.
printf '{"version":2,"dotenv":[".env",".env.staging"]}\n' > .secrets.json
rm -f .secrets-files
run bash -c "ls $SECRETS_DIR/staleblob/external/*.gradle-properties.age"
[ "$status" -eq 0 ] # precondition: the v1 blob exists in the store
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/staleblob/external/*.properties.age"
[ "$status" -eq 0 ] # twinned despite not being declared anywhere
}
@test "migrate in a project with no manifest and no store blobs is a clean no-op" {
make_v1_store
local dir="$WORK_DIR/nomanifest"; mkdir -p "$dir"; cd "$dir"
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
[[ "$output" == *"no v1 properties blobs"* ]] || false
}
@test "migrate on an already-v2 store is a no-op" {
init_with_remote
create_project_dir alreadyv2
run "$SECRETS_BIN" migrate
[ "$status" -eq 0 ]
[[ "$output" == *"already format v2"* ]] || false
}
@test "migrate unknown flag dies with usage" {
init_with_remote
create_project_dir mgflag
run "$SECRETS_BIN" migrate --bogus
[ "$status" -eq 1 ]
[[ "$output" == *"Unknown migrate flag"* ]] || false
}
@test "migrate leaves dotenv and file blobs untouched" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
m_file_src
create_project_dir mixproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\nfile ~/keystores/upload.keystore\n' > .secrets-files
"$SECRETS_BIN" push mixproj >/dev/null 2>&1
local envblob; envblob=$(ls "$SECRETS_DIR/mixproj/".env.age)
local fileblob; fileblob=$(ls "$SECRETS_DIR/mixproj/external/"*.file.age)
local envsum; envsum=$(cksum "$envblob")
local filesum; filesum=$(cksum "$fileblob")
"$SECRETS_BIN" migrate >/dev/null 2>&1
[ "$(cksum "$envblob")" = "$envsum" ] # dotenv blob unchanged
[ "$(cksum "$fileblob")" = "$filesum" ] # file blob unchanged
}
# ─── migrate --finalize (increment 3) ─────────────────────────────────
@test "finalize refuses when verify --all is not green" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir failverify
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push failverify >/dev/null 2>&1
m_make_v1_only failverify
"$SECRETS_BIN" migrate >/dev/null 2>&1
# corrupt the v2 twin so verify --all fails
printf 'garbage' > "$SECRETS_DIR/failverify/external/"*.properties.age
run "$SECRETS_BIN" migrate --finalize --yes
[ "$status" -eq 1 ]
[[ "$output" == *"not green"* ]] || false
# marker not stamped; v1 blob still present
[ ! -f "$SECRETS_DIR/.secrets-format" ]
run bash -c "ls $SECRETS_DIR/failverify/external/*.gradle-properties.age"
[ "$status" -eq 0 ]
}
@test "finalize refuses an un-twinned v1 blob (project not migrated)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir untwinned
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push untwinned >/dev/null 2>&1
m_make_v1_only untwinned
# do NOT migrate — leave the v1 blob with no twin
run "$SECRETS_BIN" migrate --finalize --yes
[ "$status" -eq 1 ]
[[ "$output" == *"no v2 twin"* ]] || false
run bash -c "ls $SECRETS_DIR/untwinned/external/*.gradle-properties.age"
[ "$status" -eq 0 ]
}
@test "finalize green path drops v1, keeps v2, stamps the marker" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir finproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push finproj >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1
run "$SECRETS_BIN" migrate --finalize --yes
[ "$status" -eq 0 ]
[ "$(cat "$SECRETS_DIR/.secrets-format")" = "2" ]
run bash -c "ls $SECRETS_DIR/finproj/external/*.properties.age"
[ "$status" -eq 0 ]
run bash -c "ls $SECRETS_DIR/finproj/external/*.gradle-properties.age 2>/dev/null"
[ "$status" -ne 0 ]
}
@test "finalize cuts a recovery tag before deleting v1 blobs" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir tagproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push tagproj >/dev/null 2>&1
m_make_v1_only tagproj
"$SECRETS_BIN" migrate >/dev/null 2>&1
"$SECRETS_BIN" migrate --finalize --yes >/dev/null 2>&1
local tag; tag=$(git -C "$SECRETS_DIR" tag | grep '^pre-v2-migrate-')
[ -n "$tag" ]
# the tagged commit still contains the v1 blob (tag cut before delete)
run bash -c "git -C $SECRETS_DIR ls-tree -r --name-only $tag | grep -q gradle-properties.age"
[ "$status" -eq 0 ]
}
@test "finalize without --yes aborts when not confirmed" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir confproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push confproj >/dev/null 2>&1
m_make_v1_only confproj
"$SECRETS_BIN" migrate >/dev/null 2>&1
run bash -c "echo '' | $SECRETS_BIN migrate --finalize"
[ "$status" -eq 1 ]
[[ "$output" == *"aborted"* ]] || false
[ ! -f "$SECRETS_DIR/.secrets-format" ]
}
@test "verify is green during the migration window (v2 twin is not a spurious orphan)" {
# Regression: after copy-forward the store is still v1, so verify computed the
# external blob path as .gradle-properties.age and flagged the .properties.age
# twin as an orphan, failing verify mid-migration. The orphan set now accounts
# for both suffix forms.
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir windowverify
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push windowverify >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1
run "$SECRETS_BIN" verify windowverify
[ "$status" -eq 0 ]
}
@test "v1 client still reads during the migration window (after copy-forward, before finalize)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir windowproj
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push windowproj >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1
# store is still v1 (markerless); pull uses the .gradle-properties.age blob
rm "$HOME/.gradle/gradle.properties"
run "$SECRETS_BIN" pull windowproj
[ "$status" -eq 0 ]
grep -q '^beaconClerkPkTest=pk_test_abc$' "$HOME/.gradle/gradle.properties"
}
@test "post-finalize pull reads the v2 blob" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir postfin
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push postfin >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1
"$SECRETS_BIN" migrate --finalize --yes >/dev/null 2>&1
rm "$HOME/.gradle/gradle.properties"
run "$SECRETS_BIN" pull postfin
[ "$status" -eq 0 ]
grep -q '^beaconClerkPkTest=pk_test_abc$' "$HOME/.gradle/gradle.properties"
}
@test "finalize on a v1 dotenv-only store stamps the marker (no v1 blobs to drop)" {
make_v1_store
create_project_dir dotenvfin
"$SECRETS_BIN" push dotenvfin >/dev/null 2>&1
# no gradle-properties externals → v1count==0 path; no confirmation needed
run "$SECRETS_BIN" migrate --finalize
[ "$status" -eq 0 ]
[[ "$output" == *"no v1 blobs"* ]] || false
[ "$(cat "$SECRETS_DIR/.secrets-format")" = "2" ]
}
@test "migrate with a positional argument dies" {
init_with_remote
create_project_dir mgpos
run "$SECRETS_BIN" migrate someproject
[ "$status" -eq 1 ]
[[ "$output" == *"no project argument"* ]] || false
}
@test "finalize on an already-v2 store is a no-op" {
init_with_remote
create_project_dir finv2
run "$SECRETS_BIN" migrate --finalize --yes
[ "$status" -eq 0 ]
[[ "$output" == *"already format v2"* ]] || false
}
@test "migrate --status flags a project that needs migrating" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir needsmig
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push needsmig >/dev/null 2>&1 # v1 blob, no twin yet
m_make_v1_only needsmig
run "$SECRETS_BIN" migrate --status
[ "$status" -ne 0 ] # not finalize-ready
[[ "$output" == *"needsmig"* ]] || false
[[ "$output" == *"NEEDS MIGRATE"* ]] || false
[[ "$output" == *"Not finalize-ready"* ]] || false
}
@test "migrate --status reports finalize-ready once every blob is twinned" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir readymig
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push readymig >/dev/null 2>&1
"$SECRETS_BIN" migrate >/dev/null 2>&1 # create the twin
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ]
[[ "$output" == *"Finalize-ready"* ]] || false
}
@test "migrate --status counts v2-only externals (old clients not served)" {
make_v1_store
m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
create_project_dir v2onlyext
printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
"$SECRETS_BIN" push v2onlyext >/dev/null 2>&1 # v2-only (fresh, no v1 twin)
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ] # no v1 blobs -> finalize-ready
[[ "$output" == *"v2-only"* ]] || false # surfaced as v2-only coverage
}
@test "migrate --status on an already-v2 store says nothing to do" {
init_with_remote
create_project_dir v2status
run "$SECRETS_BIN" migrate --status
[ "$status" -eq 0 ]
[[ "$output" == *"v2"* ]] || false
[[ "$output" == *"nothing to migrate"* ]] || false
}
@test "_store_format reads a garbage marker as v1 (strict parse)" {
make_v1_store
create_project_dir garbagemarker
# a non-"2" marker (e.g. a truncated/garbled value) must read as v1, not v2
printf 'v2-ish-garbage\n' > "$SECRETS_DIR/.secrets-format"
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"format: v1"* ]] || false
}

407
test/recipients.bats Normal file
View file

@ -0,0 +1,407 @@
#!/usr/bin/env bats
load test_helper
# A throwaway second identity for "another teammate".
make_second_identity() {
age-keygen -o "$TEST_TMPDIR/bob.txt" 2>/dev/null
BOB_PUB=$(age-keygen -y "$TEST_TMPDIR/bob.txt")
}
@test "push with only-self recipients.txt encrypts to the store key (born-multi)" {
init_with_remote
# init now seeds recipients.txt with self — born-multi store.
[ -e "$SECRETS_DIR/recipients.txt" ]
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
# Blob must still decrypt with the store's own key.
run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "push with a hand-written recipients.txt encrypts to every listed key" {
init_with_remote
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '# self\n%s\n# bob\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
# Bob (a recipient) can decrypt the pushed blob with HIS key.
run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
# And the store key still can too.
run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "push refuses a recipients.txt with an invalid key" {
init_with_remote
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\nnot-an-age-key\n' "$STORE_PUB" > "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid recipient"* ]] || false
}
@test "push refuses a symlinked recipients.txt" {
init_with_remote
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\n' "$STORE_PUB" > "$TEST_TMPDIR/elsewhere.txt"
# Remove the born-multi recipients.txt so we can replace it with a symlink.
rm -f "$SECRETS_DIR/recipients.txt"
ln -s "$TEST_TMPDIR/elsewhere.txt" "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"symlink"* ]] || false
}
@test "recipients list on a legacy store shows the single derived key" {
init_with_remote
# Simulate a legacy store by removing the born-multi recipients.txt.
rm -f "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" recipients list
[ "$status" -eq 0 ]
[[ "$output" == *"single-key"* ]] || false
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
[[ "$output" == *"$STORE_PUB"* ]] || false
}
@test "recipients list shows names and keys from recipients.txt" {
init_with_remote
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '# alice\n%s\n# bob\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" recipients list
[ "$status" -eq 0 ]
[[ "$output" == *"recipients: 2"* ]] || false
[[ "$output" == *"alice"* ]] || false
[[ "$output" == *"bob"* ]] || false
}
@test "reencrypt re-encrypts existing blobs to a newly added recipient line" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push # single-key blob (project name = "myproj"; blob at $SECRETS_DIR/myproj/.env.age)
[ "$status" -eq 0 ]
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt"
# Bob cannot read the old single-key blob yet.
run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -ne 0 ]
run "$SECRETS_BIN" reencrypt
[ "$status" -eq 0 ]
# Now he can.
run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "rekey on a multi-recipient store keeps recipients and the same key" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push
before=$(cat "$SECRETS_DIR/key.txt")
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" rekey
[ "$status" -eq 0 ]
# No new keypair was generated.
[ "$(cat "$SECRETS_DIR/key.txt")" = "$before" ]
# Both recipients can decrypt.
run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "rekey on a legacy store still rotates to a new key (unchanged)" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push
before=$(cat "$SECRETS_DIR/key.txt")
# Simulate a legacy store by removing the born-multi recipients.txt.
rm -f "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" rekey
[ "$status" -eq 0 ]
[ "$(cat "$SECRETS_DIR/key.txt")" != "$before" ]
run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "recipients add bootstraps a legacy store and re-encrypts" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push
# Simulate a legacy store (no recipients.txt) so recipients add triggers the
# bootstrap branch (if [ ! -e "$RECIPIENTS_FILE" ]) rather than the append path.
rm -f "$SECRETS_DIR/recipients.txt"
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
[ "$status" -eq 0 ]
[ -e "$SECRETS_DIR/recipients.txt" ]
# recipients.txt now has self + bob (2 keys).
run "$SECRETS_BIN" recipients list
[[ "$output" == *"recipients: 2"* ]] || false
[[ "$output" == *"bob"* ]] || false
# Existing blob re-encrypted: bob can read it.
run age -d -i "$TEST_TMPDIR/bob.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "recipients add rejects a non-age key" {
init_with_remote
run "$SECRETS_BIN" recipients add "ssh-ed25519 AAAAfoo"
[ "$status" -ne 0 ]
[[ "$output" == *"valid age recipient"* ]] || false
}
@test "recipients add rejects a duplicate" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
[ "$status" -eq 0 ]
run "$SECRETS_BIN" recipients add "$BOB_PUB"
[ "$status" -ne 0 ]
[[ "$output" == *"already present"* ]] || false
}
@test "recipients add rejects an unsafe --name" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name 'bob; rm -rf ~'
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid --name"* ]] || false
}
@test "recipients add rejects a whitespace-only --name" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name ' '
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid --name"* ]] || false
}
@test "recipients add --name with no value errors" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name
[ "$status" -ne 0 ]
[[ "$output" == *"--name requires a value"* ]] || false
}
@test "recipients rm removes a recipient and re-encrypts to the rest" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
run "$SECRETS_BIN" recipients rm bob
[ "$status" -eq 0 ]
run "$SECRETS_BIN" recipients list
[[ "$output" == *"recipients: 1"* ]] || false
# Store key still reads its own blobs.
run age -d -i "$SECRETS_DIR/key.txt" "$SECRETS_DIR/myproj/.env.age"
[ "$status" -eq 0 ]
}
@test "recipients rm refuses to remove the last recipient" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob # store = self + bob
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
run "$SECRETS_BIN" recipients rm bob # back to self only
[ "$status" -eq 0 ]
run "$SECRETS_BIN" recipients rm "$STORE_PUB" # would be the last
[ "$status" -ne 0 ]
[[ "$output" == *"last recipient"* ]] || false
}
@test "recipients rm of your own key requires --yes" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
run "$SECRETS_BIN" recipients rm "$STORE_PUB"
[ "$status" -ne 0 ]
[[ "$output" == *"your own key"* ]] || false
run "$SECRETS_BIN" recipients rm "$STORE_PUB" --yes
[ "$status" -eq 0 ]
}
@test "recipients rm of a non-existent target errors" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
run "$SECRETS_BIN" recipients rm carol
[ "$status" -ne 0 ]
[[ "$output" == *"No recipient matches"* ]] || false
}
@test "init seeds recipients.txt with the new store key (born-multi)" {
run "$SECRETS_BIN" init
[ "$status" -eq 0 ]
[ -e "$SECRETS_DIR/recipients.txt" ]
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
run cat "$SECRETS_DIR/recipients.txt"
[[ "$output" == *"$STORE_PUB"* ]] || false
}
@test "which reports the recipient count" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
create_project_dir myproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"recipients: 2"* ]] || false
[[ "$output" == *"bob"* ]] || false
}
@test "which reports single-key for a legacy store" {
init_with_remote
rm -f "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"recipients: single-key"* ]] || false
}
@test "verify --all passes on a healthy multi-recipient store" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob # re-encrypts to 2
run "$SECRETS_BIN" verify --all
[ "$status" -eq 0 ]
}
@test "verify flags a blob whose recipient count drifted" {
init_with_remote
create_project_dir myproj
run "$SECRETS_BIN" push # single-key blob (1 stanza)
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
# Declare 2 recipients but do NOT re-encrypt — the on-disk blob still has 1.
printf '%s\n%s\n' "$STORE_PUB" "$BOB_PUB" > "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" verify --all
[ "$status" -ne 0 ]
[[ "$output" == *"recipient"* ]] || false
}
@test "SECURITY: a dangling symlink recipients.txt is refused, not silently ignored" {
init_with_remote
rm -f "$SECRETS_DIR/recipients.txt"
ln -s "$TEST_TMPDIR/does-not-exist.txt" "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"symlink"* ]] || false
}
@test "SECURITY: recipients.txt with shell metacharacters is rejected, no execution" {
init_with_remote
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\nage1$(touch %s/pwned)\n' "$STORE_PUB" "$TEST_TMPDIR" > "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[ ! -e "$TEST_TMPDIR/pwned" ]
[[ "$output" == *"Invalid recipient"* ]] || false
}
@test "SECURITY: recipients.txt line that looks like an extra age flag is rejected" {
init_with_remote
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\n-i /etc/passwd\n' "$STORE_PUB" > "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid recipient"* ]] || false
# The push must have aborted before encrypting — no blob should exist.
[ ! -e "$SECRETS_DIR/myproj/.env.age" ]
}
@test "SECURITY: control/ANSI characters in recipients.txt are rejected" {
init_with_remote
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\nage1%b\n' "$STORE_PUB" 'aaaa\033[31mevil' > "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid recipient"* ]] || false
}
@test "SECURITY: recipients add rejects a key with embedded whitespace" {
init_with_remote
run "$SECRETS_BIN" recipients add "age1aaaa bbbb"
[ "$status" -ne 0 ]
[[ "$output" == *"valid age recipient"* ]] || false
}
# ── Fix 1: validate-before-mutate ────────────────────────────────────────────
@test "recipients rm dies without mutating a hand-corrupted recipients.txt" {
init_with_remote
make_second_identity
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob # valid: self + bob
# Corrupt the file by hand.
printf 'age1-not-a-valid-key\n' >> "$SECRETS_DIR/recipients.txt"
before=$(cat "$SECRETS_DIR/recipients.txt")
run "$SECRETS_BIN" recipients rm bob
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid recipient"* ]] || false
# File unchanged (no half-mutation).
[ "$(cat "$SECRETS_DIR/recipients.txt")" = "$before" ]
}
@test "recipients add dies without mutating a hand-corrupted recipients.txt" {
init_with_remote
make_second_identity
printf 'age1-not-a-valid-key\n' >> "$SECRETS_DIR/recipients.txt" # init seeded self; now corrupt
before=$(cat "$SECRETS_DIR/recipients.txt")
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name bob
[ "$status" -ne 0 ]
[[ "$output" == *"Invalid recipient"* ]] || false
[ "$(cat "$SECRETS_DIR/recipients.txt")" = "$before" ]
}
# ── Fix 4: reencrypt advisory on a legacy store ───────────────────────────────
@test "reencrypt on a legacy store prints a single-key advisory" {
init_with_remote
rm -f "$SECRETS_DIR/recipients.txt"
create_project_dir myproj
run "$SECRETS_BIN" push
run "$SECRETS_BIN" reencrypt
[ "$status" -eq 0 ]
[[ "$output" == *"single-key"* ]] || false
}
# ── Fix 5: ambiguous-name rm coverage ────────────────────────────────────────
@test "recipients rm by an ambiguous name is refused" {
init_with_remote
make_second_identity
age-keygen -o "$TEST_TMPDIR/carol.txt" 2>/dev/null
CAROL_PUB=$(age-keygen -y "$TEST_TMPDIR/carol.txt")
run "$SECRETS_BIN" recipients add "$BOB_PUB" --name dup
run "$SECRETS_BIN" recipients add "$CAROL_PUB" --name dup
run "$SECRETS_BIN" recipients rm dup
[ "$status" -ne 0 ]
[[ "$output" == *"matches"* ]] || false
}
@test "SECURITY: a symlinked recipients.txt is refused on add and rm too" {
init_with_remote
make_second_identity
STORE_PUB=$(age-keygen -y "$SECRETS_DIR/key.txt")
printf '%s\n' "$STORE_PUB" > "$TEST_TMPDIR/elsewhere.txt"
rm -f "$SECRETS_DIR/recipients.txt"
ln -s "$TEST_TMPDIR/elsewhere.txt" "$SECRETS_DIR/recipients.txt"
run "$SECRETS_BIN" recipients add "$BOB_PUB"
[ "$status" -ne 0 ]
[[ "$output" == *"symlink"* ]] || false
}

65
test/run-security.sh Executable file
View file

@ -0,0 +1,65 @@
#!/usr/bin/env bash
# Local-only security regression suite. Uses attack-payload fixtures on purpose.
# Do NOT ask hosted AI agents to run this script or to perform equivalent red-team review.
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
cd "$ROOT"
if ! command -v bats >/dev/null 2>&1; then
echo "bats-core is required: brew install bats-core" >&2
exit 1
fi
# Subset of the full suite: adversarial regressions + explicit SECURITY tests +
# closely related path/injection rails. Keeps the run focused and fast.
FILTER='SECURITY|F1:|F2:|F3:|F4:|F5:|command injection does not execute|outside HOME is refused|symlinked target is refused|shell metacharacters|command-substitution|symlinked .secrets-store is skipped|symlinked .secrets-files is ignored|symlinked .secrets.json is refused'
echo "Security regression suite (operator-local only)"
echo "Repository policy: see .ship-policy.json"
echo ""
bats --filter "$FILTER" test/
echo ""
echo "All filtered security regression tests passed."
echo ""
if [ ! -t 0 ]; then
echo "Refusing non-interactive sign-off. Re-run in a terminal and complete operator certification." >&2
exit 1
fi
read -r -p "Operator name: " OPERATOR
if [ -z "${OPERATOR//[[:space:]]/}" ]; then
echo "Operator name is required." >&2
exit 1
fi
read -r -p "Type SIGNOFF to certify you ran this suite locally: " CONFIRM
if [ "$CONFIRM" != "SIGNOFF" ]; then
echo "Sign-off aborted (expected exactly SIGNOFF)." >&2
exit 1
fi
SIGNOFF_DIR="$ROOT/.gstack"
mkdir -p "$SIGNOFF_DIR"
COMMIT="$(git rev-parse HEAD 2>/dev/null || echo unknown)"
TS="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
SIGNOFF_FILE="$SIGNOFF_DIR/security-signoff.json"
# Escape operator name for JSON (minimal — names should not contain quotes).
OPERATOR_JSON="${OPERATOR//\\/\\\\}"
OPERATOR_JSON="${OPERATOR_JSON//\"/\\\"}"
cat >"$SIGNOFF_FILE" <<EOF
{
"operator": "$OPERATOR_JSON",
"signed_at": "$TS",
"commit": "$COMMIT",
"suite": "test/run-security.sh",
"filter": "$FILTER"
}
EOF
echo "Sign-off recorded at $SIGNOFF_FILE (gitignored — local only)."

File diff suppressed because it is too large Load diff

373
test/sync.bats Normal file
View file

@ -0,0 +1,373 @@
#!/usr/bin/env bats
# EGB-1230 / EGB-1231: store sync — loud failures on `pull`, and a `secrets
# sync` verb that reconciles a diverged store instead of dead-ending.
#
# bash 3.2 gotcha (see CLAUDE.md): every standalone [[ ]] assertion MUST end
# with `|| false`, or a failing assertion does not fail the test.
load test_helper
# ─── fixtures ──────────────────────────────────────────────────────────
#
# A "peer" is a second clone of the same bare remote. Committing + pushing
# from the peer is how we put the store under test *behind* its remote
# without touching the store itself.
peer_commit_and_push() {
local name="${1:-peer-file}" content="${2:-peer}"
local peer="$TEST_TMPDIR/peer"
if [ ! -d "$peer" ]; then
git clone "$REMOTE_DIR" "$peer" >/dev/null 2>&1
fi
( cd "$peer" && git pull >/dev/null 2>&1 || true )
echo "$content" > "$peer/$name"
( cd "$peer" && git add -A && git commit -m "peer: $name" >/dev/null && git push >/dev/null 2>&1 )
}
# Give the store a local-only commit (store becomes "ahead").
store_local_commit() {
local name="${1:-local-file}"
echo "local" > "$SECRETS_DIR/$name"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: $name" >/dev/null
}
# Leave an uncommitted modification in the store working tree ("dirty").
store_dirty() {
echo "scratch" > "$SECRETS_DIR/${1:-dirty-file}"
git -C "$SECRETS_DIR" add -A >/dev/null 2>&1 || true
}
store_head() { git -C "$SECRETS_DIR" rev-parse HEAD; }
# init_with_remote (shared helper) commits --allow-empty, which leaves the
# files `secrets init` wrote — .gitignore, .secrets-format, recipients.txt —
# untracked. That is a genuinely dirty store, so tests that assert on the
# "nothing to reconcile" path must land them first.
init_clean_store() {
init_with_remote
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "store: initial files" >/dev/null 2>&1 || true
git -C "$SECRETS_DIR" push >/dev/null 2>&1
}
# ─── EGB-1230: pull must never fail silently ───────────────────────────
@test "EGB-1230: pull surfaces git's diagnosis when the store can't fast-forward" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt # now diverged: ahead 1, behind 1
create_project_dir divproj
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
# The failure is named, not silent — this is the whole bug.
[[ "$output" == *"Store sync failed"* ]] || false
[[ "$output" == *"$SECRETS_DIR"* ]] || false
}
@test "EGB-1230: pull's sync failure points at secrets sync" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir divproj
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}
@test "EGB-1230: pull writes the diagnosis to stderr, not just stdout" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir divproj
run bash -c "'$SECRETS_BIN' pull 2>&1 1>/dev/null"
[[ "$output" == *"Store sync failed"* ]] || false
}
@test "EGB-1230: a clean store still pulls normally" {
init_with_remote
create_project_dir cleanproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
rm -f .env .env.staging
run "$SECRETS_BIN" pull
[ "$status" -eq 0 ]
[ -f .env ]
}
@test "EGB-1230: pull's sync is fast-forward only (no silent merge commit)" {
init_with_remote
create_project_dir ffproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
local before; before=$(store_head)
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
# A merge commit would have moved HEAD. Nothing was integrated.
[ "$(store_head)" = "$before" ]
}
# ─── EGB-1231: which reports store state ───────────────────────────────
@test "EGB-1231: which reports ahead/behind/dirty for a diverged store" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
create_project_dir whichproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"remote:"* ]] || false
[[ "$output" == *"ahead 1"* ]] || false
[[ "$output" == *"behind 1"* ]] || false
}
@test "EGB-1231: which reports an in-sync store as up to date" {
init_clean_store
create_project_dir syncedproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"remote:"* ]] || false
[[ "$output" == *"up to date"* ]] || false
}
@test "EGB-1231: which stays quiet about the remote when none is configured" {
run "$SECRETS_BIN" init
create_project_dir noremote
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" != *"remote:"* ]] || false
}
@test "EGB-1231: which reports a dirty store working tree" {
init_with_remote
store_dirty scratch.age
create_project_dir dirtyproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"modified"* ]] || false
}
# ─── EGB-1231: secrets sync reconciles ─────────────────────────────────
@test "EGB-1231: sync rebases a diverged store onto the remote" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
# Both sides' work survives the reconcile.
[ -f "$SECRETS_DIR/remote-only.txt" ]
[ -f "$SECRETS_DIR/local-only.txt" ]
}
@test "EGB-1231: sync leaves the store able to pull again" {
init_with_remote
create_project_dir recovered
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
rm -f .env .env.staging
run "$SECRETS_BIN" pull
[ "$status" -eq 0 ]
[ -f .env ]
}
@test "EGB-1231: sync fast-forwards a store that is only behind" {
init_with_remote
peer_commit_and_push remote-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ -f "$SECRETS_DIR/remote-only.txt" ]
}
@test "EGB-1231: sync stashes and restores a dirty working tree" {
init_with_remote
peer_commit_and_push remote-only.txt
echo "uncommitted work" > "$SECRETS_DIR/scratch.age"
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ -f "$SECRETS_DIR/remote-only.txt" ]
# The local uncommitted blob edit is NOT lost.
[ -f "$SECRETS_DIR/scratch.age" ]
[ "$(cat "$SECRETS_DIR/scratch.age")" = "uncommitted work" ]
}
@test "EGB-1231: sync on an already-clean store reports no work and changes nothing" {
init_clean_store
local before; before=$(store_head)
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ "$(store_head)" = "$before" ]
[[ "$output" == *"up to date"* ]] || false
}
@test "EGB-1231: sync dies directed when the store has no remote" {
run "$SECRETS_BIN" init
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[[ "$output" == *"no remote"* ]] || false
}
# ─── EGB-1231: --dry-run changes nothing ───────────────────────────────
@test "EGB-1231: sync --dry-run reports state without mutating the store" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
local before; before=$(store_head)
run "$SECRETS_BIN" sync --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"ahead 1"* ]] || false
[[ "$output" == *"behind 1"* ]] || false
[ "$(store_head)" = "$before" ]
[ ! -f "$SECRETS_DIR/remote-only.txt" ]
}
@test "EGB-1231: sync --dry-run does not push local commits" {
init_with_remote
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --dry-run
[ "$status" -eq 0 ]
# The remote never received the local commit.
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" != *"local: local-only.txt"* ]] || false
}
# ─── EGB-1231: the push gate ───────────────────────────────────────────
@test "EGB-1231: sync --yes pushes reconciled local commits to the remote" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" == *"local: local-only.txt"* ]] || false
}
@test "EGB-1231: sync without confirmation reconciles locally but does not push" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
# No tty and no --yes: the push gate must not open on its own.
run bash -c "'$SECRETS_BIN' sync < /dev/null"
[ "$status" -eq 0 ]
# Local reconcile happened...
[ -f "$SECRETS_DIR/remote-only.txt" ]
# ...but nothing was published to the shared store.
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" != *"local: local-only.txt"* ]] || false
}
@test "EGB-1231: sync says how to publish when the push gate stays closed" {
init_with_remote
store_local_commit local-only.txt
run bash -c "'$SECRETS_BIN' sync < /dev/null"
[ "$status" -eq 0 ]
[[ "$output" == *"--yes"* ]] || false
}
# ─── EGB-1231: conflicts restore the store as found ────────────────────
@test "EGB-1231: a rebase conflict leaves the store exactly as it was" {
init_with_remote
# Both sides edit the same path — a guaranteed rebase conflict.
peer_commit_and_push contested.txt "from-remote"
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
echo "from-local" > "$SECRETS_DIR/contested.txt"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null
local before; before=$(store_head)
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[ "$(store_head)" = "$before" ]
# No half-finished rebase left behind for the user to trip over.
[ ! -d "$SECRETS_DIR/.git/rebase-merge" ]
[ ! -d "$SECRETS_DIR/.git/rebase-apply" ]
[ "$(cat "$SECRETS_DIR/contested.txt")" = "from-local" ]
}
@test "EGB-1231: a rebase conflict names the conflicting files" {
init_with_remote
peer_commit_and_push contested.txt "from-remote"
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
echo "from-local" > "$SECRETS_DIR/contested.txt"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[[ "$output" == *"contested.txt"* ]] || false
}
@test "EGB-1231: sync never force-pushes or hard-resets" {
# Guard rail on the implementation itself: this store holds the only
# copy of encrypted secrets, so the destructive git verbs must not
# appear anywhere in the sync path.
run bash -c "sed -n '/^cmd_sync()/,/^}/p' '$SECRETS_BIN'"
[[ "$output" != *"--force"* ]] || false
[[ "$output" != *"reset --hard"* ]] || false
[[ "$output" != *"stash drop"* ]] || false
}
# ─── EGB-1231: the push path's advice is no longer a dead end ──────────
@test "EGB-1231: push's fast-forward failure points at secrets sync" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir pushproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}
# ─── help ──────────────────────────────────────────────────────────────
@test "EGB-1231: sync is documented in help" {
run "$SECRETS_BIN" help
[ "$status" -eq 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}

View file

@ -19,6 +19,14 @@ setup() {
# not be a real ancestor of /tmp). EGB-281 F9. # not be a real ancestor of /tmp). EGB-281 F9.
export HOME="$TEST_TMPDIR" export HOME="$TEST_TMPDIR"
# Provide git author identity so `git commit` works with the fresh temp HOME
# (no ~/.gitconfig is present in the isolated dir). GIT_* env vars override
# any global config and survive the HOME redirect.
export GIT_AUTHOR_NAME="Test User"
export GIT_AUTHOR_EMAIL="test@example.com"
export GIT_COMMITTER_NAME="Test User"
export GIT_COMMITTER_EMAIL="test@example.com"
# Secrets repo lives in temp # Secrets repo lives in temp
export SECRETS_DIR="$TEST_TMPDIR/secrets-repo" export SECRETS_DIR="$TEST_TMPDIR/secrets-repo"

115
test/upgrade.bats Normal file
View file

@ -0,0 +1,115 @@
#!/usr/bin/env bats
# EGB-716: `secrets upgrade` verb — self-update (git pull --ff-only) + skew re-check.
#
# These tests never touch the real tool checkout. Each test relocates a COPY of
# the script into a throwaway git repo wired to a bare upstream, so $SCRIPT_DIR
# (computed from BASH_SOURCE) resolves to the fake tool repo and the pull/fetch
# operate there.
load test_helper
# Create a fake tool repo at $TOOL (script copy + VERSION), wired to a bare
# upstream at $TOOL_REMOTE, at version $1. cd's into $TOOL (under $HOME so
# resolve_store's walk-up stays bounded and never strays to a real store).
setup_tool_repo() {
TOOL="$TEST_TMPDIR/tool"
TOOL_REMOTE="$TEST_TMPDIR/tool-remote.git"
mkdir -p "$TOOL"
cp "$SECRETS_BIN" "$TOOL/secrets"
echo "$1" > "$TOOL/VERSION"
git -c init.defaultBranch=main init -q "$TOOL"
git -C "$TOOL" add -A
git -C "$TOOL" -c user.email=t@t -c user.name=t commit -qm "v$1"
git -c init.defaultBranch=main init --bare -q "$TOOL_REMOTE"
git -C "$TOOL" remote add origin "$TOOL_REMOTE"
git -C "$TOOL" push -q -u origin HEAD:main
cd "$TOOL"
}
# Publish a newer VERSION to the upstream (as a different clone would).
advance_tool_remote() {
local clone="$TEST_TMPDIR/tool-pub"
rm -rf "$clone"
git clone -q "$TOOL_REMOTE" "$clone"
echo "$1" > "$clone/VERSION"
git -C "$clone" -c user.email=t@t -c user.name=t commit -qam "v$1"
git -C "$clone" push -q origin HEAD:main
rm -rf "$clone"
}
@test "upgrade --check reports an available update without changing VERSION (EGB-716)" {
setup_tool_repo 0.1.0.0
advance_tool_remote 0.2.0.0
run "$TOOL/secrets" upgrade --check
[ "$status" -eq 0 ]
[[ "$output" == *"Update available"* ]] || false
[[ "$output" == *"0.1.0.0"* ]] || false
# --check must not pull: local VERSION is untouched.
[ "$(cat "$TOOL/VERSION")" = "0.1.0.0" ]
}
@test "upgrade --check is clean when already current (EGB-716)" {
setup_tool_repo 0.2.0.0
run "$TOOL/secrets" upgrade --check
[ "$status" -eq 0 ]
[[ "$output" == *"up to date"* ]] || false
}
@test "upgrade fast-forwards and reports old -> new (EGB-716)" {
setup_tool_repo 0.1.0.0
advance_tool_remote 0.2.0.0
run "$TOOL/secrets" upgrade
[ "$status" -eq 0 ]
[[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false
[ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ]
}
@test "upgrade is a no-op when already at the latest (EGB-716)" {
setup_tool_repo 0.2.0.0
run "$TOOL/secrets" upgrade
[ "$status" -eq 0 ]
[[ "$output" == *"up to date"* ]] || false
[ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ]
}
@test "upgrade refuses when the tool dir is not a git checkout (EGB-716)" {
local d="$HOME/plain-tool"
mkdir -p "$d"
cp "$SECRETS_BIN" "$d/secrets"
echo 0.1.0.0 > "$d/VERSION"
cd "$d"
run "$d/secrets" upgrade
[ "$status" -eq 1 ]
[[ "$output" == *"git checkout"* ]] || false
}
@test "upgrade rejects an unknown flag (EGB-716)" {
setup_tool_repo 0.1.0.0
run "$TOOL/secrets" upgrade --bogus
[ "$status" -eq 1 ]
[[ "$output" == *"Unknown upgrade flag"* ]] || false
}
@test "upgrade re-checks store skew and confirms the client caught up (EGB-716)" {
setup_tool_repo 0.1.0.0
advance_tool_remote 0.9.0.0
# A store last written by a newer client than our starting version.
git -c init.defaultBranch=main init -q "$SECRETS_DIR"
echo 0.8.0.0 > "$SECRETS_DIR/.secrets-writer-version"
run "$TOOL/secrets" upgrade
[ "$status" -eq 0 ]
[[ "$output" == *"v0.1.0.0 -> v0.9.0.0"* ]] || false
# New client (0.9.0.0) is now ahead of the store's last writer (0.8.0.0).
[[ "$output" == *"at or ahead"* ]] || false
}
@test "upgrade still notes when the store is ahead of the upgraded client (EGB-716)" {
setup_tool_repo 0.1.0.0
advance_tool_remote 0.2.0.0
git -c init.defaultBranch=main init -q "$SECRETS_DIR"
echo 0.9.0.0 > "$SECRETS_DIR/.secrets-writer-version"
run "$TOOL/secrets" upgrade
[ "$status" -eq 0 ]
[[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false
[[ "$output" == *"still ahead"* ]] || false
}

106
test/version.bats Normal file
View file

@ -0,0 +1,106 @@
#!/usr/bin/env bats
# EGB-713 version-skew nudge: writer-version stamp, numeric comparator, skew
# warning, `which` surface. bash 3.2: every standalone [[ ]] ends with || false.
load test_helper
VERSION_FILE() { echo "$(cd "$(dirname "$SECRETS_BIN")" && pwd)/VERSION"; }
# ─── comparator contract ──────────────────────────────────────────────
@test "version comparator orders 0.7.0.0 < 0.10.0.0 numerically (not lexically)" {
run bash -c '
_version_gt() {
local a="$1" b="$2" i ai bi; local -a af bf
IFS=. read -r -a af <<< "$a"; IFS=. read -r -a bf <<< "$b"
for i in 0 1 2 3; do
ai=${af[$i]:-0}; ai=${ai//[!0-9]/}; [ -n "$ai" ] || ai=0
bi=${bf[$i]:-0}; bi=${bi//[!0-9]/}; [ -n "$bi" ] || bi=0
if [ "$((10#$ai))" -gt "$((10#$bi))" ]; then return 0; fi
if [ "$((10#$ai))" -lt "$((10#$bi))" ]; then return 1; fi
done; return 1
}
_version_gt 0.10.0.0 0.7.0.0 && echo "10gt7"
_version_gt 0.7.0.0 0.10.0.0 || echo "7not_gt_10"
_version_gt 0.7.0.0 0.7.0.0 || echo "equal_not_gt"
_version_gt 1.0.0.0 0.9.9.9 && echo "major_wins"
'
[ "$status" -eq 0 ]
[[ "$output" == *"10gt7"* ]] || false
[[ "$output" == *"7not_gt_10"* ]] || false
[[ "$output" == *"equal_not_gt"* ]] || false
[[ "$output" == *"major_wins"* ]] || false
}
# ─── stamp on write ───────────────────────────────────────────────────
@test "push stamps the store writer-version with the client version" {
init_with_remote
create_project_dir wvstamp
"$SECRETS_BIN" push wvstamp >/dev/null 2>&1
[ -f "$SECRETS_DIR/.secrets-writer-version" ]
run cat "$SECRETS_DIR/.secrets-writer-version"
[ "$output" = "$(cat "$(VERSION_FILE)")" ]
}
@test "writer-version stamp is monotonic (a push never lowers a higher stamp)" {
init_with_remote
create_project_dir wvmono
printf '9.9.9.9\n' > "$SECRETS_DIR/.secrets-writer-version"
"$SECRETS_BIN" push wvmono >/dev/null 2>&1
run cat "$SECRETS_DIR/.secrets-writer-version"
[ "$output" = "9.9.9.9" ]
}
@test "writer-version stamp is committed, not gitignored" {
init_with_remote
create_project_dir wvcommit
"$SECRETS_BIN" push wvcommit >/dev/null 2>&1
run bash -c "git -C $SECRETS_DIR ls-files | grep -qx .secrets-writer-version"
[ "$status" -eq 0 ]
}
# ─── skew warning on command ──────────────────────────────────────────
@test "a store written by a newer version warns on a command (non-fatal)" {
init_with_remote
create_project_dir skewwarn
"$SECRETS_BIN" push skewwarn >/dev/null 2>&1
printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" list
[ "$status" -eq 0 ]
[[ "$output" == *"last written by secrets v99.0.0.0"* ]] || false
[[ "$output" == *"Update your secrets tool"* ]] || false
}
@test "a store at the same/older version is silent" {
init_with_remote
create_project_dir noskew
"$SECRETS_BIN" push noskew >/dev/null 2>&1
run "$SECRETS_BIN" list
[ "$status" -eq 0 ]
[[ "$output" != *"Update your secrets tool"* ]] || false
}
@test "a store with no writer-version marker is silent (legacy store)" {
init_with_remote
create_project_dir legacynostamp
"$SECRETS_BIN" push legacynostamp >/dev/null 2>&1
rm -f "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" list
[ "$status" -eq 0 ]
[[ "$output" != *"Update your secrets tool"* ]] || false
}
# ─── which surface ────────────────────────────────────────────────────
@test "which prints the store writer-version and a behind note" {
init_with_remote
create_project_dir whichwv
"$SECRETS_BIN" push whichwv >/dev/null 2>&1
printf '99.0.0.0\n' > "$SECRETS_DIR/.secrets-writer-version"
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"written-by: v99.0.0.0"* ]] || false
[[ "$output" == *"behind"* ]] || false
}

300
test/workspaces.bats Normal file
View file

@ -0,0 +1,300 @@
#!/usr/bin/env bats
# EGB-1232: workspace discovery must not be npm-only.
#
# Two defects, both in the pattern-source layer:
# 1. pnpm declares workspaces in pnpm-workspace.yaml, not package.json's
# `workspaces` key — so both call sites came up empty on every pnpm
# monorepo (silently, in the plain-push path).
# 2. the jq filter `.workspaces // .workspaces.packages` short-circuits on
# yarn's truthy object form, iterating the object's values and yielding
# the pattern array itself instead of the patterns.
#
# bash 3.2 gotcha (see CLAUDE.md): every standalone [[ ]] assertion MUST end
# with `|| false`, or a failing assertion does not fail the test.
load test_helper
# ─── fixtures ──────────────────────────────────────────────────────────
# A monorepo root with two workspace packages, each holding a .env.
# The workspace *declaration* is left to the caller — that is what varies.
# NB: the workspace re-scan (_maybe_workspace_env_files) runs only for
# projects that already have a .secrets.json — push's root-scan-only
# behaviour on a first push is by design (EGB-677 E13), and EGB-1232 is
# about the fallback that covers it never engaging. So every fixture
# carries a manifest, matching the reported repro.
make_monorepo() {
local dir="$WORK_DIR/${1:-mono}"
mkdir -p "$dir/apps/web" "$dir/apps/admin"
echo "VITE_CLERK_PUBLISHABLE_KEY=pk_test_web" > "$dir/apps/web/.env"
echo "VITE_CLERK_PUBLISHABLE_KEY=pk_test_admin" > "$dir/apps/admin/.env"
printf '{\n "version": 2,\n "dotenv": [],\n "external": []\n}\n' > "$dir/.secrets.json"
cd "$dir"
}
declare_pnpm() {
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "apps/*"
- "packages/*"
YAML
echo '{"name":"mono","private":true}' > package.json
}
declare_npm() {
echo '{"name":"mono","private":true,"workspaces":["apps/*"]}' > package.json
}
declare_yarn_object() {
echo '{"name":"mono","private":true,"workspaces":{"packages":["apps/*"]}}' > package.json
}
# ─── the pnpm gap (the reported bug) ───────────────────────────────────
@test "EGB-1232: push discovers workspace env files in a pnpm monorepo" {
init_with_remote
make_monorepo pnpmrepo
declare_pnpm
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" == *"apps/admin/.env"* ]] || false
}
@test "EGB-1232: push actually vaults pnpm workspace env files" {
init_with_remote
make_monorepo pnpmreal
declare_pnpm
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
rm -f apps/web/.env apps/admin/.env
run "$SECRETS_BIN" pull
[ "$status" -eq 0 ]
[ -f apps/web/.env ]
[ -f apps/admin/.env ]
# The value that was empty in the .env.example — the actual impact.
[[ "$(cat apps/web/.env)" == *"pk_test_web"* ]] || false
}
@test "EGB-1232: push -w works on a pnpm monorepo" {
init_with_remote
make_monorepo pnpmw
declare_pnpm
run "$SECRETS_BIN" push -w
[ "$status" -eq 0 ]
[[ "$output" != *"No workspaces field"* ]] || false
}
@test "EGB-1232: pnpm-workspace.yaml is read without requiring a package.json" {
init_with_remote
make_monorepo pnpmnopkg
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "apps/*"
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
}
# ─── pnpm YAML parsing edge cases ──────────────────────────────────────
@test "EGB-1232: pnpm parsing stops at the next top-level key" {
init_with_remote
make_monorepo pnpmkeys
# pnpm 10+ carries sibling top-level keys. They must not leak in as globs.
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "apps/*"
onlyBuiltDependencies:
- esbuild
- sharp
catalog:
react: ^18.0.0
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" != *"esbuild"* ]] || false
[[ "$output" != *"sharp"* ]] || false
}
@test "EGB-1232: pnpm parsing handles quotes, bare entries, and comments" {
init_with_remote
make_monorepo pnpmquotes
mkdir -p docs && echo "DOCS=1" > docs/.env
cat > pnpm-workspace.yaml <<'YAML'
# which packages belong to this workspace
packages:
- "apps/*" # double quoted, with a trailing comment
- 'docs' # single quoted, no glob
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" == *"docs/.env"* ]] || false
}
@test "EGB-1232: a pnpm-workspace.yaml with no packages key discovers nothing and says so" {
init_with_remote
make_monorepo pnpmempty
cat > pnpm-workspace.yaml <<'YAML'
onlyBuiltDependencies:
- esbuild
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"pnpm-workspace.yaml"* ]] || false
}
# ─── the yarn object-form defect ───────────────────────────────────────
@test "EGB-1232: yarn's object workspaces form is expanded, not iterated" {
init_with_remote
make_monorepo yarnobj
declare_yarn_object
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" == *"apps/admin/.env"* ]] || false
}
@test "EGB-1232: yarn object form works under -w" {
init_with_remote
make_monorepo yarnobjw
declare_yarn_object
run "$SECRETS_BIN" push -w
[ "$status" -eq 0 ]
[[ "$output" != *"No workspaces field"* ]] || false
}
# ─── npm regression guard ──────────────────────────────────────────────
@test "EGB-1232: npm's array workspaces form still works" {
init_with_remote
make_monorepo npmarr
declare_npm
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" == *"apps/admin/.env"* ]] || false
}
@test "EGB-1232: package.json workspaces wins when both sources are present" {
init_with_remote
make_monorepo bothsrc
mkdir -p only-pnpm && echo "P=1" > only-pnpm/.env
echo '{"name":"mono","workspaces":["apps/*"]}' > package.json
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "only-pnpm"
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"apps/web/.env"* ]] || false
[[ "$output" != *"only-pnpm/.env"* ]] || false
}
# ─── diagnostics: the silence is the bug ───────────────────────────────
@test "EGB-1232: a monorepo-shaped root with no readable workspace source warns" {
init_with_remote
make_monorepo shaped
# Looks like a monorepo (pnpm-workspace.yaml present) but declares nothing.
echo "# nothing useful here" > pnpm-workspace.yaml
# A root .env so the push SUCCEEDS — the dangerous case is a green push
# that silently skipped every workspace, which is what bit thatsbait.
echo "ROOT=1" > .env
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
[[ "$output" == *"WARNING"* ]] || false
[[ "$output" == *"workspace"* ]] || false
}
@test "EGB-1232: the warning tells you how to recover" {
init_with_remote
make_monorepo shapedfix
echo "# nothing useful here" > pnpm-workspace.yaml
echo "ROOT=1" > .env
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
[[ "$output" == *"secrets add"* ]] || false
}
@test "EGB-1232: a plain single-package project never warns about workspaces" {
init_with_remote
create_project_dir plainproj
printf '{\n "version": 2,\n "dotenv": [],\n "external": []\n}\n' > .secrets.json
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
[[ "$output" != *"workspace"* ]] || false
}
@test "EGB-1232: push -w names pnpm-workspace.yaml when that is the file present" {
init_with_remote
make_monorepo wnames
echo '{"name":"mono"}' > package.json
echo "# no packages key" > pnpm-workspace.yaml
run "$SECRETS_BIN" push -w
[ "$status" -ne 0 ]
[[ "$output" == *"pnpm-workspace.yaml"* ]] || false
}
@test "EGB-1232: push -w still names package.json when no pnpm file exists" {
init_with_remote
make_monorepo wnamespkg
echo '{"name":"mono"}' > package.json
run "$SECRETS_BIN" push -w
[ "$status" -ne 0 ]
[[ "$output" == *"package.json"* ]] || false
}
# ─── path safety ───────────────────────────────────────────────────────
@test "EGB-1232: a pnpm packages entry cannot escape the project root" {
init_with_remote
make_monorepo escapee
echo "OUTSIDE=1" > "$WORK_DIR/outside.env"
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "../"
- "/etc"
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[[ "$output" != *"outside.env"* ]] || false
[[ "$output" != *"/etc/"* ]] || false
}
@test "EGB-1232: a pnpm packages entry with shell metacharacters is refused" {
init_with_remote
make_monorepo metachars
cat > pnpm-workspace.yaml <<'YAML'
packages:
- "apps/*; touch /tmp/egb1232-pwned"
YAML
run "$SECRETS_BIN" push --dry-run
[ "$status" -eq 0 ]
[ ! -f /tmp/egb1232-pwned ]
}