# secrets Encrypted env file sync between machines using `age` key-file encryption + a private git repo. ## Quick Start ```bash brew install age ./secrets init # Create ~/.secrets repo + generate age key cd ~/my-project && ./secrets push # Encrypt .env* files, commit, push # On other machine: cd ~/my-project && ./secrets pull # Pull + decrypt .env* files ``` ## Testing ```bash brew install bats-core bats test/ # runs secrets.bats + manifest.bats + migrate.bats + upgrade.bats + recipients.bats ./test/run-security.sh # security regression subset + operator sign-off (see below) ``` **bash 3.2 assertion gotcha:** bats runs under system bash 3.2, where a failing `[[ ]]` mid-test does NOT fail the test (the ERR trap skips `[[` compound commands). Every standalone `[[ ... ]]` assertion MUST end with `|| false`. Single-bracket `[ ]` assertions are unaffected. ## Security review policy (mandatory for agents) This repo contains **defensive** security regression tests with deliberate attack-payload fixtures (shell injection strings, symlink escape cases, path rails). Hosted models (including Opus) block AI-driven "think like an attacker" / red-team / security-specialist review of that material under Anthropic Usage Policy (real-time cyber safeguards). **Read `.ship-policy.json` before any `/ship`, `/review`, or `/autoplan` work.** When that file is present (it always is in this repo): 1. **Never** dispatch red-team subagents, security specialist subagents, or adversarial review prompts ("think like an attacker", "find security holes", "chaos engineer", exploit hunting) against this codebase. 2. **Never** run `./test/run-security.sh` or filter bats to SECURITY/adversarial tests on behalf of the user — that is operator-local work only. 3. **Before ship/PR:** ask the human operator to run `./test/run-security.sh` locally and complete the SIGNOFF prompt. Record lives at `.gstack/security-signoff.json` (gitignored). Proceed with ship only after they confirm sign-off for the current `HEAD` (or note in the PR that sign-off is pending). 4. Normal `./ship` steps may continue (tests via `bats test/`, eng review, changelog, PR) — only the AI security/red-team/adversarial passes are opted out. Gstack `/ship` reads the same policy via `gstack-ship-policy` (sources `.ship-policy.json` at repo root): Step 9 blocks when manual sign-off is missing, skips security specialist + red team, and Step 11 skips adversarial review. ## Architecture Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, sync, upgrade. - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable) - Storage: Private git repo at `~/.secrets/` - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `/.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. - Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves. - Store sync + divergence (EGB-1230/EGB-1231): the store is a git repo, so a clone can end up ahead of and behind its remote at once. **EGB-1230:** `cmd_pull`'s sync used to be `git pull >/dev/null 2>&1` under `set -euo pipefail` — a store that couldn't fast-forward killed the script there with git's exit 128 and nothing on either stream (a banner, no files, no reason; invisible in a pipeline). It now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. That sync is **`--ff-only`**, matching the push path — a plain `git pull` could quietly manufacture a merge commit in the store, and divergence is now resolved in exactly one place. **EGB-1231:** `_store_git_state` emits `ahead\tbehind\tdirty` (from `rev-list --left-right --count @{u}...HEAD` plus `status --porcelain`) and `_format_store_state` renders it; `cmd_which` prints a `remote:` line from them — offline-safe (reports against the last fetch), silent with no remote/upstream. `cmd_sync` is the reconcile verb the CLI was missing: fetch → report state → stash (`push -u`) → `rebase @{u}` → restore stash → `ensure_store_protections` (rebased-in history may lack `.gitignore`, and a store missing the `key.txt` line would stage the private key — same reasoning as push) → **confirmation-gated** `git push` of local commits. The gate (`_sync_confirm_push`) reads `/dev/tty` and requires a tty, so it stays CLOSED in scripts/CI rather than publishing to a shared store by default; `--yes` opens it, `--dry-run` reports and returns before any mutation. Non-destructive by construction: no merge, no `--force`, no `reset --hard`, no `stash drop`. A rebase conflict collects the conflicting paths BEFORE `rebase --abort` (the abort clears them), restores the stash, and dies — store byte-identical to how it was found. `_sync_restore_stash` never drops the stash on a failed pop; it tells the operator where their only copy lives. `cmd_sync` does not `_stamp_writer_version`: it replays existing commits rather than authoring content, and the stamp is specified to ride a store-committing `git add -A`. Test suite: `test/sync.bats` (25 tests), including a grep over the `cmd_sync` body asserting the destructive git verbs never appear in it. - Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR//` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`. - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below - Workspaces (EGB-1232): `--workspaces` and the plain-push workspace re-scan both resolve patterns through ONE source — `_workspace_patterns()`. It reads `package.json` `.workspaces` via `$WORKSPACES_JQ` (type-aware: handles npm's array AND yarn's object `{packages:[...]}` form) and falls back to `pnpm-workspace.yaml`'s `packages:` block when package.json declares none. **Two defects fixed:** (1) both call sites were package.json-only, so no pnpm monorepo ever resolved a workspace — and `_maybe_workspace_env_files` failed *silently* (`jq -e '.workspaces' ... || return 0`), making auto-discovery inert and `push` print "Nothing new to add", indistinguishable from a repo with nothing new; it bit the same repo twice. (2) the old filter `.workspaces // .workspaces.packages | .[]` short-circuits on yarn's truthy object, iterating the object's values and yielding the pattern ARRAY as a single token. Note a naive reorder does NOT fix it — `.workspaces.packages` errors on an array; hence the `if type == "object"` form. `_pnpm_workspace_packages()` is a deliberate non-parser (block sequence only, stops at the next top-level key so pnpm 10's `onlyBuiltDependencies:`/`catalog:` can't leak in as globs, strips quotes/inline comments, refuses a symlinked file). Patterns are validated by `_valid_workspace_pattern` before they reach the unquoted `for pattern in $patterns` glob expansion (no absolute/`..`/metacharacters/whitespace; pnpm `!` negations skipped) — same posture as `.secrets-store`/`.secrets-files`. `_looks_like_monorepo` + `_workspace_source` turn the old silent return into a warning that names the real file, and `get_workspaces`'s error names `pnpm-workspace.yaml` when that's what's present instead of blaming package.json. jq is required only when package.json is the source. **Scope note:** the workspace re-scan still runs only for projects that already have a `.secrets.json` — push's root-scan-only behavior on a first push is by design (EGB-677 E13), and EGB-1232 is about the fallback that covers it never engaging. Tests: `test/workspaces.bats` (18). - Safety: Pre-commit hook rejects plaintext secret files (`.env`, `.dev.vars`, `gradle.properties`) - Multi-recipient (EGB-283): a store-scoped, committed `recipients.txt` (age `-R` format, `# name` comments) lets one store encrypt every blob to N age keys — one per team member. Managed via `secrets recipients add/rm/list`; absence of the file ⇒ legacy single-key behavior (recipients = the pubkey derived from `key.txt`). The file is parsed by us (never `age -R `) into a validated `RECIPIENT_ARGS` array (native age X25519 only, `age1[0-9a-z]{58}`; SSH recipients rejected; symlinked file refused) — same conservative posture as `.secrets-store`/`.secrets-files`. `_load_recipients` populates the array; every encrypt site routes through it. Any recipient change re-encrypts the WHOLE store in one commit via the shared `_reencrypt_all` engine (also used by the new `secrets reencrypt` and by `rekey` on a multi-recipient store, where rekey re-encrypts to the set with NO new keypair; legacy stores keep rekey's generate-new-keypair behavior). `init` seeds `recipients.txt` born-multi. `which` prints `recipients: N`; `verify`/`verify --all` assert each blob's age recipient-stanza count equals `recipients.txt`'s length. Removal takes effect going forward — git history stays readable by an old key, so rotate genuinely-sensitive values. Decryption is unchanged (each member uses their own `key.txt`). - Portability: must run on system bash 3.2 (macOS) — no associative arrays or bash-4 features ## Project Structure ``` secrets # CLI script (~2000 lines bash) hooks/pre-commit # Pre-commit hook template test/ secrets.bats # bats-core test suite (140 tests) manifest.bats # EGB-677 .secrets.json manifest tests (83 tests) migrate.bats # EGB-703 store-format-v2 migration tests (35 tests) upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests) sync.bats # EGB-1230/1231 store sync + divergence reconcile tests (25 tests) workspaces.bats # EGB-1232 npm/yarn/pnpm workspace discovery tests (18 tests) recipients.bats # EGB-283 multi-recipient age encryption tests (34 tests) test_helper.bash # Shared setup/teardown README.md # User-facing documentation CLAUDE.md # This file ``` ## Key file `~/.secrets/key.txt` is the age identity (private key). It is gitignored and must be copied manually to each machine once. ## Multi-store resolution The active store directory is picked by `resolve_store()` using these rules, highest precedence first: 1. `--store ` flag (parsed in the main pre-pass into `STORE_OVERRIDE`). 2. `.secrets-store` file in cwd or any ancestor, walk-up bounded by `$HOME` (never reads `$HOME/.secrets-store` itself or anything above). 3. `SECRETS_DIR` env var (legacy escape hatch). 4. `~/.secrets` default. `resolve_store` mutates BOTH `SECRETS_DIR` and `KEY_FILE` so the existing single-store code paths just work. `STORE_SOURCE` reports which rule won. `_LAST_FOUND_AT` (when rule 2 fires) holds the path of the file that was read. `.secrets-store` parsing is deliberately conservative: first non-empty non-comment line wins, no shell expansion (no `$VAR`, `$()`, backticks). Bare names map via `_expand_store_path`: `work` → `$HOME/.secrets-work`, `default` → `$HOME/.secrets`. An optional remote URL after the spec on the same line is captured as `_REMOTE_URL` and passed through to `check_initialized`, which uses it to fill in a runnable `git clone ` in the missing-store error (EGB-282). The URL is parsed via `read -r spec rest` (no `set -- $line`, no glob expansion) and then **sanitized**: any URL containing shell metacharacters (`;&|<>$\`(){}*?!"'\\`), control characters (incl. ANSI escapes), or whitespace is dropped with a stderr warning. The directed error then falls back to the `` placeholder. This matters because the rendered `git clone` line is meant to be copy-pasted by a teammate — without sanitization, `work evil.git;rm -rf ~` would render verbatim and execute the payload on paste. Internal flow: `_parse_secrets_store_file` returns `\t`; `_find_secrets_store_file` returns `\t\t`; `resolve_store` splits the 3-tuple via `IFS=$'\t' read -r ...`. ## External files (.secrets-files) — EGB-531 `.secrets-files` is a committed, project-root manifest declaring keys to sync from files **outside** the project (motivating case: `~/.gradle/gradle.properties`, which Android Studio GUI builds read but terminal env vars can't reach). One entry per line: ` ...`. Two types: `gradle-properties` (named-key merge) and `file` (EGB-652 — whole-file verbatim sync, binary-safe, built for the Beacon Android upload keystore; no keys, restored at mode 600 with a `.secrets-bak` backup of a divergent existing target, basename restriction waived but all other path rails apply). Still no plugin-dispatch framework — each type is a concrete `case` branch (deliberate scope cut). Key design decisions (all driven by /autoplan review): - **Wire-in is at command scope** (`cmd_push`/`cmd_pull`), via `push_external_files` / `pull_external_files`, **not** inside `push_dir_to_project` / `pull_project_to_dir` (those loop per-workspace and `pull_project_to_dir` uses stdout as a data channel). - **Storage:** blobs live in `$SECRETS_DIR//external/.gradle-properties.age`. The `external/` subdir keeps them out of the legacy non-recursive `*.age` / `.*.age` globs the dotenv `pull` path uses, so a dotenv pull can never decrypt an external blob into cwd. `cmd_rekey` and `cmd_list` instead walk the **entire** project tree (`find -type f`), so they cover both `external/.age` and nested manifest dotenv blobs (`/.age`) — rekey MUST recurse, or any nested/external blob is orphaned under the old key after rotation = data loss (EGB-677 regression test: "rekey re-encrypts a nested manifest dotenv blob"). **EGB-701 cleanups:** (1) the *legacy* (manifest-less) `pull` keeps its non-recursive globs but now **warns** when nested `/.age` blobs exist that those globs can't see (it excludes `external/`, which `pull_external_files` handles) — so a manifest-less pull never silently under-restores; the fix the warning points at is committing a `.secrets.json`. (2) `cmd_which`, push, and pull share one external extractor (`_json_external_entries`), so `which` applies the same `properties`→`gradle-properties` normalization and skip-with-warning rules the sync path does (it shows exactly what will sync, not a stale raw projection). (3) the two external-manifest read guards are factored into `_json_readable` (plain regular file, silent) / `_legacy_readable` (warn-and-skip on a symlinked legacy manifest). `` = manifest path token with non-`[A-Za-z0-9._-]` chars → `_`, plus a `cksum` suffix of the original path so paths that clean to the same string (`a/b` vs `a_b`) don't collide. Machine-independent (derived from the committed manifest token, not the expanded path). `cmd_list --json` (EGB-699) emits the same recursive walk as a machine-readable object (`{store, projects[].entries[]}`, each entry `dotenv`→`path` or `external`→`subtype`+`path`) for tooling/CI (feeds EGB-671); jq assembles it so paths escape correctly and stdout stays pure JSON (the human store hint is suppressed; jq is a hard dep only in `--json` mode). - **Merge is pure bash, no `sed`/regex** (`merge_gradle_keys`): exact-string key comparison (avoids `beaconClerkPk` vs `beaconClerkPkTest` substring bug), value treated as opaque literal (survives `& \ /` in values). Updates a managed key in place at its first occurrence, collapses duplicates, appends new keys, preserves unrelated lines/comments/order. Continuation lines (trailing odd backslashes, tracked by `_trailing_bs_odd`) are never matched as keys. Atomic write: temp in the same dir → `chmod` to match (or `600` on create) → `mv`. Backs up to `.secrets-bak` before each merge. - **Properties separator parsing** (`_props_get`): key ends at the first `=`, `:`, or whitespace (after lstrip); handles `key=value`, `key = value`, `key:value`, `key value`; last definition wins. - **Security:** the write target comes from a committed file, so `_validate_external_target_path` locks it down — basename must be `gradle.properties`, must resolve inside `$HOME` (deepest-existing-ancestor resolved, symlink target/parent refused, `..` rejected). This blocks a malicious manifest from appending decrypted keys to `~/.gitconfig`/`~/.bashrc`. `_parse_secrets_files_manifest` rejects shell metacharacters/control chars in path and keys (path allows `[A-Za-z0-9/._~-]` only; keys allow `[A-Za-z0-9._-]` + space), mirrors the `.secrets-store` posture (no shell expansion, symlinked manifest skipped). - **Plaintext tradeoff (accepted, documented):** merged keys are permanent plaintext in the target; `secrets clear` does not remove them. Fine for the Clerk *publishable* keys this was built for; not for high-value secrets (use `secrets run` + `.env`). ## Deploy Configuration - Platform: NONE (distributed via `git clone` from the private Forgejo at `git.dev.egbt.com`) - Production URL: N/A (no live service) - Release model: merge to `main` is the release. Optionally tagged with `v`. - Verification after merge: a fresh `git clone` should produce a working `secrets which` against an isolated `$HOME`. No canary URL. - Staging: none. - Rollback: revert the merge commit on `main` (and delete the tag) to roll back. ## Forge operations (self-hosted Forgejo) The remote is a private Forgejo instance at `https://git.dev.egbt.com` (migrated off Codeberg 2026-09-08). `gh`/`glab` do NOT work here. Use `tea` (login name: `egbt`, user `brian`) for forge operations when a skill's platform detection comes up "unknown": **Always pass `--login egbt --repo egbt/secrets` explicitly.** `tea`'s repo autodetection fails here ("remote repository required"), and this machine also has a leftover `codeberg` login pointing at the *old* forge (`https://codeberg.org`) that `tea` will silently fall back to in non-interactive mode — which would target the wrong server. Confirm with `tea logins list` if a command errors. - PRs: `tea pr create --login egbt --repo egbt/secrets --base main --head --title ... --description ...` / `tea pr merge --login egbt --repo egbt/secrets` - Releases: `tea releases create --login egbt --repo egbt/secrets --tag v --title "v" --note ...` (convention: one release per tag, title `v`) - Issues/status: `tea issues --login egbt --repo egbt/secrets`, `tea pr list --login egbt --repo egbt/secrets` - **SSH is on port 2222**, not 22 (port 22 is the host's own sshd). Clone/remote URLs must be `ssh://git@git.dev.egbt.com:2222/egbt/secrets.git`. A bare `git@git.dev.egbt.com:egbt/secrets.git` will fail with "Permission denied (publickey)" because it hits the wrong daemon. - The host resolves to a Tailscale address — the forge is reachable only on the VPN. Off-net, push/pull/`tea` all fail to connect; that is expected, not a credentials problem. - `FORGEJO_URL` and `FORGEJO_TOKEN` (API token for user `brian`) live in `~/.zshenv` for direct API calls. - CI: the instance has an Actions runner available, but no workflow is configured for this repo yet. The bats suite run locally is still the merge gate. ## Environment variable `SECRETS_DIR` overrides the default `~/.secrets` location (useful for testing). Per-project bindings via `.secrets-store` file beat this env var; use `--store ` for one-shot overrides that beat everything. ## Skill routing When the user's request matches an available skill, invoke it via the Skill tool. When in doubt, invoke the skill. Key routing rules: - Product ideas/brainstorming → invoke /office-hours - Strategy/scope → invoke /plan-ceo-review - Architecture → invoke /plan-eng-review - Design system/plan review → invoke /design-consultation or /plan-design-review - Full review pipeline → invoke /autoplan - Bugs/errors → invoke /investigate - QA/testing site behavior → invoke /qa or /qa-only - Code review/diff check → invoke /review - Visual polish → invoke /design-review - Ship/deploy/PR → invoke /ship or /land-and-deploy (after reading `.ship-policy.json`; no AI adversarial/red-team/security-specialist review in this repo) - Save progress → invoke /context-save - Resume context → invoke /context-restore - Author a backlog-ready spec/issue → invoke /spec