From dd6a025fa05bb45061f680151fc373c9b8379209 Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Tue, 8 Sep 2026 14:20:20 -0700 Subject: [PATCH 1/2] fix: surface store sync failures + add `secrets sync` (EGB-1230, EGB-1231) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit EGB-1230: `cmd_pull` synced the store with `git pull >/dev/null 2>&1` under `set -euo pipefail`. A store that couldn't fast-forward killed the script at that line with git's exit 128 and nothing on stdout or stderr — a banner, no restored files, no reason, and invisible in a pipeline. The sync now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. It is now `--ff-only` to match the push path, so a plain pull can no longer manufacture a merge commit in the store. EGB-1231: once a store diverged there was no way out — push demanded a fast-forward and pointed at pull, which couldn't fast-forward either, so the advice looped and recovery meant hand-running git next to encrypted blobs. Adds `secrets sync`: fetch, stash, rebase onto the remote, restore the stash, then a confirmation-gated push of local commits (`--yes` to skip the prompt, `--dry-run` to report only). Non-destructive by construction — no merge, no force-push, no `reset --hard`, no `stash drop`; a rebase conflict names the conflicting files and leaves the store exactly as found. `secrets which` gains a `remote:` line reporting ahead/behind/dirty, and push's dead-end message now points at `sync`. test/sync.bats: 25 new tests. Full suite 353/353 green. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB --- CHANGELOG.md | 41 ++++++ CLAUDE.md | 4 +- README.md | 29 ++++ VERSION | 2 +- secrets | 240 ++++++++++++++++++++++++++++++- test/sync.bats | 373 +++++++++++++++++++++++++++++++++++++++++++++++++ 6 files changed, 681 insertions(+), 8 deletions(-) create mode 100644 test/sync.bats diff --git a/CHANGELOG.md b/CHANGELOG.md index d37cd05..7e7cde1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,47 @@ All notable changes to this project will be documented in this file. The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme. +## [0.7.6.0] - 2026-09-08 + +### Added + +- **`secrets sync` — reconcile a diverged store (EGB-1231)** — the store is a + git repo, and once a clone was both ahead and behind its remote the CLI had + no way out: `push` demanded a fast-forward and pointed at `pull`, which + could not fast-forward either, so the advice looped and recovery meant + hand-running git next to a directory of encrypted blobs. `secrets sync` + fetches, stashes uncommitted blob edits, rebases local commits onto the + remote, restores the stash, and then asks before publishing local commits to + the shared store. `--yes` skips the prompt (scripts/CI); `--dry-run` reports + ahead/behind/dirty and what would happen, changing nothing. Deliberately + non-destructive: no merge, no force-push, no `reset --hard`, no `stash + drop`. A rebase conflict aborts, restores the stash, names the conflicting + files, and leaves the store exactly as found. +- **Store state in `secrets which` (EGB-1231)** — a new `remote:` line reports + the store's `ahead N, behind N, N modified` (or `up to date`) against its + upstream, with a `(run: secrets sync)` hint when there is anything to + reconcile. Offline-safe (reports against the last fetch, never reaches the + network) and silent for a local-only store or one with no upstream. + +### Fixed + +- **`secrets pull` no longer fails silently when the store can't sync + (EGB-1230)** — the store sync was `git pull >/dev/null 2>&1` under `set -euo + pipefail`, so a store that could not fast-forward killed the script at that + line with git's exit 128 and *nothing* on stdout or stderr. The user saw a + banner, no restored files, and no reason — indistinguishable from a project + with nothing to pull, and easy to lose entirely in a pipeline. The sync is + now guarded, git's output is captured and surfaced as the diagnosis, and the + error names the store path and points at `secrets sync`. + +### Changed + +- **`secrets pull`'s store sync is now fast-forward only**, matching the push + path. A plain `git pull` could quietly manufacture a merge commit in the + store; divergence is now resolved in exactly one place — `secrets sync`. +- **The push path's dead-end advice** ("Run 'secrets pull' first, then retry + push") now points at `secrets sync` and includes git's own output. + ## [0.7.5.0] - 2026-06-24 ### Added diff --git a/CLAUDE.md b/CLAUDE.md index a7afbda..8bfe715 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -56,13 +56,14 @@ skips security specialist + red team, and Step 11 skips adversarial review. ## Architecture -Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, upgrade. +Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, sync, upgrade. - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable) - Storage: Private git repo at `~/.secrets/` - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`) - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `/.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints. - Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at , version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves. +- Store sync + divergence (EGB-1230/EGB-1231): the store is a git repo, so a clone can end up ahead of and behind its remote at once. **EGB-1230:** `cmd_pull`'s sync used to be `git pull >/dev/null 2>&1` under `set -euo pipefail` — a store that couldn't fast-forward killed the script there with git's exit 128 and nothing on either stream (a banner, no files, no reason; invisible in a pipeline). It now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. That sync is **`--ff-only`**, matching the push path — a plain `git pull` could quietly manufacture a merge commit in the store, and divergence is now resolved in exactly one place. **EGB-1231:** `_store_git_state` emits `ahead\tbehind\tdirty` (from `rev-list --left-right --count @{u}...HEAD` plus `status --porcelain`) and `_format_store_state` renders it; `cmd_which` prints a `remote:` line from them — offline-safe (reports against the last fetch), silent with no remote/upstream. `cmd_sync` is the reconcile verb the CLI was missing: fetch → report state → stash (`push -u`) → `rebase @{u}` → restore stash → `ensure_store_protections` (rebased-in history may lack `.gitignore`, and a store missing the `key.txt` line would stage the private key — same reasoning as push) → **confirmation-gated** `git push` of local commits. The gate (`_sync_confirm_push`) reads `/dev/tty` and requires a tty, so it stays CLOSED in scripts/CI rather than publishing to a shared store by default; `--yes` opens it, `--dry-run` reports and returns before any mutation. Non-destructive by construction: no merge, no `--force`, no `reset --hard`, no `stash drop`. A rebase conflict collects the conflicting paths BEFORE `rebase --abort` (the abort clears them), restores the stash, and dies — store byte-identical to how it was found. `_sync_restore_stash` never drops the stash on a failed pop; it tells the operator where their only copy lives. `cmd_sync` does not `_stamp_writer_version`: it replays existing commits rather than authoring content, and the stamp is specified to ride a store-committing `git add -A`. Test suite: `test/sync.bats` (25 tests), including a grep over the `cmd_sync` body asserting the destructive git verbs never appear in it. - Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR//` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`. - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below - Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq` @@ -97,6 +98,7 @@ test/ manifest.bats # EGB-677 .secrets.json manifest tests (83 tests) migrate.bats # EGB-703 store-format-v2 migration tests (35 tests) upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests) + sync.bats # EGB-1230/1231 store sync + divergence reconcile tests (25 tests) recipients.bats # EGB-283 multi-recipient age encryption tests (34 tests) test_helper.bash # Shared setup/teardown README.md # User-facing documentation diff --git a/README.md b/README.md index 4ebb17b..f838ad5 100644 --- a/README.md +++ b/README.md @@ -193,6 +193,9 @@ secrets clear | `secrets recipients add [--name N]` | Add a recipient key to the store and immediately re-encrypt every blob to the new set | | `secrets recipients rm [--yes]` | Remove a recipient and re-encrypt the store; `--yes` required when removing your own key | | `secrets reencrypt` | Re-encrypt every blob to the current recipients (idempotent — useful after a manual edit or partial failure) | +| `secrets sync` | Reconcile a store that has diverged from its remote: stash local blob edits, rebase onto the remote, restore the stash, then offer to publish your local commits. Never merges, force-pushes, or hard-resets | +| `secrets sync --dry-run` | Report the store's ahead/behind/dirty state and what a reconcile would do; changes nothing | +| `secrets sync --yes` | Reconcile and publish local commits without the confirmation prompt (for scripts) | | `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks | | `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing | @@ -211,6 +214,32 @@ Store-format v2 is **additive** — an upgraded client reads either blob suffix And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent. +### When the store diverges + +The store is a git repo, so two machines pushing at once can leave your clone +both ahead and behind its remote. `secrets push` needs a fast-forward and +`secrets pull` won't silently merge, so both stop and tell you to run: + +```bash +secrets sync +``` + +`sync` fetches, stashes any uncommitted blob edits, rebases your local commits +onto the remote, restores the stash, and then *asks* before publishing your +commits to the shared store (`--yes` skips the prompt; `--dry-run` just +reports). If the rebase conflicts, it aborts, restores your stash, names the +conflicting files, and leaves the store exactly as it found it — nothing in +the path force-pushes, hard-resets, or drops a stash. + +`secrets which` now reports the same state up front, so you can see it coming: + +``` +store: /Users/you/.secrets +source: default +format: v2 +remote: ahead 1, behind 11, 3 modified (run: secrets sync) +``` + ### Automatic project detection When you run `secrets push` or `secrets pull` without specifying a project name, the tool figures out which project you're in by: diff --git a/VERSION b/VERSION index e413a4a..29e2792 100644 --- a/VERSION +++ b/VERSION @@ -1 +1 @@ -0.7.5.0 +0.7.6.0 diff --git a/secrets b/secrets index da8f4be..8d947dc 100755 --- a/secrets +++ b/secrets @@ -1339,6 +1339,65 @@ ensure_store_protections() { fi } +# ─── Store git state (EGB-1230 / EGB-1231) ───────────────────────────── + +# True when the store has an 'origin' remote. Everything below is a no-op +# without one — a local-only store can never be out of sync with anything. +_store_has_remote() { + git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 +} + +# Emit "\t\t" for the store against its upstream. +# Reads only what git already knows — the caller decides whether to fetch +# first, so `which` stays cheap and offline-safe while `sync` sees fresh +# counts. Returns 1 (emitting nothing) when there is no upstream to compare +# against, which is the normal state for a local-only store. +_store_git_state() { + _store_has_remote || return 1 + git -C "$SECRETS_DIR" rev-parse --abbrev-ref '@{u}' >/dev/null 2>&1 || return 1 + local counts ahead behind dirty + # --left-right --count prints "\t" for @{u}...HEAD. + counts=$(git -C "$SECRETS_DIR" rev-list --left-right --count '@{u}...HEAD' 2>/dev/null) || return 1 + behind=$(printf '%s' "$counts" | awk '{print $1}') + ahead=$(printf '%s' "$counts" | awk '{print $2}') + dirty=$(git -C "$SECRETS_DIR" status --porcelain 2>/dev/null | grep -c . || true) + # `grep -c` on empty input exits 1 under `set -e`; normalize to a number. + [ -n "$dirty" ] || dirty=0 + printf '%s\t%s\t%s\n' "${ahead:-0}" "${behind:-0}" "$dirty" +} + +# Render a state triple as one human line: "ahead 1, behind 11, 3 modified", +# or "up to date" when there is genuinely nothing to reconcile. +_format_store_state() { + local ahead="$1" behind="$2" dirty="$3" parts="" + [ "$ahead" -gt 0 ] 2>/dev/null && parts="ahead $ahead" + [ "$behind" -gt 0 ] 2>/dev/null && parts="${parts:+$parts, }behind $behind" + [ "$dirty" -gt 0 ] 2>/dev/null && parts="${parts:+$parts, }$dirty modified" + printf '%s' "${parts:-up to date}" +} + +# Sync the store from its remote before reading or writing blobs. +# +# EGB-1230: this used to be `git pull >/dev/null 2>&1` under `set -euo +# pipefail`, so a store that could not fast-forward killed the script at +# that line with git's exit 128 and NOTHING on either stream — the user saw +# a banner, no files, and no reason. git's output is the diagnosis; capture +# it and put it in the error. +# +# Fast-forward only, matching the push path: a plain `git pull` can quietly +# manufacture a merge commit in the store. Divergence gets resolved in +# exactly one place — `secrets sync`. +_store_sync_pull() { + _store_has_remote || return 0 + local out + if ! out=$(git -C "$SECRETS_DIR" pull --ff-only 2>&1); then + die "Store sync failed ($SECRETS_DIR): +$(printf '%s\n' "$out" | sed 's/^/ /') +The store has diverged from its remote or has local changes. +Run: secrets sync" + fi +} + # ─── Subcommands ─────────────────────────────────────────────────────── cmd_init() { @@ -1595,8 +1654,14 @@ commit_and_push_secrets() { local message="$1" if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then - if ! git -C "$SECRETS_DIR" pull --ff-only 2>/dev/null; then - die "Fast-forward pull failed. Run 'secrets pull' first, then retry push." + # EGB-1231: this used to point at `secrets pull`, which cannot resolve a + # diverged store either — the advice looped. `secrets sync` is the one + # command that reconciles. + local ff_out + if ! ff_out=$(git -C "$SECRETS_DIR" pull --ff-only 2>&1); then + die "Store sync failed ($SECRETS_DIR): +$(printf '%s\n' "$ff_out" | sed 's/^/ /') +Run 'secrets sync' to reconcile, then retry push." fi fi @@ -1865,10 +1930,8 @@ cmd_pull() { info "Pulling secrets for project: $project" echo_store_if_non_default - # Pull latest - if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then - git -C "$SECRETS_DIR" pull >/dev/null 2>&1 - fi + # Pull latest. Guarded and unsilenced — see _store_sync_pull (EGB-1230). + _store_sync_pull # ── Manifest-driven pull (EGB-677 stage 1) ── # With a .secrets.json present, the manifest decides what restores and @@ -2621,6 +2684,19 @@ cmd_which() { # EGB-700 (folded into EGB-703): surface the store format so users can tell # v1 from v2 during the migration window. v1 = legacy store, no format marker. echo "format: v$(_store_format)" + # EGB-1231: the one thing `which` never reported was the thing that + # actually blocks push and pull — whether the clone is ahead/behind/dirty + # relative to its remote. Offline-safe: reports against the last fetch, + # never reaches the network, and stays silent for a local-only store. + local _st _a _b _d + if _st=$(_store_git_state); then + IFS=$'\t' read -r _a _b _d <<< "$_st" + if [ "$_a" -gt 0 ] || [ "$_b" -gt 0 ] || [ "$_d" -gt 0 ]; then + echo "remote: $(_format_store_state "$_a" "$_b" "$_d") (run: secrets sync)" + else + echo "remote: $(_format_store_state "$_a" "$_b" "$_d")" + fi + fi local _wv; _wv=$(_store_writer_version) if [ -n "$_wv" ]; then local _cv; _cv=$(_client_version) @@ -2703,6 +2779,154 @@ cmd_which() { # After a real update it best-effort re-checks the store's writer-version skew # against the NEW on-disk version, so the operator sees whether the EGB-713 # nudge is now cleared (the new code itself takes effect on the next command). +# Put a stashed working tree back. Never drops the stash on failure — the +# entry is the only copy of the operator's uncommitted blob edits. +_sync_restore_stash() { + [ "$1" = true ] || return 0 + local out + if out=$(git -C "$SECRETS_DIR" stash pop 2>&1); then + info "Restored your local store changes." + return 0 + fi + echo "WARNING: could not restore your stashed store changes:" >&2 + printf '%s\n' "$out" | sed 's/^/ /' >&2 + echo "They are SAFE and still stashed. Inspect with:" >&2 + echo " git -C \"$SECRETS_DIR\" stash list" >&2 + echo " git -C \"$SECRETS_DIR\" stash show -p" >&2 + return 1 +} + +# The push gate (EGB-1231): publishing local commits to a store other people +# pull from is an outward-facing act, so it needs an explicit yes. No tty +# means no confirmation — the gate stays closed rather than opening by +# default in scripts and CI. +_sync_confirm_push() { + [ -t 0 ] && [ -e /dev/tty ] || return 1 + printf "Publish %s local commit(s) to the shared store? [y/N] " "$1" > /dev/tty 2>/dev/null || return 1 + local ans="" + read -r ans < /dev/tty 2>/dev/null || return 1 + case "$ans" in [Yy]*) return 0 ;; *) return 1 ;; esac +} + +# EGB-1231: reconcile a store that has diverged from its remote. +# +# This is the command that was missing. Before it, `push` demanded a +# fast-forward and told you to run `pull`, and `pull` could not fast-forward +# either — the advice looped and the only way out was hand-running git next +# to a directory of encrypted blobs, which is where someone reaches for +# `reset --hard` and destroys the local work. +# +# Deliberately non-destructive: stash (never drop), rebase (never merge, +# never force), abort-and-restore on any conflict. Every failure path leaves +# the store exactly as it was found. +cmd_sync() { + check_cmd git + resolve_store + check_initialized + + local dry_run=false assume_yes=false + while [ $# -gt 0 ]; do + case "$1" in + --dry-run|-n) dry_run=true; shift ;; + --yes|-y) assume_yes=true; shift ;; + -*) die "Unknown sync flag: $1. Usage: secrets sync [--dry-run] [--yes]" ;; + *) die "secrets sync takes no arguments (got: $1)" ;; + esac + done + + if ! _store_has_remote; then + die "Store has no remote configured — nothing to reconcile ($SECRETS_DIR). +Wire one with: git -C \"$SECRETS_DIR\" remote add origin " + fi + + info "Reconciling store: $SECRETS_DIR" + + local out + if ! out=$(git -C "$SECRETS_DIR" fetch origin 2>&1); then + die "Fetch failed ($SECRETS_DIR): +$(printf '%s\n' "$out" | sed 's/^/ /')" + fi + + local st ahead behind dirty + if ! st=$(_store_git_state); then + die "The store's branch has no upstream to reconcile against ($SECRETS_DIR). +Set one with: git -C \"$SECRETS_DIR\" branch --set-upstream-to origin/" + fi + IFS=$'\t' read -r ahead behind dirty <<< "$st" + echo " state: $(_format_store_state "$ahead" "$behind" "$dirty")" + + if [ "$dry_run" = true ]; then + if [ "$behind" -gt 0 ]; then + echo " would: rebase $ahead local commit(s) onto origin ($behind to integrate)" + elif [ "$ahead" -gt 0 ]; then + echo " would: publish $ahead local commit(s)" + fi + [ "$dirty" -gt 0 ] && echo " would: stash and restore $dirty uncommitted change(s)" + info "Dry run — nothing changed." + return 0 + fi + + if [ "$ahead" -eq 0 ] && [ "$behind" -eq 0 ]; then + info "Nothing to reconcile." + return 0 + fi + + # Stash before touching history: a rebase refuses to start on a dirty tree, + # and those uncommitted .age edits may be the only copy of a secret. + local stashed=false + if [ "$dirty" -gt 0 ]; then + if ! out=$(git -C "$SECRETS_DIR" stash push -u -m "secrets sync" 2>&1); then + die "Could not stash local store changes ($SECRETS_DIR): +$(printf '%s\n' "$out" | sed 's/^/ /')" + fi + stashed=true + info "Stashed $dirty uncommitted change(s)." + fi + + if [ "$behind" -gt 0 ]; then + if ! out=$(git -C "$SECRETS_DIR" rebase '@{u}' 2>&1); then + # Collect the conflicting paths BEFORE aborting — the abort clears them. + local conflicts + conflicts=$(git -C "$SECRETS_DIR" diff --name-only --diff-filter=U 2>/dev/null || true) + git -C "$SECRETS_DIR" rebase --abort >/dev/null 2>&1 || true + _sync_restore_stash "$stashed" || true + die "Store reconcile hit a conflict — the store is unchanged. +Conflicting files: +$(printf '%s\n' "${conflicts:-(see git output below)}" | sed 's/^/ /') +$(printf '%s\n' "$out" | sed 's/^/ /') +Resolve by hand in $SECRETS_DIR, or ask the other machine to push again." + fi + info "Rebased onto origin." + fi + + _sync_restore_stash "$stashed" || true + + # Same reasoning as the push path: rebased-in history may lack .gitignore, + # and a store missing the key.txt line would stage the private key. + ensure_store_protections + + # Re-read state — the rebase changed it. + st=$(_store_git_state) || st=$'0\t0\t0' + IFS=$'\t' read -r ahead behind dirty <<< "$st" + + if [ "$ahead" -gt 0 ]; then + if [ "$assume_yes" != true ] && ! _sync_confirm_push "$ahead"; then + info "Reconciled locally. $ahead local commit(s) NOT published." + echo " Publish them with: secrets sync --yes" + return 0 + fi + if ! out=$(git -C "$SECRETS_DIR" push 2>&1); then + die "Push failed ($SECRETS_DIR): +$(printf '%s\n' "$out" | sed 's/^/ /')" + fi + info "Published $ahead local commit(s)." + fi + + st=$(_store_git_state) || st=$'0\t0\t0' + IFS=$'\t' read -r ahead behind dirty <<< "$st" + info "Store reconciled: $(_format_store_state "$ahead" "$behind" "$dirty")" +} + cmd_upgrade() { local check_only=0 while [ $# -gt 0 ]; do @@ -3243,6 +3467,9 @@ Usage: secrets recipients add KEY [--name N] Add a recipient and re-encrypt the store secrets recipients rm KEY|NAME [--yes] Remove a recipient and re-encrypt the store secrets reencrypt Re-encrypt every blob to the current recipients + secrets sync Reconcile a diverged store with its remote + secrets sync --dry-run Report ahead/behind/dirty; change nothing + secrets sync --yes Reconcile and publish local commits without prompting secrets which Show the active store, manifest, and external entries secrets where Alias for `which` secrets status Alias for `which` @@ -3419,6 +3646,7 @@ case "${1:-help}" in migrate) shift; cmd_migrate "$@" ;; recipients) shift; cmd_recipients "$@" ;; which|where|status) cmd_which ;; + sync) shift; cmd_sync "$@" ;; upgrade) shift; cmd_upgrade "$@" ;; help|--help|-h) cmd_help ;; *) die "Unknown command: $1. Run 'secrets help' for usage." ;; diff --git a/test/sync.bats b/test/sync.bats new file mode 100644 index 0000000..3cc90e8 --- /dev/null +++ b/test/sync.bats @@ -0,0 +1,373 @@ +#!/usr/bin/env bats +# EGB-1230 / EGB-1231: store sync — loud failures on `pull`, and a `secrets +# sync` verb that reconciles a diverged store instead of dead-ending. +# +# bash 3.2 gotcha (see CLAUDE.md): every standalone [[ ]] assertion MUST end +# with `|| false`, or a failing assertion does not fail the test. + +load test_helper + +# ─── fixtures ────────────────────────────────────────────────────────── +# +# A "peer" is a second clone of the same bare remote. Committing + pushing +# from the peer is how we put the store under test *behind* its remote +# without touching the store itself. + +peer_commit_and_push() { + local name="${1:-peer-file}" content="${2:-peer}" + local peer="$TEST_TMPDIR/peer" + if [ ! -d "$peer" ]; then + git clone "$REMOTE_DIR" "$peer" >/dev/null 2>&1 + fi + ( cd "$peer" && git pull >/dev/null 2>&1 || true ) + echo "$content" > "$peer/$name" + ( cd "$peer" && git add -A && git commit -m "peer: $name" >/dev/null && git push >/dev/null 2>&1 ) +} + +# Give the store a local-only commit (store becomes "ahead"). +store_local_commit() { + local name="${1:-local-file}" + echo "local" > "$SECRETS_DIR/$name" + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "local: $name" >/dev/null +} + +# Leave an uncommitted modification in the store working tree ("dirty"). +store_dirty() { + echo "scratch" > "$SECRETS_DIR/${1:-dirty-file}" + git -C "$SECRETS_DIR" add -A >/dev/null 2>&1 || true +} + +store_head() { git -C "$SECRETS_DIR" rev-parse HEAD; } + +# init_with_remote (shared helper) commits --allow-empty, which leaves the +# files `secrets init` wrote — .gitignore, .secrets-format, recipients.txt — +# untracked. That is a genuinely dirty store, so tests that assert on the +# "nothing to reconcile" path must land them first. +init_clean_store() { + init_with_remote + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "store: initial files" >/dev/null 2>&1 || true + git -C "$SECRETS_DIR" push >/dev/null 2>&1 +} + +# ─── EGB-1230: pull must never fail silently ─────────────────────────── + +@test "EGB-1230: pull surfaces git's diagnosis when the store can't fast-forward" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt # now diverged: ahead 1, behind 1 + + create_project_dir divproj + run "$SECRETS_BIN" pull + + [ "$status" -ne 0 ] + # The failure is named, not silent — this is the whole bug. + [[ "$output" == *"Store sync failed"* ]] || false + [[ "$output" == *"$SECRETS_DIR"* ]] || false +} + +@test "EGB-1230: pull's sync failure points at secrets sync" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + create_project_dir divproj + run "$SECRETS_BIN" pull + + [ "$status" -ne 0 ] + [[ "$output" == *"secrets sync"* ]] || false +} + +@test "EGB-1230: pull writes the diagnosis to stderr, not just stdout" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + create_project_dir divproj + run bash -c "'$SECRETS_BIN' pull 2>&1 1>/dev/null" + + [[ "$output" == *"Store sync failed"* ]] || false +} + +@test "EGB-1230: a clean store still pulls normally" { + init_with_remote + create_project_dir cleanproj + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + + rm -f .env .env.staging + run "$SECRETS_BIN" pull + [ "$status" -eq 0 ] + [ -f .env ] +} + +@test "EGB-1230: pull's sync is fast-forward only (no silent merge commit)" { + init_with_remote + create_project_dir ffproj + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + local before; before=$(store_head) + run "$SECRETS_BIN" pull + [ "$status" -ne 0 ] + # A merge commit would have moved HEAD. Nothing was integrated. + [ "$(store_head)" = "$before" ] +} + +# ─── EGB-1231: which reports store state ─────────────────────────────── + +@test "EGB-1231: which reports ahead/behind/dirty for a diverged store" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1 + + create_project_dir whichproj + run "$SECRETS_BIN" which + + [ "$status" -eq 0 ] + [[ "$output" == *"remote:"* ]] || false + [[ "$output" == *"ahead 1"* ]] || false + [[ "$output" == *"behind 1"* ]] || false +} + +@test "EGB-1231: which reports an in-sync store as up to date" { + init_clean_store + create_project_dir syncedproj + run "$SECRETS_BIN" which + + [ "$status" -eq 0 ] + [[ "$output" == *"remote:"* ]] || false + [[ "$output" == *"up to date"* ]] || false +} + +@test "EGB-1231: which stays quiet about the remote when none is configured" { + run "$SECRETS_BIN" init + create_project_dir noremote + run "$SECRETS_BIN" which + + [ "$status" -eq 0 ] + [[ "$output" != *"remote:"* ]] || false +} + +@test "EGB-1231: which reports a dirty store working tree" { + init_with_remote + store_dirty scratch.age + create_project_dir dirtyproj + run "$SECRETS_BIN" which + + [ "$status" -eq 0 ] + [[ "$output" == *"modified"* ]] || false +} + +# ─── EGB-1231: secrets sync reconciles ───────────────────────────────── + +@test "EGB-1231: sync rebases a diverged store onto the remote" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + + # Both sides' work survives the reconcile. + [ -f "$SECRETS_DIR/remote-only.txt" ] + [ -f "$SECRETS_DIR/local-only.txt" ] +} + +@test "EGB-1231: sync leaves the store able to pull again" { + init_with_remote + create_project_dir recovered + run "$SECRETS_BIN" push + [ "$status" -eq 0 ] + + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + + rm -f .env .env.staging + run "$SECRETS_BIN" pull + [ "$status" -eq 0 ] + [ -f .env ] +} + +@test "EGB-1231: sync fast-forwards a store that is only behind" { + init_with_remote + peer_commit_and_push remote-only.txt + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + [ -f "$SECRETS_DIR/remote-only.txt" ] +} + +@test "EGB-1231: sync stashes and restores a dirty working tree" { + init_with_remote + peer_commit_and_push remote-only.txt + echo "uncommitted work" > "$SECRETS_DIR/scratch.age" + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + [ -f "$SECRETS_DIR/remote-only.txt" ] + # The local uncommitted blob edit is NOT lost. + [ -f "$SECRETS_DIR/scratch.age" ] + [ "$(cat "$SECRETS_DIR/scratch.age")" = "uncommitted work" ] +} + +@test "EGB-1231: sync on an already-clean store reports no work and changes nothing" { + init_clean_store + local before; before=$(store_head) + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + [ "$(store_head)" = "$before" ] + [[ "$output" == *"up to date"* ]] || false +} + +@test "EGB-1231: sync dies directed when the store has no remote" { + run "$SECRETS_BIN" init + run "$SECRETS_BIN" sync --yes + + [ "$status" -ne 0 ] + [[ "$output" == *"no remote"* ]] || false +} + +# ─── EGB-1231: --dry-run changes nothing ─────────────────────────────── + +@test "EGB-1231: sync --dry-run reports state without mutating the store" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + local before; before=$(store_head) + run "$SECRETS_BIN" sync --dry-run + + [ "$status" -eq 0 ] + [[ "$output" == *"ahead 1"* ]] || false + [[ "$output" == *"behind 1"* ]] || false + [ "$(store_head)" = "$before" ] + [ ! -f "$SECRETS_DIR/remote-only.txt" ] +} + +@test "EGB-1231: sync --dry-run does not push local commits" { + init_with_remote + store_local_commit local-only.txt + + run "$SECRETS_BIN" sync --dry-run + [ "$status" -eq 0 ] + + # The remote never received the local commit. + run git -C "$REMOTE_DIR" log --oneline + [[ "$output" != *"local: local-only.txt"* ]] || false +} + +# ─── EGB-1231: the push gate ─────────────────────────────────────────── + +@test "EGB-1231: sync --yes pushes reconciled local commits to the remote" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + run "$SECRETS_BIN" sync --yes + [ "$status" -eq 0 ] + + run git -C "$REMOTE_DIR" log --oneline + [[ "$output" == *"local: local-only.txt"* ]] || false +} + +@test "EGB-1231: sync without confirmation reconciles locally but does not push" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + # No tty and no --yes: the push gate must not open on its own. + run bash -c "'$SECRETS_BIN' sync < /dev/null" + [ "$status" -eq 0 ] + + # Local reconcile happened... + [ -f "$SECRETS_DIR/remote-only.txt" ] + # ...but nothing was published to the shared store. + run git -C "$REMOTE_DIR" log --oneline + [[ "$output" != *"local: local-only.txt"* ]] || false +} + +@test "EGB-1231: sync says how to publish when the push gate stays closed" { + init_with_remote + store_local_commit local-only.txt + + run bash -c "'$SECRETS_BIN' sync < /dev/null" + [ "$status" -eq 0 ] + [[ "$output" == *"--yes"* ]] || false +} + +# ─── EGB-1231: conflicts restore the store as found ──────────────────── + +@test "EGB-1231: a rebase conflict leaves the store exactly as it was" { + init_with_remote + # Both sides edit the same path — a guaranteed rebase conflict. + peer_commit_and_push contested.txt "from-remote" + git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1 + echo "from-local" > "$SECRETS_DIR/contested.txt" + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null + + local before; before=$(store_head) + run "$SECRETS_BIN" sync --yes + + [ "$status" -ne 0 ] + [ "$(store_head)" = "$before" ] + # No half-finished rebase left behind for the user to trip over. + [ ! -d "$SECRETS_DIR/.git/rebase-merge" ] + [ ! -d "$SECRETS_DIR/.git/rebase-apply" ] + [ "$(cat "$SECRETS_DIR/contested.txt")" = "from-local" ] +} + +@test "EGB-1231: a rebase conflict names the conflicting files" { + init_with_remote + peer_commit_and_push contested.txt "from-remote" + git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1 + echo "from-local" > "$SECRETS_DIR/contested.txt" + git -C "$SECRETS_DIR" add -A + git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null + + run "$SECRETS_BIN" sync --yes + [ "$status" -ne 0 ] + [[ "$output" == *"contested.txt"* ]] || false +} + +@test "EGB-1231: sync never force-pushes or hard-resets" { + # Guard rail on the implementation itself: this store holds the only + # copy of encrypted secrets, so the destructive git verbs must not + # appear anywhere in the sync path. + run bash -c "sed -n '/^cmd_sync()/,/^}/p' '$SECRETS_BIN'" + [[ "$output" != *"--force"* ]] || false + [[ "$output" != *"reset --hard"* ]] || false + [[ "$output" != *"stash drop"* ]] || false +} + +# ─── EGB-1231: the push path's advice is no longer a dead end ────────── + +@test "EGB-1231: push's fast-forward failure points at secrets sync" { + init_with_remote + peer_commit_and_push remote-only.txt + store_local_commit local-only.txt + + create_project_dir pushproj + run "$SECRETS_BIN" push + + [ "$status" -ne 0 ] + [[ "$output" == *"secrets sync"* ]] || false +} + +# ─── help ────────────────────────────────────────────────────────────── + +@test "EGB-1231: sync is documented in help" { + run "$SECRETS_BIN" help + [ "$status" -eq 0 ] + [[ "$output" == *"secrets sync"* ]] || false +} From f1cf9d9e7efbff45ce4fde808bf931530e43644c Mon Sep 17 00:00:00 2001 From: Brian Majewski Date: Tue, 8 Sep 2026 14:31:16 -0700 Subject: [PATCH 2/2] docs: correct Codeberg tea invocation in CLAUDE.md `tea` autodetection fails in this repo, and the machine carries a second login (`egbt`) pointing at a different forge that tea silently falls back to in non-interactive mode. Document the explicit `--login codeberg --repo egbt/secrets` form for every verb. Co-Authored-By: Claude Opus 5 (1M context) Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB --- CLAUDE.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index 8bfe715..ff8405f 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -150,10 +150,17 @@ The remote is Codeberg (Forgejo) — `gh`/`glab` do NOT work here. Use `tea` (login name: `codeberg`, user `egbt`) for forge operations when a skill's platform detection comes up "unknown": -- PRs: `tea pr create --base main --title ... --description ...` / `tea pr merge ` -- Releases: `tea releases create --tag v --title "v — " --note ...` +**Always pass `--login codeberg --repo egbt/secrets` explicitly.** `tea`'s +repo autodetection fails here ("remote repository required"), and this machine +also has an `egbt` login pointing at a *different* forge +(`https://git.dev.egbt.com`) that `tea` will silently fall back to in +non-interactive mode — which would target the wrong server. Confirm with +`tea logins list` if a command errors. + +- PRs: `tea pr create --login codeberg --repo egbt/secrets --base main --head --title ... --description ...` / `tea pr merge --login codeberg --repo egbt/secrets` +- Releases: `tea releases create --login codeberg --repo egbt/secrets --tag v --title "v — " --note ...` (convention: one release per tag, title `v — `) -- Issues/status: `tea issues`, `tea pr list` +- Issues/status: `tea issues --login codeberg --repo egbt/secrets`, `tea pr list --login codeberg --repo egbt/secrets` - No CI on this repo: the bats suite run locally is the merge gate. ## Environment variable