fix: surface store sync failures + add secrets sync (EGB-1230, EGB-1231)

EGB-1230: `cmd_pull` synced the store with `git pull >/dev/null 2>&1` under
`set -euo pipefail`. A store that couldn't fast-forward killed the script at
that line with git's exit 128 and nothing on stdout or stderr — a banner, no
restored files, no reason, and invisible in a pipeline. The sync now routes
through `_store_sync_pull`, which guards the pull, captures git's output as
the diagnosis, and dies naming the store path and `secrets sync`. It is now
`--ff-only` to match the push path, so a plain pull can no longer manufacture
a merge commit in the store.

EGB-1231: once a store diverged there was no way out — push demanded a
fast-forward and pointed at pull, which couldn't fast-forward either, so the
advice looped and recovery meant hand-running git next to encrypted blobs.
Adds `secrets sync`: fetch, stash, rebase onto the remote, restore the stash,
then a confirmation-gated push of local commits (`--yes` to skip the prompt,
`--dry-run` to report only). Non-destructive by construction — no merge, no
force-push, no `reset --hard`, no `stash drop`; a rebase conflict names the
conflicting files and leaves the store exactly as found. `secrets which` gains
a `remote:` line reporting ahead/behind/dirty, and push's dead-end message now
points at `sync`.

test/sync.bats: 25 new tests. Full suite 353/353 green.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BrUoYuUMoTj91rzV4vxGPB
This commit is contained in:
Brian Majewski 2026-09-08 14:20:20 -07:00
parent 2558ea3c23
commit dd6a025fa0
6 changed files with 681 additions and 8 deletions

View file

@ -5,6 +5,47 @@ All notable changes to this project will be documented in this file.
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to a four-digit MAJOR.MINOR.PATCH.MICRO version scheme.
## [0.7.6.0] - 2026-09-08
### Added
- **`secrets sync` — reconcile a diverged store (EGB-1231)** — the store is a
git repo, and once a clone was both ahead and behind its remote the CLI had
no way out: `push` demanded a fast-forward and pointed at `pull`, which
could not fast-forward either, so the advice looped and recovery meant
hand-running git next to a directory of encrypted blobs. `secrets sync`
fetches, stashes uncommitted blob edits, rebases local commits onto the
remote, restores the stash, and then asks before publishing local commits to
the shared store. `--yes` skips the prompt (scripts/CI); `--dry-run` reports
ahead/behind/dirty and what would happen, changing nothing. Deliberately
non-destructive: no merge, no force-push, no `reset --hard`, no `stash
drop`. A rebase conflict aborts, restores the stash, names the conflicting
files, and leaves the store exactly as found.
- **Store state in `secrets which` (EGB-1231)** — a new `remote:` line reports
the store's `ahead N, behind N, N modified` (or `up to date`) against its
upstream, with a `(run: secrets sync)` hint when there is anything to
reconcile. Offline-safe (reports against the last fetch, never reaches the
network) and silent for a local-only store or one with no upstream.
### Fixed
- **`secrets pull` no longer fails silently when the store can't sync
(EGB-1230)** — the store sync was `git pull >/dev/null 2>&1` under `set -euo
pipefail`, so a store that could not fast-forward killed the script at that
line with git's exit 128 and *nothing* on stdout or stderr. The user saw a
banner, no restored files, and no reason — indistinguishable from a project
with nothing to pull, and easy to lose entirely in a pipeline. The sync is
now guarded, git's output is captured and surfaced as the diagnosis, and the
error names the store path and points at `secrets sync`.
### Changed
- **`secrets pull`'s store sync is now fast-forward only**, matching the push
path. A plain `git pull` could quietly manufacture a merge commit in the
store; divergence is now resolved in exactly one place — `secrets sync`.
- **The push path's dead-end advice** ("Run 'secrets pull' first, then retry
push") now points at `secrets sync` and includes git's own output.
## [0.7.5.0] - 2026-06-24
### Added

View file

@ -56,13 +56,14 @@ skips security specialist + red team, and Step 11 skips adversarial review.
## Architecture
Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, upgrade.
Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, recipients, reencrypt, sync, upgrade.
- Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable)
- Storage: Private git repo at `~/.secrets/`
- Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`)
- Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `<project>/<relpath>.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints.
- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age``.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at <path>, version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-<sha>` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves.
- Store sync + divergence (EGB-1230/EGB-1231): the store is a git repo, so a clone can end up ahead of and behind its remote at once. **EGB-1230:** `cmd_pull`'s sync used to be `git pull >/dev/null 2>&1` under `set -euo pipefail` — a store that couldn't fast-forward killed the script there with git's exit 128 and nothing on either stream (a banner, no files, no reason; invisible in a pipeline). It now routes through `_store_sync_pull`, which guards the pull, captures git's output as the diagnosis, and dies naming the store path and `secrets sync`. That sync is **`--ff-only`**, matching the push path — a plain `git pull` could quietly manufacture a merge commit in the store, and divergence is now resolved in exactly one place. **EGB-1231:** `_store_git_state` emits `ahead\tbehind\tdirty` (from `rev-list --left-right --count @{u}...HEAD` plus `status --porcelain`) and `_format_store_state` renders it; `cmd_which` prints a `remote:` line from them — offline-safe (reports against the last fetch), silent with no remote/upstream. `cmd_sync` is the reconcile verb the CLI was missing: fetch → report state → stash (`push -u`) → `rebase @{u}` → restore stash → `ensure_store_protections` (rebased-in history may lack `.gitignore`, and a store missing the `key.txt` line would stage the private key — same reasoning as push) → **confirmation-gated** `git push` of local commits. The gate (`_sync_confirm_push`) reads `/dev/tty` and requires a tty, so it stays CLOSED in scripts/CI rather than publishing to a shared store by default; `--yes` opens it, `--dry-run` reports and returns before any mutation. Non-destructive by construction: no merge, no `--force`, no `reset --hard`, no `stash drop`. A rebase conflict collects the conflicting paths BEFORE `rebase --abort` (the abort clears them), restores the stash, and dies — store byte-identical to how it was found. `_sync_restore_stash` never drops the stash on a failed pop; it tells the operator where their only copy lives. `cmd_sync` does not `_stamp_writer_version`: it replays existing commits rather than authoring content, and the stamp is specified to ride a store-committing `git add -A`. Test suite: `test/sync.bats` (25 tests), including a grep over the `cmd_sync` body asserting the destructive git verbs never appear in it.
- Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR/<project>/` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`.
- External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below
- Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq`
@ -97,6 +98,7 @@ test/
manifest.bats # EGB-677 .secrets.json manifest tests (83 tests)
migrate.bats # EGB-703 store-format-v2 migration tests (35 tests)
upgrade.bats # EGB-716 `secrets upgrade` self-update tests (8 tests)
sync.bats # EGB-1230/1231 store sync + divergence reconcile tests (25 tests)
recipients.bats # EGB-283 multi-recipient age encryption tests (34 tests)
test_helper.bash # Shared setup/teardown
README.md # User-facing documentation

View file

@ -193,6 +193,9 @@ secrets clear
| `secrets recipients add <age1…> [--name N]` | Add a recipient key to the store and immediately re-encrypt every blob to the new set |
| `secrets recipients rm <key\|name> [--yes]` | Remove a recipient and re-encrypt the store; `--yes` required when removing your own key |
| `secrets reencrypt` | Re-encrypt every blob to the current recipients (idempotent — useful after a manual edit or partial failure) |
| `secrets sync` | Reconcile a store that has diverged from its remote: stash local blob edits, rebase onto the remote, restore the stash, then offer to publish your local commits. Never merges, force-pushes, or hard-resets |
| `secrets sync --dry-run` | Report the store's ahead/behind/dirty state and what a reconcile would do; changes nothing |
| `secrets sync --yes` | Reconcile and publish local commits without the confirmation prompt (for scripts) |
| `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks |
| `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing |
@ -211,6 +214,32 @@ Store-format v2 is **additive** — an upgraded client reads either blob suffix
And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent.
### When the store diverges
The store is a git repo, so two machines pushing at once can leave your clone
both ahead and behind its remote. `secrets push` needs a fast-forward and
`secrets pull` won't silently merge, so both stop and tell you to run:
```bash
secrets sync
```
`sync` fetches, stashes any uncommitted blob edits, rebases your local commits
onto the remote, restores the stash, and then *asks* before publishing your
commits to the shared store (`--yes` skips the prompt; `--dry-run` just
reports). If the rebase conflicts, it aborts, restores your stash, names the
conflicting files, and leaves the store exactly as it found it — nothing in
the path force-pushes, hard-resets, or drops a stash.
`secrets which` now reports the same state up front, so you can see it coming:
```
store: /Users/you/.secrets
source: default
format: v2
remote: ahead 1, behind 11, 3 modified (run: secrets sync)
```
### Automatic project detection
When you run `secrets push` or `secrets pull` without specifying a project name, the tool figures out which project you're in by:

View file

@ -1 +1 @@
0.7.5.0
0.7.6.0

240
secrets
View file

@ -1339,6 +1339,65 @@ ensure_store_protections() {
fi
}
# ─── Store git state (EGB-1230 / EGB-1231) ─────────────────────────────
# True when the store has an 'origin' remote. Everything below is a no-op
# without one — a local-only store can never be out of sync with anything.
_store_has_remote() {
git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1
}
# Emit "<ahead>\t<behind>\t<dirty>" for the store against its upstream.
# Reads only what git already knows — the caller decides whether to fetch
# first, so `which` stays cheap and offline-safe while `sync` sees fresh
# counts. Returns 1 (emitting nothing) when there is no upstream to compare
# against, which is the normal state for a local-only store.
_store_git_state() {
_store_has_remote || return 1
git -C "$SECRETS_DIR" rev-parse --abbrev-ref '@{u}' >/dev/null 2>&1 || return 1
local counts ahead behind dirty
# --left-right --count prints "<behind>\t<ahead>" for @{u}...HEAD.
counts=$(git -C "$SECRETS_DIR" rev-list --left-right --count '@{u}...HEAD' 2>/dev/null) || return 1
behind=$(printf '%s' "$counts" | awk '{print $1}')
ahead=$(printf '%s' "$counts" | awk '{print $2}')
dirty=$(git -C "$SECRETS_DIR" status --porcelain 2>/dev/null | grep -c . || true)
# `grep -c` on empty input exits 1 under `set -e`; normalize to a number.
[ -n "$dirty" ] || dirty=0
printf '%s\t%s\t%s\n' "${ahead:-0}" "${behind:-0}" "$dirty"
}
# Render a state triple as one human line: "ahead 1, behind 11, 3 modified",
# or "up to date" when there is genuinely nothing to reconcile.
_format_store_state() {
local ahead="$1" behind="$2" dirty="$3" parts=""
[ "$ahead" -gt 0 ] 2>/dev/null && parts="ahead $ahead"
[ "$behind" -gt 0 ] 2>/dev/null && parts="${parts:+$parts, }behind $behind"
[ "$dirty" -gt 0 ] 2>/dev/null && parts="${parts:+$parts, }$dirty modified"
printf '%s' "${parts:-up to date}"
}
# Sync the store from its remote before reading or writing blobs.
#
# EGB-1230: this used to be `git pull >/dev/null 2>&1` under `set -euo
# pipefail`, so a store that could not fast-forward killed the script at
# that line with git's exit 128 and NOTHING on either stream — the user saw
# a banner, no files, and no reason. git's output is the diagnosis; capture
# it and put it in the error.
#
# Fast-forward only, matching the push path: a plain `git pull` can quietly
# manufacture a merge commit in the store. Divergence gets resolved in
# exactly one place — `secrets sync`.
_store_sync_pull() {
_store_has_remote || return 0
local out
if ! out=$(git -C "$SECRETS_DIR" pull --ff-only 2>&1); then
die "Store sync failed ($SECRETS_DIR):
$(printf '%s\n' "$out" | sed 's/^/ /')
The store has diverged from its remote or has local changes.
Run: secrets sync"
fi
}
# ─── Subcommands ───────────────────────────────────────────────────────
cmd_init() {
@ -1595,8 +1654,14 @@ commit_and_push_secrets() {
local message="$1"
if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then
if ! git -C "$SECRETS_DIR" pull --ff-only 2>/dev/null; then
die "Fast-forward pull failed. Run 'secrets pull' first, then retry push."
# EGB-1231: this used to point at `secrets pull`, which cannot resolve a
# diverged store either — the advice looped. `secrets sync` is the one
# command that reconciles.
local ff_out
if ! ff_out=$(git -C "$SECRETS_DIR" pull --ff-only 2>&1); then
die "Store sync failed ($SECRETS_DIR):
$(printf '%s\n' "$ff_out" | sed 's/^/ /')
Run 'secrets sync' to reconcile, then retry push."
fi
fi
@ -1865,10 +1930,8 @@ cmd_pull() {
info "Pulling secrets for project: $project"
echo_store_if_non_default
# Pull latest
if git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1; then
git -C "$SECRETS_DIR" pull >/dev/null 2>&1
fi
# Pull latest. Guarded and unsilenced — see _store_sync_pull (EGB-1230).
_store_sync_pull
# ── Manifest-driven pull (EGB-677 stage 1) ──
# With a .secrets.json present, the manifest decides what restores and
@ -2621,6 +2684,19 @@ cmd_which() {
# EGB-700 (folded into EGB-703): surface the store format so users can tell
# v1 from v2 during the migration window. v1 = legacy store, no format marker.
echo "format: v$(_store_format)"
# EGB-1231: the one thing `which` never reported was the thing that
# actually blocks push and pull — whether the clone is ahead/behind/dirty
# relative to its remote. Offline-safe: reports against the last fetch,
# never reaches the network, and stays silent for a local-only store.
local _st _a _b _d
if _st=$(_store_git_state); then
IFS=$'\t' read -r _a _b _d <<< "$_st"
if [ "$_a" -gt 0 ] || [ "$_b" -gt 0 ] || [ "$_d" -gt 0 ]; then
echo "remote: $(_format_store_state "$_a" "$_b" "$_d") (run: secrets sync)"
else
echo "remote: $(_format_store_state "$_a" "$_b" "$_d")"
fi
fi
local _wv; _wv=$(_store_writer_version)
if [ -n "$_wv" ]; then
local _cv; _cv=$(_client_version)
@ -2703,6 +2779,154 @@ cmd_which() {
# After a real update it best-effort re-checks the store's writer-version skew
# against the NEW on-disk version, so the operator sees whether the EGB-713
# nudge is now cleared (the new code itself takes effect on the next command).
# Put a stashed working tree back. Never drops the stash on failure — the
# entry is the only copy of the operator's uncommitted blob edits.
_sync_restore_stash() {
[ "$1" = true ] || return 0
local out
if out=$(git -C "$SECRETS_DIR" stash pop 2>&1); then
info "Restored your local store changes."
return 0
fi
echo "WARNING: could not restore your stashed store changes:" >&2
printf '%s\n' "$out" | sed 's/^/ /' >&2
echo "They are SAFE and still stashed. Inspect with:" >&2
echo " git -C \"$SECRETS_DIR\" stash list" >&2
echo " git -C \"$SECRETS_DIR\" stash show -p" >&2
return 1
}
# The push gate (EGB-1231): publishing local commits to a store other people
# pull from is an outward-facing act, so it needs an explicit yes. No tty
# means no confirmation — the gate stays closed rather than opening by
# default in scripts and CI.
_sync_confirm_push() {
[ -t 0 ] && [ -e /dev/tty ] || return 1
printf "Publish %s local commit(s) to the shared store? [y/N] " "$1" > /dev/tty 2>/dev/null || return 1
local ans=""
read -r ans < /dev/tty 2>/dev/null || return 1
case "$ans" in [Yy]*) return 0 ;; *) return 1 ;; esac
}
# EGB-1231: reconcile a store that has diverged from its remote.
#
# This is the command that was missing. Before it, `push` demanded a
# fast-forward and told you to run `pull`, and `pull` could not fast-forward
# either — the advice looped and the only way out was hand-running git next
# to a directory of encrypted blobs, which is where someone reaches for
# `reset --hard` and destroys the local work.
#
# Deliberately non-destructive: stash (never drop), rebase (never merge,
# never force), abort-and-restore on any conflict. Every failure path leaves
# the store exactly as it was found.
cmd_sync() {
check_cmd git
resolve_store
check_initialized
local dry_run=false assume_yes=false
while [ $# -gt 0 ]; do
case "$1" in
--dry-run|-n) dry_run=true; shift ;;
--yes|-y) assume_yes=true; shift ;;
-*) die "Unknown sync flag: $1. Usage: secrets sync [--dry-run] [--yes]" ;;
*) die "secrets sync takes no arguments (got: $1)" ;;
esac
done
if ! _store_has_remote; then
die "Store has no remote configured — nothing to reconcile ($SECRETS_DIR).
Wire one with: git -C \"$SECRETS_DIR\" remote add origin <url>"
fi
info "Reconciling store: $SECRETS_DIR"
local out
if ! out=$(git -C "$SECRETS_DIR" fetch origin 2>&1); then
die "Fetch failed ($SECRETS_DIR):
$(printf '%s\n' "$out" | sed 's/^/ /')"
fi
local st ahead behind dirty
if ! st=$(_store_git_state); then
die "The store's branch has no upstream to reconcile against ($SECRETS_DIR).
Set one with: git -C \"$SECRETS_DIR\" branch --set-upstream-to origin/<branch>"
fi
IFS=$'\t' read -r ahead behind dirty <<< "$st"
echo " state: $(_format_store_state "$ahead" "$behind" "$dirty")"
if [ "$dry_run" = true ]; then
if [ "$behind" -gt 0 ]; then
echo " would: rebase $ahead local commit(s) onto origin ($behind to integrate)"
elif [ "$ahead" -gt 0 ]; then
echo " would: publish $ahead local commit(s)"
fi
[ "$dirty" -gt 0 ] && echo " would: stash and restore $dirty uncommitted change(s)"
info "Dry run — nothing changed."
return 0
fi
if [ "$ahead" -eq 0 ] && [ "$behind" -eq 0 ]; then
info "Nothing to reconcile."
return 0
fi
# Stash before touching history: a rebase refuses to start on a dirty tree,
# and those uncommitted .age edits may be the only copy of a secret.
local stashed=false
if [ "$dirty" -gt 0 ]; then
if ! out=$(git -C "$SECRETS_DIR" stash push -u -m "secrets sync" 2>&1); then
die "Could not stash local store changes ($SECRETS_DIR):
$(printf '%s\n' "$out" | sed 's/^/ /')"
fi
stashed=true
info "Stashed $dirty uncommitted change(s)."
fi
if [ "$behind" -gt 0 ]; then
if ! out=$(git -C "$SECRETS_DIR" rebase '@{u}' 2>&1); then
# Collect the conflicting paths BEFORE aborting — the abort clears them.
local conflicts
conflicts=$(git -C "$SECRETS_DIR" diff --name-only --diff-filter=U 2>/dev/null || true)
git -C "$SECRETS_DIR" rebase --abort >/dev/null 2>&1 || true
_sync_restore_stash "$stashed" || true
die "Store reconcile hit a conflict — the store is unchanged.
Conflicting files:
$(printf '%s\n' "${conflicts:-(see git output below)}" | sed 's/^/ /')
$(printf '%s\n' "$out" | sed 's/^/ /')
Resolve by hand in $SECRETS_DIR, or ask the other machine to push again."
fi
info "Rebased onto origin."
fi
_sync_restore_stash "$stashed" || true
# Same reasoning as the push path: rebased-in history may lack .gitignore,
# and a store missing the key.txt line would stage the private key.
ensure_store_protections
# Re-read state — the rebase changed it.
st=$(_store_git_state) || st=$'0\t0\t0'
IFS=$'\t' read -r ahead behind dirty <<< "$st"
if [ "$ahead" -gt 0 ]; then
if [ "$assume_yes" != true ] && ! _sync_confirm_push "$ahead"; then
info "Reconciled locally. $ahead local commit(s) NOT published."
echo " Publish them with: secrets sync --yes"
return 0
fi
if ! out=$(git -C "$SECRETS_DIR" push 2>&1); then
die "Push failed ($SECRETS_DIR):
$(printf '%s\n' "$out" | sed 's/^/ /')"
fi
info "Published $ahead local commit(s)."
fi
st=$(_store_git_state) || st=$'0\t0\t0'
IFS=$'\t' read -r ahead behind dirty <<< "$st"
info "Store reconciled: $(_format_store_state "$ahead" "$behind" "$dirty")"
}
cmd_upgrade() {
local check_only=0
while [ $# -gt 0 ]; do
@ -3243,6 +3467,9 @@ Usage:
secrets recipients add KEY [--name N] Add a recipient and re-encrypt the store
secrets recipients rm KEY|NAME [--yes] Remove a recipient and re-encrypt the store
secrets reencrypt Re-encrypt every blob to the current recipients
secrets sync Reconcile a diverged store with its remote
secrets sync --dry-run Report ahead/behind/dirty; change nothing
secrets sync --yes Reconcile and publish local commits without prompting
secrets which Show the active store, manifest, and external entries
secrets where Alias for `which`
secrets status Alias for `which`
@ -3419,6 +3646,7 @@ case "${1:-help}" in
migrate) shift; cmd_migrate "$@" ;;
recipients) shift; cmd_recipients "$@" ;;
which|where|status) cmd_which ;;
sync) shift; cmd_sync "$@" ;;
upgrade) shift; cmd_upgrade "$@" ;;
help|--help|-h) cmd_help ;;
*) die "Unknown command: $1. Run 'secrets help' for usage." ;;

373
test/sync.bats Normal file
View file

@ -0,0 +1,373 @@
#!/usr/bin/env bats
# EGB-1230 / EGB-1231: store sync — loud failures on `pull`, and a `secrets
# sync` verb that reconciles a diverged store instead of dead-ending.
#
# bash 3.2 gotcha (see CLAUDE.md): every standalone [[ ]] assertion MUST end
# with `|| false`, or a failing assertion does not fail the test.
load test_helper
# ─── fixtures ──────────────────────────────────────────────────────────
#
# A "peer" is a second clone of the same bare remote. Committing + pushing
# from the peer is how we put the store under test *behind* its remote
# without touching the store itself.
peer_commit_and_push() {
local name="${1:-peer-file}" content="${2:-peer}"
local peer="$TEST_TMPDIR/peer"
if [ ! -d "$peer" ]; then
git clone "$REMOTE_DIR" "$peer" >/dev/null 2>&1
fi
( cd "$peer" && git pull >/dev/null 2>&1 || true )
echo "$content" > "$peer/$name"
( cd "$peer" && git add -A && git commit -m "peer: $name" >/dev/null && git push >/dev/null 2>&1 )
}
# Give the store a local-only commit (store becomes "ahead").
store_local_commit() {
local name="${1:-local-file}"
echo "local" > "$SECRETS_DIR/$name"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: $name" >/dev/null
}
# Leave an uncommitted modification in the store working tree ("dirty").
store_dirty() {
echo "scratch" > "$SECRETS_DIR/${1:-dirty-file}"
git -C "$SECRETS_DIR" add -A >/dev/null 2>&1 || true
}
store_head() { git -C "$SECRETS_DIR" rev-parse HEAD; }
# init_with_remote (shared helper) commits --allow-empty, which leaves the
# files `secrets init` wrote — .gitignore, .secrets-format, recipients.txt —
# untracked. That is a genuinely dirty store, so tests that assert on the
# "nothing to reconcile" path must land them first.
init_clean_store() {
init_with_remote
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "store: initial files" >/dev/null 2>&1 || true
git -C "$SECRETS_DIR" push >/dev/null 2>&1
}
# ─── EGB-1230: pull must never fail silently ───────────────────────────
@test "EGB-1230: pull surfaces git's diagnosis when the store can't fast-forward" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt # now diverged: ahead 1, behind 1
create_project_dir divproj
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
# The failure is named, not silent — this is the whole bug.
[[ "$output" == *"Store sync failed"* ]] || false
[[ "$output" == *"$SECRETS_DIR"* ]] || false
}
@test "EGB-1230: pull's sync failure points at secrets sync" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir divproj
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}
@test "EGB-1230: pull writes the diagnosis to stderr, not just stdout" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir divproj
run bash -c "'$SECRETS_BIN' pull 2>&1 1>/dev/null"
[[ "$output" == *"Store sync failed"* ]] || false
}
@test "EGB-1230: a clean store still pulls normally" {
init_with_remote
create_project_dir cleanproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
rm -f .env .env.staging
run "$SECRETS_BIN" pull
[ "$status" -eq 0 ]
[ -f .env ]
}
@test "EGB-1230: pull's sync is fast-forward only (no silent merge commit)" {
init_with_remote
create_project_dir ffproj
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
local before; before=$(store_head)
run "$SECRETS_BIN" pull
[ "$status" -ne 0 ]
# A merge commit would have moved HEAD. Nothing was integrated.
[ "$(store_head)" = "$before" ]
}
# ─── EGB-1231: which reports store state ───────────────────────────────
@test "EGB-1231: which reports ahead/behind/dirty for a diverged store" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
create_project_dir whichproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"remote:"* ]] || false
[[ "$output" == *"ahead 1"* ]] || false
[[ "$output" == *"behind 1"* ]] || false
}
@test "EGB-1231: which reports an in-sync store as up to date" {
init_clean_store
create_project_dir syncedproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"remote:"* ]] || false
[[ "$output" == *"up to date"* ]] || false
}
@test "EGB-1231: which stays quiet about the remote when none is configured" {
run "$SECRETS_BIN" init
create_project_dir noremote
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" != *"remote:"* ]] || false
}
@test "EGB-1231: which reports a dirty store working tree" {
init_with_remote
store_dirty scratch.age
create_project_dir dirtyproj
run "$SECRETS_BIN" which
[ "$status" -eq 0 ]
[[ "$output" == *"modified"* ]] || false
}
# ─── EGB-1231: secrets sync reconciles ─────────────────────────────────
@test "EGB-1231: sync rebases a diverged store onto the remote" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
# Both sides' work survives the reconcile.
[ -f "$SECRETS_DIR/remote-only.txt" ]
[ -f "$SECRETS_DIR/local-only.txt" ]
}
@test "EGB-1231: sync leaves the store able to pull again" {
init_with_remote
create_project_dir recovered
run "$SECRETS_BIN" push
[ "$status" -eq 0 ]
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
rm -f .env .env.staging
run "$SECRETS_BIN" pull
[ "$status" -eq 0 ]
[ -f .env ]
}
@test "EGB-1231: sync fast-forwards a store that is only behind" {
init_with_remote
peer_commit_and_push remote-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ -f "$SECRETS_DIR/remote-only.txt" ]
}
@test "EGB-1231: sync stashes and restores a dirty working tree" {
init_with_remote
peer_commit_and_push remote-only.txt
echo "uncommitted work" > "$SECRETS_DIR/scratch.age"
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ -f "$SECRETS_DIR/remote-only.txt" ]
# The local uncommitted blob edit is NOT lost.
[ -f "$SECRETS_DIR/scratch.age" ]
[ "$(cat "$SECRETS_DIR/scratch.age")" = "uncommitted work" ]
}
@test "EGB-1231: sync on an already-clean store reports no work and changes nothing" {
init_clean_store
local before; before=$(store_head)
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
[ "$(store_head)" = "$before" ]
[[ "$output" == *"up to date"* ]] || false
}
@test "EGB-1231: sync dies directed when the store has no remote" {
run "$SECRETS_BIN" init
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[[ "$output" == *"no remote"* ]] || false
}
# ─── EGB-1231: --dry-run changes nothing ───────────────────────────────
@test "EGB-1231: sync --dry-run reports state without mutating the store" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
local before; before=$(store_head)
run "$SECRETS_BIN" sync --dry-run
[ "$status" -eq 0 ]
[[ "$output" == *"ahead 1"* ]] || false
[[ "$output" == *"behind 1"* ]] || false
[ "$(store_head)" = "$before" ]
[ ! -f "$SECRETS_DIR/remote-only.txt" ]
}
@test "EGB-1231: sync --dry-run does not push local commits" {
init_with_remote
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --dry-run
[ "$status" -eq 0 ]
# The remote never received the local commit.
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" != *"local: local-only.txt"* ]] || false
}
# ─── EGB-1231: the push gate ───────────────────────────────────────────
@test "EGB-1231: sync --yes pushes reconciled local commits to the remote" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
run "$SECRETS_BIN" sync --yes
[ "$status" -eq 0 ]
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" == *"local: local-only.txt"* ]] || false
}
@test "EGB-1231: sync without confirmation reconciles locally but does not push" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
# No tty and no --yes: the push gate must not open on its own.
run bash -c "'$SECRETS_BIN' sync < /dev/null"
[ "$status" -eq 0 ]
# Local reconcile happened...
[ -f "$SECRETS_DIR/remote-only.txt" ]
# ...but nothing was published to the shared store.
run git -C "$REMOTE_DIR" log --oneline
[[ "$output" != *"local: local-only.txt"* ]] || false
}
@test "EGB-1231: sync says how to publish when the push gate stays closed" {
init_with_remote
store_local_commit local-only.txt
run bash -c "'$SECRETS_BIN' sync < /dev/null"
[ "$status" -eq 0 ]
[[ "$output" == *"--yes"* ]] || false
}
# ─── EGB-1231: conflicts restore the store as found ────────────────────
@test "EGB-1231: a rebase conflict leaves the store exactly as it was" {
init_with_remote
# Both sides edit the same path — a guaranteed rebase conflict.
peer_commit_and_push contested.txt "from-remote"
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
echo "from-local" > "$SECRETS_DIR/contested.txt"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null
local before; before=$(store_head)
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[ "$(store_head)" = "$before" ]
# No half-finished rebase left behind for the user to trip over.
[ ! -d "$SECRETS_DIR/.git/rebase-merge" ]
[ ! -d "$SECRETS_DIR/.git/rebase-apply" ]
[ "$(cat "$SECRETS_DIR/contested.txt")" = "from-local" ]
}
@test "EGB-1231: a rebase conflict names the conflicting files" {
init_with_remote
peer_commit_and_push contested.txt "from-remote"
git -C "$SECRETS_DIR" fetch origin >/dev/null 2>&1
echo "from-local" > "$SECRETS_DIR/contested.txt"
git -C "$SECRETS_DIR" add -A
git -C "$SECRETS_DIR" commit -m "local: contested" >/dev/null
run "$SECRETS_BIN" sync --yes
[ "$status" -ne 0 ]
[[ "$output" == *"contested.txt"* ]] || false
}
@test "EGB-1231: sync never force-pushes or hard-resets" {
# Guard rail on the implementation itself: this store holds the only
# copy of encrypted secrets, so the destructive git verbs must not
# appear anywhere in the sync path.
run bash -c "sed -n '/^cmd_sync()/,/^}/p' '$SECRETS_BIN'"
[[ "$output" != *"--force"* ]] || false
[[ "$output" != *"reset --hard"* ]] || false
[[ "$output" != *"stash drop"* ]] || false
}
# ─── EGB-1231: the push path's advice is no longer a dead end ──────────
@test "EGB-1231: push's fast-forward failure points at secrets sync" {
init_with_remote
peer_commit_and_push remote-only.txt
store_local_commit local-only.txt
create_project_dir pushproj
run "$SECRETS_BIN" push
[ "$status" -ne 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}
# ─── help ──────────────────────────────────────────────────────────────
@test "EGB-1231: sync is documented in help" {
run "$SECRETS_BIN" help
[ "$status" -eq 0 ]
[[ "$output" == *"secrets sync"* ]] || false
}