feat: secrets upgrade verb — self-update + re-check version skew (EGB-716)

Pairs the EGB-713 skew WARNING with a fix path. `secrets upgrade` fast-forwards
the tool's own git checkout (git -C "$SCRIPT_DIR" pull --ff-only; never merges or
rewrites local commits), reports vOLD -> vNEW, then best-effort re-checks the
store's recorded writer-version against the new version so the operator sees
whether the nudge is cleared. `secrets upgrade --check` reports availability
without pulling. Thin and explicit: no auto-update, no background polling
(security tool). Directed errors for not-a-checkout / no-upstream / diverged /
offline. cmd_upgrade never calls check_initialized (it's about the tool, not
the store); the skew re-check is silent unless a store with a writer-version
resolves.

Wired into the dispatcher (upgrade) shift; cmd_upgrade "$@") and cmd_help.
Tests: test/upgrade.bats (8) run a relocated script copy in a throwaway git
repo with a bare upstream, so the real checkout is never touched.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

CLAUDE.md

@@ -56,13 +56,13 @@ skips security specialist + red team, and Step 11 skips adversarial review.
 
 ## Architecture
 
-Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate.
+Single bash script (`secrets`) with subcommands: init, push, pull, list, rm, rekey, verify, migrate, upgrade.
 
 - Encryption: `age` with key files (not passphrases — age passphrases are non-scriptable)
 - Storage: Private git repo at `~/.secrets/`
 - Convention: Tracks `.env`, `.env.*`, and `.dev.vars` (not `.envrc`, `.environment-*`)
 - Manifest (EGB-677 stage 1): committed `.secrets.json` is the source of truth for what syncs — `dotenv[]` (project-relative, nested ok, `@` allowed; rail rejects `..`/absolute/symlink) + `external[]` (`properties`/`file`). Push discovery auto-adds (gated by committed `options.autoAdd`, default ON; `--frozen`/`--dry-run` overrides), bootstraps the manifest on first push (written only after ≥1 blob encrypts), and absorbs a legacy `.secrets-files` (gradle-properties → `properties`; on pull the legacy file is superseded with a warning). Store layout: nested dotenv entries land at `<project>/<relpath>.age` (relpath preserved — the store self-describes where a file restores). jq is a hard dep only when a manifest exists/is written; manifest-less projects run jq-free (manifest features skipped with a notice). `check_cmd` prints platform-aware install hints.
-- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at <path>, version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-<sha>` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka.
+- Store format (EGB-677 stage 2 / EGB-703): the store is self-describing via a committed one-line `$SECRETS_DIR/.secrets-format` file (`2`). Absence ⇒ v1 (every store predating EGB-703). v2's only on-disk change vs v1 is the external `properties` blob suffix: `.gradle-properties.age` → `.properties.age` (matching the manifest `type`); dotenv and `file` blobs are unchanged. `_store_format()` reads the marker. **Additive v2 (EGB-712):** reads resolve a `properties` blob by trying `.properties.age` then falling back to `.gradle-properties.age` (`_resolve_external_blob_read`); writes dual-write a `properties` external only when a v1 twin already exists in the store (`_external_blob_write_targets`), so existing externals keep old clients fresh while brand-new externals are written v2-only (a gentle forcing function). Blob location no longer depends on the marker — the old `_external_blob_suffix` is gone. `init` stamps a fresh store v2 (born-v2). `secrets which` prints the store-format line `format: vN`, and (EGB-700) when a `.secrets.json` is present the manifest header line also carries its schema version (`manifest (.secrets.json at <path>, version N):`). **Migration is copy-forward and non-destructive:** `secrets migrate --dry-run` (per project, reports old→new, writes nothing) → `secrets migrate` (per project, manifest-free: enumerates the store's `*.gradle-properties.age` blobs directly — same source of truth as `--finalize` — and writes their `.properties.age` twins, so a legacy `.secrets-files`-only project with no `.secrets.json` migrates cleanly and no store blob is left un-twinned; idempotent; EGB-710) → `secrets migrate --finalize` (store-wide; the ONLY destructive step — gates on `verify --all` green + every v1 blob having a v2 twin, cuts a `pre-v2-migrate-<sha>` recovery tag, stamps the marker, then drops v1 blobs; refuses without `--yes`/operator confirmation since a lagging v1 client against a finalized store stops seeing `properties` externals until it upgrades). `secrets migrate --status` is a read-only survey that walks every project in the store and reports each one's v2 readiness (v2-ready / migrated / NEEDS MIGRATE, plus a `v2-only` count of externals old clients can't read), exiting non-zero while any v1 blob is un-twinned so it gates the path to `--finalize` (EGB-710/EGB-712). **Under additive v2 (EGB-712) `--finalize` is now OPTIONAL GC, not a required milestone:** because upgraded clients dual-write existing externals and read-fall-back, *not* finalizing never cuts anyone off — finalize only reclaims the duplicate v1 blobs and stays deferrable indefinitely (defusing the cross-machine coordination gate). dotenv and `file` blobs are identical across formats, so they always propagate to old clients; only a brand-new `properties` external is v2-only. **Version-skew nudge (EGB-713):** a committed `$SECRETS_DIR/.secrets-writer-version` records the highest client `VERSION` that has written to the store (monotonic; stamped via `_stamp_writer_version` right before each store-committing `git add -A` — push/rekey/migrate/finalize — never on read paths, so it always rides a commit and never dangles to break `pull --ff-only`). `check_initialized` calls `_check_store_version_skew`, which warns once per invocation (stderr, non-fatal, `set -e`-safe) when the store's stamp is numerically greater than `_client_version` (read from `$SCRIPT_DIR/VERSION`); `secrets which` prints the `written-by:` line. Stores with no stamp (pre-EGB-713) are silent. The deliberate flatten-to-basename naming the EGB-677 CEO plan sketched was dropped as lossy (it discards the restore relpath that makes the store self-describing) — see the EGB-703 eureka. **Upgrade verb (EGB-716):** `secrets upgrade` is the fix path paired with the EGB-713 skew *warning* — it `git -C "$SCRIPT_DIR" pull --ff-only`s the tool's own checkout (fast-forward only, never merges/rewrites local commits), reports `vOLD -> vNEW`, then best-effort re-checks `_store_writer_version` against the new on-disk version so the operator sees whether the nudge is cleared (the new code takes effect next invocation). `secrets upgrade --check` does `git fetch` + `rev-list --count HEAD..@{u}` and reports availability without pulling. Deliberately thin: no auto-update, no background polling (security tool). Directed errors for not-a-git-checkout / no-upstream / diverged / offline. `cmd_upgrade` never calls `check_initialized` (it's about the tool, not the store); the skew re-check is silent unless a store with a writer-version resolves.
 - Verify (EGB-698): `secrets verify` is a read-only integrity check. Default mode (current project) cross-checks `$PWD/.secrets.json` against `$SECRETS_DIR/<project>/` both ways (declared-but-missing blobs + orphaned blobs) and decrypt-tests every blob (dotenv + external) by streaming plaintext to `/dev/null` (never written to disk). `secrets verify --all` decrypt-tests every blob in every project (integrity only — the store carries no manifests, so consistency can't be checked store-wide). Both recurse the whole project tree (`find -type f`, same as rekey/list). Exits non-zero on any finding so it can gate the stage-2 `migrate --finalize` and CI. The store deliberately holds no manifest — `.secrets.json` is committed in each project's own repo and read from `$PWD`.
 - External files: `.secrets-files` manifest tracks designated keys from files outside the project (e.g. `~/.gradle/gradle.properties`, merged not overwritten — EGB-531) and whole binary files (type `file`, e.g. an Android upload keystore — EGB-652); see below
 - Workspaces: `--workspaces` flag reads `package.json` workspaces, requires `jq`
@@ -78,6 +78,7 @@ test/
   secrets.bats       # bats-core test suite (140 tests)
   manifest.bats      # EGB-677 .secrets.json manifest tests (83 tests)
   migrate.bats       # EGB-703 store-format-v2 migration tests (35 tests)
+  upgrade.bats       # EGB-716 `secrets upgrade` self-update tests (8 tests)
   test_helper.bash   # Shared setup/teardown
 README.md            # User-facing documentation
 CLAUDE.md            # This file

README.md

@@ -187,6 +187,8 @@ secrets clear
 | `secrets migrate [--dry-run]` | Copy-forward this project's encrypted blobs to store format v2 (non-destructive; manifest-free; `--dry-run` previews) |
 | `secrets migrate --status` | Survey every project's v2 readiness; exits non-zero until the whole store is finalize-ready |
 | `secrets migrate --finalize` | **Optional GC** — drop the old v1 blobs and mark the store pure v2. Never required: upgraded clients dual-write and read-fall-back, so not finalizing never cuts anyone off |
+| `secrets upgrade` | Self-update the tool: `git pull --ff-only` on the `secrets` checkout, report old → new version, then re-check store version-skew. No auto-update, no background checks |
+| `secrets upgrade --check` | Report whether an update is available (without pulling); changes nothing |
 
 ### Upgrading: do teammates on an older `secrets` get new secrets?
 

secrets

@@ -2375,6 +2375,90 @@ cmd_which() {
   fi
 }
 
+# `secrets upgrade [--check]` (EGB-716) — self-update the TOOL checkout.
+#
+# Pairs the EGB-713 skew WARNING with a fix path. Deliberately thin and explicit
+# (no auto-update, no background polling — this is a security tool): it only
+# fast-forwards the tool's own git checkout, never merges or rewrites local
+# commits. --check reports whether an update is available and changes nothing.
+# After a real update it best-effort re-checks the store's writer-version skew
+# against the NEW on-disk version, so the operator sees whether the EGB-713
+# nudge is now cleared (the new code itself takes effect on the next command).
+cmd_upgrade() {
+  local check_only=0
+  while [ $# -gt 0 ]; do
+    case "$1" in
+      --check) check_only=1; shift ;;
+      -*) die "Unknown upgrade flag: $1. Usage: secrets upgrade [--check]" ;;
+      *)  die "Unexpected argument to upgrade: $1. Usage: secrets upgrade [--check]" ;;
+    esac
+  done
+
+  check_cmd git
+
+  # Self-update only works on a git checkout of the tool.
+  if ! git -C "$SCRIPT_DIR" rev-parse --is-inside-work-tree >/dev/null 2>&1; then
+    die "secrets at $SCRIPT_DIR is not a git checkout, so it can't self-update.
+  Re-install by cloning the tool repo, e.g.: git clone <tool-remote> <dir>"
+  fi
+
+  # Need a tracking branch to compare against / pull from.
+  local upstream
+  upstream=$(git -C "$SCRIPT_DIR" rev-parse --abbrev-ref --symbolic-full-name '@{u}' 2>/dev/null || true)
+  if [ -z "$upstream" ]; then
+    die "No upstream tracking branch for the secrets checkout at $SCRIPT_DIR.
+  Set one with: git -C \"$SCRIPT_DIR\" branch --set-upstream-to=origin/main"
+  fi
+
+  local oldver; oldver=$(_client_version)
+
+  if [ "$check_only" = 1 ]; then
+    if ! git -C "$SCRIPT_DIR" fetch --quiet 2>/dev/null; then
+      die "Couldn't reach the tool remote to check for updates (offline?).
+  Try again when connected, or run: git -C \"$SCRIPT_DIR\" fetch"
+    fi
+    local behind; behind=$(git -C "$SCRIPT_DIR" rev-list --count "HEAD..$upstream" 2>/dev/null || echo 0)
+    if [ "${behind:-0}" -gt 0 ]; then
+      info "Update available: $behind commit(s) behind $upstream (you're on v$oldver)."
+      info "Apply it with: secrets upgrade"
+    else
+      info "secrets is up to date (v$oldver)."
+    fi
+    return 0
+  fi
+
+  info "Updating secrets at $SCRIPT_DIR ..."
+  # Fast-forward only: never merge or rewrite local commits.
+  if ! git -C "$SCRIPT_DIR" pull --ff-only 2>&1; then
+    die "Update failed (see git output above).
+  Likely a local change or a diverged branch in $SCRIPT_DIR.
+  Inspect with: git -C \"$SCRIPT_DIR\" status"
+  fi
+
+  local newver; newver=$(_client_version)
+  if [ "$oldver" = "$newver" ]; then
+    info "Already up to date (v$newver)."
+  else
+    info "Upgraded: v$oldver -> v$newver"
+    info "The new version takes effect on your next 'secrets' command."
+  fi
+
+  # Best-effort EGB-713 skew re-check against the NEW version. Silent unless a
+  # store with a recorded writer-version resolves.
+  resolve_store 2>/dev/null || true
+  if [ -n "${SECRETS_DIR:-}" ] && [ -f "$SECRETS_DIR/$WRITER_VERSION_FILE_NAME" ]; then
+    local sv; sv=$(_store_writer_version)
+    if [ -n "$sv" ]; then
+      if _version_gt "$sv" "$newver"; then
+        info "Note: the store was last written by v$sv — still ahead of v$newver. Another machine may run a newer client."
+      else
+        info "Your client (v$newver) is now at or ahead of the store's last writer (v$sv)."
+      fi
+    fi
+  fi
+  return 0
+}
+
 # Decrypt-test one blob with the current key. Plaintext is streamed to
 # /dev/null and never written to disk (read-only contract). Returns 0 if the
 # blob decrypts, non-zero otherwise.
@@ -2783,6 +2867,8 @@ Usage:
   secrets which                 Show the active store, manifest, and external entries
   secrets where                 Alias for `which`
   secrets status                Alias for `which`
+  secrets upgrade               Self-update the tool (git pull --ff-only) + recheck skew
+  secrets upgrade --check       Report whether an update is available; change nothing
 
 Tracked files: .env, .env.*, .dev.vars
 
@@ -2952,6 +3038,7 @@ case "${1:-help}" in
   verify) shift; cmd_verify "$@" ;;
   migrate) shift; cmd_migrate "$@" ;;
   which|where|status) cmd_which ;;
+  upgrade) shift; cmd_upgrade "$@" ;;
   help|--help|-h) cmd_help ;;
   *)      die "Unknown command: $1. Run 'secrets help' for usage." ;;
 esac

test/upgrade.bats

@@ -0,0 +1,115 @@
+#!/usr/bin/env bats
+# EGB-716: `secrets upgrade` verb — self-update (git pull --ff-only) + skew re-check.
+#
+# These tests never touch the real tool checkout. Each test relocates a COPY of
+# the script into a throwaway git repo wired to a bare upstream, so $SCRIPT_DIR
+# (computed from BASH_SOURCE) resolves to the fake tool repo and the pull/fetch
+# operate there.
+
+load test_helper
+
+# Create a fake tool repo at $TOOL (script copy + VERSION), wired to a bare
+# upstream at $TOOL_REMOTE, at version $1. cd's into $TOOL (under $HOME so
+# resolve_store's walk-up stays bounded and never strays to a real store).
+setup_tool_repo() {
+  TOOL="$TEST_TMPDIR/tool"
+  TOOL_REMOTE="$TEST_TMPDIR/tool-remote.git"
+  mkdir -p "$TOOL"
+  cp "$SECRETS_BIN" "$TOOL/secrets"
+  echo "$1" > "$TOOL/VERSION"
+  git -c init.defaultBranch=main init -q "$TOOL"
+  git -C "$TOOL" add -A
+  git -C "$TOOL" -c user.email=t@t -c user.name=t commit -qm "v$1"
+  git -c init.defaultBranch=main init --bare -q "$TOOL_REMOTE"
+  git -C "$TOOL" remote add origin "$TOOL_REMOTE"
+  git -C "$TOOL" push -q -u origin HEAD:main
+  cd "$TOOL"
+}
+
+# Publish a newer VERSION to the upstream (as a different clone would).
+advance_tool_remote() {
+  local clone="$TEST_TMPDIR/tool-pub"
+  rm -rf "$clone"
+  git clone -q "$TOOL_REMOTE" "$clone"
+  echo "$1" > "$clone/VERSION"
+  git -C "$clone" -c user.email=t@t -c user.name=t commit -qam "v$1"
+  git -C "$clone" push -q origin HEAD:main
+  rm -rf "$clone"
+}
+
+@test "upgrade --check reports an available update without changing VERSION (EGB-716)" {
+  setup_tool_repo 0.1.0.0
+  advance_tool_remote 0.2.0.0
+  run "$TOOL/secrets" upgrade --check
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"Update available"* ]] || false
+  [[ "$output" == *"0.1.0.0"* ]] || false
+  # --check must not pull: local VERSION is untouched.
+  [ "$(cat "$TOOL/VERSION")" = "0.1.0.0" ]
+}
+
+@test "upgrade --check is clean when already current (EGB-716)" {
+  setup_tool_repo 0.2.0.0
+  run "$TOOL/secrets" upgrade --check
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"up to date"* ]] || false
+}
+
+@test "upgrade fast-forwards and reports old -> new (EGB-716)" {
+  setup_tool_repo 0.1.0.0
+  advance_tool_remote 0.2.0.0
+  run "$TOOL/secrets" upgrade
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false
+  [ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ]
+}
+
+@test "upgrade is a no-op when already at the latest (EGB-716)" {
+  setup_tool_repo 0.2.0.0
+  run "$TOOL/secrets" upgrade
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"up to date"* ]] || false
+  [ "$(cat "$TOOL/VERSION")" = "0.2.0.0" ]
+}
+
+@test "upgrade refuses when the tool dir is not a git checkout (EGB-716)" {
+  local d="$HOME/plain-tool"
+  mkdir -p "$d"
+  cp "$SECRETS_BIN" "$d/secrets"
+  echo 0.1.0.0 > "$d/VERSION"
+  cd "$d"
+  run "$d/secrets" upgrade
+  [ "$status" -eq 1 ]
+  [[ "$output" == *"git checkout"* ]] || false
+}
+
+@test "upgrade rejects an unknown flag (EGB-716)" {
+  setup_tool_repo 0.1.0.0
+  run "$TOOL/secrets" upgrade --bogus
+  [ "$status" -eq 1 ]
+  [[ "$output" == *"Unknown upgrade flag"* ]] || false
+}
+
+@test "upgrade re-checks store skew and confirms the client caught up (EGB-716)" {
+  setup_tool_repo 0.1.0.0
+  advance_tool_remote 0.9.0.0
+  # A store last written by a newer client than our starting version.
+  git -c init.defaultBranch=main init -q "$SECRETS_DIR"
+  echo 0.8.0.0 > "$SECRETS_DIR/.secrets-writer-version"
+  run "$TOOL/secrets" upgrade
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"v0.1.0.0 -> v0.9.0.0"* ]] || false
+  # New client (0.9.0.0) is now ahead of the store's last writer (0.8.0.0).
+  [[ "$output" == *"at or ahead"* ]] || false
+}
+
+@test "upgrade still notes when the store is ahead of the upgraded client (EGB-716)" {
+  setup_tool_repo 0.1.0.0
+  advance_tool_remote 0.2.0.0
+  git -c init.defaultBranch=main init -q "$SECRETS_DIR"
+  echo 0.9.0.0 > "$SECRETS_DIR/.secrets-writer-version"
+  run "$TOOL/secrets" upgrade
+  [ "$status" -eq 0 ]
+  [[ "$output" == *"v0.1.0.0 -> v0.2.0.0"* ]] || false
+  [[ "$output" == *"still ahead"* ]] || false
+}