secrets/docs/superpowers/plans/2026-06-08-egb-710-migrate-guided-flow.md

EGB-710: secrets migrate guided flow (manifest-free copy-forward + --status survey) Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Replace the dead-end "No .secrets.json — cd into a project that has a manifest" error in secrets migrate with a manifest-free copy-forward that just works, plus a secrets migrate --status survey that tells the operator exactly which projects still need migrating.

Architecture: The per-project copy-forward step (_migrate_project) currently reads $PWD/.secrets.json to find properties externals, then looks for their v1 blobs. This makes it die on a legacy .secrets-files-only project (the real EGB-710 repro) and — worse — silently skips any store blob the manifest doesn't declare, leaving it un-twinned so --finalize later refuses it. The fix: enumerate the store ($SECRETS_DIR/<project>/external/*.gradle-properties.age) directly and copy each to its .properties.age twin via suffix-swap — the exact logic _migrate_finalize already uses. No manifest needed, dotenv/file-only projects become a clean no-op, and migrate twins precisely the blobs finalize will demand twins for. A new --status mode walks every project dir and reports per-project readiness + whether the store is finalize-ready.

Tech Stack: Single bash 3.2 script (secrets), age, git, jq (unaffected here). Tests: bats-core (test/migrate.bats). Every standalone [[ ]] assertion ends with || false (bash 3.2 ERR-trap gotcha).

---

Background facts (verified against HEAD 633d19e)

• _migrate_project() lives at secrets:2135; it dies at secrets:2142-2148 on missing $PWD/.secrets.json, then iterates _json_external_entries "$manifest" filtered to gradle-properties (secrets:2154-2174).
• _migrate_finalize() (secrets:2196) already enumerates blobs manifest-free: find "$SECRETS_DIR" -type f -name '*.gradle-properties.age' and derives the twin as new="${f%.gradle-properties.age}.properties.age" (secrets:2212-2217). The per-project step will mirror this, scoped to one project dir.
• cmd_migrate() flag parser: secrets:2267-2285.
• derive_project_name "" (secrets:101) needs no manifest — it uses the git remote basename or basename "$PWD".
• info()/die(): secrets:36-37. ensure_store_protections is called post-write in the existing copy-forward.
• Output-contract strings existing tests depend on (must be preserved): "would migrate" (migrate.bats:80), "0 blob(s) would be copy-forwarded" (migrate.bats:94), "1 already present" (migrate.bats:121, idempotent re-run prints $already already present), "already format v2" (migrate.bats:139).
• The one test that codifies the OLD dead-end — "migrate with no manifest in cwd dies with a directed message" (migrate.bats:126) — is the behavior we are intentionally changing; it gets rewritten in Task 1.
• No test asserts the non-dry-run "nothing to migrate" wording (grep confirmed), so that message is free to change. We keep the substring no v1 properties blobs regardless for safety.
• VERSION is 0.6.0.1; bump to 0.6.1.0 (new subcommand flag + behavior change). MANIFEST_VERSION stays 2 (no schema change).

File structure

• Modify: secrets — rewrite _migrate_project (secrets:2135-2191), add _migrate_status, extend cmd_migrate flag parsing + dispatch (secrets:2267-2285), update cmd_help migrate lines (secrets:2308-2309).
• Modify: test/migrate.bats — rewrite the dead-end test; add manifest-free, finalize-consistency, and --status tests.
• Modify: CLAUDE.md — update the migrate paragraph (Architecture → Store format) to note manifest-free copy-forward + --status.
• Modify: README.md — migrate usage/help.
• Modify: VERSION → 0.6.1.0; CHANGELOG.md — new entry.

---

Task 1: Manifest-free per-project copy-forward

Files:

• Modify: secrets — _migrate_project() (secrets:2135-2191)

• Test: test/migrate.bats

• Step 1: Write the failing test — migrate works with no .secrets.json (the EGB-710 repro)

Add to test/migrate.bats (after the existing copy-forward tests, ~line 124):

@test "migrate copy-forwards a v1 properties blob with no .secrets.json (manifest-free)" {
  # The EGB-710 repro: a legacy project has a v1 properties blob in the store
  # but no .secrets.json (it predates the manifest). migrate must NOT dead-end.
  make_v1_store
  m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
  create_project_dir nomanifestblob
  printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
  "$SECRETS_BIN" push nomanifestblob >/dev/null 2>&1
  rm -f .secrets.json                       # simulate a pre-manifest project
  run "$SECRETS_BIN" migrate
  [ "$status" -eq 0 ]
  run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.properties.age"
  [ "$status" -eq 0 ]                        # v2 twin written despite no manifest
  run bash -c "ls $SECRETS_DIR/nomanifestblob/external/*.gradle-properties.age"
  [ "$status" -eq 0 ]                        # v1 kept (non-destructive)
}

• Step 2: Run it to confirm it fails

Run: bats test/migrate.bats -f "manifest-free" Expected: FAIL — current code dies with "No .secrets.json in …" (status 1), so the first [ "$status" -eq 0 ] fails.

• Step 3: Rewrite _migrate_project to enumerate the store, not the manifest

Replace the body of _migrate_project() (secrets:2135-2191, from _migrate_project() { through its closing }) with:

_migrate_project() {
  local dry_run="$1"
  if [ "$(_store_format)" = "2" ]; then
    info "Store is already format v2 — nothing to migrate."
    return 0
  fi
  local project; project=$(derive_project_name "")
  local pdir="$SECRETS_DIR/$project"

  # Source of truth for the copy-forward is the STORE, not a project manifest.
  # Every v1 properties blob is a `*.gradle-properties.age` file whose v2 twin
  # is the same name with the `.properties.age` suffix (the only on-disk change
  # v2 makes). Enumerating the store — exactly as `_migrate_finalize` does —
  # means migrate twins precisely the blobs finalize will demand twins for, with
  # no manifest dependency. This is why a legacy `.secrets-files`-only project
  # (no `.secrets.json` yet) migrates cleanly instead of dead-ending, and why a
  # store blob the manifest no longer declares still gets a twin.
  local moved=0 already=0 would=0 v1count=0 f new
  while IFS= read -r f; do
    [ -f "$f" ] || continue
    v1count=$((v1count + 1))
    new="${f%.gradle-properties.age}.properties.age"
    if [ -f "$new" ]; then
      already=$((already + 1))
      continue
    fi
    if [ "$dry_run" = true ]; then
      echo "would migrate: ${f#"$SECRETS_DIR"/} -> $(basename "$new")"
      would=$((would + 1))
    else
      cp "$f" "$new"
      moved=$((moved + 1))
    fi
  done < <(find "$pdir/external" -type f -name '*.gradle-properties.age' 2>/dev/null)

  if [ "$dry_run" = true ]; then
    echo "migrate --dry-run: $would blob(s) would be copy-forwarded for '$project' (writes nothing); $already already present. v1 blobs are kept until 'secrets migrate --finalize'."
    return 0
  fi
  if [ "$moved" -eq 0 ] && [ "$already" -eq 0 ]; then
    info "Nothing to migrate for '$project' — no v1 properties blobs in the store (already v2-shaped). If you expected one, run 'secrets push' first, then re-run 'secrets migrate'."
    return 0
  fi
  ensure_store_protections
  git -C "$SECRETS_DIR" add -A
  git -C "$SECRETS_DIR" commit -m "migrate: copy-forward v2 twins for $project" >/dev/null 2>&1 || true
  # Push the twins so a --finalize on another machine sees them (finalize
  # refuses any v1 blob without a twin). Mirrors push/rekey's push behavior.
  git -C "$SECRETS_DIR" remote get-url origin >/dev/null 2>&1 && git -C "$SECRETS_DIR" push >/dev/null 2>&1 || true
  info "Copy-forward for '$project': $moved new v2 twin(s), $already already present. v1 blobs kept (non-destructive). Run 'secrets migrate --finalize' once every project is migrated and every machine is upgraded."
}

Notes for the implementer:

• This removes the only in-script caller of _check_manifest_file/_json_external_entries inside migrate; both remain defined and used by push/pull/verify, so do not delete them.

• ${f#"$SECRETS_DIR"/} keeps the dry-run line's store-relative form (matches the old $project/external/... style closely enough; the test only checks the substring would migrate).

• The moved==0 && already==0 branch keeps the substring no v1 properties blobs. The idempotent re-run path (moved==0, already>0) falls through to the final info printing $already already present, preserving the 1 already present contract.

• Step 4: Run the new test + the full migrate suite

Run: bats test/migrate.bats Expected: the new "manifest-free" test PASSES; existing dry-run/copy-forward/idempotent/finalize tests still PASS; the "migrate with no manifest in cwd dies" test (migrate.bats:126) now FAILS (we fix it in Step 5).

• Step 5: Update the test that codified the old dead-end

Replace the test at test/migrate.bats:126-132 ("migrate with no manifest in cwd dies with a directed message") with:

@test "migrate in a project with no manifest and no store blobs is a clean no-op" {
  make_v1_store
  local dir="$WORK_DIR/nomanifest"; mkdir -p "$dir"; cd "$dir"
  run "$SECRETS_BIN" migrate
  [ "$status" -eq 0 ]
  [[ "$output" == *"no v1 properties blobs"* ]] || false
}

• Step 6: Run the full suite to confirm green

Run: bats test/migrate.bats Expected: all PASS.

• Step 7: Commit

git add secrets test/migrate.bats
git commit -m "fix: migrate copy-forward is manifest-free, no dead-end on legacy projects (EGB-710)"

---

Task 2: Finalize-consistency regression test (store blob not in manifest)

Files:

• Test: test/migrate.bats

This proves the latent-bug fix: the old manifest-driven migrate skipped store blobs the manifest didn't declare, leaving them un-twinned so --finalize refused them. The rewrite twins them.

• Step 1: Write the test

Add to test/migrate.bats:

@test "migrate twins a store blob even when the manifest no longer declares it" {
  make_v1_store
  m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
  create_project_dir staleblob
  printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
  "$SECRETS_BIN" push staleblob >/dev/null 2>&1
  # The blob is now in the store. Drop the external from the project's manifest
  # entirely (and remove the legacy file) so NO manifest declares it.
  printf '{"version":2,"dotenv":[".env",".env.staging"]}\n' > .secrets.json
  rm -f .secrets-files
  run "$SECRETS_BIN" migrate
  [ "$status" -eq 0 ]
  run bash -c "ls $SECRETS_DIR/staleblob/external/*.properties.age"
  [ "$status" -eq 0 ]   # twinned despite not being declared anywhere
}

• Step 2: Run it

Run: bats test/migrate.bats -f "no longer declares" Expected: PASS (the Task 1 rewrite already makes this green — this test guards against regressing back to manifest-driven enumeration).

• Step 3: Commit

git add test/migrate.bats
git commit -m "test: migrate twins undeclared store blobs (finalize-consistency, EGB-710)"

---

Task 3: secrets migrate --status survey

Files:

• Modify: secrets — add _migrate_status(); extend cmd_migrate (secrets:2267-2285)

• Test: test/migrate.bats

• Step 1: Write the failing tests

Add to test/migrate.bats:

@test "migrate --status flags a project that needs migrating" {
  make_v1_store
  m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
  create_project_dir needsmig
  printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
  "$SECRETS_BIN" push needsmig >/dev/null 2>&1   # v1 blob, no twin yet
  run "$SECRETS_BIN" migrate --status
  [ "$status" -ne 0 ]                            # not finalize-ready
  [[ "$output" == *"needsmig"* ]] || false
  [[ "$output" == *"NEEDS MIGRATE"* ]] || false
  [[ "$output" == *"Not finalize-ready"* ]] || false
}

@test "migrate --status reports finalize-ready once every blob is twinned" {
  make_v1_store
  m_gradle_src $'beaconClerkPkTest=pk_test_abc\n'
  create_project_dir readymig
  printf 'gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest\n' > .secrets-files
  "$SECRETS_BIN" push readymig >/dev/null 2>&1
  "$SECRETS_BIN" migrate >/dev/null 2>&1         # create the twin
  run "$SECRETS_BIN" migrate --status
  [ "$status" -eq 0 ]
  [[ "$output" == *"Finalize-ready"* ]] || false
}

@test "migrate --status on an already-v2 store says nothing to do" {
  init_with_remote
  create_project_dir v2status
  run "$SECRETS_BIN" migrate --status
  [ "$status" -eq 0 ]
  [[ "$output" == *"v2"* ]] || false
}

• Step 2: Run them to confirm they fail

Run: bats test/migrate.bats -f "status" Expected: FAIL — --status is an unknown flag today (die "Unknown migrate flag: --status…", status 1), so the assertions fail.

• Step 3: Add _migrate_status (place it just before _migrate_finalize, ~secrets:2195)

# Read-only survey: walk every project dir in the store and report each one's
# v2-readiness from the blobs on disk (no manifest, no decryption). Exits
# non-zero when any v1 properties blob lacks a v2 twin (i.e. the store is not
# yet finalize-ready) so it can gate scripting, mirroring `verify`'s posture.
_migrate_status() {
  if [ "$(_store_format)" = "2" ]; then
    info "Store format: v2 (finalized) — nothing to migrate."
    return 0
  fi
  echo "Store format: v1 (not finalized). Per-project migration status:"
  local any_untwinned=0 dir project v1 untwinned f new
  for dir in "$SECRETS_DIR"/*/; do
    [ -d "$dir" ] || continue
    project=$(basename "$dir")
    case "$project" in .*) continue ;; esac
    v1=0; untwinned=0
    while IFS= read -r f; do
      [ -f "$f" ] || continue
      v1=$((v1 + 1))
      new="${f%.gradle-properties.age}.properties.age"
      [ -f "$new" ] || untwinned=$((untwinned + 1))
    done < <(find "$dir" -type f -name '*.gradle-properties.age' 2>/dev/null)
    if [ "$v1" -eq 0 ]; then
      echo "  $project: v2-ready (no v1 properties blobs)"
    elif [ "$untwinned" -eq 0 ]; then
      echo "  $project: migrated ($v1 v1 blob(s), all twinned)"
    else
      echo "  $project: NEEDS MIGRATE ($untwinned of $v1 v1 blob(s) un-twinned) — cd into the project and run 'secrets migrate'"
      any_untwinned=1
    fi
  done
  echo
  if [ "$any_untwinned" -eq 1 ]; then
    echo "Not finalize-ready: migrate the projects marked NEEDS MIGRATE, then run 'secrets migrate --finalize'."
    return 1
  fi
  echo "Finalize-ready: every v1 properties blob has a v2 twin. Run 'secrets migrate --finalize' once every machine is upgraded."
  return 0
}

• Step 4: Wire --status into cmd_migrate

In cmd_migrate() (secrets:2267), add the status local and parse + dispatch. Replace:

  local dry_run=false finalize=false force=false
  while [ $# -gt 0 ]; do
    case "$1" in
      --dry-run)     dry_run=true; shift ;;
      --finalize)    finalize=true; shift ;;
      --yes)         force=true; shift ;;
      -*)            die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --finalize] [--yes]" ;;
      *)             die "migrate takes no project argument. Run it from inside a project (copy-forward) or use --finalize (store-wide)." ;;
    esac
  done
  if [ "$finalize" = true ]; then
    _migrate_finalize "$force"
  else
    _migrate_project "$dry_run"
  fi

with:

  local dry_run=false finalize=false force=false status=false
  while [ $# -gt 0 ]; do
    case "$1" in
      --dry-run)     dry_run=true; shift ;;
      --finalize)    finalize=true; shift ;;
      --status)      status=true; shift ;;
      --yes)         force=true; shift ;;
      -*)            die "Unknown migrate flag: $1. Usage: secrets migrate [--dry-run | --status | --finalize] [--yes]" ;;
      *)             die "migrate takes no project argument. Run it from inside a project (copy-forward), or use --status / --finalize (store-wide)." ;;
    esac
  done
  if [ "$status" = true ]; then
    _migrate_status
  elif [ "$finalize" = true ]; then
    _migrate_finalize "$force"
  else
    _migrate_project "$dry_run"
  fi

• Step 5: Run the status tests + full suite

Run: bats test/migrate.bats Expected: all PASS.

• Step 6: Commit

git add secrets test/migrate.bats
git commit -m "feat: secrets migrate --status surveys per-project v2 readiness (EGB-710)"

---

Task 4: Docs, help text, version, changelog

Files:

• Modify: secrets — cmd_help (secrets:2308-2309)

• Modify: CLAUDE.md, README.md, VERSION, CHANGELOG.md

• Step 1: Update cmd_help migrate lines

In cmd_help() replace the two migrate lines (secrets:2308-2309):

  secrets migrate [--dry-run]   Copy-forward this project's blobs to store format v2
  secrets migrate --finalize    Drop v1 blobs and mark the store v2 (after verify)

with:

  secrets migrate [--dry-run]   Copy-forward this project's v1 blobs to store format v2
  secrets migrate --status      Survey every project's v2 readiness (finalize gate)
  secrets migrate --finalize    Drop v1 blobs and mark the store v2 (after verify)

• Step 2: Update CLAUDE.md migrate paragraph

In the "Store format (EGB-677 stage 2 / EGB-703)" bullet, update the migration description: copy-forward is now manifest-free — secrets migrate enumerates the project's *.gradle-properties.age store blobs directly (same source of truth as --finalize), so a legacy .secrets-files-only project migrates without a .secrets.json and no store blob is left un-twinned. Add secrets migrate --status to the workflow line as the read-only survey that reports per-project readiness and gates --finalize. Reference EGB-710.

• Step 3: Update README.md

Find the migrate section/help block and add the --status line and the manifest-free note, mirroring the help text.

• Step 4: Bump VERSION

Set VERSION to 0.6.1.0.

• Step 5: Add CHANGELOG.md entry

Insert above ## [0.6.0.1]:

## [0.6.1.0] - 2026-06-08

### Changed

- **`secrets migrate` copy-forward is now manifest-free (EGB-710)** — the
  per-project step enumerates the store's `*.gradle-properties.age` blobs
  directly (the same source of truth `--finalize` uses) instead of reading
  `.secrets.json`. A legacy `.secrets-files`-only project now migrates cleanly
  instead of dead-ending with "No .secrets.json", and a store blob the manifest
  no longer declares still gets a v2 twin (so `--finalize` won't refuse it).
  Running migrate in a project with no v1 properties blobs is a clean no-op.

### Added

- **`secrets migrate --status`** — a read-only survey that walks every project
  in the store and reports its v2 readiness (v2-ready / migrated / NEEDS
  MIGRATE), then whether the store as a whole is finalize-ready. Exits non-zero
  while any v1 blob is un-twinned, so it can gate the path to `--finalize`.

• Step 6: Run the full bats suite (all files)

Run: bats test/ Expected: all PASS (secrets.bats + manifest.bats + migrate.bats). Confirm the count went up by the tests added here.

• Step 7: Commit

git add secrets CLAUDE.md README.md VERSION CHANGELOG.md
git commit -m "docs: migrate --status + manifest-free copy-forward; bump 0.6.1.0 (EGB-710)"

---

Self-review against the EGB-710 spec

• AC: dotenv-only project exits 0 "already v2-shaped" → Task 1 (moved==0 && already==0 branch, substring no v1 properties blobs). Test: "no manifest and no store blobs is a clean no-op" (Task 1 Step 5).
• AC: .secrets-files-only project with v1 blobs migrates instead of the generic error → the manifest-free rewrite makes it just migrate (better than guiding to push first). Test: "manifest-free" (Task 1 Step 1). Note the rewrite supersedes the ticket's "offer to run absorb then migrate" branch — migrate no longer needs the manifest, so there's nothing to prompt for; the only destructive step (--finalize) keeps its existing /dev/tty confirmation.
• AC: interactive prompts read /dev/tty, degrade non-interactively, bash 3.2 → no new prompt is introduced (copy-forward is non-destructive); --finalize's existing /dev/tty confirm is untouched. --status is pure read-only output. All new code is bash 3.2 (no associative arrays, [[ ]] only in tests with || false).
• AC: no behavior change to copy-forward/finalize safety gates → finalize untouched; copy-forward stays non-destructive (v1 kept, commit + push twins). Verified by the unchanged finalize tests (migrate.bats:168-214).
• AC: bats coverage per branch → Tasks 1-3 add: manifest-free migrate, no-op no-blobs, undeclared-blob twinning, --status needs-migrate / finalize-ready / already-v2.
• Stretch: --status survey → Task 3, delivered.
• Sequencing: independent of the EGB-709 gate → confirmed; this only touches migrate ergonomics and helps operators reach all-v2. EGB-703 safety posture (recovery tag, verify-green gate, twin-before-drop) is preserved.

Operator-local follow-up (not part of this plan)

This repo's .ship-policy.json opts out of AI security/red-team passes; before any PR, ask the operator to run ./test/run-security.sh and complete the SIGNOFF. After landing, the operator can re-run their real-store migration for onefinalmessage et al. (secrets migrate per project → secrets migrate --finalize), which is the path to clearing the EGB-709 v2 gate.