secrets/README.md

# secrets

A command-line tool for sharing secret files (API keys, database passwords, tokens) between your machines and teammates — without ever putting them in your project's git history.

## The problem

Most projects have files like `.env`, `.env.staging`, or `.dev.vars` that contain sensitive credentials. These files should never be committed to your project's git repository because:

- Anyone with access to the repo can see them (even if you delete them later — git keeps history forever)
- Automated tools, CI pipelines, and compromised dependencies can read plaintext files from your project directory
- There's no safe built-in way to share these files between your laptop, your desktop, or a teammate's machine

People end up sharing secrets over Slack, email, or sticky notes. When a key changes, someone forgets to update, and things break.

## What this tool does

`secrets` encrypts your secret files and stores them in a separate, private git repository. Only someone with the encryption key can read them.

```mermaid
flowchart TD
    subgraph project["Your project (~/myapp/)"]
        direction TB
        p1[".env (plaintext)"]
        p2[".env.staging (plaintext)"]
        p3[".dev.vars (plaintext)"]
    end

    subgraph store["Your secrets store (~/.secrets/)"]
        direction TB
        s1["myapp/.env.age (encrypted)"]
        s2["myapp/.env.staging.age"]
        k["key.txt (never uploaded)"]
    end

    gh["GitHub (private)"]

    project -->|secrets push<br/>encrypt| store
    store -->|git push| gh
    gh -->|git pull| store
    store -->|secrets pull<br/>decrypt| project
```

- **Encrypted at rest** — files are encrypted with [age](https://github.com/FiloSottile/age), a modern encryption tool. Without the key, the files are unreadable.
- **Synced via git** — the encrypted files are stored in a private git repository that syncs between machines. You never interact with this repo directly — `secrets push` and `secrets pull` handle it.
- **Minimal exposure** — `secrets run` keeps plaintext files on disk only while your command is running, then deletes them automatically.

### What files are tracked

| Pattern | Example | Source |
|---------|---------|--------|
| `.env` | `SECRET_KEY=abc123` | Standard environment file |
| `.env.*` | `.env.staging`, `.env.production` | Environment-specific variants |
| `.dev.vars` | `CF_API_TOKEN=xyz` | Cloudflare Wrangler local secrets |

Files like `.envrc` (direnv) and `.environment-*` are intentionally **not** tracked.

Beyond project files, `secrets` can also sync files that live *outside* the project — designated keys from `~/.gradle/gradle.properties` (merged without clobbering unrelated keys), or whole files like an Android upload keystore. See [External files (Gradle properties)](#external-files-gradle-properties).

## Prerequisites

- **macOS** (uses Homebrew for installation)
- **git** (already installed on most Macs — type `git --version` to check)
- **age** (the encryption tool — installed in step 1 below)

## Setup

### First machine (one-time setup)

```bash
# 1. Install the encryption tool
brew install age

# 2. Download the secrets tool (this repo — contains only the CLI, no secret files)
git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets

# 3. Make the 'secrets' command available everywhere
#    Add this line to your shell config file (~/.zshrc on Mac):
export PATH="$HOME/dev/secrets:$PATH"
#    Then restart your terminal, or run:
source ~/.zshrc

# 4. Initialize your encrypted secrets store
#    This creates a folder at ~/.secrets/ with your encryption key
secrets init

# 5. Create a PRIVATE repository on GitHub to store your encrypted secrets
#    Go to github.com/new, name it something like 'my-secrets', and make sure
#    "Private" is selected. Then connect it:
cd ~/.secrets
git remote add origin git@github.com:<you>/my-secrets.git
git push -u origin main
```

> **Important:** Step 5 creates a *separate* private repo for your encrypted secrets. This is different from the `secrets` tool repo you cloned in step 2. The tool repo can be public — it contains no secrets. The `~/.secrets/` repo must be private.

### Additional machines

On each new machine (your desktop, a teammate's laptop, etc.):

```bash
# 1. Install prerequisites and the tool (same as steps 1-3 above)
brew install age
git clone https://codeberg.org/egbt/secrets.git ~/dev/secrets
export PATH="$HOME/dev/secrets:$PATH"  # add to ~/.zshrc

# 2. Clone the encrypted secrets repo
git clone git@github.com:<you>/my-secrets.git ~/.secrets

# 3. Copy the encryption key from your first machine
#    This is the only step that requires direct machine-to-machine transfer.
#    Choose one method:
#
#    Option A: AirDrop (Mac to Mac)
#      On your first machine, right-click ~/.secrets/key.txt → Share → AirDrop
#      Save it to ~/.secrets/key.txt on the new machine
#
#    Option B: Secure copy over SSH
#      scp first-machine:~/.secrets/key.txt ~/.secrets/key.txt
#
#    Option C: USB drive
#      Copy key.txt to a USB drive, transfer it, delete from USB after

# 4. Pull your secrets into any project
cd ~/myapp
secrets pull
```

> **The key file (`~/.secrets/key.txt`) is the only thing that needs to be transferred manually.** It never leaves your machines — it's excluded from git, never uploaded, never transmitted over the internet. Anyone with this file can decrypt all your secrets, so treat it like a password.

### Sharing with teammates

To share secrets with a teammate, they need:

1. Access to your private `my-secrets` GitHub repo (add them as a collaborator)
2. A copy of `key.txt` (send it to them directly — AirDrop, USB, or in-person)

Everyone on the team uses the same key. When anyone runs `secrets push`, the encrypted files are updated and everyone else can `secrets pull` to get the latest version.

## Usage

### Daily workflow

```bash
# Start of your work session — pull the latest secrets into your project
cd ~/myapp
secrets pull

# ... code, test, deploy ...

# If you changed any secret files, push the updates
secrets push

# End of session — remove plaintext secrets from disk (optional but recommended)
secrets clear
```

### Command reference

| Command | What it does |
|---------|-------------|
| `secrets init` | Create the `~/.secrets/` repo and generate an encryption key |
| `secrets push` | Encrypt secret files in the current directory and upload them |
| `secrets push --frozen` | Sync only what `.secrets.json` declares (skip auto-add) |
| `secrets push --dry-run` | Show what would be added/synced without changing anything |
| `secrets pull` | Download and decrypt secret files into the current directory |
| `secrets add <path>` | Declare a project-relative file in `.secrets.json` |
| `secrets clear` | Delete plaintext secret files from the current directory |
| `secrets run <command>` | Pull secrets, run a command, then clear secrets when it exits |
| `secrets list` | Show all projects that have stored secrets |
| `secrets list --json` | Same listing as a machine-readable JSON object (`{store, projects[].entries[]}`, each entry `dotenv`/`external`) for tooling and CI. JSON goes to stdout; notices to stderr |
| `secrets rm <project>` | Delete a project's secrets from the store |
| `secrets rekey` | Generate a new encryption key and re-encrypt everything |
| `secrets verify [project]` | Check the current project's `.secrets.json` against the store (missing/orphaned blobs) and decrypt every blob. `[project]` overrides the store directory name; the manifest is still read from the current directory |
| `secrets verify --all` | Decrypt-test every blob in every project — a store-wide integrity sweep |
| `secrets migrate [--dry-run]` | Copy-forward this project's encrypted blobs to store format v2 (non-destructive; manifest-free; `--dry-run` previews) |
| `secrets migrate --status` | Survey every project's v2 readiness; exits non-zero until the whole store is finalize-ready |
| `secrets migrate --finalize` | **Optional GC** — drop the old v1 blobs and mark the store pure v2. Never required: upgraded clients dual-write and read-fall-back, so not finalizing never cuts anyone off |

### Upgrading: do teammates on an older `secrets` get new secrets?

Store-format v2 is **additive** — an upgraded client reads either blob suffix and keeps the old (v1) suffix alive for externals that already existed, so you almost never have to coordinate an upgrade:

| Secret type | Old client gets it? |
|---|---|
| `.env` / `.env.*` / `.dev.vars` | **Yes, always** (blob name is identical across formats) |
| whole-file external (`file`) | **Yes, always** |
| `properties` external that already existed | **Yes** (dual-written so old clients stay fresh) |
| brand-new `properties` external | **No — must upgrade `secrets`** (the gentle forcing function) |

"Upgrade your secrets" = `git pull` the tool clone (binary ≥ 0.6.0.0) and/or `secrets migrate` the store. A read-only teammate only needs the tool `git pull`.

And you'll be told when you're behind: if a store was last written by a newer `secrets` than the one you're running, any command prints a one-line nudge to stderr (non-fatal) — and `secrets which` shows the store's `written-by:` version. Stores written by older builds (no version stamp) stay silent.

### Automatic project detection

When you run `secrets push` or `secrets pull` without specifying a project name, the tool figures out which project you're in by:

1. Checking the current directory's git remote (e.g., `origin` → `github.com/you/myapp.git` → `myapp`)
2. Falling back to the directory name (e.g., `/Users/you/myapp` → `myapp`)

You can also specify a name explicitly: `secrets push myapp`.

### The manifest (`.secrets.json`)

Every project gets a committed `.secrets.json` at its root declaring exactly what syncs — the single source of truth `push` and `pull` operate from (requires `jq`):

```json
{
  "version": 2,
  "options": { "autoAdd": true },
  "dotenv": [".env", ".env.staging", "packages/web/.env.development"],
  "external": [
    { "type": "properties", "path": "~/.gradle/gradle.properties",
      "keys": ["beaconClerkPkTest"] },
    { "type": "file", "path": "~/keystores/upload.keystore" }
  ]
}
```

You rarely write it by hand:

- **Auto-add (default):** `secrets push` discovers conventional files (`.env`, `.env.*`, `.dev.vars` — plus `package.json` workspace dirs once a manifest exists) and adds them to the manifest with an `==>` notice. Commit the manifest so other machines pick it up.
- **Explicit mode:** set `"options": {"autoAdd": false}` (a committed, team-shared setting) and `push` only syncs declared entries, warning about undeclared files. `secrets add <path>` is then the only manifest writer. Per-invocation: `push --frozen` (declared-only once) and `push --dry-run` (preview).
- `dotenv` paths are project-relative — nested monorepo paths like `packages/@acme/web/.env` are welcome; `..`, absolute paths, and symlinked manifests are refused.
- On the other machine, `secrets pull` restores exactly what the committed manifest declares, creating nested directories as needed.

Projects without a manifest keep working exactly as before (and work without `jq`); the first `push` bootstraps one for you.

### secrets run

`secrets run` is a **pull → run → clear** pipeline: it runs `secrets pull` to decrypt the latest files into your project, executes your command, then runs `secrets clear` when that command finishes. Plaintext `.env` / `.dev.vars` files exist only while your command is running.

That matters because anything on disk can be read by other processes. `secrets run` keeps that window as small as possible—useful for dev servers, deploys, and one-off scripts.

```mermaid
flowchart TD
    subgraph run["secrets run"]
        direction TB
        A[secrets pull] --> B["Your command"]
        B --> C[secrets clear]
    end
```

`secrets clear` is hooked to **shell exit**, so it runs after **success**, **non-zero exit**, or **Ctrl-C** (SIGINT).

**Syntax**

```bash
secrets run <command> [args...]
secrets run -w <command> [args...]    # monorepo: all workspaces (needs jq)
secrets run -- <command>              # if the command starts with -
```

Use **`--`** when the program you are running begins with a dash so it is not parsed as a `secrets` flag.

**Examples**

```bash
secrets run npm start              # .env only while the dev server runs
secrets run wrangler deploy        # .dev.vars only during deploy
```

**`package.json` scripts** so the whole team gets the same behavior by default:

```json
{
  "scripts": {
    "dev": "secrets run react-router dev --port 5173",
    "deploy": "secrets run wrangler deploy"
  }
}
```

Stopping the dev server (or any failing command) ends the process; the `EXIT` trap clears secrets afterward.

If you prefer to keep decrypted files on disk for a long editing session, use `secrets pull` and `secrets clear` manually instead.

### Multiple stores

By default, all your encrypted secrets live in one store at `~/.secrets/`. That works great if you have one set of secrets shared across machines. If you want **separate stores** — for example, work secrets isolated from personal projects, or one store per client — `secrets` supports that without any special setup.

A "store" is just a directory with its own `.git` repo, age key, and remote. You can have as many as you want.

#### How a store gets picked

When you run `secrets push` or `secrets pull`, the tool resolves the active store using the first matching rule (highest precedence first):

```
1. --store <dir>          flag passed on the command line
2. .secrets-store         file in the current directory or any ancestor up to $HOME
3. SECRETS_DIR            environment variable (legacy escape hatch)
4. ~/.secrets             default
```

Run `secrets which` from any project directory to see which rule won and which store is active. Aliases `secrets where` and `secrets status` do the same thing.

#### Set up a second store on this machine

```bash
# Create a fresh store at ~/.secrets-work with its own age key
secrets --store work init

# Connect it to a separate private GitHub repo
cd ~/.secrets-work
git remote add origin git@github.com:<you>/work-secrets.git
git push -u origin main
```

The bare name `work` expands to `$HOME/.secrets-work`. Use `secrets --store /any/abs/path init` if you want a custom location.

#### Bind a project to a non-default store

In any project directory, write a `.secrets-store` file with the store's name (or path) and commit it:

```bash
cd ~/myapp
echo work > .secrets-store
git add .secrets-store
git commit -m "use work secrets store"
```

For teammates who haven't set up the store yet, you can include the store's git remote URL on the same line so they don't have to ask you for it:

```bash
echo "work git@github.com:acme/work-secrets.git" > .secrets-store
```

That second token (whitespace-separated) is optional, ignored when the store already exists locally, and used as a copy-paste-ready hint in the missing-store error message when it doesn't. See "Joining a teammate's bound project" below.

After that, every `secrets push` / `secrets pull` from this project (or any subdirectory) automatically uses `~/.secrets-work`. Teammates who clone the project get the same binding for free — the file is in the repo.

When you push or pull from a non-default store, `secrets` echoes which one is active so you can spot mistakes immediately:

```
==> Pushing secrets for project: myapp
==> Store: /Users/you/.secrets-work (from .secrets-store file (~/myapp/.secrets-store))
```

#### Joining a teammate's bound project

If you clone a project that has a committed `.secrets-store: work` file but you don't have `~/.secrets-work` set up locally, `secrets pull` will tell you exactly what to do.

When the original setter included the remote URL in `.secrets-store` (recommended), the error fills in the actual `git clone` command for you to copy-paste:

```
ERROR: Store not initialized: /Users/you/.secrets-work
  Resolved from: .secrets-store file (~/myapp/.secrets-store)
  This path doesn't exist on this machine yet.

  If you're joining a teammate's existing store:
    git clone git@github.com:acme/work-secrets.git /Users/you/.secrets-work
    # then copy their key.txt to /Users/you/.secrets-work/key.txt

  If you want a fresh new store at this path:
    secrets --store /Users/you/.secrets-work init
```

If the URL wasn't in the file, the error shows `<their-store-remote>` as a placeholder — you'll need to ask the teammate who set it up. Either way, you need two things to finish onboarding:

1. **The git remote URL** of the work-secrets repo — clone it to `~/.secrets-work` (or wherever the `.secrets-store` file resolves to on your machine). The `.secrets-store` file may already include this for you.
2. **The age key file** (`key.txt`) — same as standing up any new machine. AirDrop, scp, or USB.

Once both are in place, `secrets pull` works.

#### Undo or change a binding

```bash
# Stop using a non-default store for this project
rm .secrets-store
git commit -am "go back to default secrets store"

# Or change which store the project is bound to
echo personal > .secrets-store
git commit -am "switch to personal secrets"
```

### Monorepo support

For projects with multiple packages (monorepos using `package.json` workspaces), add the `-w` flag to operate on all workspaces at once:

```bash
cd ~/myapp                  # has package.json with "workspaces": ["apps/*", "packages/*"]
secrets push -w             # encrypts secrets from root + each workspace
secrets pull -w             # decrypts into root + each workspace directory
secrets clear -w            # clears secrets from root + each workspace
secrets run -w turbo dev    # pull all, run command, clear all on exit
```

Inside `~/.secrets/`, workspace secrets are organized by path:

```shell
~/.secrets/
  myapp/
    .env.age                        # root project secrets
    apps/web/.env.staging.age       # web app workspace
    apps/api/.env.age               # api workspace
```

Requires `jq` (`brew install jq`).

### External files (Gradle properties)

Some credentials don't live in your project at all. Android builds, for example, read keys from `~/.gradle/gradle.properties` — a global file, outside any project, shared by every Gradle project on your machine (the project's own `gradle.properties` is git-tracked, so it's the wrong home for secrets). `secrets` can sync specific keys from such a file without touching the unrelated keys around them.

You declare what to sync in the `external` array of your committed `.secrets.json`:

```json
{
  "version": 2,
  "external": [
    { "type": "properties", "path": "~/.gradle/gradle.properties",
      "keys": ["beaconClerkPkTest", "beaconClerkPkLive"] }
  ]
}
```

- **type** — `properties` (sync named keys from a Java-properties-style file) or `file` (sync the whole file — see below).
- **path** — absolute or `~/`-relative; must resolve inside `$HOME`. For `properties` the basename must end in `.properties`.
- **keys** — the property names to sync (`properties` only). Only these keys are read on push and merged on pull; everything else in the file is left alone. `file` entries take no keys.

> **Legacy `.secrets-files`:** older projects declared these entries in a line-based `.secrets-files`. It still parses, and the next `secrets push` absorbs its entries into `.secrets.json` (type `gradle-properties` becomes `properties`) — after that the legacy file is superseded and can be deleted.

#### Syncing to a second machine

On the machine that already has the keys set, add the entry to `.secrets.json` (create the file if the project doesn't have one yet):

```bash
cd ~/myapp
cat > .secrets.json <<'EOF'
{
  "version": 2,
  "external": [
    { "type": "properties", "path": "~/.gradle/gradle.properties",
      "keys": ["beaconClerkPkTest", "beaconClerkPkLive"] }
  ]
}
EOF
git add .secrets.json && git commit -m "sync gradle Clerk keys"
secrets push
# ==> Extracted 2 key(s) from ~/.gradle/gradle.properties
```

`secrets push` extracts just those keys, encrypts them, and stores them under `<project>/external/` in your secrets repo. Run `secrets which` from the project to confirm the manifest parsed.

On the other machine (after the usual key + repo setup):

```bash
cd ~/myapp
secrets pull
# ==> Merged 2 key(s) into ~/.gradle/gradle.properties (beaconClerkPkTest, beaconClerkPkLive)
```

`secrets pull` merges those keys into the local `~/.gradle/gradle.properties`, leaving every other key untouched. If a managed key already exists, its value is updated in place; comments, ordering, and unrelated entries are preserved. The file is backed up to `gradle.properties.secrets-bak` before each merge.

> **Note:** unlike `.env` files, merged Gradle keys are written as **permanent plaintext** into the target file — `secrets clear` does **not** remove them. This is appropriate for publishable / low-secrecy values (like Clerk publishable keys, `pk_*`). For high-value secrets that should never sit on disk, use `secrets run` with a `.env` instead.

#### Whole files (`file` type)

Some external secrets are whole binary files — an Android upload keystore, a certificate. The `file` type syncs the file verbatim (binary-safe, encrypted with age like everything else):

```json
{ "type": "file", "path": "~/keystores/beacon-upload.keystore" }
```

On `secrets push` the file is encrypted into `<project>/external/`. On `secrets pull` it is restored to the same path with mode `600`; if a different version already exists there, it is backed up to `<name>.secrets-bak` first. The same path rules apply (inside `$HOME`, no `..`, no symlinks). Like merged Gradle keys, restored files are permanent plaintext on disk — `secrets clear` does not remove them.

## Safety features

- **`secrets run` auto-clears** — plaintext files are deleted when the command exits, errors, or is interrupted with Ctrl-C
- **Pre-commit hook** — a git hook in `~/.secrets/` prevents accidentally committing plaintext secret files to the encrypted store
- **Key is never uploaded** — `key.txt` is gitignored and never leaves your machine via git
- **Encryption is file-level** — each secret file is independently encrypted. A corrupted file doesn't affect others.

## Key rotation

If you suspect your key has been compromised, or a teammate leaves the team:

```bash
secrets rekey
```

This generates a new key and re-encrypts all secrets (including `.env` and other dotfiles). After rekeying:

1. Copy the new `~/.secrets/key.txt` to every machine and teammate
2. Old encrypted files remain in git history (encrypted with the old key, which should be discarded)

For complete rotation with no historical exposure, create a fresh `~/.secrets/` repo.

> **Recovering from a broken rekey (pre-0.2.1.0):** Older versions of `rekey` skipped dotfiles (`.env`, `.dev.vars`) when re-encrypting, leaving their blobs on the *old* key while `key.txt` was replaced. If `secrets pull` now fails with `no identity matched any of the recipients`, those blobs are still encrypted to a key you no longer have. Restore the **old** `key.txt` from another machine that hasn't rekeyed, `secrets pull` to recover the plaintext, then `secrets rekey` again on 0.2.1.0 or later.

## Environment variables

| Variable | Default | Purpose |
|----------|---------|---------|
| `SECRETS_DIR` | `~/.secrets` | Override the secrets store location (legacy; prefer `--store` or a `.secrets-store` file — see "Multiple stores" above) |

## Troubleshooting

**"Key file not found"** — You need `~/.secrets/key.txt`. Either run `secrets init` (first machine) or copy it from a machine that has it.

**"Not initialized"** — Run `secrets init` to create the `~/.secrets/` directory.

**"Found an existing key ... but no repo"** — You copied `key.txt` into `~/.secrets` and then ran `secrets init`. On a second machine you should *clone* your existing secrets repo, not re-initialize it (`init` is only for the very first machine). The error prints the exact `git clone` command to run — copy-paste it, or see [Additional machines](#additional-machines). When your project's `.secrets-store` file declares a remote URL, the command is filled in with the real URL.

**"No secret files found"** — You're in a directory that doesn't have `.env`, `.env.*`, or `.dev.vars` files. Make sure you're in the right project directory.

**"Project not found"** — The project name doesn't match anything in `~/.secrets/`. Run `secrets list` to see what's stored. The name is usually derived from your directory name or git remote.

**"Fast-forward pull failed"** — Someone else pushed secrets while you had local changes. Run `secrets pull` first, then retry your push.

**".secrets.json: invalid JSON" / "manifest version N is not supported"** — The committed manifest is malformed or written by a newer `secrets`. The error names the file; fix the syntax, or update the tool (`git pull` in the tool's clone).

**"'jq' is not installed"** — Manifest features need `jq`. The error prints the install command for your platform. Manifest-less projects work without it.

**Not sure the store is intact?** — Run `secrets verify` in a project to check its `.secrets.json` against the store (declared-but-missing blobs and orphaned blobs) and decrypt-test every blob with your current key. Use `secrets verify --all` for a store-wide decrypt sweep across every project. It's read-only — plaintext is streamed to `/dev/null`, never written to disk — and exits non-zero if anything is wrong, so it's safe to run in CI.

## Development

```bash
# Run the test suite (237 tests across three files)
brew install bats-core
bats test/

# Security regression subset — operator-local only (attack-payload fixtures).
# Required before ship; records sign-off in .gstack/security-signoff.json.
./test/run-security.sh
```

Hosted AI agents must not run the security script or perform red-team/adversarial
review on this repo — see `.ship-policy.json` and `CLAUDE.md`.