feat: 'file' external type — whole-file sync for .secrets-files (EGB-652)
Built for the Beacon Android upload keystore: binary files outside the project root can now ride the encrypted store. - manifest: 'file <path>' (no keys; keys present = rejected loudly) - push: encrypts the file verbatim (age is binary-safe) - pull: restores next to target (atomic same-fs mv), mode 600, TOCTOU symlink recheck, divergent existing target backed up to .secrets-bak - _validate_external_target_path parameterized by type (basename restriction stays gradle-properties-only; $HOME/../symlink rails apply to both) - README/help/CLAUDE docs + 7 bats tests (133/133 pass)
This commit is contained in:
Brian Majewski
parent
ab45c4c94b
commit
e0474f3ce9
5 changed files with 226 additions and 40 deletions
132
secrets
132
secrets
|
|
@ -330,16 +330,24 @@ collect_env_files() {
|
|||
#
|
||||
# push extracts only the named keys → encrypts a subset blob under
|
||||
# <project>/external/. pull decrypts it and MERGES those keys into <path>,
|
||||
# preserving every unrelated line. Only type `gradle-properties` is
|
||||
# supported (the type token leaves room for future types without a plugin
|
||||
# framework). Parsing mirrors `.secrets-store`: no shell expansion, no
|
||||
# symlink following, conservative sanitization. The merge is pure bash
|
||||
# (no sed/regex) with exact-string key matching, so it is safe against
|
||||
# values containing `& \ /` and against substring keys
|
||||
# preserving every unrelated line. Parsing mirrors `.secrets-store`: no
|
||||
# shell expansion, no symlink following, conservative sanitization. The
|
||||
# merge is pure bash (no sed/regex) with exact-string key matching, so it
|
||||
# is safe against values containing `& \ /` and against substring keys
|
||||
# (beaconClerkPk vs beaconClerkPkTest).
|
||||
#
|
||||
# A second type, `file` (EGB-652), syncs a WHOLE file outside the project
|
||||
# root (e.g. an Android upload keystore) — binary-safe via age, no keys:
|
||||
#
|
||||
# file ~/keystores/beacon-upload.keystore
|
||||
#
|
||||
# push encrypts the file verbatim; pull restores it (mode 600, existing
|
||||
# target backed up to <name>.secrets-bak first). Same path rules as
|
||||
# gradle-properties (inside $HOME, no `..`, no symlinks) minus the
|
||||
# basename restriction.
|
||||
|
||||
SECRETS_FILES_NAME=".secrets-files"
|
||||
SUPPORTED_EXTERNAL_TYPE="gradle-properties"
|
||||
SUPPORTED_EXTERNAL_TYPES="gradle-properties file"
|
||||
|
||||
# True if the string ends in an odd number of backslashes (a Java/Gradle
|
||||
# properties line-continuation). Used to skip continuation lines when
|
||||
|
|
@ -366,14 +374,28 @@ _parse_secrets_files_manifest() {
|
|||
case "$line" in '#'*) continue ;; esac
|
||||
# `read -r` does not glob-expand and keeps the key list intact in $mkeys.
|
||||
read -r mtype mpath mkeys <<< "$line"
|
||||
if [ "$mtype" != "$SUPPORTED_EXTERNAL_TYPE" ]; then
|
||||
echo "WARNING: $file line $lineno: unknown type '$mtype' (supported: $SUPPORTED_EXTERNAL_TYPE). Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
if [ -z "$mpath" ] || [ -z "$mkeys" ]; then
|
||||
echo "WARNING: $file line $lineno: expected '<type> <path> <key> [key...]'. Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
case "$mtype" in
|
||||
gradle-properties)
|
||||
if [ -z "$mpath" ] || [ -z "$mkeys" ]; then
|
||||
echo "WARNING: $file line $lineno: expected 'gradle-properties <path> <key> [key...]'. Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
;;
|
||||
file)
|
||||
if [ -z "$mpath" ]; then
|
||||
echo "WARNING: $file line $lineno: expected 'file <path>'. Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
if [ -n "$mkeys" ]; then
|
||||
echo "WARNING: $file line $lineno: 'file' entries take no keys (got '$mkeys'). Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
;;
|
||||
*)
|
||||
echo "WARNING: $file line $lineno: unknown type '$mtype' (supported: $SUPPORTED_EXTERNAL_TYPES). Skipping." >&2
|
||||
continue
|
||||
;;
|
||||
esac
|
||||
# Path: allow alnum and / . _ ~ - only; reject everything else (blocks
|
||||
# \$ \` ; & | ( ) * ? whitespace etc.) and reject `..` traversal.
|
||||
case "$mpath" in
|
||||
|
|
@ -394,14 +416,16 @@ _parse_secrets_files_manifest() {
|
|||
done < "$file"
|
||||
}
|
||||
|
||||
# Validate a writable external target path. Refuses anything that isn't a
|
||||
# `gradle.properties` resolving inside $HOME, blocks `..`, and refuses
|
||||
# symlinks (target file or its parent dir) — the path comes from a
|
||||
# committed file, so it is attacker-controllable. Returns 0 if safe.
|
||||
# Validate a writable external target path. Requires the path to resolve
|
||||
# inside $HOME, blocks `..`, and refuses symlinks (target file or its
|
||||
# parent dir) — the path comes from a committed file, so it is
|
||||
# attacker-controllable. For type `gradle-properties` the basename must be
|
||||
# 'gradle.properties' (type `file` allows any basename — the whole file is
|
||||
# replaced, never merged). Returns 0 if safe.
|
||||
_validate_external_target_path() {
|
||||
local p="$1"
|
||||
local p="$1" mtype="${2:-gradle-properties}"
|
||||
local base; base=$(basename "$p")
|
||||
if [ "$base" != "gradle.properties" ]; then
|
||||
if [ "$mtype" = "gradle-properties" ] && [ "$base" != "gradle.properties" ]; then
|
||||
echo "ERROR: $SECRETS_FILES_NAME: target basename must be 'gradle.properties' (got '$base'). Refusing." >&2
|
||||
return 1
|
||||
fi
|
||||
|
|
@ -563,13 +587,22 @@ push_external_files() {
|
|||
while IFS=$'\t' read -r mtype mpath mkeys; do
|
||||
[ -n "$mtype" ] || continue
|
||||
local expanded; expanded=$(_expand_store_path "$mpath")
|
||||
if ! _validate_external_target_path "$expanded"; then
|
||||
if ! _validate_external_target_path "$expanded" "$mtype"; then
|
||||
die "Refusing unsafe external target in $SECRETS_FILES_NAME: $mpath"
|
||||
fi
|
||||
if [ ! -f "$expanded" ]; then
|
||||
echo "WARNING: source '$mpath' not found on this machine; skipping." >&2
|
||||
continue
|
||||
fi
|
||||
if [ "$mtype" = "file" ]; then
|
||||
# EGB-652: whole-file sync — encrypt the file verbatim (binary-safe).
|
||||
mkdir -p "$SECRETS_DIR/$project/external"
|
||||
local fslug; fslug=$(_secrets_files_slug "$mpath")
|
||||
age -r "$pubkey" -o "$SECRETS_DIR/$project/external/$fslug.file.age" "$expanded"
|
||||
info "Encrypted file $mpath"
|
||||
pushed=$((pushed + 1))
|
||||
continue
|
||||
fi
|
||||
local tmp; tmp=$(mktemp)
|
||||
local found=0 k v
|
||||
for k in $mkeys; do
|
||||
|
|
@ -620,7 +653,7 @@ pull_external_files() {
|
|||
while IFS=$'\t' read -r mtype mpath mkeys; do
|
||||
[ -n "$mtype" ] || continue
|
||||
local expanded; expanded=$(_expand_store_path "$mpath")
|
||||
if ! _validate_external_target_path "$expanded"; then
|
||||
if ! _validate_external_target_path "$expanded" "$mtype"; then
|
||||
echo "WARNING: skipping unsafe external target: $mpath" >&2
|
||||
continue
|
||||
fi
|
||||
|
|
@ -630,6 +663,30 @@ pull_external_files() {
|
|||
echo "WARNING: $SECRETS_FILES_NAME names '$mpath' but no encrypted data exists in the store yet. Run 'secrets push' on a machine that has these keys. Skipping." >&2
|
||||
continue
|
||||
fi
|
||||
if [ "$mtype" = "file" ]; then
|
||||
# EGB-652: whole-file restore — decrypt next to the target (same
|
||||
# filesystem -> atomic mv), back up any existing target, mode 600,
|
||||
# TOCTOU symlink recheck before the write into $HOME.
|
||||
local fdir; fdir=$(dirname "$expanded")
|
||||
mkdir -p "$fdir"
|
||||
local ftmp; ftmp=$(mktemp "$fdir/.secrets-file.XXXXXX") || { echo "WARNING: mktemp failed for $mpath; skipping." >&2; continue; }
|
||||
if ! age -d -i "$KEY_FILE" -o "$ftmp" "$blob"; then
|
||||
rm -f "$ftmp"
|
||||
die "Decryption failed for external target $mpath."
|
||||
fi
|
||||
chmod 600 "$ftmp" 2>/dev/null || true
|
||||
if [ -L "$expanded" ]; then
|
||||
rm -f "$ftmp"
|
||||
echo "WARNING: $expanded became a symlink; skipping restore." >&2
|
||||
continue
|
||||
fi
|
||||
if [ -f "$expanded" ] && ! cmp -s "$expanded" "$ftmp"; then
|
||||
cp "$expanded" "$expanded.secrets-bak" 2>/dev/null || true
|
||||
fi
|
||||
mv "$ftmp" "$expanded"
|
||||
info "Restored file $expanded"
|
||||
continue
|
||||
fi
|
||||
local tmp; tmp=$(mktemp)
|
||||
if ! age -d -i "$KEY_FILE" -o "$tmp" "$blob"; then
|
||||
rm -f "$tmp"
|
||||
|
|
@ -1409,23 +1466,28 @@ Workspaces:
|
|||
are stored under <monorepo>/ directly. Requires jq.
|
||||
|
||||
External files (.secrets-files):
|
||||
Sync specific keys from files OUTSIDE the project root (e.g. global
|
||||
Gradle properties). Create a committed .secrets-files in the project
|
||||
root, one entry per line:
|
||||
Sync files (or specific keys from files) OUTSIDE the project root.
|
||||
Create a committed .secrets-files in the project root, one entry per
|
||||
line:
|
||||
|
||||
# <type> <path> <keys...>
|
||||
gradle-properties ~/.gradle/gradle.properties beaconClerkPkTest beaconClerkPkLive
|
||||
file ~/keystores/beacon-upload.keystore
|
||||
|
||||
On push, the named keys are extracted from <path> and encrypted under
|
||||
<project>/external/ in the store. On pull, they are MERGED back into
|
||||
<path>, leaving all unrelated keys untouched. Only type
|
||||
'gradle-properties' is supported; the target basename must be
|
||||
'gradle.properties' and must resolve inside $HOME. Run 'secrets which'
|
||||
from the project to confirm the manifest parsed.
|
||||
gradle-properties: on push, the named keys are extracted from <path>
|
||||
and encrypted under <project>/external/ in the store. On pull, they
|
||||
are MERGED back into <path>, leaving all unrelated keys untouched.
|
||||
The target basename must be 'gradle.properties'.
|
||||
|
||||
Note: merged keys are written as permanent plaintext into the target
|
||||
file (suitable for publishable/low-secrecy values). 'secrets clear'
|
||||
does NOT remove them.
|
||||
file: the whole file is encrypted verbatim (binary-safe — keystores,
|
||||
certificates). On pull it is restored with mode 600; an existing
|
||||
divergent target is backed up to <name>.secrets-bak first. No keys.
|
||||
|
||||
All targets must resolve inside $HOME (no '..', no symlinks). Run
|
||||
'secrets which' from the project to confirm the manifest parsed.
|
||||
|
||||
Note: pulled external targets are permanent plaintext on disk —
|
||||
'secrets clear' does NOT remove them.
|
||||
|
||||
Environment:
|
||||
SECRETS_DIR Path to secrets repo (default: ~/.secrets). See also
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue