config-password-sanitisation

Swap the real secrets in a config file for stable placeholders so it can be pasted into a ticket, a chat, a mailing list or an LLM prompt — and swap them back afterwards, so the answer you get returned is a file you can actually deploy.

sanitise.sh --init                       # write ~/.config/sanitise/map, then edit it
sanitise.sh config.yaml                  # -> config-sanitised.yaml       (safe to hand over)
sanitise.sh -r config-sanitised.yaml     # -> config-sanitised-unsanitised.yaml

The point of stable placeholders is the round trip. Blanking secrets out with sed -i 's/password: .*/password: REDACTED/' is a one-way door: whoever helps you edits the redacted file, hands it back, and now you have to re-apply their changes to the real one by hand — a merge, done by eye, in the one file where a mistake locks you out of your own database. With a map, the same secret always becomes the same placeholder, so -r reconstructs the deployable file byte for byte.

The map file

The secrets live in the map, never in the script — which is why the script can be in this repository and the map cannot.

s3cr3t-db-pass!	password1
hunter2	password2
p@ss*w[ord?	password3

One replacement per line: the real secret, a single TAB, then its placeholder. A tab because it is the one character a password essentially never contains, unlike a space, =, :, # or a quote — no quoting rules to learn and no secret you cannot express. A line starting with # that contains no tab is a comment, so a secret may itself start with #. See sanitise.map.example.

--init writes a template with mode 600 and creates the directory if needed. The map is looked for at --map, else $SANITISE_MAP, else ${XDG_CONFIG_HOME:-~/.config}/sanitise/map; the current directory is deliberately not searched, so a map cannot be picked up by accident from a repository you happen to be standing in. If the map is readable by anyone but you, the tool says so on every run.

Substitution that survives its own output

Replacements are applied in one left-to-right pass, taking the leftmost match and, where several start at the same position, the longest of them. Text that has been replaced is never looked at again.

That is not the obvious implementation, and the obvious one is wrong in two ways that a config file will find:

• A short secret matching inside a placeholder that was just written. Apply the pairs one after another — sed -e … -e, or a loop over the map — and a secret like word will happily match the middle of the password2 a previous pair produced. Sorting the pairs by length does not help; nothing helps except not rescanning.
• A short secret biting off the front of a longer one. With top and topsecret both in the map, whichever is tried first wins, and if that is top then topsecret never matches again. Leftmost-longest picks topsecret, and the same rule makes password10 survive a map that also contains password1 on the way back.

Secrets are matched literally, so p@ss*w[ord? is a password and not a glob.

Three checks, because a silent failure here is a leaked credential

After sanitising, before you upload anything:

• Did anything survive? Every secret is grepped for in the output. If one is still there the tool says do NOT upload this file and exits 2.
• Was a placeholder already in the input? If password1 appears in the config for its own reasons, reversing turns it into a real secret on a line that never had one — a credential invented out of nothing, in a file you are about to deploy. Reported, exit 2.
• Is the map self-defeating? Duplicate placeholders (whose reverse would be ambiguous), duplicate secrets, and placeholders containing one of your secrets are all refused or flagged when the map loads, not after the file is written.

A per-secret count goes to stderr — password1: 3 — because the count you did not expect is how you find out that the map is stale, or that a secret is in the file twice as often as you thought.

$ sanitise.sh config.yaml
password1: 1
password2: 1
config-sanitised.yaml: 2 replacement(s)

Output files are never overwritten. If config-sanitised.yaml exists you get config-sanitised-2.yaml, then -3, and the file is created with noclobber rather than tested-then-written, so two runs at once cannot land on the same name. -o FILE overrides the name (--force to overwrite, -o - for stdout).

The reversed file is created mode 600 whatever your umask is: it holds the real secrets. The sanitised file is not, because handing it to someone is the entire point.

As a pre-commit hook

--check reports which secrets appear in the files you give it and exits 2 if any do, writing nothing:

$ sanitise.sh --check config.yaml
config.yaml:1,7: secret for placeholder password1
config.yaml:2: secret for placeholder password2

It prints the file, the lines and the placeholder — never the matched line. A secret scanner that echoes the secret into a terminal, a CI log or a hook's output has just leaked it somewhere new.

#!/bin/sh
# .git/hooks/pre-commit
files=$(git diff --cached --name-only --diff-filter=ACM)
[ -n "$files" ] && exec sanitise.sh --check $files

Tests

uv run tests/test_sanitise.py   # self-contained
pytest tests/                   # if pytest is already available

The tests drive the real CLI end-to-end in a temporary directory, including both overlap hazards, the glob-metacharacter secret, the file modes and every map-file error.

Limits

• Line-based, so a secret cannot contain a newline, and the input must be text — no NUL bytes.
• The input must be a regular file, not a pipe: it is read twice, once to transform and once to check what came out.
• It is not a secret detector. It replaces the strings you list and nothing else; a password you forgot to put in the map goes through untouched. --check has the same blind spot. Something like gitleaks or detect-secrets finds unknown secrets by pattern; this tool round-trips known ones.
• Structure is not understood. It is a string substitution, not a YAML/JSON/INI parser, which is exactly why it works on all of them — and why a secret that appears base64-encoded, URL-escaped or split across lines is not found.
• Pure bash 4.3+, so it is slow on large files (a few thousand lines a second). Configs are small; do not point it at a log.