Git and GitHub the Safe Way, Ch. 4: Secrets Never Go In
Part of Git and GitHub the Safe Way, a short book on git, GitHub, and the habits that keep a repository from hurting you.
Promise one. A committed secret is the most expensive mistake in this book, because it is the only one git cannot undo: a key that was public for four minutes may already be in someone’s scraper, in a fork, in a CI log. So the work happens before the commit, in layers, and the last layer is knowing what to do when all the others failed.
What counts
More than you think. Passwords and API keys, obviously. Also: tokens of every kind, private SSH keys, the .env file that holds the database address with the password in it, cloud credentials files, certificates with private halves, OAuth client secrets, webhook URLs (chapter 17 of the Go book explains why a webhook URL is a password), session cookies pasted into a test, a customer export used as “sample data”, a colleague’s email address in a fixture. The rule that covers all of them: if someone could use it to act as you, or it is about a real person, it is a secret.
Where they hide: configuration files, test fixtures, example scripts, notebooks, a TODO comment with “temporary” next to it, the log file you committed for a bug report, and history, where a secret removed from the current files three months ago still sits in the commit that added it.
Layer 1: keep the file out of the folder git watches
The settings a program needs live in a file git never sees, and a copy with the values blanked lives in the repository to show the shape. Chapter 3’s .gitignore is the mechanism:
.env
in .gitignore, and a committed .env.example:
AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
DATABASE_URL=postgres://app:PASSWORD@localhost:5432/app
A new contributor copies .env.example to .env, fills it in, and git ignores it from then on. Write the .gitignore line before the first commit of the project, not after: .gitignore stops git from noticing a file; it does nothing about a file already tracked.
Layer 2: look at what you stage
Chapter 3’s habit, git add -p and git diff --staged, is layer two. Most committed secrets were never looked at; they came in with git add ..
Layer 3: a scanner that checks
Eyes miss things. A secret scanner knows the shapes of hundreds of providers’ keys. TruffleHog goes one step further than most: when it finds something key-shaped, it tries the key against the provider and tells you whether it is live. That changes what you do next, and it is why this book uses it.
Install it: brew install trufflehog on macOS. On Linux the project’s install script, curl -sSfL https://raw.githubusercontent.com/trufflesecurity/trufflehog/main/scripts/install.sh | sh -s -- -b /usr/local/bin; download the script and read it first if you would rather know what runs as you. On Windows there is no package manager entry; download the release archive from the releases page, unpack trufflehog.exe somewhere on your PATH (chapter 0.2 of the Go book explains PATH), or run it through Docker as the README shows.
Here is the mistake, caught. A real-looking .env is staged by accident (the values in this demonstration were randomly generated):
git add .env
git status --short
A .env
Scan what is staged. --since-commit HEAD means “what is newer than the last commit,” which includes the staging area, and --fail makes the exit status non-zero when anything is found:
trufflehog git file://. --since-commit HEAD --fail
Verification issue: lookup db.internal: no such host
Detector Type: Postgres
Decoder Type: PLAIN
Raw result: (hidden)
Sslmode: <unset>
Host: db.internal:5432
Username: app
Database: app
Commit: Staged
File: .env
Line: 3
Found unverified result
Detector Type: AWS
Decoder Type: PLAIN
Raw result: (hidden)
Resource_type: Access key
Commit: Staged
File: .env
Line: 1
exit status 183
Read what it did. It recognised a database connection string and tried to connect to the host to see if the password worked; the host does not exist, so it reports an issue rather than a verdict. It recognised an AWS access key and asked AWS whether the key was valid; AWS said no, because the key is random, so the result is “unverified”. Every finding comes in one of three kinds: verified (the provider confirmed the credential works, which means you are leaking a live key right now), unverified (it looks like a key but the provider rejected it), and unknown (it looks like a key and the check could not be completed). Exit status 183 is TruffleHog’s “I found something,” distinct from a crash.
Take the file back off the list, add the ignore line and the example, and scan again:
git restore --staged .env
git add .gitignore .env.example
trufflehog git file://. --since-commit HEAD --fail
exit status 0
What a live key looks like
TruffleHog’s authors keep a public repository of deliberately live test credentials so you can see the difference without leaking anything of your own. This is the README’s own first exercise:
git clone https://github.com/trufflesecurity/test_keys.git
trufflehog git file://test_keys --results=verified,unknown
Found verified result
Detector Type: AWS
Decoder Type: PLAIN
Raw result: (hidden)
Commit: 0416560b1330d8ac42045813251d85c688717eaf
Email: counter <hello@trufflesec.com>
File: new_key
Line: 2
Found verified result
Detector Type: URI
Decoder Type: PLAIN
Raw result: (hidden)
Commit: 77b2a3e56973785a52ba4ae4b8dac61d4bac016f
Email: counter <counter@counters-MacBook-Air.local>
File: keys
Line: 3
--results=verified,unknown hides the unverified kind. That is the setting TruffleHog recommends for day-to-day use, and it is a real trade: you hear only about credentials that work or could not be checked, and never about dead keys, test strings, or things that merely look like keys. Random fakes, like the ones in the first scan above, sail through it. Keep that in mind when you read the next section.
Run it on every commit
A hook is a script git runs at a moment you name; pre-commit runs before every commit and, if it exits non-zero, the commit does not happen. Keep hooks in a folder that is committed, so they travel with the project. This is the form TruffleHog’s pre-commit guide gives, in .githooks/pre-commit:
#!/bin/sh
# Refuse the commit if a staged change holds a credential that verifies against its provider.
exec trufflehog git file://. --since-commit HEAD --results=verified,unknown --fail --trust-local-git-config
Make it executable (chmod +x .githooks/pre-commit; Git Bash on Windows does this too) and tell git where the hooks live:
git config core.hooksPath .githooks
git add .githooks
git commit -m "Add a pre-commit secret scan"
Each collaborator runs that one git config line once after cloning; a line in the README is the usual reminder.
Two details in that command line are not optional, and I know because I tested the hook without them. --trust-local-git-config is what lets TruffleHog see the commit git is in the middle of making. Without it, a git commit -am (stage and commit in one step) went through with the hook reporting nothing, and produced an empty commit: the edit was left behind, unstaged, and the log said it was committed. The guide’s other instruction follows from the same mechanism: run git add and then git commit as two steps, and avoid -am with this hook. And --results=verified,unknown is the recommended setting; drop it if you would rather the hook refuse anything key-shaped, at the cost of refusing test fixtures too.
Now stage a file holding one of those live test keys, and try to commit it:
git add deploy.env
git commit -m "Add deploy settings"
Found verified result
Detector Type: AWS
Decoder Type: PLAIN
Raw result: (hidden)
Commit: Staged
File: deploy.env
Line: 1
exit status 1
The commit did not happen, the file is still staged, and nothing reached history. git restore --staged deploy.env, delete the file, carry on. The same hook let a randomly generated key through (Add fake settings, committed without a word), which is the trade from the previous section made concrete: the hook is a net for live credentials; the glance from chapter 3 is still the net for everything else.
One more thing scanners do that will surprise you: the example keys in a provider’s own documentation (AWS’s famous AKIAIOSFODNN7EXAMPLE) are deliberately ignored, so that every tutorial on the internet does not light up. If you test your setup with one of those and nothing fires, the setup is fine; test with a random string of the right shape, or with the test_keys repository.
Layer 4: let GitHub refuse the push
Every layer so far runs on your machine. GitHub’s push protection runs on theirs: a push containing a secret in a format GitHub recognises is rejected before it lands, for every contributor, hook or no hook. Chapter 9 switches it on (two clicks under Settings, Advanced Security); it is mentioned here because it is the net under the net.
When one gets in anyway
It will, once. The order matters more than the commands.
Rotate first. Within minutes, before anything else: generate a new key at the provider, put it where the program reads it, and revoke the old one. The old key is dead from that moment, and nothing else in this section is urgent any more. If the secret was a password a person uses elsewhere, change it there too. TruffleHog’s findings include a rotation guide link for most providers; follow it.
Then tell whoever needs to know. Your team, the owner of the system the key opened, and if customer data was involved, whoever handles that where you work. “I leaked a key, it is rotated, here is the window” is a sentence people respect.
Then clean the history, so the dead key does not keep triggering scanners and questions. Scan the whole repository to find every commit that holds it:
trufflehog git file://.
Found unverified result
Detector Type: Github
Decoder Type: PLAIN
Raw result: (hidden)
Rotation_guide: https://howtorotate.com/docs/tutorials/github/
Version: 2
Token_type: Personal Access Token (classic)
Commit: 868fd7ae36b858a76958d65785c34d4395f77f20
Email: Ada Gopher <ada@example.com>
File: mail.toml
Line: 1
Rewriting history is the one time this book reaches for a tool that changes old commits. git filter-repo is the maintained tool for it (install with brew install git-filter-repo, pip install git-filter-repo, or your package manager). To remove one file from every commit it ever appeared in, then confirm:
git filter-repo --invert-paths --path mail.toml --force
trufflehog git file://.
Parsed 5 commits
HEAD is now at 4604ec0 Add a pre-commit secret scan
New history written in 0.04 seconds; now repacking/cleaning...
...
exit status 0
Every commit after the removed one got a new fingerprint; that is a rewrite, and it means a force push of every affected branch (chapter 8’s --force-with-lease, for each branch git branch -r lists), every collaborator re-cloning, and every open pull request against those branches needing attention. It is disruptive on purpose. Understand what it does not do: forks still have the old commits, GitHub keeps them reachable by id for a while, and any scraper that saw the push has its copy. That is why the key was rotated first. History cleaning is housekeeping; rotation is the fix.
Conclusion
Four layers, each cheaper than the next: keep the file out of the watched folder, look at what you stage, let a scanner that checks with the provider refuse the commit, let GitHub refuse the push. And the day they all fail, rotate within minutes, say so, then clean the history knowing that cleaning is not undoing. Next: branches and pull requests, the shape that lets someone else look before anything lands.