A Hardcoded Secret Starts Its Clock When It Lands, Not When You Notice

Toyota published part of the T-Connect source to a public GitHub repository in December 2017. An access key to the data server sat in that code. Nobody looked at the repository again until 15 September 2022, when someone finally noticed it was public; the keys were changed two days later, and the disclosure put 296,019 customers inside the blast radius (GitGuardian's write-up, BleepingComputer).

The interesting number is not the roughly 1,750 days of exposure. It is that the exposure was total on day one and no worse on the last day. Nothing about the key's reachability degraded over time; the only thing that grew was the number of people who had had an opportunity to find it.

That is the frame I want to put on this, because it changes what "fixing it" means. Every remediation you can perform operates on the repository. None of them operate on the copies. Work out what is already true the moment the push completes, and the right answer falls out on its own.

Minute zero

Here is what exists the instant git push returns, before you have read the diff, before CI has finished, before anyone has reviewed the PR:

Node H is the one that matters. Once a credential reaches a third party's disk, it is no longer a property of your repository — it is a property of the provider that honours it. Deleting your copy does nothing to their copy.

Palo Alto Unit 42 measured the gap for AWS IAM keys specifically. Their report on the Elektra-Leak campaign states: "We found that the actor was able to detect and use the exposed IAM credentials within five minutes of their initial exposure on GitHub", and that AWS's own automatic quarantine landed at around the two-minute mark — fast, and still not fast enough to stop a full mining operation from spinning up (Unit 42).

Five minutes is shorter than a code review.

git rm removed exactly one copy

Let me commit a fake key and then "remove" it the way most people do — edit the file, commit the fix, move on. Throwaway repo, invented value:

mkdir art && cd art
git init -q -b main
git config user.email dev@example.com && git config user.name dev
mkdir src
printf 'export const RESEND_KEY = "re_INVENTED_3Qv7Kd2NpLxAo9Rb";\n' > src/mailer.ts
git add -A && git commit -qm "feat: mailer"
 
printf 'export const RESEND_KEY = process.env.RESEND_API_KEY;\n' > src/mailer.ts
git add -A && git commit -qm "fix: read key from env"
 
grep -r INVENTED src/ || echo "no match"

no match

The working tree is clean. Three commands get it back:

git log --oneline -S 're_INVENTED' -- src/mailer.ts
git show "$(git rev-list --max-parents=0 HEAD)":src/mailer.ts
git rev-parse "$(git rev-list --max-parents=0 HEAD)":src/mailer.ts

7d1d0c5 fix: read key from env
64b15ba feat: mailer
 
export const RESEND_KEY = "re_INVENTED_3Qv7Kd2NpLxAo9Rb";
 
8affd6c1b0ec55a9faaf920f9d7924c1f4fb86c3

git log -S (the pickaxe) finds every commit where the occurrence count of that string changed, so it names both the commit that added it and the commit that removed it. git show reads the file at the old tree. And the last line is the part people underestimate: the blob has a stable address. git cat-file -p 8affd6c… prints the secret from any clone of that repository, forever, without needing to know which commit or which path it came from.

Amending instead of committing a fix does not change the outcome — it just moves the object somewhere less obvious. Separate throwaway repo, same idea:

git init -q -b main
printf "export const k = 'ghp_INVENTED_TOKEN_abc123def456';\n" > token.ts
git add -A && git commit -qm "oops"
printf "export const k = process.env.GH_TOKEN;\n" > token.ts
git add -A && git commit -q --amend -m "use env"
 
git log --oneline
git reflog --all
git fsck --unreachable --no-reflogs | head -3

8f5605b use env
 
8f5605b refs/heads/main@{0}: commit (amend): use env
1ab503f refs/heads/main@{1}: commit (initial): oops
8f5605b HEAD@{0}: commit (amend): use env
1ab503f HEAD@{1}: commit (initial): oops
 
unreachable blob 6bc63a8596bf44d238b14a8b2a2321abd72f04ef
unreachable commit 1ab503f7a32bfe136c7c55eeb22c7cfd8a72010f
unreachable tree df15df7e9c7cb524ecf726b9c38d0641283dd97f

The amended-away commit is off the branch and still recoverable from the reflog for 90 days by default, and git fsck will hand you the blob directly.

Rewriting history moves the problem, it does not close it

The industrial fix is git filter-repo (or BFG). It genuinely works on the repository it runs in. Watch what it does to the identifiers, though:

printf 're_INVENTED_3Qv7Kd2NpLxAo9Rb==>REMOVED\n' > /tmp/rep.txt
git log --format='%h %s'
git filter-repo --replace-text /tmp/rep.txt --force
git log --format='%h %s'
git grep -I 're_INVENTED' $(git rev-list --all) || echo "no match on any ref"
git cat-file -p 8affd6c1b0ec55a9faaf920f9d7924c1f4fb86c3

7d1d0c5 fix: read key from env
64b15ba feat: mailer
 
551e924 fix: read key from env
d9bf887 feat: mailer
 
no match on any ref
fatal: Not a valid object name 8affd6c1b0ec55a9faaf920f9d7924c1f4fb86c3

Locally, that is a real clean-up — filter-repo repacks and expires the old objects, so even the blob hash stops resolving. But every commit hash changed, which is the tell. A rewritten history is a different history, and everyone who already has the old one keeps it. I cloned the repo before running filter-repo, exactly the way a colleague or a CI runner would have:

cd ../art-colleague
git grep -I 're_INVENTED' $(git rev-list --all)

64b15bad62f9da7e428360a41c55059e64989457:src/mailer.ts:export const RESEND_KEY = "re_INVENTED_3Qv7Kd2NpLxAo9Rb";

Old hash, old blob, still there. That clone never learned the rewrite happened, and nothing you run in your repository can reach it. The same applies to forks, to mirrors, to a CI cache, and — on GitHub specifically — to cached pull request views, which survive a force-push. GitHub's own documentation is blunt about the ordering: "if the sensitive data is something you should not have disclosed (e.g. password/token/credential) … then as a first step you need to revoke and/or rotate that secret. Once the secret is revoked or rotated, it can no longer be used for access, and that may be sufficient to solve your problem." Their support team will only help scrub cached views "in cases where we determine that the risk can't be mitigated by rotating affected credentials" (GitHub Docs).

The vendor whose object store it is treats rotation as the fix and rewriting as the optional part. So should you.

.gitignore does not untrack a file that is already tracked

This is the second-most-common way a .env gets committed: someone adds the file, then adds the pattern, then assumes the pattern applied retroactively.

git init -q -b main
printf 'API_KEY=INVENTED_v1_placeholder\n' > .env
git add .env && git commit -qm "track .env by mistake"
printf '.env\n' > .gitignore
git add .gitignore && git commit -qm "add gitignore"
printf 'API_KEY=INVENTED_v2_still_tracked\n' > .env
git status --short
git check-ignore -v .env; echo "exit=$?"
git check-ignore -v --no-index .env

 M .env
exit=1
.gitignore:1:.env	.env

Three things in that output. The file still shows as modified, so git will keep committing it. git check-ignore reports nothing and exits 1, because ignore rules are not consulted for tracked paths — which is a genuinely confusing way to be told "your pattern is fine, it just isn't being used". Only --no-index shows that the pattern does match. The actual fix is to remove it from the index:

git rm --cached -q .env && git commit -qm "untrack .env"
printf 'API_KEY=INVENTED_v3_now_ignored\n' > .env
git status --short
git log -p --oneline -- .env | grep -E '^\+API_KEY'

+API_KEY=INVENTED_v1_placeholder

Empty status: the file is finally ignored. And the first line of the file is still sitting in history, which is minute zero all over again.

This site's own repository shows the same mechanic from the other direction. Its .gitignore has a bare .env* on line 35 with no negation, plus bare *.md and *.json lines further down. That makes .env unstageable by accident — good — but it also makes .env.example unstageable on purpose, so the template had to be force-added:

printf '.env*\n' > .gitignore
git add .env.example
git add -f .env.example && git ls-files

The following paths are ignored by one of your .gitignore files:
.env.example
hint: Use -f if you really want to add them.
hint: Turn this message off by running
hint: "git config advice.addIgnoredFile false"
.env.example

The last line is git ls-files after the forced add. Putting !.env.example on the line after .env* is the cleaner spelling — I checked that the negation resolves, since git refuses to re-include a file whose parent directory is excluded and people get burned by that:

git init -q && touch .env .env.local .env.example
printf '.env*\n!.env.example\n' > .gitignore
for f in .env .env.local .env.example; do
  printf "%-14s " "$f"; git check-ignore -q "$f" && echo IGNORED || echo "NOT ignored"
done

.env           IGNORED
.env.local     IGNORED
.env.example   NOT ignored

Either spelling keeps the secret out. The negation version means nobody has to remember -f, which is worth something on a team.

Moving a key to an environment variable is not automatically moving it to the server

In a Next.js app, NEXT_PUBLIC_* variables are substituted into the client bundle at build time. This is documented, widely known, and still leaks keys, because the name reads like a namespace rather than a warning. I built a minimal Next.js 16.1.6 app to show exactly where the value ends up.

.env, all three values invented:

NEXT_PUBLIC_ANALYTICS_ID=INVENTED_PUBLIC_ID_12345
NEXT_PUBLIC_OPENAI_API_KEY=INVENTED_LEAKY_KEY_zzz999
OPENAI_API_KEY=INVENTED_SERVER_ONLY_KEY_qqq777

"use client";
export default function Page() {
  return (
    <div>
      <span>{process.env.NEXT_PUBLIC_ANALYTICS_ID}</span>
      <button onClick={() => fetch("https://api.example.com", {
        headers: { Authorization: `Bearer ${process.env.NEXT_PUBLIC_OPENAI_API_KEY}` },
      })}>go</button>
      <span>{process.env.OPENAI_API_KEY ? "server" : "no-server"}</span>
    </div>
  );
}

After next build, grep the static chunks that ship to the browser:

for v in INVENTED_PUBLIC_ID_12345 INVENTED_LEAKY_KEY_zzz999 INVENTED_SERVER_ONLY_KEY_qqq777; do
  echo "$v -> $(grep -rl "$v" .next/static | wc -l) client file(s)"
done
grep -o '.\{40\}INVENTED_LEAKY_KEY_zzz999.\{10\}' .next/static/chunks/afe91e9c71cfd609.js

INVENTED_PUBLIC_ID_12345 -> 1 client file(s)
INVENTED_LEAKY_KEY_zzz999 -> 1 client file(s)
INVENTED_SERVER_ONLY_KEY_qqq777 -> 0 client file(s)
 
le.com",{headers:{Authorization:"Bearer INVENTED_LEAKY_KEY_zzz999"}}),child

The key is a string literal in a file served from your CDN. The unprefixed one is not in the bundle at all. Note that it is a build-time substitution, not a lookup: an earlier version of my page only tested process.env.NEXT_PUBLIC_OPENAI_API_KEY for truthiness, and Turbopack folded the comparison away, so the value never appeared in the output. Absence from the bundle is therefore not evidence of safety — it can just mean this build happened to optimise the reference out, and the next build will not.

The clock on that one starts at deploy, and the copies are every browser cache that fetched the chunk.

Eight invented credentials, three scanners

I put eight invented credentials in one file and ran three tools against it. The file has five vendor-shaped values (AWS ID, AWS secret, Stripe, GitHub PAT, Slack bot), one Postgres URL with an inline password, and two 40-character random strings with no recognisable prefix — one named hmacSigningKey, one named rotationSeed.

Commands, with values redacted in the output:

gitleaks dir --no-banner --redact -f json -r /dev/stdout src/keys.ts
trufflehog filesystem src/keys.ts --no-update --no-verification --results=verified,unknown,unverified --json
git-secrets --register-aws && git-secrets --scan src/keys.ts

Three things I did not expect before running this.

The union of the two serious scanners still misses rotationSeed. Gitleaks caught hmacSigningKey under its generic-api-key rule at entropy 5.32 — but that rule keys off the variable name containing something like key or token, and rotationSeed has identical entropy and an innocent name, so it falls through. Generic secrets are not a rare corner: GitGuardian's 2025 report puts them at 58% of all leaked credentials (GitGuardian).

Gitleaks missed the database URL entirely and trufflehog caught it with a Postgres detector. Running one scanner is a coverage decision, not a hygiene checkbox.

TruffleHog attempts live verification by default — it makes network calls against the provider to see whether a credential is valid. That is useful for triage and surprising the first time you run it inside a locked-down CI environment. --no-verification turns it off.

On the article's other common recommendation: GitHub's push protection for users is on by default and blocks pushes containing recognised secrets to public repositories, but repository-level push protection "is disabled by default" and requires GitHub Secret Protection (the product that used to be bundled as GitHub Advanced Security) to be enabled by an admin (GitHub Docs). If you are on a private repo and nobody has turned it on, nothing is standing between a git push and minute zero.

The order of operations, and the part people skip

Issue first, deploy second, revoke third. Reversing the first and third steps is how a credential rotation becomes an outage, and an outage is how a rotation gets rolled back at 2am by someone restoring the old key from — where else — the commit it was leaked in.

Which brings me to the caveat, because rotation is the right answer and it is also where the second mistake lives.

Creating a new credential is not revoking the old one. On most providers those are separate operations on separate screens, and the old value keeps working until you explicitly kill it. That gap is where the number in GitGuardian's reporting comes from: 70% of the secrets they saw leaked in 2022 were still valid when they retested in January 2025, and re-running the same check in January 2026 still found over 64% valid (2025 report, 2026 report). Those are not credentials nobody noticed. A large share of them belong to teams that rotated, shipped the new key, saw everything working, and never went back to press revoke.

So the check that closes the incident is not "the new key is deployed". It is the negative one: take the old value, send one request with it, and confirm the provider rejects it. Then read the provider's audit log for the window between the commit timestamp and that rejection, because that window is the only honest measure of what the leak cost you — and unlike the commit, it is a window you can actually close.