Quick answer: Secret scanning tools catch hardcoded credentials before they reach production. They're a necessary baseline — but AI agents introduce an exposure surface that git-layer scanning was never designed to cover. Here's what scanning handles, what it misses, and how to close the gap.
Quick answer
Secret scanning catches credentials that land in committed files, git history, and CI pipelines. It won't catch credentials passed in agent prompts, stored in conversation context, included in tool-call parameters, or written to agent logs. Both layers matter — scanning protects git, upstream controls protect the agent.
How traditional secret scanning works
Secret scanning tools — gitleaks, truffleHog, GitHub's native scanning — operate by pattern-matching on text. They know what a Stripe secret key looks like, what an AWS access key ID looks like, what a GitHub personal access token looks like. They check those patterns against:
- Committed file contents
- Git history (including files that were deleted)
- Pull request diffs before merge
- Pre-commit hooks if you've configured them
When a match is detected, the tool flags the file and line, and optionally blocks the push or alerts the team. The model is solid for its intended purpose: stopping credentials from being committed to a repository where they'd be visible to anyone with access.
GitHub's secret scanning goes further — it works with some providers to automatically revoke detected tokens before they can be used. That's a meaningful safeguard when it's available.
What scanning misses with AI agents
The problem isn't that secret scanning is broken. It's that agents create exposure paths that don't involve git at all.
Credentials in prompts. When a developer pastes an API key into a prompt — "use this Stripe key to test the webhook" — that credential enters the agent's conversation context. It may be sent to the model provider's API, stored in conversation history, exported in logs, or included in traces. None of that touches a git repository. Scanning never sees it.
Agent memory and context windows. Some agents maintain memory across sessions. If a credential is mentioned once, it may persist in memory stores or embeddings that outlive the conversation. That's a different attack surface from committed code — and a different remediation path if something goes wrong.
Tool-call parameters. Agents that call external services via MCP or function calling often pass parameters as structured data. If a credential is included in a tool-call argument — either explicitly or because the agent inferred it from context — it may be logged by the tool, the agent framework, or the model provider. Again, no git commit required.
Conversation exports. Many AI coding tools let users export or share conversations. A conversation that includes a credential is a credential leak, regardless of whether the underlying code was ever committed.
Config files outside version control. mcp.json, .cursor/settings, and similar agent configuration files are sometimes excluded from .gitignore by default or added to it after the fact. If a credential was present before the file was excluded, it's in git history. If the file was always untracked, scanning misses it entirely — but the credential is still on disk.
The complementary approach
Scanning is necessary. It's not sufficient on its own when your team uses agents.
The complementary controls live upstream — before credentials reach the agent's context at all:
Don't put raw credentials in prompts. Agents don't need your actual API key to work with a service. They need a credential they can use. A scoped credential from a broker means the agent gets a token with limited permissions and a logged access record — not your master key. See what happens when you paste an API key into an agent's prompt for why this matters specifically.
Scope what agents can access. An agent that only has access to the permissions it actually needs can't expose what it can't reach. This isn't a secret scanning concern — it's an access control concern. The environment variables and AI agent security post covers why env vars alone don't give you this.
Audit what agents log. Review how your agent framework handles tool-call logging, conversation exports, and memory persistence. If credentials appear in those logs — even transiently — you have an exposure you won't find in a git scan.
Lock down config files. MCP configuration files in particular often contain credentials directly. MCP config file security covers this in detail, but the short version: config files with API keys belong in a secrets manager, not on disk in plaintext.
Tooling overview
For the git layer, three tools are worth knowing:
gitleaks — open source, fast, runs as a pre-commit hook or in CI. Good default for teams that want self-hosted scanning without a vendor dependency.
truffleHog — scans git history deeply, including older commits that simpler tools skip. Useful for auditing repositories that predate a scanning policy.
GitHub secret scanning — built into GitHub, no configuration required for public repositories, available on private repositories with GitHub Advanced Security. Covers the broadest set of token types and integrates with provider revocation for supported services.
For the agent layer, the tooling category is credential brokers rather than scanners. Instead of detecting credentials after they've been committed, brokers issue scoped credentials on demand — so agents never hold your master key in the first place. Each access is logged with the agent's identity, the service, and the timestamp. Gazebo's free secrets scanner is a quick way to check whether your existing agent config files contain credentials that shouldn't be there.
A combined posture
These two layers aren't alternatives — they address different parts of the problem.
Secret scanning handles: hardcoded credentials in committed files, git history, and CI pipelines.
Upstream agent controls handle: credentials in prompts, conversation context, tool-call parameters, agent logs, and config files that never reach git.
Running both means a credential that's accidentally committed gets caught by scanning. A credential that's passed to an agent gets a scoped token with an audit trail instead of a master key with none. The gap between those two surfaces is where most agent-related credential exposure actually happens.
Setting up the combined posture
Theory is easy. Here's the actual configuration.
Git layer: gitleaks in CI
Add this to .github/workflows/secrets.yml:
name: Secret Scanning
on: [push, pull_request]
jobs:
gitleaks:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
with:
fetch-depth: 0
- uses: gitleaks/gitleaks-action@v2
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
fetch-depth: 0 is non-negotiable. Without it, you're scanning only the latest commit. History leaks are real — don't skip them.
For pre-commit hooks, add .gitleaks.toml to the repo root:
[extend]
useDefault = true
[[allowlists]]
description = "Test fixtures"
paths = [
"tests/fixtures/.*"
]
Then wire it up:
# .pre-commit-config.yaml
repos:
- repo: https://github.com/gitleaks/gitleaks
rev: v8.18.0
hooks:
- id: gitleaks
Run pre-commit install once per developer machine. Done. Commits with matching patterns get blocked locally before they ever hit CI.
Enable GitHub's native secret scanning separately — it's not redundant. It catches token types gitleaks doesn't pattern-match, and it handles the provider revocation side for supported services. Go to Settings → Code security → Secret scanning → Enable. Turn on push protection while you're there. It blocks the push at the GitHub layer, not just alerts after the fact.
Agent layer: what to actually configure
This is less about tooling and more about policy enforced in your setup scripts and onboarding docs.
MCP config files. Your mcp.json should never contain a raw credential. The pattern that works:
{
"mcpServers": {
"stripe": {
"command": "npx",
"args": ["-y", "@gazebo/mcp-server"],
"env": {
"GAZEBO_TOKEN": "${GAZEBO_TOKEN}"
}
}
}
}
The environment variable gets resolved at runtime from your secrets manager or shell environment. The config file itself contains no secret. It's safe to commit.
Cursor and similar tools. Add .cursor/ to .gitignore globally, not per-repo:
git config --global core.excludesFile ~/.gitignore_global
echo ".cursor/" >> ~/.gitignore_global
echo ".continue/" >> ~/.gitignore_global
This is a one-time setup step that belongs in your team's dev environment docs.
Logging audit. Check your agent framework's default logging level. Most frameworks log tool-call parameters at DEBUG level. That means any credential passed as a tool argument ends up in your log aggregator. Set explicit redaction rules in your log pipeline for known secret patterns, or better — ensure credentials never enter tool-call parameters directly.
Incident response: credential in an agent log or conversation export
Someone exports a conversation. A credential is in it. Here's the playbook.
Step 1: Revoke immediately. Don't investigate first. Revoke the credential at the source — the provider's dashboard, the secrets manager, wherever it was issued. Time between discovery and revocation is the exposure window. Minimize it.
Step 2: Determine what the credential could access. Pull the permission scope. If it was a master key, assume full access to that service. If it was a scoped credential, check what operations it permitted. This determines blast radius.
Step 3: Check for access during the exposure window. Query the provider's access logs for the credential. Look for requests that weren't made by your systems. Most providers expose this — AWS CloudTrail, Stripe's API logs, GitHub's token activity. If you used a credential broker, the broker's audit log covers this step.
Step 4: Identify the conversation export or log file's reach. Who can see it? Was it shared externally? Exported to a third-party tool? Stored in a logging service with broad access? Revocation handles the credential, but the log file itself may need to be purged from wherever it landed.
Step 5: Understand how it got there. Was a raw credential pasted into a prompt? Did the agent infer it from an environment variable and include it in a tool-call response? Did a config file get inadvertently included in agent context? The answer determines which upstream control was missing.
Step 6: Close the gap before reissuing. Issue a replacement credential only after the upstream control is in place. Reissuing into the same setup produces the same incident.
Step 7: Document the timeline. When was the credential created, when did it appear in the log or export, when was it discovered, when was it revoked, what access occurred in between. This is the incident record. You'll need it for any compliance obligations and for the internal postmortem.
The difference between a painful incident and a catastrophic one is usually step 1. Revoke first. Everything else is investigation.
Gazebo issues scoped credentials to agents on demand — each agent gets a token limited to the services and operations it actually needs, and every access is logged. See how it works or get started free.