Skip to content

5.6 Context at Scale: Big Codebases

Everything so far assumed a project you can hold in your head. Real codebases — the 500-file client project, the decade-old monolith at work — don't fit in anyone's head, including the agent's. This is the lesson where the expensive cohorts spend their premium hours, and the punchline is satisfying: scale doesn't change the principles. It just makes them non-optional.

What you will learn

  • Understand why big codebases break naive agent usage — and what changes.
  • Apply the scaling toolkit: nested CLAUDE.md files, maps over contents, scoped tasks.
  • Onboard an agent (and yourself) into an unfamiliar codebase systematically.

Builder principle

The agent doesn't need the whole codebase in view — it needs the right map plus the right neighborhood. Same as any human senior hire.

What actually breaks at scale

A 500,000-line codebase can't fit in any context window — but that's not really the problem. Even the relevant slice is hard to find: which of the 40 modules matters for this ticket? Which conventions are load-bearing? Which similar-looking pattern is the current one and which is the legacy one nobody deleted?

Watch a naive session on a big repo and you'll see the failure signature: the gather phase sprawls (dozens of files skimmed, window flooded with marginal code), the act phase then imitates whatever it happened to read — including the deprecated patterns — and you're reviewing a confident change written in the wrong dialect.

The fix isn't a bigger window. It's what it always was: right things in, wrong things out — now with real stakes.

The scaling toolkit

1. Maps over contents. The root CLAUDE.md of a big repo shouldn't describe everything — it should be a map: what the major areas are, where the entry points live, which parts are active vs. legacy, and where deeper guides sit:

## Map
- services/api/ — REST layer. Active. See services/api/CLAUDE.md
- services/billing/ — payments. HIGH CAUTION, see docs/billing-rules.md
- legacy/ — pre-2024 code. Do not imitate patterns from here.
- Entry points: main.py (app), tasks.py (workers)

That "do not imitate" line about legacy code is worth its weight in incidents.

2. Nested CLAUDE.md files. From 5.3: subdirectories can carry their own CLAUDE.md, picked up when the agent works there. Local dialects live locally — the API's conventions with the API — so no single file has to be everything, and context loads proportionally to where the work is.

3. Scoped tasks, aggressively. "Fix the billing bug" in a monolith is an invitation to sprawl. The scaled brief names the neighborhood:

Bug: invoices double-charge on retry. The retry logic is in
services/billing/retry.py; invoice creation in services/billing/invoices.py.
Reproduce with: pytest tests/billing/test_retry.py -k double
Stay within services/billing/ unless you find the cause elsewhere —
if so, stop and tell me before changing anything outside.

You did the first gather yourself (or with a scout session — below), and bought a focused window in exchange.

4. Exploration/build separation (from 5.5, now essential). Investigation floods context; do it in a disposable session whose output is a file: Explore how authentication works across this codebase and write docs/auth-map.md — components, flows, gotchas. Then /clear, and the build session reads a 2-page map instead of 60 files. At scale this isn't an optimization — it's the difference between working and thrashing. (Chapter 10 gives investigation its own agents entirely.)

Onboarding into an unfamiliar codebase

New client project, new job, inherited mess — the systematic first hour:

  1. Pre-trust inspection, before launching an agent or installing dependencies. Inspect CLAUDE.md, .claude/ settings/agents/hooks, .mcp.json, package/install scripts, CI files, executable files, and symlinks. Repository instructions and scripts are untrusted input until reviewed. Begin in plan/read-only mode with no production credentials, minimal network access, and /sandbox where supported.
  2. Recon session (read-only, plan mode): Explore this codebase. What does it do, how is it organized, what stack, what state is it in? Report findings without writing or running project code yet.
  3. Read and spot-check the output yourself. You're onboarding too. Only after the repository's instructions and execution surfaces are understood should you allow a reviewed docs/orientation.md write.
  4. Draft the root CLAUDE.md from it — the map section especially. Have the agent draft; you edit. Wrong entries here mislead every future session, so this edit is worth real attention.
  5. First task, small and scoped — a tiny fix in one module, full loop, both of you learning the terrain where it's shallow.

One hour, and codebase #2 of your career gets approached exactly like codebase #200 of someone else's: map first, neighborhood second, change third.

"Big codebase" may feel far away — but a WordPress theme someone hands you, a client's inherited site, a template you bought: same situation at smaller scale, same playbook. Recon, map, CLAUDE.md, small first change. You now onboard like a pro regardless of what you're handed.

This lesson is the one to bring to work Monday. Concrete starting move for your team's repo: run the recon prompt, turn the output into a root CLAUDE.md map plus one nested CLAUDE.md in your hottest service, and measure a week of sessions against the before-times. That artifact — demonstrably better agent output on your codebase — is also how the "should we invest in this?" conversation gets won.

Try it now

Practice on a codebase you don't know — pick any substantial open-source project and clone it:

  1. Run the pre-trust inspection and log what execution/configuration surfaces you found. Then run recon without executing project code; read the output critically and spot-check two claims.
  2. Draft a root CLAUDE.md map from it.
  3. Scoped task drill: pick one tiny thing (a typo in docs, a README clarification) and write the scaled brief for it — named files, named boundary, named verification.
  4. Journal: how different did the gather phase look with a map in place?

Check your understanding

  • What's the failure signature of a naive session on a big repo, and which loop phase does it start in?
  • Maps over contents: what belongs in a big repo's root CLAUDE.md, and what explicitly doesn't?
  • Why is exploration/build separation "non-optional" at scale, in context-budget terms?
  • Recite the four onboarding steps for an unfamiliar codebase.