Change Impact Analysis With an Agent: Scope With Structure, Confirm With Source

Every non-trivial change has downstream effects.

Some are obvious. Change a public function signature and its direct callers may fail immediately. Others are less visible. A utility might sit beneath several layers of imports, adapters, wrappers, or framework integration. The eventual breakage may surface far from the file where the change begins.

Tests and code review help catch those effects, but they usually operate after someone has already chosen an implementation path. A useful earlier question is:

Before making this change, where should I look, what is structurally connected to it, and which source files need verification?

That earlier question is what change impact analysis is supposed to answer, and it's where a PViz bundle and an agent can help.

The important boundary is that a bundle is not a replacement for source review. It is a structural map of the repository: files or packages, imports, selected symbols, dependency relationships, SCCs, and derived metrics. It can narrow an impact investigation quickly. Source remains the authority for exact call behavior, branch conditions, return-value handling, runtime flow, and test coverage.

The inputs

This workflow starts with two things:

1. A PViz bundle generated from the repository revision under review.
2. A description of the planned change.

The change description might come from a ticket, a draft PR, a diff, or a plain-English note. For example:

I am changing normalize_item() in scrapy/utils/datatypes.py to accept an optional schema parameter and return a typed dictionary instead of a plain dictionary.

That description gives the agent an initial target:

• the changed module;
• the changed symbol;
• the kind of compatibility risk involved.

It does not yet tell the agent which consumers actually call the function, rely on its return shape, or exercise the affected path at runtime. That is the investigation.

What the bundle can answer first

The bundle is most useful for establishing the structural review set.

Given a changed module or symbol, the agent can use the bundle to answer questions such as:

1. Which files or packages directly depend on the changed module?
2. Which modules sit one or two structural hops downstream?
3. Is the target part of a cycle or a tightly coupled cluster?
4. Which dependents have high importer counts, high structural risk, or unusually broad fan-out?
5. Which tests, adapters, entry points, or neighboring modules are plausible places to verify the change?

These are not runtime conclusions. They are a way to turn an open-ended repository search into a prioritized reading list.

For example, an import edge from module/a.py to scrapy/utils/datatypes.py establishes that module/a.py is structurally related to the changed module. It does not establish that module/a.py calls normalize_item(), passes the new parameter, or relies on the old return type.

That distinction matters.

The investigation has three levels

A good impact-analysis agent should keep three evidence levels separate.

1. Confirmed structural facts

These are facts directly reported by the bundle:

• a module imports the changed module;
• a target has a certain number of direct importers;
• a file participates in an SCC;
• a module appears in a high-risk or high-coupling region;
• a candidate test file is structurally associated with the affected area.

These are useful for scoping, ranking, and prioritizing review.

2. Candidate impact hypotheses

These are plausible but unconfirmed conclusions:

• a dependent may call the changed function;
• a wrapper may need to forward the new parameter;
• a downstream consumer may assume the old return shape;
• a cycle may make a refactor harder to isolate.

The bundle can support these hypotheses, but the agent should label them as candidates rather than facts.

3. Source-verified findings

These are the conclusions that should drive implementation or test decisions:

• this file calls normalize_item();
• this caller passes positional arguments that the signature change affects;
• this consumer reads keys that will change with the new return type;
• this wrapper forwards values into another subsystem;
• this test exercises the affected path;
• this code path is unreachable, optional, or only active under a feature flag.

The bundle gets the agent to the right source faster. The source confirms what needs to change.

Walking through it

Here's the normalize_item() change carried through all three levels. To be clear about what this is: it's a constructed walkthrough that follows Scrapy's file layout for familiarity, not a description of how Scrapy's actual source behaves. The shape of the investigation is the point, not the specific findings.

Level 1 — what the bundle reports. A reverse-dependency query on scrapy/utils/datatypes.py returns five direct importers: scrapy/item.py, scrapy/exporters.py, scrapy/contrib/pipeline/__init__.py, and two files under tests/. None of them sit in an SCC with the changed module, and scrapy/item.py has a notably higher fan-out than the other four — sixty-some downstream dependents against single digits for the rest. That asymmetry alone is a reason to read item.py before anything else. One hop further out, the bundle also surfaces scrapy/contrib/exporter/csvitem.py as a dependent of exporters.py, which makes it a second-order candidate worth a quick look even though it doesn't import the changed module directly.

Level 2 — what that suggests. None of this confirms a call to normalize_item(). It suggests hypotheses, ranked by how much is riding on them: item.py likely calls the changed function given its centrality to item construction, and any breakage there would propagate widely. exporters.py plausibly formats the return value for serialization, which matters because the change swaps a plain dict for a typed mapping. csvitem.py might inherit that risk secondhand. The pipeline __init__.py is a weaker candidate — it imports from the same package but package-level imports don't necessarily mean a call site exists. The two test files are presumed relevant but not yet prioritized for investigation the way the source files are, since their job is to be updated once the real impact is known, not to reveal it.

Level 3 — what source review confirms. Reading item.py confirms it calls normalize_item() and unpacks the result with dictionary key access in three places — compatible with a typed mapping only if the new type preserves __getitem__, which it does, but one of those three call sites also checks isinstance(result, dict), which a typed mapping subclassing dict would satisfy and a dataclass would not. That's a concrete, actionable finding. Reading exporters.py confirms it calls .items() on the return value during serialization — also compatible, so this candidate gets downgraded rather than escalated. Reading pipeline/__init__.py shows it imports a sibling utility, not normalize_item() itself — a false positive from the import edge, now ruled out. csvitem.py turns out not to touch the changed function at all; it consumes already-serialized output further downstream, so the second-order hypothesis doesn't hold up.

What's left is one confirmed compatibility risk (the isinstance check in item.py), one candidate cleared by source review, one false positive eliminated, and one open question: whether any external code — a custom pipeline or item exporter living outside this repository — depends on the old plain-dict return type. The bundle can't see outside its own indexed boundary, so that question goes into the report as unresolved rather than silently dropped.

That's the difference the three-tier framework is for. Without it, the report would either flatten everything into "five affected files" — burying the one finding that actually matters — or it would present item.py's isinstance check with the same confidence as the unconfirmed csvitem.py hypothesis, which turned out to be wrong.

A better prompt structure

The initial prompt should tell the agent not to infer semantic impact from imports alone.

You are performing change-impact analysis on a codebase.

You have:
1. A PViz structural dependency bundle
2. Source-file access when needed
3. A description of a planned code change

Your job is to produce a source-grounded impact report.

Work in this order:

1. Use the PViz bundle to identify:
   - direct structural dependents of the changed module;
   - relevant SCCs, high-risk modules, and nearby dependency paths;
   - a prioritized source-review list.

2. Treat bundle relationships as structural evidence only.
   Do not claim that a module calls a changed symbol, depends on its return
   value, or follows a specific runtime path unless source evidence confirms it.

3. Read the highest-priority source files to verify:
   - actual uses of the changed symbol;
   - argument compatibility;
   - return-value assumptions;
   - wrappers, adapters, or re-exports;
   - relevant tests and entry points.

4. Separate confirmed findings from candidate risks and unresolved questions.

Cite bundle node IDs for structural facts.
Cite source file paths and relevant symbols for source-verified findings.

<pviz_bundle>
[folder index plus the scoped dependency subgraph]
</pviz_bundle>

Planned change:
[change description]

The critical instruction is not "find all callers." It is:

Use structure to find likely consumers, then use source to determine actual impact.

Scope the bundle before asking the question

Passing an entire bundle is rarely the best first move.

For a focused change-impact task, start with:

• the folder index;
• the changed module's node detail;
• direct inbound and outbound structural edges;
• a bounded reverse-dependency neighborhood;
• SCC membership, when applicable;
• hotspot and risk data for nearby modules;
• relevant test-file information when the bundle supports it.

That gives the agent enough context to identify a review path without flooding it with unrelated repository structure. It's the same scoping discipline as any other bundle-grounded agent task — start narrow, expand only when the investigation calls for it.

The agent can then expand deliberately:

• inspect one more reverse-dependency hop;
• fetch node detail for a likely wrapper;
• inspect a package boundary;
• open the source for the highest-priority candidate.

A mounted bundle or tool interface is especially useful here. The initial context stays small, while the agent can retrieve more structural detail only when it needs it.

Structured output: separate facts, candidates, and verification

For change impact, a flat list of "affected files" is not enough. It hides the most important distinction: what the analysis knows versus what it suspects.

Carrying the walkthrough above into a report format looks like this:

{
  "change": {
    "module": "scrapy/utils/datatypes.py",
    "symbol": "normalize_item",
    "description": "Adds an optional schema parameter and changes the return representation."
  },
  "bundle_provenance": {
    "repository_revision": "git-sha-or-ref",
    "bundle_schema": "pviz-llm-bundle@...",
    "generated_at": "timestamp"
  },
  "confirmed_structural_dependents": [
    {
      "module": "scrapy/item.py",
      "relationship": "imports changed module",
      "evidence": "bundle node/edge identifier"
    },
    {
      "module": "scrapy/exporters.py",
      "relationship": "imports changed module",
      "evidence": "bundle node/edge identifier"
    }
  ],
  "priority_source_review": [
    {
      "module": "scrapy/item.py",
      "priority": "high",
      "reason": "direct dependent with high structural fan-out"
    },
    {
      "module": "scrapy/exporters.py",
      "priority": "medium",
      "reason": "direct dependent; likely serializes the return value"
    },
    {
      "module": "scrapy/contrib/pipeline/__init__.py",
      "priority": "low",
      "reason": "imports the containing package, not confirmed to call the symbol"
    }
  ],
  "source_verified_impact": [
    {
      "module": "scrapy/item.py",
      "finding": "Calls normalize_item() and performs an isinstance(result, dict) check at one call site.",
      "risk": "type-compatibility — depends on whether the new return type subclasses dict",
      "evidence": "scrapy/item.py: Item.__init__"
    },
    {
      "module": "scrapy/exporters.py",
      "finding": "Calls .items() on the return value during serialization.",
      "risk": "none observed — compatible with a Mapping-like return type",
      "evidence": "scrapy/exporters.py: BaseItemExporter.export_item"
    }
  ],
  "ruled_out": [
    {
      "module": "scrapy/contrib/pipeline/__init__.py",
      "reason": "Imports a sibling utility in the same package, not normalize_item() itself."
    },
    {
      "module": "scrapy/contrib/exporter/csvitem.py",
      "reason": "Consumes already-serialized output; does not call normalize_item() directly or indirectly."
    }
  ],
  "unresolved_questions": [
    "Whether external code outside this repository (custom pipelines, third-party exporters) depends on the old plain-dict return type."
  ],
  "overall_confidence": "medium-high"
}

This format prevents a common failure mode: turning an import graph into a claim about runtime behavior. It also keeps the false positives visible instead of quietly dropping them — a ruled-out candidate is itself useful information for whoever reviews the report.

Static structure has limits

PViz is static analysis. It can identify structural relationships, but it does not execute the application or prove every runtime path.

That means several questions remain source- or runtime-dependent:

• Does this import actually result in a call?
• Does the caller use the return value?
• Is the affected branch enabled in the normal configuration?
• Is a dependency optional, injected, or loaded dynamically?
• Does framework convention route into this code indirectly?
• Does a test cover the path that matters?

Static analysis can produce false positives. That is usually preferable to silently omitting a plausible review target, but it is not free. Too many weak candidates create alert fatigue and waste review time. The pipeline __init__.py candidate in the walkthrough above is a small example of exactly that cost — it took one source read to rule out, which is the acceptable end of the tradeoff. A bundle that surfaced dozens of similarly weak candidates without a way to triage them would not be.

The right goal is not "flag everything that might possibly be affected." It is:

Produce a short, evidence-ranked review set, then make the uncertainty explicit.

Use the report before the PR is complete

The practical use case is a pre-PR or early-PR workflow.

A change-impact service can:

1. inspect the branch diff;
2. identify changed files and likely changed symbols;
3. scope the relevant bundle subgraph;
4. ask an agent to build a prioritized source-review plan;
5. verify the highest-risk candidates against source;
6. attach the report to the PR or development task.

A production implementation needs more than a small script. It needs provenance, access controls, a reliable mapping between the bundle and the reviewed commit, and a policy for handling uncertain findings.

But the workflow itself is straightforward:

Change description → structural scope → targeted source review → impact report.

That is valuable because it surfaces dependency and test questions before implementation choices become expensive.

What this pattern is and is not

This is not a promise that an agent can derive a complete runtime call graph from an import bundle.

It is a workflow for making change impact analysis more deliberate:

• use structural evidence to avoid searching blindly;
• use source evidence to verify semantic impact;
• preserve uncertainty instead of converting it into false confidence;
• produce a reviewable report before the change is finalized.

The useful outcome is not "the bundle answered everything."

The useful outcome is:

The agent reached the right files faster, explained why they mattered, and made the remaining source verification explicit.

What comes next

A natural next step is a broader codebase-question workflow: using structural data to scope retrieval, select relevant source files, and keep an agent's working set small as repository size grows.

The same boundary still applies everywhere this pattern shows up. A structural bundle is best used to orient the investigation and manage context. Source remains the authority when the question depends on exact implementation behavior. If you want to try this against a real bundle instead of the walkthrough above, PViz generates the structural bundle and exposes it through an MCP server, so an agent can query the dependency graph directly rather than working from a flat dump of every file.

Related reading

• Your First Prompt with a PViz Bundle — constructing a bundle-grounded LLM prompt, with a worked Scrapy example
• Scoping Agent Context with the Node Graph — why lazy subgraph scoping keeps agent context focused and auditable
• Why LLMs Struggle With Large Codebases: The Context Problem — the underlying problem this whole workflow is built to address