AI Can Find More Bugs Than You Can Fix. Now What?

An agent finishes exploring your app. Projects failed. Members failed. Billing failed. It helpfully opens three tickets, each with a screenshot of the same error toast.

Should you assign three engineers, fix one shared dependency, or investigate an authorization regression before touching anything else?

The screenshots cannot answer that. Neither can the number on the run summary.

This is a hypothetical example, not a HiTest customer incident. But it exposes a practical problem with cheaper bug discovery: every additional finding creates a decision somebody has to make. Better detection can make that queue longer even when every report is honest and useful.

The useful question is what the next engineer needs to decide, and which evidence would let them decide it. Below is a small runnable model of that process, followed by a workflow you can apply to a real report queue.

The harder problem: useful reports arriving too quickly

On October 21, PostgreSQL committer Robert Haas described increased LLM-assisted bug discovery as a reason for PostgreSQL 19's release delay and late feature reverts. His account emphasizes an imbalance: finding problems had become easier faster than fixing them had. This is a maintainer's explanation, not a controlled study establishing AI as the sole cause of the delay. Haas's account.

The next day, curl maintainer Daniel Stenberg described a related problem. In curl's current experience, largely fabricated AI reports had mostly disappeared; the difficult workload was a high volume of substantially correct reports, often made harder to process by excessive prose. That describes curl, not a measured industry-wide trend. Stenberg's account.

These accounts challenge an easy response: filter out low-quality reports and everything will be fine. A queue can contain valuable work and still exceed the capacity available to reproduce, diagnose, review, and safely ship fixes.

For teams adopting AI testing, that makes triage part of the testing system. A tool that reduces discovery time but leaves every downstream decision untouched can transfer work rather than remove it.

First, stop counting different things as “bugs”

Keep four quantities separate:

A retry adds an observation. It does not necessarily add a defect. Two failures on different pages may share a cause. Two failures with identical text may require unrelated fixes.

Even “reproducible” needs a denominator. Record attempts and conditions instead of writing “always happens” after one run. Conversely, an intermittent failure remains evidence worth preserving. Failure to reproduce it immediately is a reason to investigate uncertainty, not erase the original observation.

Your tracker should retain individual observations while allowing provisional relationships between them. Otherwise, deduplication destroys the material needed to discover that the grouping was wrong.

A runnable example: the same toast, two different problems

Imagine Projects and Members consuming a shared JSON adapter. The API now returns records under items; the adapter still expects results. Billing has a separate problem: the server rejects a role that the product's permission policy explicitly allows.

All three screens display “Could not load.” Grouping by message would merge the authorization problem into the adapter problem.

The following Python program models those conditions without a browser, network, dependencies, or AI agent. It deliberately seeds the defects. It is an executable illustration of a triage decision, not a benchmark or evidence about real-world defect frequency.

Save it as triage_example.py and run python3 triage_example.py:

from dataclasses import dataclass

@dataclass(frozen=True)
class Build:
    adapter_fixed: bool = False
    billing_fixed: bool = False


def request(page, role, build):
    if page == "billing":
        allowed = role == "owner" or (
            build.billing_fixed and role == "billing_admin"
        )
        if not allowed:
            return 403, None
    return 200, {"items": [page]}


def screen(page, role, build):
    status, payload = request(page, role, build)
    if status != 200:
        return "Could not load"
    key = "items" if page == "billing" or build.adapter_fixed else "results"
    rows = payload.get(key)
    return rows if rows is not None else "Could not load"


def check(build):
    # Expectations stay unchanged across all candidate fixes.
    return {
        "projects": screen("projects", "member", build) == ["projects"],
        "members": screen("members", "member", build) == ["members"],
        "billing_admin": screen("billing", "billing_admin", build) == ["billing"],
        "viewer_denied": request("billing", "viewer", build)[0] == 403,
    }


expected = [
    (False, False, False, True),
    (True, True, False, True),
    (False, False, True, True),
    (True, True, True, True),
]
for build, want in zip(
    [Build(), Build(adapter_fixed=True),
     Build(billing_fixed=True), Build(True, True)], expected
):
    result = check(build)
    assert tuple(result.values()) == want
    print(build, result)

The assertions encode this result matrix:

The adapter change resolves the two list failures while leaving Billing broken. The Billing change resolves its failure independently. The negative permission check makes a dangerously broad “allow everyone” repair insufficient.

This gives you two supported repair groups in this deliberately constructed model. The number comes from the seeded mechanisms and selective changes, not from clustering English descriptions.

In a real application, a patch making several tests pass is supporting evidence, not proof of a unique root cause. A broad workaround can mask multiple defects. Keep expectations unchanged, inspect the narrow change, and rerun the original cases separately. Where practical, reverting that change should restore the affected failure without restoring unrelated failures.

Open full size

Deduplicate investigation, not evidence

A useful grouping proposal says:

Projects and Members both receive the new response shape and fail in the same adapter. Investigate together. Billing returns 403 before that adapter runs; keep it separate.

A weak proposal says:

These reports look similar. Close two as duplicates.

Crash-triage systems already use signatures to reduce repeated investigation. ClusterFuzz, for example, documents stack-derived crash signatures and testcase minimization. Those are useful precedents, but a browser toast contains much less causal information than a crash stack. ClusterFuzz glossary.

Use a fingerprint to suggest related reports: build, operation, endpoint, failure stage, role, and relevant response shape. Treat the fingerprint as a search aid. A shared timestamp or exception message is weak evidence by itself.

Give a provisional group an explicit falsification condition. For the list failures, that condition is simple: if the adapter-only change leaves Members broken under the same fixture, the shared-cause hypothesis needs revision. This prevents a persuasive summary from becoming a permanent assumption. It also makes the next agent task precise: rerun a particular observation rather than broadly investigate a feature.

Record the reason for a merge and retain links to every original observation. Split the group again if a proposed fix leaves one member failing. Avoid merging reports across different tenants, permission boundaries, or data-exposure conditions merely because their UI symptoms match.

The Billing example is a denial of legitimate access, not demonstrated data leakage. A report showing a viewer receiving billing data would require a different security assessment, even if someone proposed fixing both in the same authorization function. Handle potential exposure confidentially and involve the appropriate security owner. Do not wait for routine duplicate cleanup before escalating credible high-impact evidence.

Make the next decision cheap

A long report can still omit the only information that matters: why the behavior is wrong.

“Billing returned 403” is an observation. Whether it is a defect depends on the actor and the intended permission policy. “The authorization cache is stale” is a diagnosis. It needs additional evidence.

Keep those fields separate. Here is a compact packet for the fictional Billing case:

Summary: A billing administrator cannot read its own organization's invoices.
Build/environment: exact revision and preview configuration
Fixture: organization A; billing_admin role; seeded invoice
Expected: invoice list is readable under permission-policy rule B-2
Observed: GET /billing returns 403; UI shows "Could not load"
Reproduce: reset fixture; sign in with that role; open Billing
Evidence: redacted request/response; role assignment; first-attempt trace
Attempts: record failures / attempts and any changed conditions
Diagnosis: unknown; shared list adapter is not reached
Next decision: verify policy and server-side authorization for this role

If the expected outcome has no source, label it needs requirement clarification. An agent's intuition about how a product should behave is not enough to establish a defect. A documented contract, accepted example, or product-owner decision can supply the missing expectation.

PostgreSQL's reporting guidance similarly stresses factual outcomes and self-contained reproduction inputs. That is useful discipline regardless of who discovered the issue. Bug reporting guidelines.

Minimize the reproduction by removing one step or fixture element at a time while preserving the same failure. Keep the full original trace too: a short reproducer can accidentally remove a race, account transition, or tenant boundary that changes the meaning of the report. Redact credentials and customer data before sharing artifacts.

Our browser failure report guide covers the basic packet. The extra requirement here is a clear next decision, so every incoming item does not restart the entire investigation.

Give AI a bounded job after discovery

There are useful agent tasks between “found something” and “please fix everything”:

• Replay the supplied steps against a clean, authorized fixture.
• Reduce the steps while checking the original expectation.
• Retrieve potentially related reports with the matching evidence highlighted.
• Propose a narrowly scoped failing regression check.
• Rerun every member of a provisional group after a candidate fix.

Ask for artifacts and uncertainty, not an authoritative-looking verdict. A second model agreeing with the first does not substitute for independent reproduction. Nor should an agent change the expected outcome until the test becomes green.

Set boundaries around side effects. Reproducing a billing failure should not create real charges or expose another customer's invoices. Use controlled accounts and explicit permission for actions beyond the test fixture.

Keep consequential decisions reviewable: confirming a duplicate, changing security severity, accepting a product expectation, and declaring the deployed fix verified. Automation can collect the evidence for those decisions without silently owning them.

Manage the queue you actually have

A practical queue needs more states than open and closed:

Needs reproduction → needs expectation → confirmed issue → fix in review → verified fixed.

These are possible states, not mandatory sequential gates. A well-supported report can enter directly as confirmed; a credible security concern can escalate immediately. Track suspected and confirmed duplicate relationships separately from state.

When capacity is tight, prioritize demonstrated impact and reachability: who is blocked, whether data or access boundaries are affected, what workaround exists, and whether the defect is in the current release. The most dramatic generated title should not win the queue.

For low-risk exploration, it can be sensible to pause overlapping discovery runs and spend the next agent budget reproducing the unresolved findings. That is different from suppressing reports or pausing security monitoring. Keep the intake path open and visible.

Measure progress with median time to a usable reproducer, age of unresolved high-impact issues, reopened duplicate groups, and fixes verified against their original expectations. Count reviewer time as well as machine time. A rising number of reports alongside a rising backlog may be useful discovery, but it is not yet faster repair.

What to change on your next run

Preserve every observation. Require a stated expectation. Let the agent propose relationships, then test those relationships with narrow changes and unchanged checks. Keep permission boundaries visible even when symptoms look identical.

The goal is a queue where the next engineer can quickly answer: What failed, why should it have worked, what is still unknown, and what evidence would settle the next decision?

That is how cheaper discovery becomes cheaper maintenance, instead of a larger reading assignment.

Sources

• Robert Haas, Postgres 19 is late, and it’s AI’s fault, October 21, 2026.
• Daniel Stenberg, “ignore the slop”, October 22, 2026.
• PostgreSQL, Bug Reporting Guidelines, accessed November 2, 2026.
• ClusterFuzz, Glossary, accessed November 2, 2026.