IncidentForge | Devpost

The IncidentForge Command Center: Observe → Hypothesize → Critique → Experiment → Verify.
Command Center — from incident detection to verified root cause.
Institutional Memory — verified incidents preserved as reusable intelligence for future investigations.
Live Application Connector — securely observe real application health and turn live telemetry into actionable investigations.

Inspiration

IncidentForge was inspired by a simple problem: modern applications can tell us that something is broken, but they often cannot reliably tell us why it is broken.

Traditional incident-response tools are excellent at collecting logs, metrics, traces, and alerts. The difficult part starts afterward. Engineers still have to compare possible causes, decide which hypothesis is credible, test it safely, and verify whether a proposed fix actually solved the problem.

We wanted to build an AI system that does more than generate a plausible explanation.

Our core idea was simple:

AI should propose the cause. Evidence and experiments should prove it.

What it does

IncidentForge is an AI-powered incident investigation and remediation platform built around a causal reasoning workflow:

Observe → Hypothesize → Critique → Experiment → Verify → Remediate

When an incident is detected, IncidentForge collects available evidence and generates multiple competing hypotheses instead of immediately accepting the first explanation.

An adversarial critic then challenges the leading hypothesis and defines what would falsify it.

Instead of treating correlation as causation, IncidentForge runs a controlled experiment using a deterministic digital-twin environment. The system predicts what metrics should change if the hypothesis is correct, applies an intervention, and compares the resulting metrics against the prediction.

Only when the expected conditions pass does IncidentForge mark the cause as VERIFIED.

The platform also includes a Live Application Connector, allowing users to provide an external HTTP/HTTPS application and collect read-only observations such as:

HTTP status
Availability
P95 latency
Error rate
TLS validity

These observations can be converted into investigation evidence.

A key safety principle is that external telemetry alone should not be treated as proof of an internal root cause. When there is insufficient evidence, IncidentForge can explicitly return an inconclusive investigation instead of inventing a diagnosis.

How we built it

IncidentForge combines several components into one investigation pipeline.

AI Reasoning

LLM-based reasoning is used to generate and analyze competing hypotheses. The AI is treated as a reasoning engine rather than an unquestionable source of truth.

Adversarial Critique

Each investigation includes a critic that challenges the current hypothesis and defines a falsification criterion.

Deterministic Verification

A deterministic state machine controls the investigation lifecycle and ensures that the system moves through explicit stages rather than jumping directly from observation to diagnosis.

Digital Twin & Controlled Experiments

The experiment layer simulates controlled interventions and evaluates whether predicted telemetry changes actually occur.

For example, a hypothesis may predict:

[ P95_{after} < P95_{before} \times (1-\tau) ]

where ( \tau ) is the required improvement threshold.

The investigation is only considered verified when its predefined conditions pass.

Live Application Connector

We built a secure read-only HTTP connector with SSRF protection, request limits, timeout controls, redirect validation, and response-size limits.

Investigation Interface

The frontend was designed as an operational command center rather than a generic dashboard. It visualizes:

competing hypotheses
adversarial objections
causal experiment graphs
before/after telemetry
verification results
remediation
activity history
counterfactual outcomes
institutional memory

Challenges we ran into

The biggest challenge was preventing plausible AI reasoning from becoming unsupported conclusions.

During development, we discovered that if live external telemetry was passed into a deterministic fallback incorrectly, the system could produce a believable internal database hypothesis even though the available evidence did not support it.

That forced us to redesign parts of the investigation pipeline around evidence provenance and explicit uncertainty.

We added safeguards so that:

live application evidence remains attached throughout the investigation;
external HTTP observations cannot automatically become internal infrastructure diagnoses;
hypotheses can be rejected;
experiments can be inconclusive;
the system can stop instead of fabricating an answer.

Another challenge was making the system understandable visually. We wanted judges to understand the entire causal reasoning process within seconds, so the UI became part of the storytelling: the interface shows what the AI observed, what it believed, what challenged that belief, what experiment was performed, and why the final conclusion was accepted.

Accomplishments that we're proud of

We are especially proud of building a system where verification is a first-class concept rather than simply displaying an AI-generated answer.

IncidentForge can visibly demonstrate the complete reasoning loop:

Evidence → Competing Hypotheses → Adversarial Critique → Controlled Experiment → Verification → Remediation

We also built a live application connector that extends the platform beyond predefined incidents while preserving strict safety boundaries.

Another major accomplishment was building counterfactual analysis into the workflow, allowing the system to estimate the failures avoided when an intervention is applied.

Most importantly, we designed IncidentForge so that "I don't know" is a valid outcome when the evidence is insufficient.

What we learned

We learned that building trustworthy AI for operational systems is fundamentally different from building a chatbot.

A good system cannot simply produce the most convincing explanation. It needs:

evidence provenance
explicit competing hypotheses
falsification criteria
controlled experiments
deterministic verification
safe failure states

The most important lesson was:

The AI should not be the final authority. The experiment should be.

This principle shaped both the backend architecture and the user experience of IncidentForge.

What's next for IncidentForge

Our next step is to move from controlled application observations toward deeper integrations with real observability ecosystems.

We want IncidentForge to connect with production monitoring sources, logs, traces, metrics, deployment systems, and incident-management platforms so that investigations can begin automatically from real operational signals.

We also want to expand the experiment engine with more realistic service dependencies and richer digital-twin simulations.

The long-term vision is an autonomous incident investigator that can continuously observe systems, generate competing explanations, challenge its own reasoning, run safe experiments, verify causal explanations, and recommend or execute remediation with clear evidence for every decision.

IncidentForge doesn't just ask "What went wrong?"

It asks: "What evidence would prove why it went wrong?"