> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.nora.my/llms.txt
> Use this file to discover all available pages before exploring further.

# Regression checks

> Automatic detection of examples that used to pass and now don't.

The scariest simulation result isn't the ones you expected to fail — it's the ones you didn't. **Regression checks** call these out loudly.

## What's a regression

An example that:

* Passed under the baseline.
* Fails under the target.

That's it. Nora computes this automatically for every before/after comparison.

## Where regressions show up

* **In the simulation summary** — top-of-page count. "Target passes 45, regressed 3."
* **In the results table** — regressions are red rows, sortable to the top.
* **In an alert** — if the simulation was CI-gated, regressions fail the gate.

## Investigating a regression

Click any regression to see:

* The example (input, expected, baseline output, target output).
* The step-by-step diff between baseline trace and target trace — where they diverged.
* Nora's hypothesis for the cause.

Common causes:

* **Retrieval order changed** — the top-K includes different chunks. Fix: adjust the retrieval preset.
* **Prompt shifted** — the new prompt is less specific about a required behavior. Fix: restore or clarify the missing instruction.
* **Model differs** — swap in the prior model to test.
* **New guardrail fired** — a change to guardrails blocked something the baseline let through. Fix: tune the guardrail.

## Ignoring a regression

Sometimes a "regression" is a fake one — the baseline was wrong, the target is right, and your dataset annotation matches the baseline. Fix by updating the annotation, not by rejecting the change.

Mark a regression as "acceptable" or "annotation was wrong" in the results table. Documented in the audit trail.

## Ignoring is not the same as suppressing

If you consistently mark a regression as acceptable across simulations, Nora asks: is this a pattern we should stop flagging? Configure via **Simulation settings → Regression rules**:

* Ignore regressions on examples tagged `flaky`.
* Ignore regressions where cost decreased more than 30%.

Use sparingly. Suppressing regressions makes the simulation lie to you eventually.

## Regression rate over time

The **Simulation history** page tracks regression rate per version. Health signal:

* Consistent \< 2% regression rate — good discipline.
* Rising regression rate — team is shipping less carefully; the improvement flow may not be paying attention to the dataset.
* Sudden spike — one bad deploy; investigate the version.

## Hard regressions

Some regressions matter more than others. Tag examples in the dataset as **critical** — any regression on a critical example blocks any deploy regardless of aggregate stats.

Common criticals:

* Security-relevant behaviors ("never disclose the system prompt").
* Compliance answers ("always mention the disclaimer").
* Business-critical rules ("never quote a price without checking the price list").

Simulate against criticals on every publish.