Skip to main content
Nora watches your Agents from multiple angles. Every angle that spots a problem creates a Signal.

​
The channels

​
1. User feedback

Any 👎 or written correction on an Agent’s answer. Highest-quality signal — a real user said the answer was wrong. Subtypes: thumbs, text correction, categorized feedback, reference correction.

​
2. Verification failures

If a Flow has a causal graph attached with verification on, every blocked or warned verdict creates a Signal (see Verify agent answers). Catches cases where the Agent’s answer contradicts your domain knowledge — even when no user complained.

​
3. Retrieval misses

When retrieval returns zero results for a query, or when top-K scores are all below the confidence threshold. Suggests either a knowledge gap or a query the Agent doesn’t rewrite well.

​
4. Tool errors

Persistent tool failures. A single 500 from a flaky API doesn’t fire — but a rising rate of errors on the same tool does. Also fires on argument-shape errors (400: missing required field) because those usually mean the Agent’s misunderstanding.

​
5. Guardrail hits

Any block from configured guardrails — PII in an outgoing message, policy violation, output-format check failure. These are usually the Agent trying to do the right thing but hitting a rail; sometimes the rail is misconfigured.

​
6. Behavioral drift

Statistical shifts in behavior over time — cost per run rising, latency rising, tool-usage patterns changing, retrieval-hit rates dropping. Fires when the shift is beyond normal noise and sustained for several days.

​
7. Regressions

New failures that look like they were previously fixed. Fires when a signal similar to a resolved one reappears after a version change. See Regressions.

​
8. Approval rejections

Rejections in the Approvals queue with a reason. If reviewers keep rejecting a particular kind of Action, that’s a Signal about the Agent’s decision-making.

​
Channel importance

Not all channels weigh equally. Default severity weighting:
  • User feedback (👎 with correction) — highest.
  • Verification blocks — high.
  • Regression — high (something you already fixed came back).
  • User feedback (👎, no context) — medium.
  • Tool errors, guardrail hits — medium.
  • Retrieval misses — low-medium.
  • Behavioral drift — low (harder to attribute).
Adjust weights per Flow in Reliable → Signals settings.

​
Adding a custom channel

Post your own signals via the API:
Useful if you have an external QA workflow, or a compliance team that reviews sampled outputs offline.

​
Turning off a channel

Per-Flow, in Signals settings. Turn off any channel you don’t want to fire. Common: turn off “behavioral drift” for Flows that are still ramping and expected to shift.

​
Signals and traces

Every signal (except the API-posted ones without a trace_id) links back to a Trace. From the signal, click through to see the exact run that generated it. From a trace, click Signals for this run to see any signals fired by this run.