> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.nora.my/llms.txt
> Use this file to discover all available pages before exploring further.

# nora feedback

> Submit thumbs and corrections on traces — from CLI or external apps.

Feedback is how you signal that an answer is good or bad. Every submitted feedback row becomes a Signal, which may join a cluster and eventually seed an Improvement.

The `feedback` CLI is the surface for **external apps** and **batch submission**. In-app thumbs from the chat UI use the same endpoint under the hood.

## Commands

| Command                         | Description                                         |
| ------------------------------- | --------------------------------------------------- |
| `feedback describe <flow-slug>` | Print URL, auth, and body shape for external POSTs. |
| `feedback list <trace-id>`      | List existing feedback rows for one trace.          |
| `feedback submit <trace-id>`    | Insert a feedback row from the CLI.                 |

## `feedback describe`

```bash theme={null}
nora feedback describe support-agent
```

Prints the exact HTTP contract for external feedback POSTs:

* **URL** — the endpoint to POST to.
* **Auth** — required headers or signing scheme.
* **Body shape** — the fields, their types, and which are required.
* **Response** — what you'll get back.

Copy-paste this into your external app's docs and wire up thumbs-up / thumbs-down buttons.

Add `--json` for a machine-consumable version.

## `feedback list`

```bash theme={null}
nora feedback list tr_abc
```

Every feedback row for one trace. Includes:

* `like` or `dislike`.
* Comment (if any).
* External user info (if the submitting app passed one).
* Metadata payload.
* Structured correction (if any).
* Timestamp.

Useful for aggregating multiple opinions on the same run.

## `feedback submit`

```bash theme={null}
nora feedback submit tr_abc \
  --dislike \
  --comment "Wrong refund policy quoted" \
  --external-user u_alice \
  --external-user-display-name "Alice Smith" \
  --correction '{"expected":"30 days","actual":"14 days"}' \
  --metadata '{"channel":"web-chat","urgency":"high"}'
```

Flags:

* `<trace-id>` (positional, required) — the trace to attach feedback to.
* `--like` **or** `--dislike` (exactly one, required).
* `--comment <text>` — human-readable note.
* `--external-user <id>` — your app's user identifier.
* `--external-user-display-name <name>` — human-readable name (for UI display).
* `--correction <json>` — structured fix ("what should have happened").
* `--metadata <json>` — arbitrary tags/context.

## Recipes

### Bulk-import feedback from a CSV

Say you have a CSV: `trace_id,verdict,comment,corrector_email`.

```bash theme={null}
while IFS=, read -r trace verdict comment corrector; do
  flag="--$verdict"   # --like or --dislike
  nora feedback submit "$trace" \
    $flag \
    --comment "$comment" \
    --external-user "$corrector"
done < feedback-batch.csv
```

Great for importing QA review results from an external system.

### Attach a correction from a diff

If your QA process produces a proposed correct output, submit it as `--correction`:

```bash theme={null}
nora feedback submit tr_abc \
  --dislike \
  --comment "See correction" \
  --correction '{"expected":"'"$(cat expected.txt | jq -Rs)"'"}'
```

Correction becomes the expected output when this trace is added to a dataset.

### Aggregate feedback across many traces

```bash theme={null}
for tr in $(nora traces list --flow support --status completed --json | jq -r '.[].id'); do
  count=$(nora feedback list "$tr" --json | jq 'length')
  [ "$count" -gt "0" ] && echo "$tr: $count feedback"
done
```

### Wire your app's thumbs UI

```bash theme={null}
# Get the contract
nora feedback describe support-agent

# Your app posts feedback on button press
# curl -X POST <url> -H "Authorization: ..." -d '{...}'
```

External `--external-user` values let you attribute feedback across systems without exposing your user IDs to Nora directly.

## What happens after submit

* The feedback row is stored on the trace.
* A Signal is created in the queue (see [Signals overview](/reliable/signals/overview)).
* The signal may join an existing cluster if similar patterns exist.
* The Improvement flow may propose a fix (see [From signal to fix](/reliable/signals/from-signal-to-fix)).

You don't need to do anything else — feedback rolls into the improvement loop automatically.

## Editing / deleting feedback

Feedback is immutable via the CLI. For editing (typo fixes, retracting a wrong thumbs), use the app: the reviewer who submitted can retract within 7 days.

## Related

* [Feedback on a trace](/reliable/traces/feedback) — the app-side concept.
* [Signals overview](/reliable/signals/overview) — what happens next.
* [`nora traces`](/cli/traces) — find traces to attach feedback to.