> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.nora.my/llms.txt
> Use this file to discover all available pages before exploring further.

# nora pipelines

> Foundry data pipelines from the terminal — nodes, edges, scheduling, publish, runs, logs.

Pipelines are the Foundry surface: ingest, clean, chunk, enrich, index. The CLI mirrors the visual canvas — every node, every edge, every scheduled run is scriptable.

The pipeline lifecycle is the biggest CLI surface after `flows` — plan on `flows use` for your Flow and separately reference pipelines by their IDs.

## Command groups

* **CRUD**: `pipelines list/get/create/update/delete`
* **Nodes**: `pipelines nodes add/list/update/remove`
* **Edges**: `pipelines edges connect/list/update/delete`
* **Schedule**: `pipelines schedule set/clear`
* **Runs**: `pipelines dry-run`, `pipelines run file/document/folder`
* **Blocks**: `pipelines blocks test/ingest-local`
* **Publish**: `pipelines publish/rollback`, `pipelines publications list/live`
* **Logs**: `pipelines logs list/get`
* **Bulk**: `pipelines sync`
* **Cloud ingest**: `pipelines ingest-cloud`

## Pipeline CRUD

### List / get

```bash theme={null}
nora pipelines list
nora pipelines get p_billing_ingest
```

`list` gives one row per pipeline. `get` prints full metadata plus nodes and edges.

### Create

```bash theme={null}
nora pipelines create \
  --name "Billing ingest" \
  --stages source,clean,chunk,enrich,output \
  --vector-store qdrant
```

Flags:

* `--name` (required) — display name.
* `--stages` (required) — comma list of stage names in order.
* `--id <id>` — optional explicit ID; otherwise server mints.
* `--vector-store pgvector|qdrant` — index backend for the output stage.

### Update

```bash theme={null}
nora pipelines update p_billing_ingest \
  --set-name "Billing ingest (v2)" \
  --set-vector-store qdrant
```

* `--set-name <n>`, `--set-stages <s,…>`, `--set-vector-store <backend>`.

### Delete

```bash theme={null}
nora pipelines delete p_billing_ingest
```

Nodes and edges cascade. Live pipeline versions remain reachable by ID for a grace period.

## Nodes

Nodes are the individual blocks in the pipeline canvas — each palette entry (`src_local`, `chunk_markdown`, `embed_openai`, etc.) becomes a node instance.

### List

```bash theme={null}
nora pipelines nodes list p_billing_ingest
```

Prints all nodes as JSON.

### Add

```bash theme={null}
nora pipelines nodes add p_billing_ingest \
  --palette-id chunk_markdown \
  --stage chunk \
  --title "Chunk markdown" \
  --icon scissors \
  --tone default \
  --order 2 \
  --config '{"max_tokens": 400, "overlap": 50}'
```

Flags:

* `--palette-id` (required) — the palette entry ID.
* `--stage` (required) — which pipeline stage.
* `--title`, `--icon`, `--tone` (required) — display.
* `--order` (required) — visual order within the stage.
* `--id <id>` — explicit ID; server mints if omitted.
* `--sublabel <text>` — small caption under title.
* `--config <json>` — block-specific config.
* `--notes <text>` — free-text notes.
* `--output-ports <json>` — override the palette's default output ports.

### Update

```bash theme={null}
nora pipelines nodes update n_chunk \
  --set-config '{"max_tokens": 600, "overlap": 100}' \
  --set-title "Chunk (longer)"
```

`--set-*` flags: `--set-title`, `--set-sublabel`, `--clear-sublabel`, `--set-stage`, `--set-order`, `--set-config`, `--set-notes`, `--clear-notes`, `--set-output-ports`.

### Remove

```bash theme={null}
nora pipelines nodes remove n_chunk
```

## Edges

### List / connect / update / delete

```bash theme={null}
nora pipelines edges list p_billing_ingest
nora pipelines edges connect p_billing_ingest --from-node-id n_source --to-node-id n_chunk
nora pipelines edges update e_abc --set-label "raw markdown"
nora pipelines edges delete e_abc
```

Flags on connect: `--from-node-id`, `--to-node-id` (required), `--id <id>`, `--from-port <port>`, `--label <text>`.

## Schedule

### Set

```bash theme={null}
nora pipelines schedule set p_billing_ingest \
  --cron "0 2 * * *" \
  --tz Asia/Seoul \
  --target billing \
  --sample 100
```

* `--cron <expr>` — 5-field cron.
* `--tz <IANA>` — timezone.
* `--target <tag>` — the folder tag to process on each run.
* `--sample <n>` — cap items per scheduled run.

Only flags passed change. Omit `--cron` to keep the current one, e.g. change only the target.

### Clear

```bash theme={null}
nora pipelines schedule clear p_billing_ingest
```

Wipes all schedule fields — pipeline becomes manual-only.

## Running a pipeline

### Dry-run against inline text

```bash theme={null}
nora pipelines dry-run p_billing_ingest --input "This is a test document."
# Or from a file:
nora pipelines dry-run p_billing_ingest --input @sample.md
# Or from stdin:
cat sample.md | nora pipelines dry-run p_billing_ingest --input -
```

Runs the pipeline end-to-end against the given text without persisting output. Prints results per stage.

### Run against a file

```bash theme={null}
nora pipelines run file p_billing_ingest --file ./contract.pdf
```

Multipart-uploads the file and runs the pipeline.

### Run against a stored document

```bash theme={null}
nora pipelines run document p_billing_ingest --document-id doc_abc
```

Runs against a document already in the library.

### Run against a folder

```bash theme={null}
nora pipelines run folder p_billing_ingest --tag billing --max-documents 500
```

Runs against every document tagged `billing` (up to `--max-documents`).

## Single-block testing

```bash theme={null}
nora pipelines blocks test \
  --palette-id chunk_markdown \
  --input "# Heading\n\nSome content..." \
  --config '{"max_tokens": 200}'
```

Test one palette block in isolation — no pipeline required. Useful when iterating on chunker or enricher config.

For `src_local` nodes on a running pipeline:

```bash theme={null}
nora pipelines blocks ingest-local n_source
```

Kicks off a local-filesystem scan.

## Publishing pipelines

Pipelines version and publish like Flows.

### Publish

```bash theme={null}
nora pipelines publish p_billing_ingest \
  --summary "chunker: max_tokens 400 -> 600" \
  --author-label "paul"
```

Freezes DRAFT into a live version. Scheduled runs and downstream consumers use the live version.

### Rollback

```bash theme={null}
nora pipelines rollback p_billing_ingest 5
```

Flips the live pointer to version `seq=5`.

### Publications

```bash theme={null}
nora pipelines publications list p_billing_ingest
nora pipelines publications live p_billing_ingest
```

`list` shows history. `live` fetches the currently-live payload.

## Logs

### List runs

```bash theme={null}
nora pipelines logs list p_billing_ingest \
  --status ok \
  --since 2026-07-01 \
  --limit 20
```

Filters:

* `--status ok|error`
* `--search <text>` — full-text over log payloads.
* `--since <ISO>` / `--until <ISO>`
* `--run-kind <kind>` — filter by which entry point.
* `--limit <n>` — max rows.
* `--json` — machine-readable.

### Get one

```bash theme={null}
nora pipelines logs get p_billing_ingest log_abc
```

Full per-stage detail for one run.

## Bulk sync

```bash theme={null}
nora pipelines sync p_billing_ingest --file ./pipelines/billing.json
```

Replaces the pipeline's name, stages, nodes, and edges from a JSON file. Idempotent — safe to run in CI on every deploy.

## Cloud ingest

```bash theme={null}
nora pipelines ingest-cloud p_billing_ingest n_gdrive_source
```

Kicks off a manual bulk ingest for a `src_cloud` node (Google Drive, S3, etc.). Otherwise these run only on the schedule.

## Recipes

### Bootstrap a minimal pipeline in one script

```bash theme={null}
P=$(nora pipelines create --name "Docs ingest" --stages source,chunk,enrich,output | jq -r '.id')

nora pipelines nodes add $P --palette-id src_local --stage source --title "Local" --icon folder --tone default --order 1
nora pipelines nodes add $P --palette-id chunk_markdown --stage chunk --title "Chunk MD" --icon scissors --tone default --order 1
nora pipelines nodes add $P --palette-id embed_openai --stage enrich --title "Embed" --icon spark --tone default --order 1
nora pipelines nodes add $P --palette-id out_pgvector --stage output --title "Index" --icon database --tone default --order 1

# Wire (need node IDs — capture from add)
# ...

nora pipelines publish $P --summary "initial"
```

### CI: sync pipeline from git

```bash theme={null}
nora pipelines sync p_docs_ingest --file pipelines/docs.json
nora pipelines publish p_docs_ingest --summary "$(git log -1 --pretty=%s)"
```

### Get the most recent errored run

```bash theme={null}
nora pipelines logs list p_billing_ingest --status error --limit 1 --json | jq '.[0]'
```

### Turn off a schedule for maintenance

```bash theme={null}
nora pipelines schedule clear p_billing_ingest
# Do maintenance...
nora pipelines schedule set p_billing_ingest --cron "0 2 * * *" --tz Asia/Seoul --target billing
```