> ## Documentation Index
> Fetch the complete documentation index at: https://docs.platform.nora.my/llms.txt
> Use this file to discover all available pages before exploring further.

# Source blocks

> Pull raw data in from cloud drives, databases, APIs, and files.

The **Source** stage is where a pipeline first touches the outside world. A Source block emits raw items — files, records, messages — into the pipeline for the next stages to work on.

## Source block types

### Cloud folder

Pulls files from a connected cloud drive (Google Drive, OneDrive, S3-compatible).

* **Folder** — pick from the connected account.
* **Recurse** — include subfolders. On by default.
* **File types** — restrict to `.pdf`, `.docx`, `.md`, `.csv`, etc. Skip binaries you can't process.
* **Since** — only files modified after this timestamp. Combine with a run-payload timestamp to build incremental pipelines.

Emits one item per file, with `content` (bytes), `path`, `mime_type`, `modified_at`, and `source_id` (stable across renames).

### Document store

Pulls from an existing Nora document collection. Useful when a downstream pipeline builds on data another pipeline already curated.

### Database query

Runs a SQL query on a connected Postgres/MySQL/SQL Server database, streams rows.

* **Query** — parameterized SQL. Reference run-payload with `:key`.
* **Batch size** — rows per emitted item. Larger batches are more efficient; smaller batches let downstream stages parallelize better.

Emits one item per row (or one per batch, if you set batch > 1).

### HTTP fetch

Custom HTTP GET/POST. Emits the response body.

Configure:

* URL (templatable with run-payload).
* Method, headers, body.
* **Follow pagination** — automatically loop until a stop condition. Cursor param, page count, or "until empty response".

### Webhook payload

Uses the payload from the Webhook trigger as the source. No external call — the payload is already in memory.

### Manual upload

An empty Source that lets you upload files when running the pipeline manually. Useful for testing.

## Idempotency & dedup

Every emitted item gets a stable **content hash**. Later runs against the same source recompute the hash — unchanged items are skipped end-to-end.

This means a scheduled Foundry run over a large folder is cheap: it only touches what changed.

You can force a full re-process via **Run → Rerun (ignore cache)** if the pipeline downstream changed.

## Change detection

For Watch triggers, the Source block also emits **delete events** when a source item vanishes. Downstream stages should handle deletes — often by pushing a supersede to the Output stage rather than a fresh chunk. See [Supersede](/build/knowledge/supersede).

## Limits

* **Max items per run** — default 50k. Larger runs should be split by date range.
* **Max total bytes** — default 5 GB. Raise on Enterprise plans.
* **Per-item byte cap** — default 100 MB per file. Larger items are skipped and logged.