Why chunk?
Retrieval works by finding the smallest piece of content most relevant to a query. A whole 200-page PDF is too coarse (irrelevant sections drown the signal). A single sentence is too fine (loses context). Chunks are the goldilocks unit. The right chunk size depends on your content. Docs pages: 200-500 tokens. Chat transcripts: whole conversations. Code: file or function. Nora has sensible defaults but every block is tunable.Chunkers
Markdown chunker
For markdown, HTML, PDF-with-good-structure, and Notion pages.- Splits on heading hierarchy first (never mid-section).
- Each chunk retains the heading path (
Guides > Auth > OAuth) as metadata. - Fallback: character split at ~1200 chars if a section is too long, with 100-char overlap.
Semantic chunker
Uses a small model to find natural topic boundaries. Slower and more expensive but produces better chunks for prose without clear headings.PDF chunker
Handles PDFs with layout — columns, footnotes, tables. Preserves reading order. Extracts tables as HTML or CSV depending on your preference.Spreadsheet chunker
For CSV, XLSX, Google Sheets. Two modes:- Row-wise — each row is a chunk. Good for record-like data.
- Table-summary — one chunk per sheet with a summary plus a link to the raw data. Good for large tables where retrieval is topical rather than record-lookup.
Code chunker
Language-aware. Splits at function/class boundaries. Keeps imports and file-scope context attached to every function chunk. Supported: JS/TS, Python, Rust, Go, Java, C++, C#, Ruby, PHP, SQL. Others fall back to character split.Custom chunker (inline)
Write a JS/Python function that takes the item and returns a list of chunk objects. Complete control.Chunk metadata
Every chunk gets:- Content — the text.
- Source ID — the source item it came from.
- Path — hierarchical location (e.g., heading path, sheet name, function name).
- Position — chunk index within the source.
- Custom fields — any metadata you added upstream.