Skip to main content
The Enrich stage is where chunks get the metadata retrieval will use to find them. Blocks in this stage are additive — each one attaches new fields.

​
Enrichment blocks

​
Embed

Turn each chunk’s text into a dense vector. Pick the embedding provider that fits your data (multilingual for non-English, longer-context for large chunks, smaller-dim for cost). The picker shows dimension, context window, and per-1k-token cost so you can compare. Embedding is what makes semantic search work — vectors closer in space are more relevant to each other. Every chunk gets a vector field written to the Output store.

​
Auto-tag (LLM)

Ask a small model to tag each chunk. Give it a list of possible tags and a short instruction. Common uses:
  • Domain: billing, auth, pricing, security
  • Audience: internal, customer, developer
  • Content type: how-to, reference, troubleshooting
Tags become filterable metadata. See Filters.

​
Entity extraction

Detect and normalize named entities:
  • People, organizations, locations.
  • Products, SKUs, project codes.
  • Dates and durations.
  • Custom entities via regex or a small classifier.
Each detected entity is added as a searchable field. Good for search-and-retrieve queries like “show me every chunk mentioning Acme Corp.”

​
Language detection

Detects the chunk’s language and writes it as a metadata field. Enables per-Agent language filters.

​
Summary generator

Adds a one-paragraph summary of each chunk. Retrieval can search on the summary in addition to the full chunk — sometimes better recall for short, keyword-heavy queries.

​
Sentiment / toxicity

Adds sentiment score and toxicity flags. Useful when your knowledge includes customer messages and you want to filter or route by emotional signal.

​
Custom enricher (inline)

JS/Python inline function. Takes the chunk, returns extra metadata to attach. Good for calling your own scoring services.

​
Cost management

Enrichment is where cost adds up — every LLM call, every embedding. Two levers:
  • Batch size — embedding APIs are much cheaper in batches. Default 100.
  • Skip if unchanged — on by default. A chunk with the same content hash won’t be re-enriched even if the pipeline re-runs.
The run log shows enrichment cost broken down by block.

​
Ordering

Enrichers run in the order wired. Some enrichers can consume the output of earlier ones — e.g., use extracted entities as inputs to a downstream tag classifier. When there’s no dependency, blocks run in parallel automatically for speed.

​
What Enrich does not do

Enrichment blocks do not modify the chunk text itself — only add metadata. If you need to change the text (translate, summarize into the body), do that in Normalize with a custom transform.