OpenRouter Embeddings for RAG and Search

OpenRouter embeddings provide vectors for semantic search and retrieval-augmented generation (RAG). Start with one embedding model, preserve document permissions, and evaluate retrieval before relying on generated answers.

Embeddings turn content into searchable vectors

An embedding is a numeric representation used to compare semantic similarity. In a RAG system, you embed document chunks, store vectors with metadata, embed the user query, retrieve nearby chunks, and give only relevant evidence to a generation model.

Checked October 15, 2026: available embedding models, dimensions, prices, and API support change. Use the live catalog and current embedding API documentation.

End-to-end RAG flow

1. Ingest and normalize documents.
2. Split them into meaningful chunks with stable IDs.
3. Embed chunks using one chosen model.
4. Store vectors plus source, title, permissions, and version.
5. Embed each query with the same model.
6. Retrieve top candidates and optionally rerank.
7. Generate an answer that cites supplied evidence.
8. Log retrieval and evaluation results.

Basic embedding request

{
 "model":"a-current-embedding-model",
 "input":[
  "OpenRouter can route requests across providers.",
  "A fallback chain improves availability but changes behavior."
 ]
}

import os,requests
r=requests.post(
 "https://openrouter.ai/api/v1/embeddings",
 headers={"Authorization":f"Bearer {os.environ['OPENROUTER_API_KEY']}",
          "Content-Type":"application/json"},
 json={"model":os.environ["OPENROUTER_EMBEDDING_MODEL"],"input":["first chunk","second chunk"]},
 timeout=(5,60))
r.raise_for_status()
items=r.json()["data"]
vectors=[item["embedding"] for item in items]
print(len(vectors),len(vectors[0]))

Confirm endpoint and model support in live docs before use. Set OPENROUTER_EMBEDDING_MODEL to an eligible embedding model ID from the live catalog. Store that ID and its dimension beside every index.

Chunking that preserves meaning

Fixed token chunks are simple but can split tables and procedures. Prefer headings, paragraphs, or semantic boundaries, then add modest overlap. Store parent document, section heading, page/URL, timestamps, and access-control labels.

Never mix vector spaces

Query and document vectors must come from the same embedding model and compatible settings. When changing models, create a new index or re-embed all content. Do not append new dimensions to an old collection.

Retrieval example

# Pseudocode
query_vector=embed(user_query)
results=vector_db.search(
 vector=query_vector,
 top_k=12,
 filter={"tenant_id":current_tenant,"published":True})
reranked=rerank(user_query,results)[:5]
context="\n\n".join(r.text for r in reranked)

Enforce permissions in retrieval filters, not only in the prompt. A language model cannot repair a data leak after unauthorized chunks were retrieved.

Evaluation

Layer	Metric
Retrieval	Recall@k, precision@k, MRR, relevant-source rate
Generation	Faithfulness, citation correctness, answer relevance
Operations	Latency, cost, index freshness, failure rate

Build a labeled question set from real user needs. Compare chunk size, overlap, model, top-k, reranker, and prompt one variable at a time.

Production controls

• Batch embedding requests within documented limits.
• Cache by content hash.
• Version chunks and tombstone deleted content.
• Encrypt stored text and restrict vector-store access.
• Apply tenant and document permissions before similarity ranking.
• Return “not enough evidence” when retrieval is weak.

Common failures

• Irrelevant results: improve chunk boundaries, metadata filters, or reranking.
• Good chunks, bad answer: strengthen evidence-only prompt and citations.
• Dimension mismatch: rebuild index with one model.
• Stale answers: version ingestion and remove superseded chunks.
• Permission leak: enforce server-side metadata filters.

OpenRouter embeddings FAQ

How large should chunks be?

There is no universal size. Start around coherent paragraphs/sections, then evaluate retrieval on your documents.

Do I need a vector database?

For small corpora, an in-process index can work. Use a database when persistence, filtering, scale, or multi-tenant controls matter.

Read next

See structured outputs, routing, and the complete guide.

Official references

• Embeddings API

Documentation reviewed 15 October 2026. Examples illustrate the request pattern; confirm model availability and account settings before use. No uptime or performance benchmark is claimed.