Basic RAG works in demos — production RAG requires query expansion, semantic caching, routing, fallback strategies, and monitoring patterns that the tutorials don't cover.

Retrieval quality is the most common RAG failure point and the hardest to measure — RAGAS provides the metrics framework, but building an eval pipeline that catches regressions before they hit production requires engineering work.

Shared vector stores across tenants leak data unless access control is enforced at the chunk level — the filtering strategies and architectural patterns that prevent cross-tenant retrieval in enterprise RAG.

Chunks lose their context when split from source documents — Anthropic's contextual retrieval pattern injects document-level context into each chunk before embedding, reducing retrieval failures by 35-49%.

GraphRAG adds a knowledge graph extraction layer over your corpus to answer cross-document questions that flat vector retrieval misses — but it costs 10-100x more to build and query, so the use case must justify it.

Bi-encoder similarity retrieves fast but imprecisely — cross-encoder reranking re-scores the top candidates with full query-document attention and is now a required stage in production RAG pipelines.

Vector similarity alone misses exact matches and rare terms — hybrid search combining BM25 keyword retrieval with dense embeddings using Reciprocal Rank Fusion is now the production baseline for enterprise RAG.

A practical retrospective on AI engineering in 2026 — which patterns delivered real value, which promises fell short, and what the senior engineers who navigated the year right have in common.

Internal developer tools are the highest-ROI AI application most companies underinvest in — AI-powered knowledge search, runbook automation, and incident tooling compound value every day across every engineer.

After two years of rapid tooling evolution, some parts of the AI engineering stack have stabilized — here's what you can build on confidently and where you should still expect churn.