Enterprise RAG Pipeline — Garth Puckerin

2M+

Docs Indexed

38%

Fewer Support Tickets

<1.2s

Avg Response Time

The Problem

A Fortune 500 client had 2M+ internal documents — policies, runbooks, product specs, legal contracts — spread across SharePoint, Confluence, and a legacy intranet. Their support team was spending 60% of ticket time manually locating the right document before they could even begin answering.

They needed a way to surface the right information, to the right person, instantly — without replacing their existing toolchain or retraining staff.

The Approach

I designed and built an end-to-end RAG (Retrieval-Augmented Generation) pipeline that ingested, chunked, embedded, and indexed every document into a managed vector store, then exposed it via a conversational interface.

Ingestion Pipeline
SharePoint / Confluence / Intranet
        ↓
Document Parser (PyMuPDF, Unstructured.io)
        ↓
Semantic Chunker  →  Token budget: 512 / 128 overlap
        ↓
text-embedding-3-large  →  Pinecone (cosine, 3072d)

Query Pipeline
User Question
        ↓
Query rewriter  (LLM, HyDE technique)
        ↓
Hybrid Search  →  Vector + BM25 re-rank (Cohere)
        ↓
Top-k passages  →  Context assembly
        ↓
LLM  →  Grounded answer + source citations
          

Key Engineering Decisions

HyDE (Hypothetical Document Embedding) — Before searching, the LLM generates a hypothetical ideal document, which is embedded and used as the query vector. This dramatically improved recall for abstract questions.
Hybrid search + re-ranking — Pure vector search struggled with exact-match queries (product codes, policy IDs). Adding BM25 keyword search with a Cohere re-ranker cut hallucinations by ~40%.
Streaming responses — FastAPI + Server-Sent Events delivered tokens as they generated, keeping perceived latency under 400ms for first token.
Citation grounding — Every response includes source links and confidence scores, so users can verify. This was a non-negotiable trust requirement from the client.
Namespace isolation — Pinecone namespaces partition data by department, enforcing access controls without a separate auth layer on the vector store.

Outcome

Deployed to 1,200 internal users. Within 90 days, first-contact resolution rates increased by 31%, and support ticket volume dropped 38%. The client extended the contract to cover a second business unit.