AI · 2025
Lumen AI Assistant
A 2,000-page knowledge base nobody could search. Lumen ingests every doc, chunks and embeds it, and answers agent questions in seconds with citations back to the exact paragraph.
Client
Lumen Support
Industry
Customer Support / AI
Timeline
7 weeks
My role
AI & Backend Engineer
The problem
Support staff spent hours searching a 2,000-page internal knowledge base for answers customers needed in minutes.
The solution
A document-aware assistant using embeddings search and streaming answers, with citations back to the source page.
-65%
Time to answer
92%
Answer accuracy
2k+
Docs indexed
What I built
Cited answers
Every claim links to the source chunk, so agents can verify before replying.
Hybrid search
pgvector similarity blended with full-text BM25 for names and error codes.
Streaming responses
Tokens stream to the UI so the first words appear in under a second.
Feedback loop
Thumbs-down answers are queued for review and re-indexed weekly.
Architecture
- Python + FastAPI service with async streaming endpoints
- PostgreSQL with pgvector, HNSW index over 180k chunks
- Chunker that respects headings so context never splits mid-policy
- Redis cache for repeated questions, Docker on AWS ECS
Hard problems
Hallucinated policies
Strict grounding: if no chunk scores above threshold, the model must say it doesn't know.
Stale documents
Nightly diff job re-embeds only changed chunks, cutting embedding cost by 94%.
Code from the build
A few real excerpts from the repository.
async def retrieve(question: str, k: int = 8) -> list[Chunk]:
vec = await embed(question)
rows = await db.fetch(
"""
select id, doc_id, text,
1 - (embedding <=> $1) as score,
ts_rank(tsv, plainto_tsquery($2)) as lexical
from chunks
order by (1 - (embedding <=> $1)) * 0.75
+ ts_rank(tsv, plainto_tsquery($2)) * 0.25 desc
limit $3
""",
vec, question, k,
)
return [Chunk(**r) for r in rows if r["score"] > 0.72]SYSTEM = """You answer only from the provided context.
If the context does not contain the answer, reply exactly:
"I couldn't find that in the knowledge base."
Cite every fact as [doc:{id}]."""
def build(question: str, chunks: list[Chunk]) -> list[dict]:
context = "\n\n".join(f"[doc:{c.doc_id}] {c.text}" for c in chunks)
return [
{"role": "system", "content": SYSTEM},
{"role": "user", "content": f"{context}\n\nQuestion: {question}"},
]"First-response time dropped by two thirds and new agents ramp up in days, not weeks."
How it was built
- Discovery call, scope and fixed timeline agreed up front
- Custom design system built in Figma before development
- Typed, reviewed codebase with CI checks on every commit
- Deployed with monitoring, backups and post-launch support