Skip to main content
← All insights

By

CTO @ Beautron.AI · AI Product Engineer

3 min read

AI / RAG / LLM / Architecture

RAG that earns trust: how ARA answers questions about me

Retrieval-augmented generation fails quietly when retrieval is weak and answers can't be checked. Hybrid search, context-aware chunks, citations and a retrieval eval make it trustworthy. ARA, the assistant on this site, uses all four.

I've deployed retrieval-augmented generation (RAG) agents with Azure OpenAI and LangChain for intelligent document querying. The demos are always impressive. The problems show up later, quietly: an answer that sounds right but cites nothing, or a question the system "can't find" although the document is right there. ARA, the assistant at the bottom right of this site, is a small RAG system built to avoid those failures. Here's how it works and why.

The pipeline

 Site content + these articles
            │
            ▼
 Section-aware chunks with context headers
            │
   ┌────────┴─────────┐
   ▼                  ▼
 BM25 keyword      Vector search
 search            (embeddings, optional)
   └────────┬─────────┘
            ▼
 Reciprocal rank fusion → top chunks
            ▼
 Claude answers from numbered sources,
 citing them as [1], [2] …
            ▼
 Streamed to the browser with source links

1. Chunk by meaning, and give each chunk context

Splitting text every 500 characters cuts sentences and loses context. A chunk that says "reduced costs by 30%" is useless if it doesn't say whose costs or how.

Instead, ARA splits content along its natural structure: one chunk per job, per project or per article section. It also adds a short context header to each chunk before indexing, for example "Experience — Chief Software Engineer / Architect at e-DestinAccess Inc.". That header is what lets a question like "what did he do at e-DestinAccess?" match the right bullet points.

2. Use hybrid retrieval

Vector search is great at meaning ("payments experience" finds "Stripe and Authorize.net integration") and weak at exact terms such as product names, acronyms and error codes. Keyword search (BM25) is the opposite. Running both and merging the results with reciprocal rank fusion gives you the strengths of each, with no score calibration to tune.

ARA always runs BM25, which needs no external service. When an embeddings key is configured it adds vector search with Voyage embeddings and fuses the two lists.

3. Make answers checkable

Every retrieved chunk is given to the model as a numbered source, and the instructions are strict:

  • answer only from the sources and the profile,
  • cite sources inline as [1], [2],
  • say so plainly when the sources don't contain the answer, and point to the contact form instead of guessing.

The browser shows the cited sources as links back to the exact section or article. A reader can verify any claim in one click, which is the difference between a chatbot and a tool people trust.

4. Treat the user's message as data

Anyone can type into a public chat box, including "ignore your instructions". ARA keeps its instructions in the system prompt, treats both the user's text and the retrieved content as data, stays on topic (my work, projects, skills and availability) and has hard limits on message length, history length and request rate per visitor.

5. Measure retrieval, not vibes

The most useful test in a RAG system checks retrieval, not generation. This site has a small eval: a list of realistic questions, each paired with the source that must be retrieved, run on every test pass. If a content change or a tokenizer tweak breaks "which payment gateways has he integrated?", the test fails before a visitor notices.

When not to build RAG

If your whole knowledge base fits comfortably in the model's context window, the simplest robust option is often to put all of it in the prompt and use prompt caching. There are no retrieval misses and no index to maintain. RAG earns its complexity when content outgrows the context window, changes often, or must be cited at a fine level of detail. A portfolio sits near that line. I built ARA with retrieval because citations matter here, and because the same design scales to the document sets my clients actually have.

Takeaways

  • Chunk along the content's structure and add a context header to each chunk.
  • Combine keyword and vector retrieval with rank fusion.
  • Number the sources, require citations, and show them to the user.
  • Treat user input as untrusted, and rate-limit public endpoints.
  • Keep a retrieval eval in your test suite.

Working on something similar? Let's talk — or ask ARA, the assistant in the corner, about this article.

More insights