RAG and Vector Search: Build Retrieval You Can Explain and Evaluate

QUICK ANSWER

What you need to know

Build a retrieval baseline, compare lexical and vector search, preserve access boundaries and evaluate evidence coverage with an original local example, exercises and a mini-project.

Retrieval-augmented generation is useful when an answer needs evidence from a document collection. But adding a vector database does not establish that the right passage was retrieved, that the user may read it or that the final answer is supported. A practical learning path begins with a transparent retrieval baseline and a test set whose evidence you can inspect.

This guide uses original fictional documents and a local keyword retriever before introducing vector-search decisions. No embeddings service, cloud account or model call is needed for the example. It is a deliberately limited lexical baseline, not a production vector engine or a measured claim about RAG accuracy.

Before you start

Know Python collections, simple text processing and the difference between a document and a passage within it. Understand that access permissions are an application policy, not something a model may invent. Use documents you own or fictional fixtures; do not build a public learning corpus from private workplace material.

The Google reference explains how its RAG output connects retrieval evidence with generation. The project here is provider-independent and does not reproduce that documentation's code. Start by writing five questions and the passages that support them before deciding which search implementation to install.

Learning sequence

1. Give documents stable identities, versions and access labels. Define what happens when a source changes or is removed.
2. Split content along meaningful boundaries and preserve headings or metadata needed to interpret each passage.
3. Build an inspectable lexical baseline and a relevance set. Include exact identifiers, paraphrases and unanswerable questions.
4. Compare a vector candidate on the same queries. Record the embedding model, preprocessing, similarity convention and index version.
5. Add generation only after evidence retrieval is understood. Keep source identifiers through to citations and verify important claims against actual supporting passages.

Do not optimize answer wording while the required evidence never reaches the model. Diagnose ingestion, filtering, retrieval, reranking and context assembly separately. A reranker can reorder its candidates; it cannot recover a passage excluded from the candidate set.

Worked example

Each passage below has an ID, a version, a permitted group and an active flag. Permissions are filtered before ranking. In a real service, those groups must come from authenticated trusted policy, not arbitrary labels supplied by the request. This in-memory example assumes its group argument has already been authorized.

import re

def words(text):
    return set(re.findall(r"[a-z0-9]+", text.lower()))

def retrieve(query, group, chunks, limit=2):
    if type(limit) is not int or limit < 1:
        raise ValueError("limit must be a positive integer")
    terms = words(query)
    candidates = []
    for chunk in chunks:
        if not chunk["active"] or group not in chunk["groups"]:
            continue
        score = len(terms & words(chunk["text"]))
        if score:
            candidates.append((score, chunk["id"], chunk))
    candidates.sort(key=lambda item: (-item[0], item[1]))
    return [chunk for _, _, chunk in candidates[:limit]]

chunks = [
    {"id": "leave-v2", "version": 2, "groups": {"staff"},
     "active": True, "text": "Leave balance appears in the staff portal."},
    {"id": "finance-v1", "version": 1, "groups": {"finance"},
     "active": True, "text": "Leave balance financial reserve is restricted."},
    {"id": "leave-v1", "version": 1, "groups": {"staff"},
     "active": False, "text": "Leave balance appears in the old system."},
]
hits = retrieve("leave balance", "staff", chunks)
assert [chunk["id"] for chunk in hits] == ["leave-v2"]
assert retrieve("volcano", "staff", chunks) == []
assert retrieve("leave balance", "visitor", chunks) == []
print([chunk["id"] for chunk in hits])

The result is leave-v2. The restricted finance passage and inactive old version never become candidate evidence. This is not a complete authorization system: you still need identity, source-policy enforcement, cache isolation and revocation handling. What the fixture demonstrates is the order of operations, which you can test before involving a model.

The lexical score is the number of distinct shared terms. It is transparent but brittle: synonyms, morphology and ambiguous words can defeat it. A vector representation can help retrieve semantically related content, but may underperform exact matching for requisition numbers or product codes. Compare approaches instead of assuming semantic similarity means the entity or fact is correct.

For a vector experiment, record how text becomes a vector and how similarity is calculated. With cosine similarity, vector direction rather than magnitude drives the comparison after normalization; zero-length vectors need explicit handling. Do not mix stored vectors from incompatible embedding configurations with a new query encoder. Rebuild or migrate the index with a tested compatibility plan.

Exercises and solutions

Exercise 1: Find the stage that failed

An answer omits a policy exception. The trace shows that the exception passage was not among the retrieved candidates. Should you first rewrite the answer prompt?

Solution: Inspect the retrieval path first: source ingestion, chunk boundaries, permissions, query processing and candidate selection. Prompt changes cannot reliably quote evidence that was never supplied. Add a query with known supporting passages to a retrieval-only test before deciding whether generation also needs improvement.

Exercise 2: Handle a code-like identifier

A search for policy AB-104 returns AB-140 because their descriptions are similar. How would you improve the experiment?

Solution: Add exact-identifier cases to the relevance set and compare a lexical or hybrid strategy. Keep normalization documented and retain identifiers as metadata. Evaluate the actual candidate result rather than assuming the highest semantic score must represent the right policy. A reranker can help only if the correct passage enters its candidates.

Exercise 3: Reject a decorative citation

The answer cites the correct document, but the referenced passage discusses a different topic. Is the citation valid evidence?

Solution: Not for that claim. Check whether the cited passage actually supports the statement and version. Retain passage IDs through retrieval and generation, then inspect claim-to-evidence relationships. Count grounded claims separately from links; a high citation count can coexist with unsupported answers.

Mini-project

Build a permission-aware fictional handbook search. Include two user groups, active and retired document versions, exact IDs and at least one question with no supporting answer. Create a relevance file mapping queries to acceptable supporting passages; allow more than one valid passage when appropriate.

Deliver a lexical baseline, a retrieval trace and a report showing whether required evidence appears in the selected candidate budget. An optional vector comparison must reuse the same queries, access rules and relevance judgements, and identify its embedding configuration. Do not invent vector results before running that experiment. Keep raw retrieval evaluation separate from generated-answer quality.

Completion criteria: unauthorized and inactive passages are excluded; an unknown query can return no evidence; ties are reproducible; chunk boundaries preserve necessary context; and your report includes misses, not just successful screenshots. Demonstrate a source deletion and explain how the index and caches would remove stale evidence. Clearly distinguish the simulated local behaviour from any unimplemented production guarantees.

Common mistakes

• Retrieving restricted data and asking the model not to reveal it.
• Choosing chunk size without examining the documents or evaluation queries.
• Treating retrieval similarity as factual correctness.
• Mixing embedding versions without a compatibility check.
• Scoring retrieval only through the fluency of the final answer.
• Ignoring source updates, deleted chunks and permission changes.

Self-assessment

Can you show which stage lost a required passage? Can you explain why a query retrieved a candidate and whether the requester may see it? Can you distinguish missing evidence from contradictory evidence and state when the system should abstain? If those answers are unclear, another index or another prompt will not make the system explainable.

Connect this skill to roles and jobs

Read the RAG Engineer career guide, practise RAG Engineer interview questions and explore RAG jobs. Read each employer's responsibilities rather than assuming all retrieval roles have the same scope. Return to the AI Skills overview for related capabilities.

HOW THIS WAS PREPARED

Methodology

Editorial synthesis of cited sources

AI-assisted original educational material. Explanations, code, fictional fixtures, exercise solutions and mini-project acceptance criteria were prepared for this site, not reproduced from the references. Official Google references support background concepts, not an endorsement or validation of these projects. Runnable Python examples use the standard library; the SQL fixture is checked with SQLite and is not a BigQuery deployment tutorial. Samples demonstrate bounded concepts, not production-ready systems. Measurements and scenario values are synthetic, not benchmark results. Mini-projects are suggested exercises, not claims of completed deployments. No paid model calls are needed to run the examples. Related jobs use a skill or specialization filter and may include adjacent roles. References checked 17 October 2026. No hiring guarantees or unmeasured demand rankings are implied.

Read our editorial and research policy →

EVIDENCE

Sources

1. Retrieval and generation output of RAG Engine

   Google Cloud · Checked 17 October 2026