RAG Engineer Interview Questions: 10 Detailed Answers and Scenarios

Common mistake: Assuming a fixed chunk size works equally well for every document type.

3. When might semantic search need lexical retrieval?

Answer: Exact identifiers, product codes and uncommon names may require precise lexical matching that an embedding search does not reliably prioritize. Compare lexical and semantic candidates, or combine them using an evaluated ranking strategy. Keep query normalization explainable and test near-identical identifiers. A semantically similar passage can still be completely wrong for a specific reference number.

Hypothetical scenario: A user asks about policy AB-104, but retrieval returns the similarly worded policy AB-140.

Follow-up: Which test cases would expose identifier confusion?

Common mistake: Assuming semantic similarity guarantees an exact entity match.

4. Why version the embedding model and index together?

Answer: Embeddings depend on the model and preprocessing used to create them. Mixing incompatible vectors or silently changing text normalization can make retrieval unreliable. Record model version, chunk strategy, source snapshot and index configuration. Rebuild or migrate deliberately, evaluate candidate retrieval and keep a rollback path rather than replacing query embeddings without coordinating the stored index.

Hypothetical scenario: A new query encoder is deployed while all stored document vectors still use the previous representation.

Follow-up: What compatibility evidence is needed before sharing an index?

Common mistake: Treating vectors from different embedding configurations as automatically interchangeable.

5. How do you evaluate retrieval independently?

Answer: Create queries with reviewed relevant passages or documents, allowing multiple valid evidence sources where appropriate. Measure whether needed evidence appears within the candidate budget and inspect difficult slices. Keep relevance judgements separate from generated answer quality. Explain limitations in the reference set; a passage absent from the annotations is not necessarily irrelevant.

Hypothetical scenario: Several policy sections answer the question, but the evaluation labels only one as relevant.

Follow-up: How could incomplete relevance labels distort your comparison?

Common mistake: Using answer fluency as the only retrieval metric.

6. What can reranking fix, and what can it not?

Answer: A reranker can improve the order of candidates it receives, potentially putting more useful evidence into limited context. It cannot recover a relevant passage that never entered that candidate set. Measure candidate recall before attributing gains or failures to reranking, and include its latency and cost when choosing how many candidates to retrieve and retain.

Hypothetical scenario: The correct document is filtered out before a reranker receives its top candidates.

Follow-up: Which earlier stage must change before reranking can help?

Common mistake: Adding a reranker as a universal solution to missing evidence.

7. Where should document permissions be enforced?

Answer: Apply trusted authorization before documents become model context and before snippets or citations are returned. Ensure retrieval filters derive from authenticated access rules, not user-provided labels. Recheck changing permissions and protect caches from cross-user reuse. A model instruction not to reveal secrets is insufficient once unauthorized information has already entered the request.

Hypothetical scenario: An employee searches a topic that exists in both public guidance and restricted compensation documents.

Follow-up: How would you test that caching cannot expose another user's evidence?

Common mistake: Filtering only the final answer after unauthorized passages were retrieved.

8. How should updates and deletions reach a RAG system?

Answer: Track source identity and version so ingestion can replace or remove the corresponding chunks. Reconcile indexing checkpoints with the source and monitor freshness, not only successful ingestion runs. Define how caches and citations handle removed material. Test partial failures because an update can otherwise leave old and new passages available together without clear version precedence.

Hypothetical scenario: A withdrawn policy remains searchable because the ingestion job only adds new documents.

Follow-up: What reconciliation detects stale chunks after a missed deletion event?

Common mistake: Assuming an append-only ingestion pipeline keeps the index current.

9. How do you answer when evidence is missing or contradictory?

Answer: Do not force a single confident answer. Explain what the available sources establish, identify the conflict or missing detail and ask for clarification where useful. Apply an explicit version or authority rule only when the product defines one. Evaluate these cases deliberately; appropriate abstention is a supported outcome, not automatically a failure to answer.

Hypothetical scenario: Two documents give different application deadlines and neither is marked as superseded.

Follow-up: What evidence would justify preferring one deadline?

Common mistake: Letting the model invent an authority rule to hide a source conflict.

10. What makes citations useful rather than decorative?

Answer: A citation should point to the actual supporting source and, where possible, the specific passage and version. Check that each important claim is supported, not merely that a link exists. Retain source identifiers through retrieval and generation rather than asking the model to invent URLs. Measure citation correctness separately from answer completeness and response speed.

Hypothetical scenario: An answer links to the right document but cites a paragraph unrelated to its central claim.

Follow-up: How would you test claim-level support without rewarding irrelevant links?

Common mistake: Counting the number of citations as evidence of grounding quality.

Turn practice into evidence

Build a small policy-search corpus with identifiers, document versions and access labels. Prepare queries with known supporting passages, missing answers and contradictory versions. Trace retrieval, reranking and generation separately; demonstrate that a deleted or unauthorized document cannot become answer context.

Score each response informally: 0 if you cannot explain it, 1 for a definition, 2 for an approach with checks, and 3 for justified trade-offs plus a failure-recovery plan. This is a self-study rubric, not a validated hiring assessment. Revisit weak answers and never invent project outcomes or experience.

Career roadmap and related jobs

Build the underlying skills with the RAG Engineer career guide. For a shorter starting point, read AI interview answer frameworks.

Explore relevant live jobs. Listings are filtered by RAG and can include adjacent roles. Read the employer's requirements and original posting before applying. Use one listing to select the questions most relevant to its responsibilities.