AI Engineer Interview Questions and Answer Frameworks

Define leakage as information unavailable at prediction time or contamination between training and evaluation. Discuss splitting before learned preprocessing, time-aware splits when appropriate, fitting transformers only on training data, checking duplicate entities and reviewing feature provenance. Give a concrete example and explain how leakage would distort the metric.

Check whether accuracy matches the business cost, then inspect class balance, confusion matrix, segments and recent data. Validate the labels and serving pipeline. Compare offline and online inputs, define a better metric and create error slices. The strongest answer treats this as both a model and system investigation.

Describe the gap between training fit and generalization. Responses can include better validation, simpler models, regularization, more representative data, augmentation or early stopping. Explain which evidence would lead you to each option.

Separate retrieval and generation. Create representative questions with expected evidence, measure retrieval relevance or recall, verify citation support, score answer usefulness and test refusal when evidence is absent. Track latency and cost. Include adversarial or prompt-injection cases and review important failures with humans.

Start from the required behaviour. Prompting is suitable when the base model already has the capability and instructions/context are enough. RAG is useful when answers need current or private knowledge with traceable evidence. Fine-tuning can help consistent behaviour or task adaptation, but it does not automatically provide current facts. Mention maintenance, evaluation data, privacy and cost.

Discuss incorrect tool choice, unsafe side effects, excessive loops, prompt injection, leaked secrets, stale state and unbounded cost. Mitigations include least-privilege tools, confirmation for consequential actions, typed inputs, timeouts, budgets, sandboxing, audit logs and evaluation of complete trajectories.

Clarify traffic, latency, freshness and availability. Cover input validation, feature or retrieval dependencies, model versioning, autoscaling, caching, fallbacks, observability, staged rollout and rollback. Define service metrics and model-quality signals. Google's production guidance is a useful reminder that data and operational components usually outweigh the model code.

Use input schema checks, distribution shifts, embedding or prediction changes, proxy outcomes, sampling for human review and delayed-label backfills. Establish thresholds cautiously and connect alerts to an investigation runbook. State that drift does not prove performance loss; it signals a need to investigate.

Verify the measurement and sample size, then inspect data coverage, labels and operational context. Involve domain and affected stakeholders, assess harm, compare mitigation options and document the decision. Do not promise that a single fairness metric solves the problem.