Technical practice
How would you design and evaluate prompts for a production LLM application?
What to coverThe posting calls for hands-on prompt engineering. Suggested preparation: explain how you would test prompts against representative inputs, check output quality and consistency, and weigh quality against latency and cost.
How would you design a scalable retrieval-augmented generation system?
What to coverThe role includes building scalable RAG systems. Suggested preparation: walk through ingestion, retrieval, context assembly and generation, then explain how you would evaluate answer quality and identify bottlenecks.
How would you choose and troubleshoot embeddings and vector retrieval?
What to coverThe posting specifies embedding-based systems and vector databases. Suggested preparation: discuss retrieval relevance, indexing and query choices, and how you would diagnose cases where useful source material is not retrieved.
When would you fine-tune an LLM with LoRA rather than improve its prompts or retrieval?
What to coverModel fine-tuning, including LoRA, is listed in the requirements. Suggested preparation: compare the approaches by the problem they address, the evidence needed to justify fine-tuning, and their performance, cost and maintenance implications.
How would you choose among Bedrock, SageMaker and Lambda for deployment and inference components?
What to coverThe posting names these AWS services for deployment and inference. Suggested preparation: describe a plausible division of responsibilities and explain how workload needs, latency, operational effort and cost would affect your choices; do not assume a prescribed architecture.
How would you structure a Python API for a production AI application?
What to coverThe role calls for Python fundamentals, APIs and backend components. Suggested preparation: cover request validation, error handling, testing and the boundary between the API, retrieval and model inference.
How would you diagnose a production LLM application that has become slow or unreliable?
What to coverThe posting emphasizes debugging, observability and performance monitoring. Suggested preparation: explain what you would measure across the API, retrieval and inference stages, and how those observations would guide a fix.
How would you manage cost, latency and reliability as request volume grows?
What to coverThe posting emphasizes production optimization and prefers experience with high request volumes. Suggested preparation: compare capacity, concurrency and inference-cost trade-offs, and explain how you would verify that an optimization has not reduced answer quality.
What security concerns would you address in an LLM-backed API and RAG system?
What to coverSecurity practices are part of the advertised responsibilities. Suggested preparation: discuss access to retrieved content, protection of sensitive inputs and outputs, and testing for attempts to manipulate instructions through untrusted content.
How would you evaluate a generative AI solution for quality, bias and compliance concerns?
What to coverThe posting explicitly includes evaluation, bias detection and compliance in responsible AI adoption. Suggested preparation: describe representative test cases, review of uneven or harmful outcomes, and how findings would influence release decisions or mitigations.
Behavioural practice
Tell me about a time you debugged a difficult production issue.
What to coverThe posting values debugging and production reliability. Suggested preparation: use a real example to explain the situation, your investigation and actions, and the observed result.
Describe a time you had to balance performance, reliability and cost.
What to coverThese trade-offs are explicit priorities in the role. Suggested preparation: explain the situation, the options and evidence you considered, the choice you made, and its result.
Tell me about a time you explained a technical AI or software decision to others.
What to coverThe posting calls for clear technical communication. Suggested preparation: identify the audience and situation, show how you explained the trade-offs or uncertainty, and describe the resulting decision or understanding.
Describe a time you found a quality, bias or security concern in an AI system.
What to coverThe role includes evaluation, bias detection and security practices. Suggested preparation: use a real situation to explain how you identified the concern, what action you took, and the outcome without overstating what was resolved.
Tell me about a time you moved an AI or backend component toward production use.
What to coverThe posting emphasizes production-grade applications and deployment. Suggested preparation: describe the starting state, your implementation or release actions, and the evidence you used to assess the result.