Technical practice
How would you take a new internal AI system from initial design to production?
What to coverSuggested preparation: Describe how you would define the workflow, choose a pragmatic architecture, and establish production readiness. Explain how the design would change when people across multiple functions depend on the system.
How would you measure and improve retrieval quality?
What to coverSuggested preparation: Discuss representative queries, relevance checks, and how you would diagnose missing or misleading results. Explain the trade-off between improving retrieval and adding safeguards when the available information is insufficient.
How would you enforce permissions in an AI system that retrieves internal information?
What to coverSuggested preparation: Explain where access checks belong across retrieval and response generation, including what happens when a user cannot access a source. Address how you would test that the system does not reveal restricted information.
What security boundaries would you set for an AI system with integrations?
What to coverSuggested preparation: Discuss the data and actions an integration should be allowed to access, and how you would limit an agent’s authority. Explain how you would evaluate risks introduced by untrusted content or external systems.
How would you respond when an AI system gives a confident but wrong answer?
What to coverSuggested preparation: Trace whether the failure came from source data, retrieval, generation, or an integration. Explain how you would reduce recurrence and communicate uncertainty without making the system unusable.
How would you make an internal AI system reliable for daily use at company scale?
What to coverSuggested preparation: Identify the failures worth monitoring and how you would detect a decline in quality or availability. Discuss recovery and fallback choices in light of the impact on teams that depend on the system.
When would you use an agent rather than a simpler LLM workflow?
What to coverSuggested preparation: Compare the value of multi-step tool use with its added complexity, failure modes, and oversight needs. Describe how you would keep integrations and agent actions bounded by the workflow’s requirements.
How would you make a build-versus-buy decision for an internal AI capability?
What to coverSuggested preparation: Compare options against workflow fit, trustworthiness, integration effort, and long-term ownership. Explain what evidence would justify building something custom instead of using a proven option.
What would make you continue or stop a rapid AI proof of concept?
What to coverSuggested preparation: Define a narrow question the proof of concept must answer and evidence that would change the decision. Distinguish a promising demonstration from an approach that can meet production quality, security, and reliability needs.
How would you ship frequently without weakening technical standards?
What to coverSuggested preparation: Explain which trust and production checks you would treat as non-negotiable, and where you would deliberately limit scope. Discuss how you would leave a maintainable system with clear owners after embedding with a team.
Behavioural practice
Tell me about a time you owned a technical outcome in another team’s domain.
What to coverSuggested preparation: Use a real situation, your actions with the domain team, and the result. Explain how you learned their workflow without taking ownership away from them.
Tell me about a time you explained a consequential technical decision to senior leaders.
What to coverSuggested preparation: Describe the situation, how you presented the options and trade-offs, and the resulting decision. Show how you made the implications clear without hiding uncertainty.
Tell me about a proof of concept you stopped or substantially changed.
What to coverSuggested preparation: Give the original goal, the evidence you gathered, the action you took, and the result. Explain how you avoided continuing simply because work had already been invested.
Tell me about an AI or data system failure that changed your engineering approach.
What to coverSuggested preparation: Describe what happened, your specific response, and the result without overstating your role. Focus on what the failure taught you about quality or trust and what you changed afterward.
Tell me about a time you handed a system over to lasting owners.
What to coverSuggested preparation: Explain the situation, how you prepared the receiving team to operate and improve the system, and what happened after handoff. Distinguish a completed delivery from an outcome the team could sustain.