Technical practice
How would you design instruction fine-tuning for an LLM serving a specific business context?
What to coverSuggested preparation: Explain how you would define the target tasks, prepare representative instructions and responses, and separate training from evaluation data. Discuss how you would check whether the tuned model improves relevance without degrading performance on other important tasks.
How would you adapt an LLM to a specialized domain?
What to coverSuggested preparation: Discuss how you would identify domain-specific language and tasks, assess the quality of available data, and evaluate performance in that context. Explain the trade-off between specialization and retaining broader model capabilities.
When would you use prompt engineering rather than fine-tuning?
What to coverSuggested preparation: Compare the approaches in terms of the change needed, available examples, iteration effort, and evaluation results. Explain how you would test whether a prompt change is sufficient before changing the model.
How would you design an Agentic AI solution to automate part of an enterprise process?
What to coverSuggested preparation: Describe the agent's objective, permitted actions, decision boundaries, and points where a person should review its work. Discuss how you would handle failed steps and assess whether the automation improves the process.
How would you integrate a Generative AI solution into an existing enterprise workflow?
What to coverSuggested preparation: Start with the user's task and where an AI-generated insight or action would fit into that task. Explain how you would design for dependable use, clear handoffs, and a measurable improvement to the workflow.
What evaluation framework would you use for an LLM solution?
What to coverSuggested preparation: Define measures tied to the intended task, use representative test cases, and compare results against a baseline. Include checks for relevance, failure cases, and responsible use rather than relying on a single aggregate score.
How would you investigate an LLM solution whose answers are relevant in testing but unreliable in use?
What to coverSuggested preparation: Compare failing examples with the evaluation set and trace whether the issue arises from inputs, prompts, model adaptation, or the surrounding workflow. Explain how you would test a proposed fix and watch for regressions.
What trade-offs would you consider when scaling an enterprise-grade AI solution?
What to coverSuggested preparation: Discuss how growing usage could affect response time, cost, and consistency of results. Explain how you would preserve evaluation and responsible-use checks as the solution scales.
How would you build responsible use into an LLM or agentic solution?
What to coverSuggested preparation: Identify potential harms from inaccurate outputs or inappropriate automated actions, then describe boundaries, review points, and ways to detect problems. Relate each safeguard to the solution's intended business use.
How would you show that an AI solution delivers tangible business value?
What to coverSuggested preparation: Translate the business objective into a baseline and a small set of outcome measures, such as task quality or process efficiency where relevant. Explain how you would distinguish a model improvement from an improvement that users experience in the workflow.
Behavioural practice
Tell me about a time you led the design and delivery of an AI solution.
What to coverSuggested preparation: Give the situation and your specific responsibility, then describe the design decisions and actions you led. Close with the result and the evidence used to judge it.
Describe a time you had to align an AI approach with a business objective.
What to coverSuggested preparation: Explain the original objective, any mismatch between the proposed AI capability and the need, and what you changed. State the outcome in terms of the objective rather than model performance alone.
Tell me about a time an AI solution faced a scalability or delivery constraint.
What to coverSuggested preparation: Set out the constraint and your role, then explain the trade-off you made and how you tested it. Describe the delivery outcome, including any limitation that remained.
Describe a time you addressed a responsible-use concern in an AI project.
What to coverSuggested preparation: Identify the concern and its potential impact, then explain the actions you took and why. Describe the result and what you learned about balancing usefulness with appropriate safeguards.