New opportunity

AI Architect

Guidewire · Bangalore, India

About this role

Guidewire seeks an AI Architect for its Professional Services AI Engineering team in Bangalore. The role works across AI initiatives to blueprint production-ready features and establish reusable architecture patterns for engineering Strike Teams. It focuses on prompt and LLM harness engineering, RAG, evaluation, observability and model selection, including deciding when agentic workflows, direct LLM calls, classical NLP/NLU, lightweight machine learning or fine-tuning are appropriate.

Responsibilities

Produce actionable AI feature blueprints covering architecture, data needs, model strategy, evaluation and delivery risks. Document model and architecture trade-offs using evaluation results and analysis.
Define reusable harness patterns for agent loops, tool/function calling, context construction, memory, structured outputs, retries, fallbacks and failure handling. Establish standards for prompt design, versioning, review, experimentation and regression testing.
Design evaluation frameworks using golden datasets, regression suites, error analysis, human review and LLM-as-judge approaches. Guide RAG patterns spanning document parsing, chunking, embeddings, retrieval, reranking, prompt assembly and grounded responses.
Mentor P2/P3 AI engineers, review designs, create examples and help engineers explain technical decisions. Collaborate with Product, the AI Center of Excellence, Professional Services, customer teams and systems integrator partners. Success includes consistent adoption of patterns and features that meet project targets for accuracy, groundedness, reliability, latency and cost.

Required qualifications

A bachelor's or master's degree in Computer Science, Data Science, Statistics, Machine Learning, Engineering or a related technical field, plus 8+ years of experience in AI, machine learning, data science, software engineering or solution architecture. Candidates must translate ambiguous goals into practical AI designs; understand LLMs, prompts, embeddings, retrieval, agentic workflows, structured outputs and model failure modes; and judge when non-generative methods are preferable. Strong Python/Java skills, production-code review ability, LLM harness and prompt-engineering experience, working knowledge of RAG and evaluation, familiarity with Git, testing, CI/CD and cloud delivery, and communication and mentorship skills are required.

Preferred

Production agentic systems; structured evaluation harnesses or feedback loops; hands-on AWS experience, including Bedrock, SageMaker or OpenSearch; fine-tuning, distillation or domain adaptation; Guidewire products or insurance; and experience with document-heavy or regulated insurance, finance or professional-services datasets. The posting lists the position as onsite and full-time.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

AI solution architectureLarge language modelsModel selectionPrompt engineeringLLM harness designRetrieval-augmented generationAgentic workflowsTool callingStructured outputsEmbeddingsVector searchRetrieval evaluationDocument parsingRerankingLLM evaluationGolden datasetsRegression testingLLM-as-judgeError analysisNLPNLUMachine learningModel fine-tuningPythonJavaGitCode reviewCI/CDMentorshipAWS Bedrock Аmazon SageMaker

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Actionable AI feature blueprints
Translate ambiguous goals into architecture, data needs, model strategy, evaluation plans and delivery risks that Strike Teams can implement.
Evidence-backed model selection
Use evaluation results and documented trade-offs to choose among agentic workflows, direct LLM calls, RAG, classical NLP/NLU, rules, lightweight ML and fine-tuning. The role also calls for selecting models for individual LLM calls.
Reusable harness and prompt patterns
Define patterns for tool calling, context, memory, structured outputs, retries and failure handling; set standards for prompt versioning, review, experimentation and regression testing.
Evaluation and grounded RAG
Design golden datasets, regression suites, error analysis, human review and LLM-as-judge approaches. Guide document parsing through retrieval, reranking and response assembly, with attention to accuracy, groundedness, reliability, latency and cost.
Engineering adoption and technical communication
Review designs, coach P2/P3 engineers and create examples they can implement and explain. Work across Product, the AI Center of Excellence, Professional Services, customer teams and systems integrator partners.

Likely challenges to think through

Choosing the least complex effective approach
Likely preparation challenge, not a reported problem: agentic or fine-tuned solutions may add complexity and cost where a direct call or non-generative method suffices. Prepare a decision framework that compares approaches against evaluation results and project targets.
Measuring quality without hiding failure modes
Likely preparation challenge: aggregate scores or LLM-as-judge results may miss important errors, while human review takes effort. Prepare to explain how golden sets, error taxonomies, regression tests and human review would complement one another.
Making shared patterns useful across teams
Likely preparation challenge: strict standards can improve reliability but may not fit every feature or delivery context. Prepare an example of a reusable pattern with clear defaults, documented exceptions and evidence engineers can use to explain a different choice.

Practice your answers

Technical practice

How would you turn an ambiguous AI feature request into a blueprint a Strike Team can implement?

What to coverSuggested preparation: Cover the architecture, data needs, model strategy, evaluation plan and delivery risks named in the role. Explain how you would make assumptions and trade-offs explicit before implementation.

When would you choose a direct LLM call, RAG, an agentic workflow or a non-generative approach?

What to coverSuggested preparation: Compare these options against the task, available data, failure modes and project targets for accuracy, reliability, latency and cost. Include when rules, classical NLP/NLU, lightweight ML or structured extraction would be preferable.

How would you select the LLM model for each call in an AI feature?

What to coverSuggested preparation: Describe task-specific evaluations rather than relying on a single model choice for the whole feature. Explain how you would document quality, latency and cost trade-offs and revisit the choice as results change.

What reusable harness pattern would you define for tool calling and agent loops?

What to coverSuggested preparation: Address context construction, memory, structured outputs, retries, fallbacks and failure handling, all identified in the posting. Explain where you would constrain an agent loop and how the pattern would remain usable across features.

How would you treat prompts as versioned engineering assets?

What to coverSuggested preparation: Walk through prompt design, review, controlled experimentation and regression testing. Explain how a change would be compared with a baseline before teams adopt it.

How would you build an evaluation framework for an AI feature?

What to coverSuggested preparation: Discuss golden datasets, regression suites, error taxonomies, human review and appropriate uses and limitations of LLM-as-judge. Show how evaluation results would guide architecture decisions against the feature’s stated targets.

How would you design and assess a document-based RAG pipeline?

What to coverSuggested preparation: Cover parsing, chunking, embeddings, retrieval, reranking, prompt assembly and grounded responses. Explain how you would distinguish retrieval errors from generation errors when assessing quality.

How would you investigate an AI feature that produces ungrounded answers or becomes unreliable?

What to coverSuggested preparation: Trace the problem through retrieved context, prompt or harness behavior, model output and failure handling. Propose evaluation and observability signals that would help isolate regressions without assuming a particular monitoring tool.

What would you look for when reviewing production AI code and its delivery pipeline?

What to coverSuggested preparation: Discuss Python or Java code quality, tests for harness and prompt changes, code review, Git and CI/CD practices referenced in the posting. Explain how you would check failure paths and protect reusable patterns from regressions.

How would document-heavy or regulated data affect an AI design?

What to coverSuggested preparation: This is an optional scenario drawn from the posting’s preferred experience, not a stated project. Discuss how data constraints might influence parsing, retrieval, grounding and human review, while identifying assumptions you would need to verify.

Behavioural practice

Tell me about a time you converted an ambiguous goal into an actionable technical design.

What to coverSuggested preparation: Use a real situation, your actions to clarify needs and document trade-offs, and the resulting decision or outcome. Focus on what made the design implementable for others.

Describe a time evaluation evidence changed your preferred technical approach.

What to coverSuggested preparation: Give the situation, the evidence you gathered, the action you took and the result. Explain how you communicated the trade-off rather than presenting the change as intuition alone.

Tell me about a time you helped less-experienced engineers adopt a reusable pattern.

What to coverSuggested preparation: Use situation, action and result to show how you coached, reviewed designs or created examples without relying on reporting authority. Explain how you established whether engineers could implement and explain the pattern.

Describe a cross-functional disagreement about an AI design decision.

What to coverSuggested preparation: Identify the situation and stakeholders, how you explained options and handled their concerns, and the result. Keep the example focused on a technical trade-off rather than implying any particular Guidewire process.

Tell me about a time a production issue or regression challenged an AI feature’s reliability.

What to coverSuggested preparation: Describe the situation, your actions to isolate the failure and coordinate a response, and the measurable or observable result. If relevant, explain what you changed in testing, evaluation or a reusable pattern afterward.

Sharpen your knowledge

Topics to prepare

AI architecture and method selection
The posting calls for actionable feature blueprints and data-informed choices among direct LLM calls, RAG, agents, classical NLP/NLU, rules, lightweight ML and fine-tuning. For revision, take a hypothetical ambiguous feature request and explain which approach you would choose, what data it needs and why simpler alternatives might be better.
LLM harness failure modes
The role defines reusable patterns for agent loops, tool calls, context, memory, structured outputs, retries and fallbacks. For revision, trace a hypothetical request through those components and identify where invalid outputs, failed tools or repeated calls should be handled.
RAG and grounded responses
The posting specifies document parsing, chunking, embeddings, retrieval, reranking and prompt assembly. For revision, sketch how each stage affects a response grounded in documents, then explain how you would investigate a relevant document that was not retrieved.
Evaluation and architecture trade-offs
The role uses golden datasets, regression suites, error analysis, human review and LLM-as-judge approaches to support decisions. For revision, define a small hypothetical golden set and explain how its results would inform accuracy, groundedness, reliability, latency and cost trade-offs.
Prompts as engineering assets
The posting calls for prompt design, versioning, review, experimentation and regression testing. For revision, compare two prompt revisions against the same test cases and state what evidence would justify adopting one.

Skills to revise

Write an actionable feature blueprint
Practice drafting a one-page blueprint for a hypothetical AI feature: architecture, data needs, model strategy, evaluation plan and delivery risks. This mirrors the blueprints the role creates for Strike Teams; the example feature is a preparation exercise, not a stated Guidewire project.
Implement and test a harness slice
In Python or Java, practice a small request flow with structured output validation, a tool-call boundary, retry limits and fallback behavior. These are stated harness and production-code skills; the sample implementation is a suggested way to revise them.
Run a RAG error investigation
Using a small hypothetical document set, inspect parsing and chunks, compare retrieved passages with expected evidence, and record whether an error arose in retrieval or response generation. The posting requires working knowledge of RAG and retrieval evaluation.
Build a regression review
Create a small golden set, label failure categories, compare two output versions and flag cases for human review. This practices the posting's evaluation, error-analysis and regression-suite responsibilities without assuming a particular evaluation tool.
Review production-oriented code
Practice reviewing a Python or Java change for failure handling, tests and readable trade-off notes; outline how the change would be reviewed in Git and checked through CI/CD. The posting explicitly asks for production-code review ability and familiarity with those engineering practices.
Explain and coach an architecture decision
Rehearse a brief design review that states an approach, evidence, rejected alternatives and risks, then turn the feedback into a reusable example for an engineer. This reflects the role's design reviews, mentorship of P2/P3 engineers and communication with partner teams.

Your preparation checklist

Prepare a project walkthrough
If you have a relevant project, prepare a concise account of the goal, architecture, your decisions, delivery risks and measured results. The role calls for production-ready AI designs backed by evidence; distinguish your contribution from the team’s.
Draft an AI feature blueprint
As an interview exercise, turn an ambiguous, hypothetical document-heavy request into a one-page design covering data needs, architecture, model strategy, evaluation and delivery risks. Label it as practice, not a Guidewire project.
Build a model-selection decision table
Compare direct LLM calls, RAG, agentic workflows, classical NLP/NLU, rules, lightweight ML and fine-tuning for the practice design. Record the evidence you would seek and the accuracy, latency and cost trade-offs.
Sketch a reusable harness
Prepare a diagram or pseudocode showing context construction, tool calls, structured outputs, retries, fallbacks and failure handling. Include how prompts would be versioned, reviewed and regression-tested.
Trace a grounded RAG response
Walk through parsing, chunking, embeddings, retrieval, reranking and prompt assembly for sample documents. Note where grounding could fail and how you would test retrieval and responses.
Create an evaluation plan
Specify a small proposed golden set, regression suite, error taxonomy and human-review loop. Explain when LLM-as-judge would help, how you would check its judgments, and how results would inform architecture choices.
Gather engineering and mentorship evidence
Select examples, if available, of Python or Java production-code review, testing or CI/CD, and coaching engineers without direct reporting authority. Prepare to explain one technical decision to both an engineer and a customer-facing stakeholder.
Write questions for the team
Ask how Strike Teams use shared architecture patterns, who sets project bars for accuracy, groundedness, reliability, latency and cost, and how Product, Professional Services, the AI CoE and partners resolve trade-offs.

Availability last checked: 4 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…