New opportunity

Lead Software Engineer (Python, GenAI, LLM)

EPAM Systems · Hyderabad, India

About this role

EPAM Systems seeks a Lead Software Engineer to design and build production-grade Generative AI solutions for enterprise use cases, with an emphasis on agentic workflows and multi-agent systems. The role combines software engineering with hands-on work in LLMs, RAG architectures and AI agent frameworks. The position is listed for EPAM India offices in Coimbatore, Chennai, Bangalore, Hyderabad, Pune and Gurgaon.

Responsibilities

Architect and develop scalable Generative AI applications and AI agents. Build the backend services, APIs, microservices and data pipelines that support AI-driven products.
Implement RAG and Agentic RAG, tool or function calling, planning and reflection loops, and human-in-the-loop workflows. Integrate LLMs with enterprise systems, third-party APIs, vector databases and knowledge repositories.
Optimize prompts, system instructions and agent workflows for reliability, accuracy and user experience. Monitor and evaluate model and agent performance using observability tools, metrics and user feedback.
Collaborate with Product, Engineering, Data and Design teams; document architecture, design decisions and reusable patterns; share knowledge and keep current with emerging AI technologies.

Requirements

7 to 12 years of relevant professional experience, including hands-on development with Generative AI and LLM technologies. Strong Python proficiency and experience delivering production-ready applications are required.
Experience with at least two agentic AI frameworks, such as LangChain, LangGraph, Google ADK, CrewAI, AutoGen or Microsoft Copilot extensibility. Experience with a cloud AI platform such as Azure OpenAI, AWS Bedrock or Google Vertex AI/ADK is required.
Strong backend skills spanning REST or gRPC APIs, asynchronous programming, Docker, and frameworks such as FastAPI or Flask. Practical RAG experience with vector databases such as Pinecone, Weaviate, ChromaDB or Qdrant is required.
Understanding of OpenAI GPT models, Anthropic Claude, Google Gemini and open-source LLM alternatives; expertise in prompt engineering, LLM orchestration, structured outputs, guardrails, ReAct patterns and evaluation techniques. Strong problem-solving, system design, architectural decision-making and communication skills are required for collaboration across global teams.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

PythonGenerative AILarge Language ModelsAI agentsMulti-agent systemsLangChainLangGraphGoogle ADKCrewAIAutoGenMicrosoft Copilot StudioAzure OpenAIAWS BedrockGoogle Vertex AIRAGAgentic RAGPineconeWeaviateChromaDBQdrantPrompt engineeringLLM orchestrationTool callingFunction callingReActHuman-in-the-loop workflowsLLM evaluationObservabilityGuardrailsStructured outputsystems design

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Architect production-grade agentic systems
The role centers on scalable enterprise GenAI applications, including multi-agent workflows, tool calling, planning and reflection. Prepare to explain architectural decisions and when an agentic approach is warranted.
Deliver the Python backend
The posting calls for production-ready Python applications and the APIs, microservices and data pipelines behind AI products. Prepare to discuss how you would design and operate these components, including asynchronous services.
Build retrieval and enterprise integrations
RAG and Agentic RAG, vector databases, knowledge repositories and enterprise-system integrations are explicit duties. Prepare to explain retrieval design and how tools and external data fit into an agent workflow.
Improve reliability through evaluation
The role includes optimizing prompts and workflows, applying structured outputs and guardrails, and using observability, metrics and user feedback to improve performance. Prepare to describe how you would measure accuracy and diagnose failures.
Lead through design decisions and collaboration
The posting emphasizes system design, architectural decisions, documentation, reusable patterns and work with Product, Engineering, Data and Design across global teams. Prepare to communicate the rationale and trade-offs behind a design.

Likely challenges to think through

Agent autonomy versus human control
A likely design challenge, not a stated EPAM problem, is deciding when tool use and planning can proceed automatically versus when a human-in-the-loop step is needed. Prepare a workflow example showing those boundaries and how you would assess its reliability.
Retrieval quality versus workflow complexity
Agentic RAG may offer more flexible retrieval than a simpler RAG flow, but adds steps that can make failures harder to trace. Prepare to justify when each approach is appropriate and what evaluation signals would reveal weak retrieval or answers.
Framework capability versus maintainability
The role spans multiple agent frameworks and cloud AI platforms. A plausible challenge is choosing useful orchestration features without making the system difficult to change. Prepare to explain framework-selection criteria and how you would document the decision.

Practice your answers

Technical practice

How would you architect a production-grade enterprise GenAI application?

What to coverRole focus: The posting calls for scalable applications, backend services, APIs, microservices and data pipelines. Suggested preparation: Explain component boundaries, data flow, reliability considerations and the trade-offs behind your architecture.

How would you choose and orchestrate agents for a multi-agent workflow?

What to coverRole focus: The posting names multi-agent systems and hands-on experience with at least two agentic frameworks. Suggested preparation: Compare frameworks you have used and explain how you would manage agent responsibilities, state and coordination without adding unnecessary complexity.

How would you design and troubleshoot a RAG pipeline?

What to coverRole focus: Practical RAG experience with vector databases and knowledge repositories is specified. Suggested preparation: Walk through indexing, retrieval and answer generation, then explain how you would diagnose irrelevant retrieval or unsupported answers.

When would Agentic RAG be preferable to a simpler RAG workflow?

What to coverRole focus: The posting explicitly includes Agentic RAG, planning and reflection loops. Suggested preparation: Explain when iterative retrieval or planning adds value, and weigh that against latency, complexity and opportunities for error.

How would you control tool calling and human-in-the-loop actions?

What to coverRole focus: Tool or function calling and human-in-the-loop workflows are stated responsibilities. Suggested preparation: Describe how an agent would choose and validate a tool call, handle failures and identify actions that should require human review.

How would you build a Python API for an LLM-backed service?

What to coverRole focus: The requirements mention Python, REST or gRPC, asynchronous programming, Docker, and FastAPI or Flask. Suggested preparation: Discuss API boundaries, asynchronous work, error handling and deployment trade-offs for a production-ready service.

How would you integrate an LLM application with enterprise systems and a cloud AI platform?

What to coverRole focus: The posting calls for enterprise and third-party integrations plus experience with Azure OpenAI, AWS Bedrock or Google Vertex AI/ADK. Suggested preparation: Explain interface design, data handling and how you would manage dependency failures across those integrations.

How would you evaluate which LLM to use for an enterprise use case?

What to coverRole focus: The requirements name GPT, Claude, Gemini and open-source alternatives. Suggested preparation: Describe a task-specific comparison of output quality and operational trade-offs, rather than assuming one model is best for every use case.

How would you improve prompt reliability while enforcing structured outputs and guardrails?

What to coverRole focus: Prompt engineering, system instructions, structured outputs and guardrails are all specified. Suggested preparation: Explain how you would test prompt changes, validate output structure and handle responses that fail the intended constraints.

How would you detect and improve declining agent performance?

What to coverRole focus: The role includes evaluation, observability, metrics and user feedback. Suggested preparation: Describe what you would measure, how you would trace a failure through the workflow and how you would check whether a change actually improves results.

Behavioural practice

Describe a time you aligned Product, Engineering, Data and Design on a technical approach.

What to coverRole focus: The posting calls for collaboration across these functions. Suggested preparation: Use a real situation, your actions to resolve competing needs and the resulting decision or outcome.

Tell me about a consequential architecture decision you led.

What to coverRole focus: System design, architectural decision-making and documented design decisions are part of the role. Suggested preparation: Give the situation, alternatives you considered, your action and the result, including a trade-off you accepted.

Describe a time you investigated an unreliable AI application or workflow.

What to coverRole focus: The posting emphasizes reliability, monitoring and continuous improvement of model and agent performance. Suggested preparation: Use a real situation, explain how you isolated the cause and state what changed as a result.

Tell me about a time you shared a reusable engineering pattern across teams.

What to coverRole focus: Documenting architectures and reusable patterns while sharing knowledge is a stated responsibility. Suggested preparation: Describe the situation, what you documented or taught, and the result for the teams involved.

Describe a time you assessed an emerging AI technology before adopting it.

What to coverRole focus: The posting asks for awareness of emerging AI technologies and contributions of new ideas. Suggested preparation: Give the situation, how you evaluated the technology against an actual need, and the outcome, including a decision not to adopt it if applicable.

Sharpen your knowledge

Topics to prepare

Agentic and multi-agent architecture
Revision exercise: Sketch when a single agent, multiple agents, tool calling, a ReAct loop, planning and reflection, or human review would be appropriate for an enterprise request. Explain failure and handoff points. The posting explicitly emphasizes these patterns.
RAG and Agentic RAG design
Revision exercise: Diagram retrieval from a knowledge repository through a vector database to a grounded answer, then explain what changes when an agent selects retrieval steps. The role calls for practical RAG and Agentic RAG work.
LLM selection and output reliability
Revision exercise: Compare OpenAI GPT, Claude, Gemini and open-source alternatives for a hypothetical use case; explain how prompts, structured outputs and guardrails affect reliability. The posting requests understanding of these models and expertise in these techniques.
Evaluation and observability
Revision exercise: Propose metrics and a user-feedback review for an agent that calls tools; identify how you would investigate inaccurate answers or failed calls. Monitoring and improving model and agent performance are stated responsibilities.
Production AI system design
Revision exercise: Draw an architecture connecting APIs, microservices, data pipelines, an LLM platform and enterprise systems; explain scalability decisions and document key trade-offs. The lead role calls for production-ready applications, architectural decisions and cross-team communication.

Skills to revise

Build a Python AI backend
Practice exercise: Implement a small REST or gRPC service using FastAPI or Flask, add an asynchronous LLM-facing operation, and package it with Docker. These are explicitly named backend requirements; the exercise is a preparation suggestion, not an assigned project.
Orchestrate agents in two frameworks
Practice exercise: Implement the same tool-calling workflow in two listed frameworks, such as LangGraph and CrewAI, and compare control flow and human-review points. The posting requires hands-on experience with at least two agentic AI frameworks.
Implement a retrieval pipeline
Practice exercise: Connect a small document collection to one listed vector database, retrieve relevant passages and use them in an LLM response; inspect an irrelevant-retrieval case. The posting requires practical RAG experience with vector databases.
Integrate a cloud AI platform
Practice exercise: Walk through or prototype an LLM call via Azure OpenAI, AWS Bedrock or Google Vertex AI/ADK, then explain how a backend service would handle errors and consume the result. Experience with a cloud AI platform is required; this prototype is a preparation suggestion.
Test and improve an agent workflow
Practice exercise: Create a small set of requests, record outputs and tool-call failures, then revise prompts, structured-output handling or guardrails and compare results. The posting calls for prompt optimization, evaluation and performance monitoring.

Your preparation checklist

Build a requirements evidence sheet
The posting specifies 7–12 years of relevant experience, production Python and GenAI work, at least two agentic frameworks, a cloud AI platform, and practical RAG experience. Prepare a one-page timeline with specific examples you can substantiate; mark any areas you cannot evidence.
Rehearse a production GenAI project walkthrough
Prepare a concise account of a relevant project, if available: the business problem, your architectural decisions, Python services and integrations, production constraints, and what changed after launch. Distinguish your own contributions from the team's.
Practice an enterprise agent design exercise
Sketch a proposed multi-agent application with tool calling, planning or reflection, and human approval points. Explain when that complexity is justified, how agents access enterprise systems safely, and how you would document design decisions.
Prepare a framework comparison
The role asks for hands-on experience with at least two agentic frameworks. Bring a side-by-side comparison grounded in frameworks you have used, covering orchestration, state or workflow control, testing, and limitations; avoid claiming hands-on use where you have none.
Draft a RAG evaluation plan
Diagram ingestion, retrieval, vector storage, answer generation, and an Agentic RAG extension. Propose tests for retrieval quality, answer accuracy, guardrails, and failure cases, plus metrics and user feedback for ongoing improvement.
Review backend and model trade-offs
Prepare a short technical brief on Python API design, asynchronous calls, Docker deployment, and a cloud AI platform you know. Include how you would choose among model options and handle structured outputs, latency, and reliability; these are discussion prompts, not stated project specifications.
Write questions for the employer
Bring focused questions about the enterprise use cases, existing agent and RAG architecture, evaluation practices, human-in-the-loop expectations, and collaboration across Product, Engineering, Data, and Design. These clarify responsibilities described in the posting without assuming a particular implementation.

Availability last checked: 5 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…