New opportunity

Agent Engineer

Rifa AI · Bengaluru, India

About this role

Rifa AI is seeking an Agent Engineer to own the end-to-end delivery of voice and chat agents for contact centers in regulated industries. You will be responsible for the entire lifecycle of client agents, including procedure implementation, instruction sets, system integration, and rigorous testing. Responsibilities: Own client delivery from pilot to production and continuous iteration. Engineer agent behavior by writing and maintaining versioned, tested instruction sets. Build and maintain evaluation gates using automated tests to ensure compliance and performance. Debug live conversations by analyzing system state and model instructions. Work directly with enterprise clients to manage technical requirements and business outcomes. Qualifications: 2 to 5 years of experience building and running production systems. Proficiency in Python, async programming, and relational databases. Strong written communication skills for creating precise agent instructions and client documentation. Ability to debug complex systems under pressure and work directly with clients. Preferred: Experience with LLM systems (eval frameworks, agent tooling, RAG, structured prompting), conversational AI (dialogue design, IVR, speech interfaces), regulated industries (finance, healthcare, collections), and real-time/telephony systems (WebSockets, streaming audio, Twilio). Tech Stack: Python, FastAPI, PostgreSQL, Temporal, WebSockets, Kubernetes, ArgoCD, SigNoz, DeepEval, Ory, OpenFGA. Working Arrangement: Remote role based in India.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

PythonFastAPIPostgreSQLTemporalWebSocketsLLMsOpenAIKubernetesArgoCDGitOpsSigNozDeepEvalOryOpenFGARAGConversational AIDialogue designIVR systemsChatbotsSpeech interfaces

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

End-to-End Client Delivery
You are responsible for the entire lifecycle of a client's voice or chat agent, from initial pilot to production deployment and ongoing iteration. This requires managing technical requirements, business outcomes, and direct communication with enterprise clients.
Rigorous Evaluation and Compliance
Because agents operate in regulated industries like finance and debt collection, you must build and maintain automated evaluation gates. You are responsible for ensuring agents strictly follow procedures and legal disclosures, treating instruction sets with the same rigor as production code.
System Debugging and Observability
You must be capable of analyzing complex system states to diagnose why an agent behaved a certain way. This involves deep-diving into model instructions, system logs, and conversation traces to identify and fix issues under pressure.
Precise Technical Communication
Strong written skills are essential for crafting precise agent instructions and documenting technical fixes. You must be able to translate vague client feedback into actionable technical requirements and explain complex system behaviors clearly to stakeholders.

Likely challenges to think through

Balancing Flexibility with Strict Compliance
The core difficulty is enabling an AI agent to handle natural, real-time conversations while ensuring it never violates strict regulatory constraints. Prepare by demonstrating how you design systems that enforce hard guardrails rather than relying on probabilistic model behavior.
Debugging Non-Deterministic Systems
Debugging LLM-based agents is inherently difficult because outputs can vary. You should prepare by discussing your methodology for forming and testing hypotheses in non-deterministic environments, specifically how you use evaluation suites to isolate failures.
Direct Client Interaction
As an engineer, you will be expected to own the technical conversation with enterprise clients. Prepare by practicing how you would handle high-pressure feedback, such as explaining a compliance-related error to a non-technical client while maintaining their trust.

Practice your answers

Technical practice

Designing Evaluation Gates for LLMs

What to coverExplain how you would build an automated testing suite to verify that an AI agent adheres to strict compliance procedures. Discuss the trade-offs between using deterministic unit tests versus LLM-based evaluation frameworks like DeepEval to catch hallucinations or policy violations.

Debugging Real-Time Agent Conversations

What to coverDescribe your process for investigating a production incident where an agent provided an incorrect response. Focus on how you would reconstruct the system state, analyze the specific instruction set, and use observability tools like SigNoz to isolate the root cause.

Managing State in Async Python Applications

What to coverGiven the use of FastAPI and WebSockets for real-time voice agents, how do you handle concurrency and state management? Discuss the challenges of maintaining session context across asynchronous calls and how you ensure data consistency in a PostgreSQL database.

Implementing Guardrails in Agent Workflows

What to coverHow would you architect a system to ensure an agent cannot offer payment plans outside of pre-defined client limits? Address how you would enforce these constraints at the instruction level versus the application logic level to ensure they are impossible to bypass.

Optimizing Intent Identification

What to coverIf a voice agent is frequently misinterpreting caller intent, what architectural changes would you propose to improve accuracy? Discuss the role of grounding responses in specific caller inputs rather than relying solely on model assumptions.

GitOps and Deployment Reliability

What to coverExplain the role of ArgoCD and Kubernetes in maintaining high-availability for production AI agents. How do you ensure that updates to agent instructions are safely deployed and rolled back if they negatively impact performance metrics?

Handling Long-Running Workflows

What to coverDiscuss the use of Temporal for managing background workflows in a conversational AI context. How does this approach help in maintaining the state of a long-running interaction compared to traditional request-response patterns?

Structured Prompting for Compliance

What to coverHow do you approach writing and versioning instruction sets to ensure they are precise enough for regulated industries? Suggest how you would structure these instructions to minimize ambiguity and ensure consistent adherence to legal disclosures.

Behavioural practice

Handling Vague Client Requirements

What to coverDescribe a time when you had to translate ambiguous feedback from a non-technical client into a concrete technical requirement. How did you clarify their needs and ensure the final implementation met their business objectives?

Ownership of Production Incidents

What to coverTell me about a time you were on-call for a system you built and encountered a critical failure under pressure. How did you diagnose the issue, communicate with stakeholders, and implement a fix while minimizing downtime?

Collaborating Across Disciplines

What to coverProvide an example of how you have worked with non-engineers, such as auditors or product managers, to refine a complex system. How did you ensure that technical constraints were understood while still incorporating their domain expertise?

Admitting Uncertainty

What to coverDescribe a situation where you were asked a technical question by a client or colleague and did not know the answer. How did you handle the situation while maintaining trust and ensuring you eventually provided the correct information?

Sharpen your knowledge

Topics to prepare

Evaluation-Driven Development for LLMs
The role emphasizes that evaluation methodology is the primary bottleneck for production AI agents. Exercise: Design a test suite that replays historical conversation logs to verify specific compliance constraints (e.g., mandatory disclosures). This is relevant because the company requires automated gates to prevent compliance incidents in regulated industries.
Production System Observability
The role requires debugging live conversations by analyzing system state and model instructions. Exercise: Practice tracing a complex asynchronous workflow in a distributed system to identify where a specific logic branch failed. This is relevant because you will be responsible for diagnosing why an agent deviated from its instructions during live client interactions.

Skills to revise

Asynchronous Python Development
The stack relies on FastAPI and asynchronous programming for real-time interactions. Exercise: Build a small service using FastAPI and async/await that handles concurrent WebSocket connections. This is relevant because the role involves managing real-time voice and chat streams where blocking code would degrade performance.
Structured Prompt Engineering
The role requires writing versioned, tested instruction sets that act as the agent's logic. Exercise: Draft a set of precise, rule-based instructions for a hypothetical agent and create a test case that checks for 'out-of-bounds' behavior. This is relevant because the company treats instructions as code to ensure compliance and prevent hallucinations.
Relational Database Modeling
The role involves managing state and context in PostgreSQL. Exercise: Design a database schema that tracks conversation history, agent state, and compliance metadata for a regulated workflow. This is relevant because you will need to store and query system state to debug agent performance and maintain audit trails.

Your preparation checklist

Demonstrate systematic debugging of complex systems
Prepare a concise walkthrough of a past incident where you identified a root cause in a production system. Focus on your methodology: forming a hypothesis, testing it, and implementing a fix, rather than trial-and-error changes.
Showcase experience with evaluation-driven development
Create a small project or document a past experience using automated tests to gate production releases. Explain how you defined success metrics and used them to verify system behavior, similar to how Rifa AI uses DeepEval.
Prepare examples of translating requirements into precise logic
Draft a sample set of instructions or a procedure document for an automated process. Demonstrate how you break down complex business rules into clear, versioned, and testable steps that minimize ambiguity.
Formulate questions about client-facing technical ownership
Prepare thoughtful questions for the interviewer regarding how they balance client feedback with technical constraints, and how they manage the transition from pilot to production in regulated environments.
Review your proficiency in Python and async programming
Be ready to discuss your experience with asynchronous Python patterns and relational database interactions, as these are core to the Rifa AI stack for managing real-time voice and chat agents.

Availability last checked: 18 Oct 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…