New opportunity

Engineering Manager - AI

SaaS Labs · Bengaluru, India

About this role

Role Purpose: SaaS Labs is seeking an Engineering Manager, AI to lead the AI engineering function for JustCall. This is a player-coach role focused on building production-grade AI systems, ensuring reliability, managing latency, and scaling the engineering team. Responsibilities: Lead engineering across AI Voice Agents (AIVA), AI Coaching, Conversation Intelligence, and AI Workflows. Maintain hands-on involvement in architecture, coding, debugging, and production reliability. Build scalable voice AI systems including orchestration, memory management, and eval pipelines. Define AI quality frameworks, including golden datasets, regression detection, and operational metrics. Drive technical decisions regarding model selection, prompting, fine-tuning, RAG, and infrastructure. Hire, mentor, and grow a high-performing AI engineering team in India. Partner with Product and Design to deliver reliable, production-ready AI experiences. Shape the long-term AI roadmap for multilingual support and omnichannel automation. Requirements: 8+ years of experience in distributed systems, AI/ML products, or voice infrastructure, including 3+ years in a leadership role. Proven track record of shipping production-grade AI products at scale. Expertise in building agentic systems with tool calling, memory management, and graceful degradation. Deep knowledge of eval systems, automated testing, and operational monitoring for LLM applications. Strong systems thinking regarding latency, reliability, and production debugging. Player-coach mindset with strong product intuition. Preferred Edges: Experience with voice AI platforms such as Pipecat, Retell, Vapi, LiveKit, or ElevenLabs. Familiarity with voice AI challenges like interruption handling, semantic turn detection, and accented speech recognition. Experience scaling engineering teams in India.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Distributed SystemsAI/ML Product DevelopmentVoice InfrastructureAgentic SystemsLLM OrchestrationEval PipelinesAutomated TestingRegression DetectionLatency OptimizationObservabilityPrompt EngineeringFine-tuningRAGMemory SystemsModel RoutingPipecatRetellVapiLiveKitElevenLabs

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Production-Grade AI Reliability
The primary mandate is to move AI features beyond the prototype stage. You must prioritize building robust eval pipelines, regression detection, and operational metrics to ensure voice agents and AI workflows function reliably under real-world conditions, preventing latency spikes and hallucinations.
Hands-on Technical Leadership
This is a player-coach role requiring active involvement in architecture, coding, and debugging. You are expected to maintain deep technical expertise in distributed systems and LLM orchestration while simultaneously mentoring the engineering team in India to raise the overall technical bar.
Scalable Voice AI Infrastructure
You must lead the development of scalable voice systems, focusing on real-time orchestration, memory management, and tool calling. This involves making critical architectural decisions regarding model selection, RAG, and fine-tuning to support high-volume, omnichannel customer interactions.

Likely challenges to think through

Balancing Hands-on Coding with Team Management
The role demands high-velocity shipping while managing a growing team. The challenge is to avoid delegating the most difficult technical problems while still providing the mentorship necessary to scale the team, requiring a disciplined approach to time management and technical delegation.
Managing Real-Time Voice Latency
Voice AI requires extremely low latency to feel natural. You will face the difficulty of optimizing complex agentic systems—including interruption handling and semantic turn detection—without sacrificing the reliability or quality of the AI responses.
Defining Quality in Ambiguous AI Environments
Because AI outputs can be non-deterministic, creating effective 'golden datasets' and automated testing frameworks is difficult. You must prepare to defend your methodology for measuring success and quality, ensuring that your eval systems actually catch issues before they reach enterprise customers.

Practice your answers

Technical practice

Architecting Low-Latency Voice AI Systems

What to coverExplain your approach to minimizing end-to-end latency in real-time voice agents. Discuss trade-offs between model size, inference speed, and network overhead, and how you handle interruption handling and semantic turn detection in production.

Designing AI Eval Pipelines

What to coverDescribe how you build and maintain golden datasets for regression testing. Explain how you integrate automated evaluation into the CI/CD pipeline to catch hallucinations or quality regressions before they reach customers.

Agentic System Orchestration

What to coverDiscuss the challenges of building agentic systems that utilize tool calling and memory management. Address how you implement graceful degradation when models fail or tools return unexpected results.

Production Observability for LLMs

What to coverWhat metrics do you prioritize for monitoring LLM-based applications in production? Explain how you track operational metrics beyond standard latency and error rates to ensure reliability and quality.

RAG vs. Fine-Tuning Trade-offs

What to coverWhen building AI workflows, how do you decide between RAG and fine-tuning for specific use cases? Discuss the impact of each on system architecture, maintenance, and long-term scalability.

Handling Accented Speech and Multilingual Support

What to coverWhat architectural considerations are necessary to support diverse accents and multilingual conversations in voice AI? Suggest strategies for model selection and data preparation to ensure consistent performance across global user bases.

Debugging Production AI Failures

What to coverDescribe your process for debugging a non-deterministic failure in a production AI agent. Explain how you use logs, traces, and replay mechanisms to isolate whether the issue lies in the prompt, the model, or the orchestration layer.

Model Routing and Infrastructure

What to coverHow do you approach model routing to balance cost, performance, and reliability? Explain how you manage infrastructure to support multiple models while maintaining consistent service levels.

Behavioural practice

Balancing Hands-on Coding with Leadership

What to coverProvide an example of how you have maintained a 'player-coach' role while managing a team. Describe a situation where you had to step in to solve a critical technical problem while simultaneously mentoring your team.

Driving Technical Alignment with Product

What to coverDescribe a time you had to push back on a product requirement due to technical constraints or reliability concerns. How did you communicate the trade-offs and reach a solution that satisfied both product goals and engineering standards?

Scaling Engineering Teams in High-Growth Environments

What to coverShare your experience in hiring and scaling an engineering team. Focus on how you maintained a high-performing, execution-focused culture while onboarding new members and managing rapid growth.

Managing Ambiguity in AI Roadmaps

What to coverGive an example of how you shaped a long-term technical roadmap in an environment with high ambiguity. How did you prioritize initiatives to ensure the team delivered immediate value while building for future scalability?

Sharpen your knowledge

Topics to prepare

Production-Grade AI Reliability
Focus on the transition from prototype to production. Exercise: Design a system architecture that incorporates automated regression testing, golden datasets for evaluation, and observability metrics to detect hallucinations and latency spikes in real-time. This is critical for the role's focus on ensuring AI systems do not fail in production.
Real-Time Voice AI Orchestration
Study the complexities of low-latency voice interactions. Exercise: Map out the technical requirements for handling interruption handling, semantic turn detection, and context management in a voice agent. This is directly relevant to the role's responsibility for building scalable voice AI systems using platforms like Pipecat or LiveKit.

Skills to revise

Agentic System Design
Practice building systems that utilize tool calling and memory management. Exercise: Create a workflow diagram for an agent that manages state across multiple turns, including fallback mechanisms for when a model fails to execute a tool correctly. This aligns with the requirement for building robust agentic systems.
LLM Evaluation Frameworks
Develop expertise in measuring AI performance. Exercise: Define a set of operational metrics and a testing pipeline that uses a golden dataset to score model responses for accuracy and latency. This is essential for the role's mandate to define AI quality frameworks.
Distributed Systems Debugging
Hone skills in identifying bottlenecks in complex architectures. Exercise: Simulate a production incident involving high latency in a RAG-based pipeline and document the step-by-step debugging process to isolate the issue between the model, the vector database, and the orchestration layer.

Your preparation checklist

Prepare a production-grade AI system walkthrough
Develop a concise case study of a real-time AI system you have shipped. Focus on how you managed latency, memory, and orchestration. Be prepared to explain your specific technical choices regarding model selection, RAG, or fine-tuning, and how you handled edge cases like interruptions or system failures.
Define your AI quality and eval framework
Draft a document outlining your approach to building eval pipelines for LLM applications. Include how you define golden datasets, implement regression detection, and monitor operational metrics to ensure reliability before features reach customers.
Articulate your player-coach leadership philosophy
Prepare examples of how you balance hands-on technical contributions—such as debugging production code or architecting infrastructure—with the responsibilities of hiring, mentoring, and scaling an engineering team in the Indian market.
Formulate strategic questions for the leadership team
Prepare thoughtful questions regarding the current AI roadmap, such as how the team balances speed of experimentation with long-term reliability, or how they currently handle the specific challenges of voice AI like semantic turn detection and accented speech recognition.

Availability has not yet been verified. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…