New opportunity

Prompt Engineer, Multi-Agent Systems

Fello · India

About this role

Fello is a VC-backed B2B SaaS company building an agentic real estate platform. We are seeking a Prompt Engineer to design and maintain our multi-agent systems. You will own the reliability and performance of agents that interpret, decide, and execute tasks in production. Responsibilities: Design prompts and agent contracts to ensure coherent end-to-end system behavior. Build and maintain an evaluation layer, including datasets, regression suites, and scoring rubrics. Make systems observable through tracing and versioning. Debug production issues by isolating root causes and implementing fixes. Work across multiple LLM models to optimize for reasoning, speed, and cost. Manage dynamic context assembly at runtime. Partner with product and engineering teams to translate feedback into reproducible test cases and metrics. Requirements: Hands-on experience building with LLMs in production environments. Proven experience in evaluation, including building datasets and running experiments. Fluency with LLM observability tools. Experience with multi-agent architectures, orchestration, and function calling. Experience designing or debugging voice agents, including managing latency and turn-taking. Strong systems thinking and writing skills. Nice to have: Experience with open-weight models, regulated domains, or sales technology. Benefits: Competitive salary and equity options, comprehensive health insurance, flexible remote work environment, and professional development opportunities.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Prompt EngineeringMulti-Agent SystemsLLMsEvaluation FrameworksObservabilityTracingPrompt VersioningFunction CallingStructured OutputsVoice AgentsLatency OptimizationSystems Thinking

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Building a Robust Evaluation Layer
The role requires moving beyond manual prompt tweaking to creating systematic regression suites and scoring rubrics. You must demonstrate how to curate production-derived datasets to catch regressions before they reach customers, ensuring that prompt changes are backed by quantitative evidence rather than intuition.
Managing Multi-Agent Coherence
You will be responsible for the 'contracts' between agents, defining clear boundaries for what each agent knows, decides, and hands off. Success depends on maintaining system-wide stability as the agentic surface area grows, ensuring that individual agent improvements do not break the end-to-end chain.
Optimizing Real-Time Voice Interactions
Given that voice is a core component of the platform, you must prioritize latency management and turn-taking logic. You need to balance the complexity of prompt instructions with the strict performance constraints required for natural, real-time conversational AI.
Observability and Production Debugging
You must implement and utilize tracing and versioning to isolate root causes in live production environments. This involves moving from vague feedback to reproducible test cases, requiring a disciplined approach to instrumenting systems and analyzing trace data to identify where agents fail.

Likely challenges to think through

Context Assembly Complexity
Many prompt issues are actually context assembly problems. You will face the challenge of designing logic that dynamically injects relevant data at runtime without overwhelming the model or introducing noise, requiring deep systems thinking to manage the balance between context richness and model performance.
Model-Specific Behavioral Variance
Because the system utilizes multiple LLMs to optimize for cost and speed, you will encounter the difficulty of maintaining consistent agent behavior across different model architectures. Prepare to discuss strategies for prompt portability and how to adapt instructions when switching between frontier and open-weight models.
Balancing Velocity with Rigor
The company ships weekly, creating a tension between rapid iteration and the need for rigorous testing. You must prepare to defend your process for maintaining high-quality, versioned prompt artifacts while meeting aggressive deployment schedules without sacrificing system reliability.

Practice your answers

Technical practice

Designing Multi-Agent Contracts

What to coverExplain how you define the boundaries, responsibilities, and communication protocols between agents in a multi-agent system. Discuss how you ensure coherence when one agent hands off a task to another and how you prevent cascading failures.

Building Robust Evaluation Layers

What to coverDescribe your approach to creating a regression suite for LLM-based systems. Focus on how you curate datasets from production logs, define scoring rubrics that align with business outcomes, and automate the detection of regressions.

Debugging Production Agent Behavior

What to coverWalk through your process for isolating a root cause when an agent fails in a live, multi-step interaction. Mention how you utilize tracing, prompt versioning, and logs to identify whether the failure was due to the prompt, context assembly, or model reasoning.

Optimizing Voice Agent Latency

What to coverDiscuss the specific challenges of designing prompts for real-time voice agents, particularly regarding turn-taking and latency budgets. Explain how you balance the need for high-quality reasoning with the constraints of real-time speech.

Managing Dynamic Context Assembly

What to coverExplain how you handle runtime context injection for agents. Discuss strategies for ensuring the agent receives only the most relevant information to maintain performance and cost-efficiency while avoiding context window saturation.

Model-Agnostic Prompt Engineering

What to coverDescribe your strategy for maintaining consistent agent behavior when switching between different LLM models. Address how you adapt prompts to account for varying model architectures, reasoning capabilities, and instruction-following tendencies.

Observability in LLM Pipelines

What to coverDetail the key metrics and events you track to ensure system reliability. Suggestion: Focus on how you use tracing to visualize the chain of thought and decision-making steps across multiple agents.

Structured Outputs and Function Calling

What to coverExplain how you design prompts to ensure reliable structured outputs for downstream system consumption. Discuss how you handle edge cases where the model fails to adhere to the required schema or function call format.

Behavioural practice

Translating Qualitative Feedback into Metrics

What to coverDescribe a situation where you had to turn vague stakeholder feedback about an AI's performance into a concrete, measurable test case. Explain the steps you took to define the metric and verify the fix.

Handling Systemic Regressions

What to coverShare an example of a time when a prompt change fixed a specific issue but caused unintended regressions elsewhere. How did you identify the impact, and what process changes did you implement to prevent it from recurring?

Collaborating Across Functional Teams

What to coverProvide an example of how you have partnered with product or engineering teams to improve an AI product. Focus on how you communicated technical constraints or trade-offs to non-technical stakeholders.

Prioritizing Rigor in Fast-Paced Environments

What to coverDescribe how you balance the need for rapid deployment with the requirement for rigorous testing and evaluation. How do you decide when a feature is 'production-ready' in a high-growth environment?

Sharpen your knowledge

Topics to prepare

Multi-Agent System Coherence
Focus on designing agent contracts and handoff protocols. Exercise: Map out a multi-step workflow where one agent gathers data and another executes a decision, then define the specific constraints and 'never-do' rules for each to ensure system-wide stability.
Evaluation-Driven Development
Understand the shift from prompt tweaking to rigorous testing. Exercise: Create a small dataset of 10 edge-case production scenarios and write a rubric to score agent performance, specifically focusing on how to catch regressions when changing a prompt.
Real-Time Voice Agent Constraints
Study the technical challenges of voice-based LLM interactions. Exercise: Research strategies for managing latency budgets and turn-taking logic in conversational AI to ensure natural, low-latency user experiences.

Skills to revise

LLM Observability and Tracing
Practice instrumenting agent workflows to capture full execution traces. Exercise: Use a tracing tool to visualize a multi-step agent interaction, identify a bottleneck or failure point, and document the root cause analysis process.
Dynamic Context Assembly
Develop logic for runtime prompt construction. Exercise: Design a templating system that dynamically injects relevant user data into a prompt based on specific interaction triggers, ensuring the context remains concise and relevant.
Function Calling and Structured Outputs
Master the integration of LLMs with external tools. Exercise: Build a prototype where an agent must select and execute a specific function based on user input, ensuring the output is strictly formatted for downstream system consumption.

Your preparation checklist

Develop an evaluation framework case study
Prepare a walkthrough of a project where you built an evaluation layer for an LLM system. Include how you curated datasets, defined scoring rubrics, and used regression suites to prevent prompt regressions.
Demonstrate multi-agent orchestration expertise
Prepare a technical explanation of a multi-agent system you designed. Focus on how you managed agent contracts, function calling, and handoffs between agents to ensure coherent end-to-end behavior.
Prepare a voice agent debugging scenario
Create a summary of a real-world challenge you faced with voice agents, specifically addressing latency, turn-taking, or real-time constraints. Explain the diagnostic steps you took and how you optimized the system.
Formulate questions on production observability
Prepare thoughtful questions for the team regarding their current observability stack, how they handle prompt versioning at scale, and their process for managing dynamic context assembly at runtime.
Review your experience with model-agnostic prompting
Be ready to discuss how you adapt prompts across different LLM models to balance reasoning, speed, and cost, specifically highlighting your experience with open-weight versus frontier models.

Availability has not yet been verified. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…