New opportunity

QA Engineer – AI Solutions

APPORT SOFTWARE SOLUTIONS PRIVATE LIMITED · Mumbai, India

About this role

DoubleTick is hiring a QA Engineer for its AI Solutions practice in Mumbai. The role validates enterprise AI voice bots, chat agents, and automation flows before launch. Rather than stress-testing the established core product, the engineer tests the prompt architecture, intent filters, and query logic that vary by deployment. They work alongside Forward-Deployed Engineers, AI Engineers, and Solution Engineers throughout delivery.

Responsibilities

Build and maintain test suites for AI voice and chat deployments across WhatsApp, PSTN, and SIP. Assess prompt behavior, intent classification, fallback handling, and response quality across user journeys; identify and document conversational edge cases.
Test end-to-end flows from user input and LLM processing through API callbacks and CRM or backend updates. Maintain test logs, failure reports, and regression records for each deployment.
Develop a reusable test-case playbook for patterns such as sales bots, support agents, and lead-qualification flows. Collaborate with Solution Engineers on Validation & Requirement Documents (VRDs) that turn client requirements into testable acceptance criteria and support go-live sign-off.
Participate from solutioning review through user acceptance testing and post-deployment monitoring. Flag quality risks during scoping and act as the quality gate before client launch. Success includes avoiding critical post-launch failures caused by untested conversational logic, reducing QA time through reusable tests, and producing implementation-ready VRDs.

Requirements

1–5 years of QA experience, including some exposure to API testing or conversational/AI product testing. Candidates must be able to reason about user journeys and edge cases, use Postman or equivalent tools to test webhooks and integration endpoints, produce actionable documentation, and work with multiple stakeholders in a fast-moving, ambiguous environment.

Preferred

Experience testing LLM products, chatbots, or voice bots; familiarity with WhatsApp Business API or another conversational platform; basic Python or JavaScript scripting for test automation; and exposure to Jira, Notion, or structured project-delivery tools.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

QAAPI testingPostmanTest case designConversational AI testingLLM testingPrompt testingIntent classification testingRegression testingIntegration testingWebhook testingUser acceptance testingTechnical documentationWhatsApp Business APIPythonJavaScriptJiraNotion

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Validate deployment-specific conversations
Build test suites for voice bots, chat agents, and automation flows across WhatsApp, PSTN, and SIP. Focus on prompt behavior, intent classification, fallback handling, response quality, and conversational edge cases across defined user journeys.
Verify complete integration flows
Test from user input through LLM processing to API callbacks and CRM or backend updates. Use Postman or an equivalent tool to validate webhooks and integration endpoints, and record failures and regressions per deployment.
Turn requirements into launch criteria
Collaborate with Solution Engineers on Validation & Requirement Documents (VRDs) that make client requirements testable and serve as the source of truth for go-live sign-off.
Make QA reusable and early
Maintain a standardized test-case playbook for recurring sales, support, and lead-qualification patterns. Join solutioning reviews, flag quality risks during scoping, and remain involved through UAT and post-deployment monitoring.

Likely challenges to think through

Making conversational behavior testable
Likely preparation challenge: varied phrasing and fallback paths may make a simple pass/fail checklist inadequate. Prepare to explain how you would define expected behavior for a user journey, probe edge cases, and document reproducible findings against acceptance criteria.
Locating failures across system boundaries
Likely preparation challenge: an incorrect outcome could originate in conversational logic, an API callback, or a CRM/backend update. Prepare a step-by-step tracing approach that uses test inputs, endpoint checks, and logs to isolate the failing stage.
Balancing reuse with deployment-specific risk
Likely preparation challenge: a common test library can shorten QA cycles, but each client deployment changes prompts, intent filters, and query logic. Prepare to distinguish reusable baseline cases from cases derived from that deployment’s VRD, especially before go-live sign-off.

Practice your answers

Technical practice

How would you design test cases for a lead-qualification bot's user journeys?

What to coverThe posting calls for journey-based testing and reusable cases for lead-qualification flows. Suggested preparation: describe how you would map expected paths, ambiguous answers, missing information, and exits to observable outcomes without assuming a particular client workflow.

How would you validate prompt behaviour and response quality before launch?

What to coverThe role tests deployment-specific prompt architecture and response quality against defined journeys. Suggested preparation: explain how you would compare responses with intended behaviour, record inconsistent results, and distinguish a prompt defect from an unclear requirement.

How would you test intent classification and fallback handling?

What to coverThe posting explicitly names intent accuracy and fallback handling. Suggested preparation: cover clear, overlapping, and unrecognised requests, then explain what evidence would show whether classification or fallback behaviour needs correction.

What conversational edge cases would you check across WhatsApp and voice channels?

What to coverThe role covers WhatsApp, PSTN, and SIP deployments and asks QA to uncover overlooked conversational edge cases. Suggested preparation: consider how interrupted, incomplete, or misunderstood exchanges could affect a journey, while checking each channel against its documented expected behaviour.

How would you test a webhook or integration endpoint with Postman or an equivalent tool?

What to coverAPI testing of webhook flows and integration endpoints is a must-have in the posting. Suggested preparation: explain how you would send representative requests, inspect responses and callbacks, and document both successful and failing cases against the expected contract.

How would you trace an end-to-end failure between a user's message and a CRM update?

What to coverThe advertised flow runs from user input through LLM processing and API callbacks to CRM or backend updates. Suggested preparation: describe how test logs and observed outputs could isolate the failing step before you file an actionable report.

How would you build a reusable regression playbook without overlooking deployment-specific logic?

What to coverThe posting calls for a standardised test-case playbook for sales, support, and lead-qualification patterns, alongside per-deployment regression records. Suggested preparation: separate common journey checks from prompt, intent-filter, and query-logic checks that must be adapted for each deployment.

How would you turn a client's conversational requirement into a testable VRD acceptance criterion?

What to coverVRDs are described as implementation-ready documents developed with Solution Engineers for go-live sign-off. Suggested preparation: show how you would clarify an ambiguous requirement, state the expected observable behaviour, and link it to a test case.

What evidence would you review before recommending go-live sign-off?

What to coverThe role acts as a quality gate and participates in UAT and post-deployment monitoring. Suggested preparation: discuss test results, unresolved conversational risks, integration failures, and acceptance criteria, making clear what would need resolution or an explicit decision.

When would you automate repetitive conversational tests with Python or JavaScript?

What to coverBasic Python or JavaScript scripting is listed as good to have, not a requirement. Suggested preparation: explain which repeatable checks might benefit from scripting and which response-quality judgments still need careful review.

Behavioural practice

Tell me about a time you raised a quality risk during scoping rather than waiting for testing.

What to coverThe posting expects early quality flags during solutioning. Use a real situation, your specific action with stakeholders, and the result; explain how you made an uncertain risk concrete.

Tell me about a time you resolved an ambiguous requirement with people from different functions.

What to coverThe role works alongside engineering and solution teams in an ambiguous delivery environment. Use a real situation, the questions or coordination you led, and the resulting testable agreement or remaining uncertainty.

Tell me about a time you documented a defect so others could act without a follow-up meeting.

What to coverThe posting emphasises actionable test cases, failure reports, and regression records. Give a real situation, what evidence and reproduction steps you recorded, and how the report affected the outcome.

Tell me about a time you found an edge case that a planned test path missed.

What to coverThe role calls for identifying overlooked conversational logic, not just following pass/fail checklists. Describe the real situation, how you discovered and tested the case, and what changed as a result.

Tell me about a time you handled an unresolved issue near a release or UAT milestone.

What to coverThe posting places QA at UAT and the pre-launch quality gate. Use a real situation, how you communicated the evidence and trade-off to stakeholders, and the decision and result; do not present an assumed DoubleTick release process as fact.

Sharpen your knowledge

Topics to prepare

Conversational behavior and edge cases
Map a sample support-agent journey, then identify ambiguous intents, unexpected replies, fallback points, and response-quality checks. The posting explicitly calls for validating prompt behavior, intent classification, fallbacks, and conversational edge cases.
Deployment-specific test scope
Compare two hypothetical deployments with different prompts, intent filters, and query logic; list which tests must change and which can be reused. The role focuses on validating what varies by deployment rather than stress-testing the established core product.
End-to-end integration flow
Trace a hypothetical conversation from user input through LLM processing, an API callback, and a CRM update; mark where a failure could occur and what evidence would reveal it. This is the integration path named in the posting.
Testable acceptance criteria
Turn a brief lead-qualification requirement into observable acceptance criteria, including an ambiguous response and a fallback outcome. The posting says VRDs translate client requirements into testable criteria and support go-live sign-off.
Quality across the delivery lifecycle
For a hypothetical voice-bot launch, distinguish a scoping risk, a UAT finding, a go-live blocker, and an issue to monitor afterward. The role participates from solutioning review through post-deployment monitoring and acts as a pre-launch quality gate.

Skills to revise

Design reusable conversational test cases
Write a small test suite for a sales bot with inputs, expected behavior, edge cases, and regression cases; identify which cases could be reused for another deployment. The posting calls for maintained suites and a playbook covering recurring bot patterns.
Validate webhooks with Postman or equivalent
Using a practice endpoint, send a representative request, inspect the response, and record how you would verify a callback and downstream update. Proficiency with an API testing tool for webhook flows and integration endpoints is a stated must-have.
Document actionable failures
Draft a failure report containing the journey, input, expected and observed results, relevant callback evidence, and steps to reproduce; add a regression entry. The role maintains test logs, failure reports, and regression records per deployment.
Draft a concise VRD section
Write one sample requirement, its acceptance criteria, test cases, and an unresolved scope question for review with a Solution Engineer. Collaborative, implementation-ready VRDs are an explicit responsibility.
Automate a repetitive scenario
As optional preparation, use basic Python or JavaScript to repeat a small set of conversational test inputs and compare recorded outcomes. Basic scripting for repetitive test scenarios is listed as good to have, not a must-have.

Your preparation checklist

Map the deployment quality gate
Summarize how this role validates deployment-specific prompts, intent filters, and query logic before launch, rather than stress-testing the established core product.
Prepare a QA project walkthrough
If you have a relevant example, outline the user journey, test design, defect investigation, documentation, and stakeholder decisions. Otherwise, clearly label a practice example as hypothetical.
Design a conversational test suite
For a hypothetical lead-qualification or support bot, draft cases covering prompt behavior, intent classification, fallback handling, response quality, and conversational edge cases. Include expected outcomes.
Trace an end-to-end integration test
Sketch a user input moving through LLM processing, an API callback, and a CRM or backend update. Prepare example Postman or equivalent checks for the webhook and integration endpoint, including what a failure report would capture.
Draft testable acceptance criteria
Turn a short hypothetical client request into a sample Validation & Requirement Document with clear acceptance criteria, open questions, and go-live checks. Label it as a practice artifact, not a company document.
Show how tests stay reusable
Prepare a small test-case template and regression record that could be reused across sales, support, or lead-qualification deployments. Explain how you would update them after a defect or deployment change.
Bring delivery-focused questions
Ask how Solution Engineers and QA resolve unclear requirements, how quality risks are raised during scoping, and how UAT findings and post-deployment monitoring inform go-live decisions.

Availability last checked: 4 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…