About this role
The Manager for Automation Testing will lead quality strategy and engineering efforts for AI-powered applications within Deloitte's Technology & Transformation practice. This role focuses on ensuring the accuracy, safety, reliability, and compliance of LLM-based systems through robust testing frameworks and evaluation methodologies. Responsibilities: Own end-to-end test strategy, automation roadmaps, and quality gates. Design and maintain scalable automation frameworks using Playwright with Python and BDD/Behave. Automate UI, API, integration, and regression testing. Validate AI/LLM systems for accuracy, relevance, hallucination, bias, and safety. Test prompt injection, RAG workflows, and AI agent integrations. Build LLM evaluation frameworks, benchmark datasets, and scorecards. Integrate tests into CI/CD pipelines and conduct performance testing. Mentor QA engineers and collaborate with cross-functional teams including Data Science and Security. Requirements: Bachelor's degree in Computer Science, IT, or related field. Minimum 9 years of experience in Quality Engineering and Test Automation. Strong hands-on expertise in Playwright with Python. Proven experience in testing AI, Generative AI, or LLM-powered applications. Proficiency in API testing, BDD, Git, and CI/CD practices. Knowledge of AI evaluation, prompt testing, and Responsible AI principles. Strong problem-solving, communication, and stakeholder management skills.