New opportunity

Data Engineering Lead

RhythmX AI · Bangalore, India

About this role

RhythmX AI is seeking an experienced Data Engineering Lead to join their team in Bangalore. This role is pivotal in building and managing robust data pipelines that extract and process data from health system EHR systems to support AI-driven healthcare applications. Responsibilities include developing data pipelines for large-scale healthcare data, managing sensitive data in compliance with FHIR and HL7 standards, and utilizing orchestration tools like Airflow and Temporal for complex AI-enabled workflows including RAG and GraphRAG. The lead will collaborate with data scientists and backend developers to maintain data systems and validate Gen-AI solutions using tools like RAGAs and Opik. Minimum qualifications include a Bachelor's degree in a relevant field and 10+ years of experience in data engineering with proficiency in Python, PostgreSQL, MongoDB, and Milvus. Candidates must have experience building pipelines with clinical data sources such as EHRs and HIEs. Highly preferred qualifications include expertise in the Azure technology stack, clinical data quality management, anomaly detection, and experience with large-scale recommendation systems or reinforcement learning. The role requires strong knowledge of LLMs, prompt engineering, and embedding techniques, along with hands-on experience implementing GenAI solutions in production.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

PythonMongoDBPostgreSQLApache AirflowMilvus Vector DBFHIRHL7Azure OpenAILLMPrompt EngineeringRAGGraphRAGLangChainLlamaIndexNLPData PipelinesETLAzureWeaviateTemporal

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Healthcare-Compliant Data Pipeline Engineering
The primary mandate is to build and manage robust data pipelines that ingest sensitive EHR and clinical data. You must ensure all data processing adheres strictly to FHIR and HL7 standards, requiring a deep understanding of clinical data normalization, validation, and secure exchange protocols.
GenAI Workflow Orchestration
You are responsible for implementing complex AI-enabled workflows using tools like Airflow and Temporal. This includes managing RAG and GraphRAG architectures, tool calling, and retrieval strategies to support AI-driven healthcare applications, ensuring high performance and low latency.
Production-Grade GenAI Validation
Beyond construction, you must lead the validation of GenAI solutions in production environments. This involves utilizing frameworks like RAGAs and Opik to monitor, debug, and optimize prompt engineering, embedding techniques, and overall system accuracy.

Likely challenges to think through

Balancing Clinical Data Complexity with AI Performance
Healthcare data is notoriously fragmented and variable (e.g., Epic Clarity extracts). The challenge lies in maintaining high-quality, normalized data for LLMs while meeting the strict latency requirements of real-time clinical copilots. Prepare by discussing your experience with data quality management and how you handle schema evolution in high-stakes environments.
Navigating Regulatory Constraints in GenAI
Implementing GenAI in a healthcare context requires rigorous adherence to HIPAA and data governance standards. You will face the difficulty of building innovative RAG systems that do not compromise patient privacy or data security. Prepare by highlighting your experience with PHI/PII governance and secure data handling within AI pipelines.

Practice your answers

Technical practice

Healthcare Data Interoperability Standards

What to coverExplain your experience implementing FHIR and HL7 standards within data pipelines. Discuss how you handle the inherent variability in EHR extracts, such as Epic Clarity, while maintaining data integrity and compliance.

Orchestrating Complex AI Workflows

What to coverDescribe how you utilize tools like Apache Airflow or Temporal to manage AI-native workflows. Focus on how you handle dependencies in RAG and GraphRAG pipelines, specifically regarding tool calling and retrieval accuracy.

Vector Database Optimization

What to coverGiven your experience with Milvus or Weaviate, how do you approach indexing strategies for large-scale healthcare datasets? Explain how you balance retrieval latency with the accuracy requirements of clinical AI applications.

GenAI Production Lifecycle

What to coverDetail your process for taking a GenAI solution from prototype to production. Address how you manage prompt optimization, context window constraints, and performance monitoring using tools like RAGAs or Opik.

Data Pipeline Scalability in Healthcare

What to coverHow do you design data pipelines to handle the scale of longitudinal patient data while ensuring secure, HIPAA-compliant processing? Discuss your approach to ETL, normalization, and validation for sensitive clinical information.

Embedding and Chunking Strategies

What to coverWhat criteria do you use to select chunking strategies for clinical documents? Explain how different embedding techniques impact the performance of RAG systems in a medical context.

Reinforcement Learning and Feedback Loops

What to coverIf you have experience with reinforcement learning or recommendation systems, describe how you incorporate end-user clinical feedback into the model lifecycle. How do you ensure the system learns safely from this feedback?

Anomaly Detection in Clinical Data

What to coverHow do you implement anomaly detection techniques to identify outliers or data quality issues in large-scale healthcare datasets? Provide an example of how you have used these insights to improve downstream AI model performance.

Behavioural practice

Cross-Functional Collaboration

What to coverDescribe a situation where you had to bridge the gap between data scientists and backend engineers to deploy a complex AI system. How did you align the team on technical requirements and project goals?

Navigating Technical Ambiguity

What to coverTell me about a time you faced a unique technical challenge in the healthcare domain that lacked a standard solution. How did you approach the problem-solving process and what was the outcome?

Mentorship and Team Leadership

What to coverAs a lead, how do you guide your team through the complexities of GenAI implementation? Provide an example of how you have supported a team member's growth while ensuring project deadlines were met.

Managing Stakeholder Expectations

What to coverDescribe a time you had to explain the limitations or risks of a GenAI solution to non-technical stakeholders. How did you balance their expectations with the technical realities of the system's performance?

Sharpen your knowledge

Topics to prepare

Healthcare Data Interoperability Standards
Review the FHIR and HL7 standards to understand how clinical data is structured for exchange. Exercise: Map a sample EHR extract to a FHIR resource structure to demonstrate your ability to handle clinical data compliance.
Generative AI Orchestration and Retrieval
Study RAG and GraphRAG architectures for healthcare applications. Exercise: Design a workflow diagram using Airflow or Temporal that integrates a vector database with an LLM to answer clinical queries while maintaining data provenance.
Gen-AI Production Validation
Understand frameworks for evaluating LLM performance, specifically RAGAs and Opik. Exercise: Define a set of metrics to evaluate the accuracy and latency of a RAG-based clinical assistant, focusing on minimizing hallucinations.

Skills to revise

Vector Database Implementation
Gain hands-on experience with Milvus or Weaviate. Exercise: Build a small-scale index of medical documents and perform a similarity search to demonstrate proficiency in vector retrieval techniques.
Python-based Data Pipeline Orchestration
Develop expertise in using Apache Airflow for complex data workflows. Exercise: Create a DAG that extracts data from a mock EHR source, transforms it, and loads it into a PostgreSQL database.
Prompt Engineering and Context Management
Master techniques for optimizing LLM outputs through prompt tuning and context window management. Exercise: Refine a system prompt for a clinical summarization task to improve output consistency and reduce irrelevant information.

Your preparation checklist

Prepare a production GenAI case study
Draft a technical walkthrough of a production-grade RAG or GraphRAG implementation you led. Focus on how you managed data ingestion, chunking strategies, and retrieval accuracy, specifically addressing how you handled latency and performance scaling.
Map your healthcare data experience to FHIR/HL7
Create a summary of your experience working with clinical data standards like FHIR and HL7. Be prepared to discuss how you have handled the variability of EHR extracts (e.g., Epic Clarity) and ensured compliance with PHI/PII governance.
Develop an orchestration workflow demonstration
Prepare to explain a complex data pipeline you built using Airflow or Temporal. Highlight how you integrated these tools to manage AI-enabled workflows, such as tool calling or multi-step retrieval processes.
Formulate questions on GenAI validation and monitoring
Prepare thoughtful questions regarding the company's current stack for validating GenAI outputs. Ask about their specific use cases for tools like RAGAs or Opik and how they approach model monitoring in a clinical context.
Review your vector database and Azure expertise
Be ready to discuss your hands-on experience with Milvus or Weaviate and the Azure OpenAI stack. Prepare examples of how you optimized embedding techniques and prompt tuning to improve the quality of AI-driven healthcare applications.

Availability has not yet been verified. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…