New opportunity

Senior Engineer (India Office)

Wesco · Bangalore, India

About this role

Wesco is seeking a Senior Data Engineer with AI/GenAI expertise to join the Data Platform (Foundry) team. This role focuses on building and optimizing enterprise-scale data pipelines while enabling metadata-driven, AI-driven use cases including copilots, LLM integrations, and intelligent data workflows. Responsibilities include designing and maintaining data pipelines using Databricks, Spark, ADF, and Azure services; developing bronze-to-gold transformations; and orchestrating workflows via Airflow, Control M, or ADF. The role involves enabling AI/GenAI use cases such as copilot integrations, LLM-based workflows, and intelligent data enrichment, as well as integrating external AI platforms like Anthropic and Azure OpenAI. You will ensure data reliability, performance, and operational stability, troubleshoot production issues, and implement data quality and monitoring frameworks. Required qualifications include 8-10+ years of experience in Data Engineering, strong proficiency in Databricks (Delta Lake, Spark), Azure Data Platform (ADF, Synapse), and SQL (Azure SQL, SQL Server, Teradata, DB2, Oracle). Candidates must have hands-on experience with AI/GenAI technologies, including building copilot-style experiences and working with LLMs. Strong programming skills in Python and SQL are required, along with experience in orchestration tools like Airflow or Astronomer. Familiarity with production support and incident management is essential. The role is based in Bangalore, India, and requires on-site attendance.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Data EngineeringGenerative AILLMsDatabricksDelta LakeSparkAzure Data PlatformADFAzure SynapseSQLPythonAirflowAstronomerControl METL/ELTAnthropic ClaudeOpenAIAzure SQLSQL ServerTeradataDB2OracleFivetranAuto Loader

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Architecting AI-Ready Data Pipelines
The core mandate is to evolve traditional bronze-to-gold data transformations into intelligent workflows that support LLM integrations and copilot-style experiences. You must prioritize building scalable, metadata-driven pipelines using Databricks and Azure services that can feed high-quality, enriched data into AI models.
Operational Stability and Production Excellence
With a requirement for 8-10 years of experience, you are expected to maintain high reliability and performance for enterprise-scale data. This involves implementing robust monitoring, data quality frameworks, and incident management protocols to meet strict SLAs in a production environment.
Integration of External AI Platforms
You must bridge the gap between internal data infrastructure and external AI ecosystems. This requires hands-on experience integrating platforms like Azure OpenAI and Anthropic into existing workflows, ensuring secure and efficient data enrichment for AI-driven business use cases.

Likely challenges to think through

Balancing Legacy Data Integration with GenAI Innovation
You will likely face the challenge of integrating diverse legacy SQL environments (Teradata, DB2, Oracle) with modern GenAI workflows. Prepare by demonstrating how you have successfully modernized legacy data architectures while simultaneously implementing cutting-edge AI features without compromising system stability.
Managing Complex Orchestration at Scale
The role requires managing workflows across multiple tools like Airflow, ADF, and Control M. The challenge lies in maintaining consistent orchestration logic across a hybrid environment. Prepare by discussing your strategies for standardizing pipeline orchestration and managing dependencies in complex, multi-tool data ecosystems.

Practice your answers

Technical practice

Optimizing Bronze to Gold Data Transformations

What to coverExplain your approach to designing a medallion architecture in Databricks. Discuss how you handle schema evolution, data quality checks at each layer, and performance tuning for large-scale Delta Lake tables.

Integrating LLMs into Data Pipelines

What to coverDescribe how you would architect a workflow that feeds enterprise data into an LLM, such as Azure OpenAI or Anthropic. Address how you manage context window limitations, data privacy, and the orchestration of these calls within an Airflow pipeline.

Orchestration Strategy: Airflow vs. ADF

What to coverCompare the use cases for Airflow versus Azure Data Factory for complex enterprise workflows. Suggestion: Highlight your experience with Airflow's dynamic DAG generation and how it provides better flexibility for complex dependency management compared to ADF.

Data Quality and Monitoring Frameworks

What to coverHow do you implement automated data quality checks in a production environment? Discuss tools or custom frameworks you have used to ensure reconciliation between source systems like Teradata or Oracle and the target Azure Data Platform.

Building Copilot-style Experiences

What to coverWhat is your process for preparing data to be consumed by a copilot or RAG (Retrieval-Augmented Generation) system? Focus on vectorization, metadata management, and ensuring the data retrieved is accurate and relevant.

Production Incident Management

What to coverDescribe your methodology for troubleshooting a failed production pipeline. Explain how you use logging, monitoring tools, and SLAs to identify root causes quickly and prevent data drift or downtime.

Scalable ETL/ELT Patterns

What to coverDiscuss the trade-offs between ETL and ELT when working with massive datasets in Azure Synapse or Databricks. Explain when you would choose one over the other based on transformation complexity and latency requirements.

Python for Data Engineering

What to coverProvide an example of how you use Python to automate infrastructure tasks or complex data transformations that are not easily handled by SQL alone. Focus on modularity, error handling, and code maintainability.

Behavioural practice

Collaborating with Business Stakeholders

What to coverDescribe a time you had to translate complex technical data requirements into a solution for a non-technical business stakeholder. How did you ensure the final output met their strategic needs while maintaining platform governance?

Handling Production Failures under Pressure

What to coverShare an example of a critical production issue you resolved. Focus on your communication with the team, the steps taken to mitigate the impact, and the post-mortem process to prevent recurrence.

Driving AI Adoption Strategy

What to coverGive an example of how you have contributed to a team's adoption of new technologies or AI strategies. Explain how you convinced others to adopt a new tool or methodology and the resulting impact on team efficiency.

Balancing Governance and Speed

What to coverDescribe a situation where you had to balance the need for rapid deployment with strict code quality and compliance standards. How did you ensure that your work remained audit-ready without sacrificing project timelines?

Sharpen your knowledge

Topics to prepare

Medallion Architecture (Bronze-Silver-Gold)
The role requires developing data transformations using the Medallion architecture. Exercise: Design a conceptual workflow for a retail dataset that moves raw data through these three layers, ensuring data quality checks are applied at each transition to maintain reliability.
AI/GenAI Integration Patterns
The role involves integrating LLMs and copilot-style experiences into data pipelines. Exercise: Research and document the architectural differences between using an LLM for batch data enrichment versus real-time copilot interactions, specifically focusing on latency and data security considerations.
Production Incident Management
The role requires troubleshooting production issues under strict SLAs. Exercise: Create a checklist for a post-incident review process that identifies root causes in complex Spark/Databricks pipelines and outlines steps to prevent recurrence.

Skills to revise

Databricks and Delta Lake Optimization
You must demonstrate proficiency in Databricks. Exercise: Write a Python script using PySpark to optimize a Delta Lake table, focusing on Z-Ordering and file compaction to improve query performance for large-scale datasets.
Workflow Orchestration with Airflow
Airflow is explicitly preferred for orchestration. Exercise: Build a Directed Acyclic Graph (DAG) in Python that triggers an ADF pipeline, includes a sensor to check for data availability, and sends an alert if the task fails.
LLM API Integration
The role requires integrating external AI platforms like Anthropic or Azure OpenAI. Exercise: Write a Python function that sends a structured data payload to an LLM API, handles rate limiting, and parses the JSON response back into a format suitable for a database insert.

Your preparation checklist

Architect a Bronze-to-Gold Data Pipeline
Prepare a technical walkthrough of a scalable ETL/ELT pipeline you have built using Databricks and Delta Lake. Be ready to explain how you managed schema evolution, data quality checks at each layer, and how you optimized Spark performance for large-scale datasets.
Demonstrate AI/GenAI Integration Experience
Prepare a case study detailing your experience integrating LLMs (such as Azure OpenAI or Anthropic) into data workflows. Focus on how you handled data enrichment, prompt engineering within pipelines, or the architecture of a copilot-style application, specifically addressing data privacy and latency.
Map Orchestration and Production Support Expertise
Create a summary of your experience with Airflow or Astronomer for complex workflow orchestration. Include examples of how you implemented monitoring, alerting, and incident management frameworks to maintain high availability and meet strict SLAs in a production environment.
Prepare Targeted Questions for the Data Platform Team
Develop thoughtful questions regarding Wesco's current data platform modernization strategy. Examples include asking about the specific challenges of integrating external AI platforms into their existing Azure ecosystem or how the team balances metadata-driven development with rapid AI feature deployment.

Availability has not yet been verified. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…