New opportunity

AI Infrastructure Architect

Accenture · Bengaluru, India

About this role

Accenture seeks an AI Infrastructure Architect in Bengaluru for a hands-on AI LLM Technology Architecture role. The position architects and builds custom AI infrastructure and hardware solutions, optimizing performance, power consumption, cost and scalability while advising on technology and vendor selection and full-stack integration. It also involves turning Azure-based architecture into production-quality LLM, generative AI and agentic AI components for client engagements.

Responsibilities

Build prompts, tools, agents, orchestration flows, memory and context handling, retrieval pipelines and evaluation harnesses for LLM applications.
Develop Foundry-based agents; connect enterprise tools and knowledge; implement RAG with Azure AI Search and secure API or tool calls through API Management, Logic Apps and Functions. Instrument traces, evaluations, guardrails and policy controls.
Build ingestion, parsing, chunking, enrichment, embedding, vector-search and retrieval workflows for structured and unstructured content. Implement PII detection and redaction, prompt-injection defenses, content filters, access controls, lineage and audit logging.
Work with architects, data engineers, product owners and security stakeholders; contribute code, design decisions, tested integrations, deployment runbooks, evaluation results and reusable engineering patterns. Account for industry data, processes, controls and adoption risks.

Requirements

Minimum 5 years of experience and 15 years of full-time education; a bachelor’s degree in computer science, computer engineering, data science, AI/ML, IT or a related engineering discipline. The qualifications also state typical experience profiles of 5+ years in software, data or AI engineering with 2+ years in cloud-native engineering and 1+ year in GenAI, LLM, NLP or agentic AI delivery; and 7+ years with 3+ years in cloud-native architecture or engineering and 1–2+ years in those AI areas.
Hands-on Python coding; Azure AI Foundry, Foundry Agent Service, Azure OpenAI, Azure AI Search and other listed Azure services; LLM architecture, RAG, tool calling, embeddings, vector databases and evaluation. Experience with APIs, distributed systems, CI/CD, testing, observability, secure development, infrastructure-as-code and client-facing agile delivery is required. Candidates need AI/ML or data-product experience in at least one of banking, insurance, healthcare, the public sector, retail or energy.

Preferred

Microsoft Azure Data Services; relevant Azure certifications; Microsoft Fabric, Semantic Kernel, private networking, Defender for Cloud and Responsible AI governance; exposure to LangChain, LangGraph, LlamaIndex, Haystack, MLflow, FastAPI, Docker or Kubernetes; and experience with model risk, synthetic data, human-in-the-loop review, A/B testing or GenAI cost optimization.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Large Language Models (LLMs)Generative AIAI agentsMicrosoft AzureMicrosoft FoundryAzure AI FoundryFoundry Agent ServiceAzure OpenAIAzure AI SearchPrompt FlowAzure FunctionsLogic AppsAPI ManagementEvent GridMicrosoft FabricEntra IDKey VaultAzure MonitorApplication InsightsMicrosoft Azure Data ServicesPythonRAGPrompt engineeringEmbeddingsVector searchAgent orchestrationAPI developmentDistributed systemsCI/CDAutomated testing (software)Plausible but choose normal

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Production-Grade LLM Architecture
The role requires transitioning high-level designs into functional, scalable, and secure production components. You must demonstrate proficiency in building RAG pipelines, agent orchestration, and memory management using Azure AI Foundry and related services while ensuring reliability and performance.
Enterprise Security and Governance
Implementing robust guardrails is a core priority. This includes PII redaction, prompt-injection defense, content filtering, and audit logging. You must be prepared to integrate these security controls into the full-stack AI lifecycle to meet enterprise-grade compliance standards.
Cross-Functional Stakeholder Collaboration
As a technical lead, you must bridge the gap between architects, data engineers, and security teams. Success depends on your ability to document design decisions, create reusable engineering patterns, and deliver clear deployment runbooks for client-facing projects.

Likely challenges to think through

Balancing Innovation with Domain Constraints
You will face the difficulty of applying cutting-edge GenAI to highly regulated industries like banking or healthcare. Prepare by articulating how you navigate strict data privacy, process constraints, and model risk management without stifling the performance of the AI solution.
Optimizing Complex AI Infrastructure
The role demands balancing performance, power consumption, and cost. You may be challenged to justify architectural choices regarding vendor selection or resource allocation; prepare to discuss trade-offs in cost optimization and scalability within the Azure ecosystem.

Practice your answers

Technical practice

Designing RAG Pipelines on Azure

What to coverExplain your approach to building a RAG pipeline using Azure AI Search. Discuss how you handle data ingestion, chunking strategies, and embedding generation to ensure high retrieval relevance for enterprise content.

Agentic AI Orchestration

What to coverDescribe how you design multi-agent workflows using tools like Azure AI Foundry or Semantic Kernel. Focus on how you manage state, context handling, and tool-calling mechanisms to ensure agents perform reliable actions.

Securing LLM Applications

What to coverHow do you implement defense-in-depth for generative AI? Detail your strategies for PII redaction, prompt-injection mitigation, and enforcing guardrails within an Azure-based production environment.

LLM Evaluation Frameworks

What to coverWhat metrics and methodologies do you use to evaluate LLM performance in production? Discuss how you build evaluation harnesses to monitor accuracy, latency, and cost-efficiency.

Optimizing AI Infrastructure

What to coverWhen architecting custom AI solutions, how do you balance performance, power consumption, and cost? Suggestion: Focus on your experience with cloud-native scalability and resource management in Azure.

Data Governance and Lineage

What to coverHow do you ensure auditability and lineage in an AI pipeline? Explain how you integrate logging and monitoring tools like Azure Monitor and Application Insights to track data flow and model decisions.

CI/CD for AI Components

What to coverDescribe your process for implementing CI/CD pipelines for AI models and agentic components. Address how you handle automated testing, version control for prompts, and infrastructure-as-code.

Handling Unstructured Data

What to coverWhat are the challenges in parsing and enriching unstructured enterprise content for LLM consumption? Discuss your techniques for cleaning data before embedding and vectorization.

Behavioural practice

Managing Stakeholder Expectations

What to coverDescribe a time you had to explain complex AI architecture or risks to a non-technical stakeholder. How did you ensure they understood the trade-offs regarding security, cost, or model limitations?

Navigating Domain-Specific Constraints

What to coverProvide an example of a project where you had to adapt an AI solution to meet strict industry regulations (e.g., banking or healthcare). How did you balance innovation with compliance requirements?

Collaborative Problem Solving

What to coverTell me about a time you worked with data engineers and security teams to resolve a production issue. How did you align the team to implement a fix while maintaining system stability?

Handling Technical Debt in AI

What to coverDescribe a situation where you had to refactor an existing AI component for better scalability or maintainability. What was your approach to minimizing disruption while improving the architecture?

Sharpen your knowledge

Topics to prepare

Retrieval-Augmented Generation (RAG) Architecture
Focus on the end-to-end flow of RAG systems, specifically ingestion, chunking strategies, embedding models, and vector search optimization. Exercise: Design a document processing pipeline that handles unstructured enterprise data, including PII redaction and metadata enrichment, and explain how you would evaluate retrieval accuracy.
Agentic AI and Orchestration
Understand the architecture of multi-agent systems, including tool calling, memory management, and context handling. Exercise: Map out a workflow using Azure AI Foundry or similar frameworks where an agent must decide between multiple tools to fulfill a user request, focusing on how to maintain state and handle errors.
Production AI Governance and Security
Study the implementation of guardrails, prompt-injection defenses, and audit logging in enterprise environments. Exercise: Propose a security architecture for an LLM application that includes authentication via Entra ID, content filtering, and lineage tracking to meet strict enterprise compliance standards.

Skills to revise

Azure AI Foundry and Service Integration
Gain hands-on proficiency with Azure AI Foundry, Azure OpenAI, and Azure AI Search. Exercise: Build a small prototype that connects an Azure OpenAI model to a custom knowledge base using Azure AI Search, ensuring the integration uses secure API management and monitoring via Application Insights.
Python-based LLM Engineering
Develop production-quality code for LLM applications using Python. Exercise: Write a script that implements a robust evaluation harness for an LLM, testing for specific metrics like faithfulness or relevance, and integrate it into a CI/CD pipeline to automate quality checks.
Infrastructure-as-Code and Observability
Practice deploying and monitoring scalable AI infrastructure. Exercise: Create a deployment runbook for an LLM service that includes infrastructure-as-code templates and defines key observability metrics (latency, token usage, error rates) to be tracked in Azure Monitor.

Your preparation checklist

Architect a RAG pipeline walkthrough
Prepare a technical overview of a RAG system you have built, specifically detailing your approach to data ingestion, chunking strategies, embedding models, and vector database selection within an Azure environment.
Demonstrate Azure AI Foundry expertise
Be prepared to discuss your experience using Azure AI Foundry and Prompt Flow to manage the lifecycle of LLM applications, including how you handle prompt versioning, evaluation harnesses, and agent orchestration.
Prepare security and governance case studies
Develop specific examples of how you have implemented PII redaction, prompt-injection defenses, and content guardrails in production, demonstrating your understanding of Responsible AI and enterprise security controls.
Review cloud-native engineering patterns
Refresh your knowledge on integrating Azure services like API Management, Logic Apps, and Functions into AI workflows, ensuring you can explain how to maintain observability and audit logging in distributed systems.
Formulate domain-specific questions
Prepare questions for the interviewers regarding how Accenture balances cost optimization with performance in client-facing GenAI projects, and how they manage model risk in highly regulated industries like banking or healthcare.

Availability last checked: 4 Nov 2026. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…