New opportunity

Senior Machine Learning Engineer - Responsible AI

Atlassian · Bengaluru, India

About this role

Atlassian is seeking a Senior Machine Learning Engineer to join the Responsible AI (RAI) & AI Safety engineering team in India. This role focuses on building, operating, and advancing centralized AI safety, content moderation, and guardrail platforms that power Atlassian Intelligence and Rovo. You will design and deploy systems that mitigate risks such as jailbreaks, prompt injection, harmful content, and bias in high-throughput production environments. Responsibilities include architecting safety models, building robust evaluation and red-teaming frameworks, optimizing for real-time inference latency, and ensuring global compliance with AI regulations. You will collaborate with cross-functional teams to steer technical strategy and mentor junior engineers. Required qualifications include a Bachelor’s or Master’s degree in a quantitative field and 7+ years of industry experience in production ML systems. You must have deep expertise in NLP, LLMs, transformer architectures, and fine-tuning techniques, along with proficiency in Python and experience with cloud-native deployment (AWS/GCP, Kubernetes). Preferred qualifications include experience in Trust & Safety, AI red-teaming, inference engine optimization (e.g., vLLM, TensorRT-LLM), and familiarity with AI governance frameworks like the EU AI Act or ISO 42001. This is a remote position requiring the candidate to be based in India.

For full details about this role, click “Apply on company site” to view the employer’s original posting.

Skills for this role

Machine LearningNLPLLMsTransformer ArchitecturesPrompt EngineeringFine-tuningPEFTLoRAModel AlignmentPythonKotlinJavaGoEvaluation PipelinesBenchmarkingData VisualizationAWSGCPKubernetesDockerSparkSQLTrust & SafetyContent ModerationAI Red-TeamingAdversarial RobustnessTriton Inference ServervLLMTensorRT-LLMModel Quantization

Checking your profile…

Preparation suggestions, not the employer’s actual interview questions. Always check the original posting for current requirements.

Understand the role

What matters most

Architecting Scalable AI Safety Guardrails
The primary objective is to build and maintain centralized platforms that mitigate risks like jailbreaks, prompt injection, and harmful content within high-throughput production environments. You must prioritize designing systems that balance robust safety filtering with the performance requirements of Atlassian Intelligence and Rovo.
Developing Rigorous Evaluation and Red-Teaming Frameworks
You are expected to design offline and online evaluation pipelines, including LLM-as-a-judge systems and adversarial test suites. This requires a focus on measuring precision, recall, and model robustness to ensure safety controls are effective against evolving threats and model drift.
Optimizing Inference for Real-Time Performance
A critical priority is maintaining low-latency (P50/P99) performance while running safety models inline. You will need to apply techniques such as model quantization and inference engine optimization to ensure that security measures do not degrade the user experience.
Ensuring Global Regulatory Compliance
You must align technical safety controls with enterprise standards and emerging AI regulations like the EU AI Act and ISO 42001. This involves implementing verifiable audit logging, watermarking, and governance frameworks to meet global compliance requirements.

Likely challenges to think through

Balancing Safety Sensitivity with User Experience
The core challenge is managing the trade-off between strict safety filtering (precision/recall) and system latency. To prepare, demonstrate your experience in tuning thresholds for high-consequence classification systems and your ability to optimize inference engines like vLLM or TensorRT-LLM to minimize overhead.
Defending Against Evolving Adversarial Tactics
As an AI safety engineer, you must anticipate and mitigate sophisticated threats like unauthorized tool-calling and prompt injection in agentic systems. Prepare by articulating your methodology for building adversarial test suites and your approach to continuous monitoring for emerging attack vectors.
Navigating Cross-Functional Technical Strategy
You will need to collaborate with diverse teams including Legal, Product, and AI Platform to implement safety standards. Prepare by highlighting your experience in communicating complex ML trade-offs to non-technical stakeholders and your ability to mentor junior engineers while steering technical strategy.

Practice your answers

Technical practice

Mitigating Prompt Injection and Jailbreaks

What to coverExplain your approach to designing guardrails that detect and block adversarial prompts in real-time. Discuss the trade-offs between strict filtering and user experience, and how you would handle evolving attack vectors in agentic systems.

Optimizing LLM Inference Latency

What to coverDescribe techniques for optimizing high-throughput production inference, such as model quantization, vLLM, or TensorRT-LLM. Focus on how you balance P99 latency targets with the computational requirements of safety-critical models.

Designing Evaluation Frameworks for Safety

What to coverHow do you construct robust evaluation pipelines for LLM safety? Address the use of LLM-as-a-judge, adversarial red-teaming, and how you measure precision and recall when dealing with high-consequence content moderation.

Fine-tuning and Model Alignment

What to coverDiscuss your experience with PEFT and LoRA for aligning models to specific safety policies. Explain how you ensure that fine-tuning for safety does not degrade the model's general performance or utility.

Productionizing ML Services on Kubernetes

What to coverDetail your process for deploying and scaling ML services in a cloud-native environment. Focus on containerization, managing distributed data processing with Spark, and ensuring high availability for critical safety infrastructure.

Addressing Model Drift in Production

What to coverHow do you monitor and detect drift in safety models once they are deployed? Suggest strategies for continuous improvement and retraining cycles to maintain performance against emerging threats.

AI Governance and Regulatory Compliance

What to coverExplain how you incorporate requirements from frameworks like the EU AI Act or ISO 42001 into the technical design of an AI system. Discuss the importance of audit logging and watermarking in this context.

Handling Multi-modal Safety

What to coverWhat unique challenges arise when moderating multi-modal inputs compared to text-only models? Suggest architectural considerations for building unified safety platforms that handle diverse data types.

Behavioural practice

Navigating Technical Ambiguity

What to coverDescribe a situation where you had to define a technical strategy for a complex, ambiguous problem. How did you break down the requirements and align cross-functional stakeholders on the proposed solution?

Mentorship and Technical Leadership

What to coverProvide an example of how you have mentored junior engineers or influenced the technical growth of your team. Focus on how you balance your own delivery responsibilities with the need to elevate the team's capabilities.

Cross-functional Collaboration

What to coverTell us about a time you collaborated with non-technical teams, such as Legal or Product, to implement a safety feature. How did you translate complex ML trade-offs into language that helped them make informed decisions?

Managing High-Stakes Production Incidents

What to coverDescribe a time when a production system failed or behaved unexpectedly. How did you identify the root cause, mitigate the impact, and what steps did you take to prevent recurrence?

Sharpen your knowledge

Topics to prepare

AI Safety and Adversarial Robustness
Focus on the conceptual frameworks for mitigating jailbreaks, prompt injection, and model hallucinations. Exercise: Research and document a threat model for an LLM-based agent, identifying potential failure modes and corresponding defensive strategies like input sanitization or output filtering.
AI Governance and Regulatory Compliance
Understand the requirements of frameworks like the EU AI Act and ISO 42001. Exercise: Review the core pillars of the EU AI Act and map them to technical requirements such as audit logging, model transparency, and risk management documentation.
Evaluation Frameworks for LLMs
Study methodologies for measuring model performance, including precision/recall trade-offs and LLM-as-a-judge techniques. Exercise: Design an evaluation pipeline that uses a stronger model to grade the safety outputs of a smaller, faster model, focusing on defining clear metrics for success.

Skills to revise

Inference Optimization
Gain hands-on experience with tools like vLLM or TensorRT-LLM to reduce latency in production. Exercise: Set up a local environment to benchmark the latency and throughput of a small transformer model using vLLM, comparing performance with and without quantization.
Fine-tuning and Model Alignment
Practice PEFT and LoRA techniques to align models for specific safety or moderation tasks. Exercise: Implement a LoRA fine-tuning script on a small dataset of harmful vs. safe prompts to observe how parameter-efficient methods impact model behavior.
Cloud-Native ML Deployment
Master the deployment of containerized ML services on Kubernetes. Exercise: Create a Dockerfile for a Python-based inference service and define a Kubernetes deployment manifest that includes resource limits and health checks suitable for high-throughput environments.

Your preparation checklist

Architect a high-throughput safety guardrail system
Prepare a technical design document or project walkthrough demonstrating how you would implement a low-latency guardrail system to detect prompt injection or harmful content in an LLM pipeline. Focus on the trade-offs between model accuracy, inference latency (P99), and cost-effective deployment using tools like vLLM or Triton.
Develop an adversarial red-teaming framework
Create a sample evaluation plan for testing LLM robustness. Include how you would build an 'LLM-as-a-judge' pipeline to automate the detection of jailbreaks or hallucinations, and explain how you would manage precision and recall trade-offs in a production environment.
Demonstrate production-grade ML engineering skills
Prepare to discuss your experience deploying containerized ML models on Kubernetes (AWS/GCP). Be ready to explain how you handle model drift, versioning, and monitoring in high-consequence systems, ensuring your code is clean, testable, and scalable.
Review AI governance and compliance frameworks
Familiarize yourself with the core requirements of the EU AI Act or ISO 42001. Prepare to discuss how you would integrate audit logging, watermarking, or verifiable safety controls into a product roadmap to meet enterprise regulatory standards.
Prepare thoughtful questions for the team
Formulate questions regarding the current challenges of the RAI team at Atlassian, such as how they balance user experience with strict safety guardrails, or how they prioritize emerging threats in agentic systems. This demonstrates your strategic thinking and alignment with their mission.

Availability has not yet been verified. Status can change after a check; confirm on the employer’s website. Report an issue · Our checking policy

KEEP EXPLORING

Similar AI jobs

Related skills and specializations in India. Matched to this role, not your profile.

Explore more jobs

Finding similar opportunities…